Copy CSV file to IDOC in XI

Hi,
Please explain how to copy CSV file to IDOC .
And How to define data type and how to find out field names in CSV file to IDOC .
Advance thanks ,
Laxman

if your scenario is CSV(text) file to idoc then refer below link
http://www.****************/Tutorials/XI/FileToIDoc/FiletoIDocScenario.htm
u have to use FCC from message type insted of File in above link and refer below link for how to call fcc parameters .
http://www.riyaz.net/sap/xipi-file-content-conversion-for-complex-structures/79/
remaining evarytning is same as norma scenario just you have to use fcc parameters in file sender and call parameters On Content conversion tab .
and refer SCN further information because it is very basic Question ( if ur scenario is CSV file to IDOC)

Similar Messages

Copy CSV file from windows to linux

Our oracle is installed on UNIX machine so we have to place CSV file there.
We are on windows.We are using Putty client to access UNIX server.
how to copy CSV file from windows to UNIX server?
Thanks

vis1985 wrote:
Our oracle is installed on UNIX machine so we have to place CSV file there.
We are on windows.We are using Putty client to access UNIX server.
how to copy CSV file from windows to UNIX server?Using scp (secure copy).
It is supported by Putty and ssh services are available by default on most (if not all) modern Unix servers.
Don't have a Windows machine close by, but I recall the scp command is supported by the Putty executable called "+pscp.exe+".
The syntax is very simple:
scp <from_destination> <to_destination>
example (pushing a file from Windows to Oracle Unix server)
scp c:\datafiles\accounts.csv [email protected]:/u01/files/accounts.csv This will prompt for a password.
You also can use ssh trusted certificates (RSA or DSA keys) to automate this. Using Putty you can generate a key pair that consists of a private key and public key.
You copy and paste the public key into the destination server's +$HOME/.ssh/authorized_keys+ file. This results the server in trusting that client and allows the client to connect (using ssh, scp and sftp) without having to supply a password (as the private part of the key serves as authentication). (I would however not use the oracle account for this, but a special scp-copy usage Unix account that does not support shell usage)
If you want to automate it the other way around - have the Unix server pull the file from the Windows client, then you need to install server software on that Windows machine to allow it to provide such a file copy service.
Again I suggest using ssh - OpenSSH is available for a variety of platforms, and Windows should be included. If not, there should be ssh service for Windows from Microsoft or another vendor that can be used.
Any other method than ssh will be either insecure or problematic. FTP is insecure as usernames and passwords are transmitted in clear text. Using Windows File Sharing requires Samba to be loaded and configured on the Unix server - and this is extra work installing and configuring and testing s/w that uses a low level security, that is also proprietary and could required setting on the Windows client also to make it work in a somewhat robust fashion.
The standard today is ssh. It is robust and secure. And it makes a lot of sense to use it.. and pretty good motivation for not using it ifo something else.

FILE to idoc scenario

Hi,
in one of my scenario i am sending data from CSV file to idoc(ORDERS05).
I have created on third party Technical System and assign business system and logical name(XYZ) for business system.
I have done FCC in sender CC.
Sender Partner Type is KU.
in receiver system i need to do in bound configuration for Sender Partner Type KU?
From where i created Partner Profile for KU.
regards
Mohit

check this blog this might help u:-
http://www.sdn.sap.com/irj/scn/go/portal/prtroot/docs/library/uuid/b0b355ae-0501-0010-3b83-8f2bb566fa47?quicklink=index&overridelayout=true
also look at the same:-
http://www.sdn.sap.com/irj/scn/go/portal/prtroot/docs/library/uuid/cdded790-0201-0010-6db8-beb9bb2b2660?quicklink=index&overridelayout=true
U can create partner type KU from We20 tcode in R3.
chirag

File-XI-IDOC : error in sxmb_moni because of empty idoc

Hi !
I'm converting a CSV file to IDOC segments (CREMAS04).
XI uses a lookup table to convert CSV records to IDOC segments. It is normal that some records could not be converted. When I process a CSV file that has ALL non-convertable records, I get an error in sxmb_moni :
<SAP:Category>XIAdapter</SAP:Category>
<SAP:Code area="IDOC_ADAPTER">ATTRIBUTE_IDOC_RUNTIME</SAP:Code>
<SAP:P1>MSGGUID 99889D40A4D811DBAD4A0017A4510466: Tag found instead of tag IDOC BEGIN=</SAP:P1>
And the final payload is:
<?xml version="1.0" encoding="UTF-8" ?>
<CREMAS04 />
In the integration repository, the occurance of the complex type CREMAS04 is "1..1", I modified the xsd to minOccurs=0 , maxOccurs=1 , and imported it, but it keeps showing me "1..1"...
How do I deal with these case ?? I don't want error red flags in monitor because of empty idocs...
Thanks.

Hi Bhavesh !
Thanks !
The problem is that because of mapping rules, the value of my target fields is SUPPRESS, not "empty".
For example, I have a file with this records:
A
A
B
A
"B" records should fill an IDOC segment, "A" not.
If I receive a file with only "A" records, for example 48 "A" records...then the IDOC segment, now, is receiving 48 SUPPRESS values...then the mapWithDefault doesn't work because they are SUPPRESS and not EMPTY.
Any ideas?
Thanks,
Matias.

Can't paste copied entries from online CSV file into Excel spreadsheet (chatted with zzxc on 5/3/10 at around 2.30 CST but got disconnected. did not get an answer ...)

Here is the transcript of a chat with Firefox community member zzxcon May 3/10
You are now chatting with Firefox community member zzxc
zzxc: Hello
zzxc: What happens when you attempt to download a .csv file?
seegal: hello
seegal: it doesn't copy
zzxc: how are you trying to copy?
seegal: pls bear with me I'm a slow typist. Just copy selected text
Biolizard has joined the conversation.
zzxc: ok - which text are you selecting?
seegal: I reconcile my checkbook (spreadsheet this way). I copy the items in my online bank acc and paste it to the spreadsheet
seegal: I'm using Firefox /2.0.0.19. Have no problem to do this.
zzxc: Which version of OS X?
seegal: In all newer version nothing happens when trying to paste- just doesn't paste
zzxc: Firefox 2.0.0.x is no longer supported, and hasn't been supported in over a year
zzxc: Paste into Excel, from Firefox?
seegal: Sorry, I'm ahead... /2.0.0.19
seegal: Yes. I open my bank acc in Firefox
zzxc: Which version of Excel?
zzxc: It would really help if you could tell me step by step what you're doing.
seegal: First re: your previous question: it's OS 10.4.11
seegal: About Excell: it'as 2004 version - the lasat one produced for Macs. The specific version is 11.3.7
seegal: So I open my bank acc online in Firfox (my primary browser). I copy the latest entry in the account and paste it into my Excel spreadsheet.
zzxc: so, you copy direct from the web page without downloading a CSV file?
seegal: What do you mean by downloading to CSV file? I could export from the https://chat-support.mozilla.com:9091/webchat/getimage?image=sendmessage&workgroup=support%40workgroup.chat-support.mozilla.comFirefox to the CSV file, but the other way around?
zzxc: Are you copying your bank statement directly from the web site to Excel using the clipboard?
seegal: sI don't use the clipboard. This is a mac. There is no need to do that. In PC it would be yes.
zzxc: I need to know the exact steps you're taking to get them into excel
zzxc: And I need to know what exactly goes wrong in the latest version of Firefox.
seegal: Do you have a mac there with Firefox and Excel? It would be very easy to reproduce. Imagine you open an online bank acc, select some entries , click "copy", than proceed to your already open Excel spreadsheet and click' paste". That's it!
zzxc: When this happens, do you get cryptic code pasted into Excel?
seegal: As I said before: in all newer versions starting with 3.0 when I go to Excel to "paste" from my bank acc nothing happens. It does not paste. No, I don't get a cryptic code pasted, just NOTHING.
zzxc: what if you paste into MS Word instead?
seegal: haven't tried that, the formatting most likely would be lost. Tried that with an other Exc el spreadsheet- it lost all the formating and pasted as continous text.
== This happened ==
Every time Firefox opened
== Pls. see copy of the chat above. THIS A MAC OS X. In older versions, prior to 3.0 I could copy from CSV file on the website Ibank acc) and paste directly to my Excel spreadheet to reconcile my account.

See also:
Table2Clipboard: https://addons.mozilla.org/firefox/addon/1852
TableTools: https://addons.mozilla.org/firefox/addon/2637

IDoc to csv file with required fields

All,
I have a source IDoc going to XI. For example it contains source fields SF1 (required), SF2 (required), SF3(optional) and SF4(optional).
I want to produce a target file using file content conversion. I want to produce target fields TF1(required), TF2(required), TF3(optional), and TF4(optional). In my mapping SF1 maps to TF1, SF2 maps to TF2, etc...
I want to produce a comma separated file, I am using file content conversions and I am using NameA.fieldSeparator (using a comma) in my file content conversion parameters. I have no problem when all 4 source fields are populated. In this case if the value of my source fields are: ABC 123 XYZ and 789 then I get a flat file with the result:
ABC,123,XYZ,789
The problem is when my optional fields are blank I currently get the following in my csv file:
ABC,123
When instead I want:
ABC,123,,
I know I've seen threads relating to this issue but I haven't had any success locating them. Any insight is appreciated.

Hi Shaun,
Source : SF1 (required), SF2 (required), SF3(optional) and SF4(optional).
Target: TF1(required), TF2(required), TF3(optional), and TF4(optional)
For Optional in mapping , If the source optional field is not there, then for target assign with blank constant. You will get the output as below, because at target side the element will be there with blank value and FCC will process that.
ABC,123,,
Otherwise Try with NameA.fieldFixedLengths in FCC.
Regards,
Prasanna

Mapping IDOC to CSV file - Missing blank field values in CSV file

Hello:
I am mapping an IDOC to a .csv file, using File Content Conversion. I specify comma as the value for the parameter 'fieldSeparator'. I get a .csv file, but blank field values are missing in the .csv file.
For example, if the .csv file format is <field1>,<field2>,<field3>
and if <field2> is blank, then the .csv file contains
<field1>,<field3>

Hi Bac,
As long as the XML that goes to your File Receiver Channel contains the elements, even if they don't contain values, it should work fine.
For example:
<field1>data</field1>
<field2></field2>
<field3>data2</field3>
I suspect that your XML looks like the following:
<field1>data</field1>
<field3>data2</field3>
If this is the case you can update your map. You shouldn't need to put a space, just make sure the element gets created.
Thanks,
Jesse

Dyanmic csv file downloading with out writing a copy to server

Hi,
Here is my problem, any help most appriciated..
I want to download dynamically created CSV file from the weblogic server, using jsp/servlet communication.
I developed a servlet for this and i called that servlet from my jsp page, when user clicks on download button, my servlet works fine, but it writes a copy of CSV file into c:\bea\wlserver6.0 of the server, My intention is just it's has to dowload file to the client system, with out writing a copy in the server.
Here is my code snnipits..
public void doGet(HttpServletRequest req, HttpServletResponse res) throws IOException, ServletException {
HttpSession session = req.getSession(true);
try {
resultMessageLastBalance = mtdRetrieveLastBalance(req);
rsTrxnDetailLastBalance = resultMessageLastBalance.getRecordSet("lasttentrxn");
} catch (Exception e) {
e.printStackTrace();
e.getMessage();
File csvFile;
FileOutputStream file;
//codes to generate a file in local server for downloading
Calendar calendar = Calendar.getInstance();
String strFileName = Integer.toString(calendar.get(Calendar.YEAR))+Integer.toString(calendar.get(Calendar.MONTH))+Integer.toString(calendar.get(Calendar.DATE))+Integer.toString(calendar.get(Calendar.HOUR))+Integer.toString(calendar.get(Calendar.MINUTE))+Integer.toString(calendar.get(Calendar.SECOND))+Integer.toString(calendar.get(Calendar.MILLISECOND));
try {
strFileName = DeformatAcctNo(req.getParameter("acct_trxn__acctselected")) + strFileName + ".csv";
} catch (Exception e) {}
calendar = null;
csvFile = new File(strFileName);
if (!csvFile.exists()) {
csvFile.createNewFile();
file = new FileOutputStream(csvFile);
res.setContentType("application/download");
res.setHeader("Content-Disposition","attachment;filename=" + strFileName);
javax.servlet.ServletOutputStream servletoutputstream1 = res.getOutputStream();
String s = strFileName;
dumpFile(s, servletoutputstream1);
servletoutputstream1.flush();
servletoutputstream1.close();

Hi,
Thanx a lot for ur solution..it's helful for me to send data with a file format,but i need t send data in csv file foramt.
hav a look at my complete servelt and try to suggesst some thing..
In this case csv file is creating in c:\bea\wlserver directory.my intensions sre just it jas to download to client, no copy in the server.
public class AcctStmtD2 extends HttpServlet {
     public void doGet(HttpServletRequest req, HttpServletResponse res) throws IOException, ServletException {
     //variables for retrieving last balance
     Message resultMessageLastBalance = null;
     RecordSet rsTrxnDetailLastBalance = null;
     double dblDebitTotal = 0.00;
     double dblCreditTotal = 0.00;
     HttpSession session = req.getSession(true);
Enumeration ea = session.getAttributeNames();
     while(ea.hasMoreElements()) {
     Object name = ea.nextElement();
     System.out.println("values = " + session.getAttribute((String)name));
     try {
     resultMessageLastBalance = mtdRetrieveLastBalance(req);
     rsTrxnDetailLastBalance = resultMessageLastBalance.getRecordSet("formonthtrxn");
     } catch (Exception e) {
     e.printStackTrace();
     e.getMessage();
     File csvFile;
     FileOutputStream file;
     //codes to generate a file in local server for downloading
     Calendar calendar = Calendar.getInstance();
     String strFileName = Integer.toString(calendar.get(Calendar.YEAR))+Integer.toString(calendar.get(Calendar.MONTH))+Integer.toString(calendar.get(Calendar.DATE))+Integer.toString(calendar.get(Calendar.HOUR))+Integer.toString(calendar.get(Calendar.MINUTE))+Integer.toString(calendar.get(Calendar.SECOND))+Integer.toString(calendar.get(Calendar.MILLISECOND));
     try {
          strFileName = DeformatAcctNo(req.getParameter("acct_stmt__acctselected")) + strFileName + ".csv";
     } catch (Exception e) {}
     calendar = null;
     csvFile = new File(strFileName);
     if (!csvFile.exists()) {
     csvFile.createNewFile();
     file = new FileOutputStream(csvFile);
     if (rsTrxnDetailLastBalance.getTotalRowCount() >= 1) {
     String strLastBal = "";
     String strCurrCd = "";
          try {
                    rsTrxnDetailLastBalance.moveLast();
     strLastBal = rsTrxnDetailLastBalance.getValue("bal");
                    rsTrxnDetailLastBalance.moveFirst();
     strCurrCd = "(" + GetCurrCdFromSession(session, req.getParameter("acct_stmt__acctselected")) + ")";
     file.write(("\""+req.getParameter("acct_stmt__acctselected")+"\"").getBytes());
     file.write(',');
     file.write(',');
     file.write(',');
     file.write(',');
     file.write(',');
     file.write(("\""+"Last Balance "+strCurrCd+" = "+strLastBal+"\"").getBytes());
     file.write('\n');
     file.write(("\""+"Date"+"\"").getBytes());
     file.write(',');
     file.write(("\""+"Slip No"+"\"").getBytes());
     file.write(',');
     file.write(("\""+"Description"+"\"").getBytes());
     file.write(',');
     file.write(("\""+"Debit"+" "+strCurrCd+"\"").getBytes());
     file.write(',');
     file.write(("\""+"Credit"+" "+strCurrCd+"\"").getBytes());
     file.write(',');
     file.write(("\""+"Balance"+" "+strCurrCd+"\"").getBytes());
     file.write('\n');
     } catch (Exception e) {System.out.println("!");}
while(rsTrxnDetailLastBalance.moveNext()) {
     try {
     file.write(("\""+rsTrxnDetailLastBalance.getValue("txn_dt").substring(0,12)+"\"").getBytes());
file.write(',');
     file.write(("\""+rsTrxnDetailLastBalance.getValue("slip_no")+"\"").getBytes());
     file.write(',');
     file.write(("\""+rsTrxnDetailLastBalance.getValue("dscp")+"\"").getBytes());
     file.write(',');
     file.write(("\""+rsTrxnDetailLastBalance.getValue("debit")+"\"").getBytes());
     file.write(',');
     file.write(("\""+rsTrxnDetailLastBalance.getValue("credit")+"\"").getBytes());
     file.write(',');
     file.write(("\""+rsTrxnDetailLastBalance.getValue("bal")+"\"").getBytes());
     file.write('\n');
     } catch(Exception e) {}
     try {
                         dblDebitTotal += Double.parseDouble(rsTrxnDetailLastBalance.getValue("debit"));
     } catch (Exception e) {
                         dblDebitTotal = 0;
     try {
                         dblCreditTotal += Double.parseDouble(rsTrxnDetailLastBalance.getValue("credit"));
     } catch (Exception e) {
                         dblCreditTotal = 0;
     file.write(',');
     file.write(',');
     file.write(("\""+"Total"+"\"").getBytes());
     file.write(',');
     file.write(("\""+Double.toString(dblDebitTotal)+"\"").getBytes());
     file.write(',');
     file.write(("\""+Double.toString(dblCreditTotal)+"\"").getBytes());
     } else {
     file.write(("\""+"No Record Found!"+"\"").getBytes());
     file.close();
     res.setContentType("application/download");
     res.setHeader("Content-Disposition","attachment;filename=" + strFileName);
               javax.servlet.ServletOutputStream servletoutputstream1 = res.getOutputStream();
     String s = strFileName;
     dumpFile(s, servletoutputstream1);
     servletoutputstream1.flush();
     servletoutputstream1.close();
public void doPost(HttpServletRequest req, HttpServletResponse res) throws IOException, ServletException {
          doGet(req, res);
private void dumpFile(String s, OutputStream outputstream) {
byte abyte0[] = new byte[4096];
     boolean flag = true;
     try {
     FileInputStream fileinputstream = new FileInputStream(s);
     int i;
     while((i = fileinputstream.read(abyte0)) != -1)
     outputstream.write(abyte0, 0, i);
     fileinputstream.close();
     } catch(Exception e) {}
private Message mtdRetrieveLastBalance(HttpServletRequest req) throws Exception {
     Message msgMessage = new Message();
     DAC.Parser.RecordSet objFund_Tx = msgMessage.createRecordSet("formonthtrxn");
     //Set the header fields for the record
     objFund_Tx.addHeaderFields("acctselected","String");
     objFund_Tx.addHeaderFields("formonth","String");
     objFund_Tx.addHeaderFields("foryear","String");
     //Add a new row to the recordset
     objFund_Tx.addRow();
     msgMessage.setData("recordnm", "string", "formonthtrxn");
     //Set the required fields into the recordset
     objFund_Tx.setValue("acctselected", DeformatAcctNo(req.getParameter("acct_stmt__acctselected")));
     objFund_Tx.setValue("formonth", req.getParameter("acct_stmt__formonth"));
     objFund_Tx.setValue("foryear", req.getParameter("acct_stmt__foryear"));
     objFund_Tx.setStatus("select");
     System.out.println("JSP IN :"+msgMessage);
     msgMessage = mtdOpenConnection(msgMessage);
     System.out.println("JSP OUT :"+msgMessage);
     return msgMessage;
private Message mtdOpenConnection(Message objMessage) throws Exception {
java.util.Hashtable ht = new java.util.Hashtable();
     ht.put(javax.naming.Context.INITIAL_CONTEXT_FACTORY, "weblogic.jndi.WLInitialContextFactory");
     ht.put(javax.naming.Context.PROVIDER_URL, "t3://localhost:7001");
javax.naming.InitialContext ic = new javax.naming.InitialContext(ht);
DAC.BusinessObjects.RIB.Account.AccountHome pmHome = (DAC.BusinessObjects.RIB.Account.AccountHome)PortableRemoteObject.narrow(ic.lookup("AccountEJB"),Class.forName("DAC.BusinessObjects.RIB.Account.AccountHome"));
     DAC.BusinessObjects.RIB.Account.Account pmObj = (DAC.BusinessObjects.RIB.Account.Account)PortableRemoteObject.narrow(pmHome.create(),Class.forName("DAC.BusinessObjects.RIB.Account.Account"));
     return pmObj.mtdRetrieveTrxForMonth(objMessage);
private String convertSingleDigitMonthDayToDouble(String param) throws Exception {
     if (param.length() > 1) {
     return param;
     } else {
     return "0" + param;
     //remove dashes and product name and branch code from string
     private String DeformatAcctNo(String strAcctNo) throws Exception {
          strAcctNo.trim();
          return strAcctNo.substring(0,3)+strAcctNo.substring(4,6)+strAcctNo.substring(7,12)+strAcctNo.substring(13,14);
     private     String GetCurrCdFromSession(HttpSession session, String acct_no) throws Exception {
          String strAcctDetail = "";
          String strCurrCd = "";
          Vector vList = (Vector)session.getAttribute("accountlist");
          for (int i = 0; i < vList.size(); i++) {
               strAcctDetail = (String)vList.get(i);
               if ((strAcctDetail.substring(0, acct_no.length()).equals(acct_no))) {
                    strCurrCd = strAcctDetail.substring(strAcctDetail.indexOf(":") + 1, (strAcctDetail.indexOf(":") + 4));
                    break;
          vList = null;
          strAcctDetail = null;
          return strCurrCd;
}

Read .csv file to XI and convert to IDOC

Hi XI Gurus,
We have a requirement at work where we need to import .csv files to XI ehich then XI converts it to xml and posts an IDOC to SAP. I have tried creating this in XI with File Adapter with File Content Conversion.
I gave all the things required for File Content Conversion but still my .csv file is not being picked up and read by XI. Iam not sure why this is happening but can anayone please help me with this.
Also please do let me know if anyone has a tutorial related to this scenario.
Thanks,
Mayuresh.

Hi All,
I tried checking the error via Run Time Workbench (RWB) and going to Communication Channel Monitoring. It shows that there are some errors in the Communication Channel but when i go inside it does not open an error window for me containing the error details.
Just wanted to explain my file structure and the settings i have done in the 'File Content Conversion' of the 'File Adapter' so that you guys can tell me where i am wrong:
Message Type: VENDOR_MT
VENDOR_MT
    VendorNumber
    LastName
    SearchTerm
    Currency
Address
      Street
      City
      Zip
      Country
I am using IDOC receiver at the other end (CREMAS.CREMAS03).
CSV File Structure: (Sample)
34567, Reed, Jon, AUD, Smith Street, Melbourne, 3066, AU
The settings that i have used in the File Adapter (Content Conversion tab) are:
RecordSet Structure: VendorNumber, LastName, SearchTerm, Currency, Street, City, Zip, Country
RecordSet Sequence: Ascending
ignoreRecordSetName: true
Vendor_MT.fieldNames: VendorNumber, LastName, SearchTerm, Currency
Vendor_MT.keyFieldValue: 01
Vendor_MT.fieldSeperator: ','
Vendor_MT.endSeperator: nl
Address.fieldNames: Street, City, Zip, Country
Address.keyFieldValue: 02
Address.fieldSeperator: ','
Address.endSeperator: nl
Please do let me know if the communicatioon channel settings i have made are correct. Also let me know if i am missing some major setting.
Thanks,
Mayuresh.

SQL bulk copy from csv file - Encoding

Hi Experts
This is the first time I am creating a PowerShell script and it is almost working. I just have some problems with the actual bulk import to SQL encoding from the text file since it replaces
special characters with a question mark. I have set the encoding when creating the csv file but that does not seem to reflect on the actual bulk import. I have tried difference scenarios with the encoding part but I cannot find the proper solution for that.
To shortly outline what the script does:
Connect to Active Directory fetching all user - but excluding users in specific OU's
Export all users to a csv in unicode encoding
Strip double quote text identifiers (if there is another way of handling that it will be much appreciated)
Clear all records temporary SQL table
Import records from csv file to temporary SQL table (this is where the encoding is wrong)
Update existing records in another table based on the records in the temporary table and insert new record if not found.
The script looks as the following (any suggestions for optimizing the script are very welcome):
# CSV file variables
$path = Split-Path -parent "C:\Temp\ExportADUsers\*.*"
$filename = "AD_Users.csv"
$csvfile = $path + "\" + $filename
$csvdelimiter = ";"
$firstRowColumns = $true
# Active Directory variables
$searchbase = "OU=Users,DC=fabrikam,DC=com"
$ADServer = 'DC01'
# Database variables
$sqlserver = "DB02"
$database = "My Database"
$table = "tblADimport"
$tableEmployee = "tblEmployees"
# Initialize
Write-Host "Script started..."
$elapsed = [System.Diagnostics.Stopwatch]::StartNew()
# GET DATA FROM ACTIVE DIRECTORY
# Import the ActiveDirectory Module
Import-Module ActiveDirectory
# Get all AD users not in specified OU's
Write-Host "Retrieving users from Active Directory..."
$AllADUsers = Get-ADUser -server $ADServer `
-searchbase $searchbase -Filter * -Properties * |
?{$_.DistinguishedName -notmatch 'OU=MeetingRooms,OU=Users,DC=fabrikam,DC=com' `
-and $_.DistinguishedName -notmatch 'OU=FunctionalMailbox,OU=Users,DC=fabrikam,DC=com'}
Write-Host "Users retrieved in $($elapsed.Elapsed.ToString())."
# Define labels and get specific user fields
Write-Host "Generating CSV file..."
$AllADUsers |
Select-Object @{Label = "UNID";Expression = {$_.objectGuid}},
@{Label = "FirstName";Expression = {$_.GivenName}},
@{Label = "LastName";Expression = {$_.sn}},
@{Label = "EmployeeNo";Expression = {$_.EmployeeID}} |
# Export CSV file and remove text qualifiers
Export-Csv -NoTypeInformation $csvfile -Encoding Unicode -Delimiter $csvdelimiter
Write-Host "Removing text qualifiers..."
(Get-Content $csvfile) | foreach {$_ -replace '"'} | Set-Content $csvfile
Write-Host "CSV file created in $($elapsed.Elapsed.ToString())."
# DATABASE IMPORT
[void][Reflection.Assembly]::LoadWithPartialName("System.Data")
[void][Reflection.Assembly]::LoadWithPartialName("System.Data.SqlClient")
$batchsize = 50000
# Delete all records in AD import table
Write-Host "Clearing records in AD import table..."
Invoke-Sqlcmd -Query "DELETE FROM $table" -Database $database -ServerInstance $sqlserver
# Build the sqlbulkcopy connection, and set the timeout to infinite
$connectionstring = "Data Source=$sqlserver;Integrated Security=true;Initial Catalog=$database;"
$bulkcopy = New-Object Data.SqlClient.SqlBulkCopy($connectionstring, [System.Data.SqlClient.SqlBulkCopyOptions]::TableLock)
$bulkcopy.DestinationTableName = $table
$bulkcopy.bulkcopyTimeout = 0
$bulkcopy.batchsize = $batchsize
# Create the datatable and autogenerate the columns
$datatable = New-Object System.Data.DataTable
# Open the text file from disk
$reader = New-Object System.IO.StreamReader($csvfile)
$columns = (Get-Content $csvfile -First 1).Split($csvdelimiter)
if ($firstRowColumns -eq $true) { $null = $reader.readLine()}
Write-Host "Importing to database..."
foreach ($column in $columns) {
$null = $datatable.Columns.Add()
# Read in the data, line by line
while (($line = $reader.ReadLine()) -ne $null) {
$null = $datatable.Rows.Add($line.Split($csvdelimiter))
$i++; if (($i % $batchsize) -eq 0) {
$bulkcopy.WriteToServer($datatable)
Write-Host "$i rows have been inserted in $($elapsed.Elapsed.ToString())."
$datatable.Clear()
# Add in all the remaining rows since the last clear
if($datatable.Rows.Count -gt 0) {
$bulkcopy.WriteToServer($datatable)
$datatable.Clear()
# Clean Up
Write-Host "CSV file imported in $($elapsed.Elapsed.ToString())."
$reader.Close(); $reader.Dispose()
$bulkcopy.Close(); $bulkcopy.Dispose()
$datatable.Dispose()
# Sometimes the Garbage Collector takes too long to clear the huge datatable.
[System.GC]::Collect()
# Update tblEmployee with imported data
Write-Host "Updating employee data..."
$queryUpdateUsers = "UPDATE $($tableEmployee)
SET $($tableEmployee).EmployeeNumber = $($table).EmployeeNo,
$($tableEmployee).FirstName = $($table).FirstName,
$($tableEmployee).LastName = $($table).LastName,
FROM $($tableEmployee) INNER JOIN $($table) ON $($tableEmployee).UniqueNumber = $($table).UNID
IF @@ROWCOUNT=0
INSERT INTO $($tableEmployee) (EmployeeNumber, FirstName, LastName, UniqueNumber)
SELECT EmployeeNo, FirstName, LastName, UNID
FROM $($table)"
try
Invoke-Sqlcmd -ServerInstance $sqlserver -Database $database -Query $queryUpdateUsers
Write-Host "Table $($tableEmployee) updated in $($elapsed.Elapsed.ToString())."
catch
Write-Host "An error occured when updating $($tableEmployee) $($elapsed.Elapsed.ToString())."
Write-Host "Script completed in $($elapsed.Elapsed.ToString())."

I can see that the Export-CSV exports into ANSI though the encoding has been set to UNICODE. Thanks for leading me in the right direction.
No - it exports as Unicode if set to.
Your export was wrong and is exporting nothing. Look closely at your code:
THis line exports nothing in Unicode"
Export-Csv -NoTypeInformation $csvfile -Encoding Unicode -Delimiter $csvdelimiter
There is no input object.
This line converts any file to ansi
(Get-Content $csvfile) | foreach {$_ -replace '"'} | Set-Content $csvfile
Set-Content defaults to ANSI so the output file is converted.
Since you are just dumping into a table by manually building a recorset why not just go direct. You do not need a CSV. Just dump theresults of the query to a datatable.
https://gallery.technet.microsoft.com/scriptcenter/4208a159-a52e-4b99-83d4-8048468d29dd
This script dumps to a datatable object which can now be used directly in a bulkcopy.
Here is an example of how easy this is using your script:
$AllADUsers = Get-ADUser -server $ADServer -searchbase $searchbase -Filter * -Properties GivenName,SN,EmployeeID,objectGUID |
Where{
$_.DistinguishedName -notmatch 'OU=MeetingRooms,OU=Users,DC=fabrikam,DC=com'
-and $_.DistinguishedName -notmatch 'OU=FunctionalMailbox,OU=Users,DC=fabrikam,DC=com'
} |
Select-Object @{N='UNID';E={$_.objectGuid}},
@{N='FirstName';Expression = {$_.GivenName}},
@{N='LastName';Expression = {$_.sn}},
@{N=/EmployeeNo;Expression = {$_.EmployeeID}} |
Out-DataTable
$AllDUsers is now a datatable. You can just upload it.
¯\_(ツ)_/¯

PI 7.11 CSV file

Hello All,
I have a simple scenerio IDOC to File.
We had a requirement of making a XML file of IDOC received in PI , ery simple and straight forward.
Now Target side as a requirement to place a file in CSV format.
I mean the IDOC received in PI should be placed in target directory as CSV file.
I am working on PI7.11 , can anyone help me with some info.
Thanks a lot in advance.
Regards,
Vikrant

Hi Vikrant,
>>Now Target side as a requirement to place a file in CSV format. I mean the IDOC received in PI should be placed in target directory as CSV file.
If you want the XMLized IDoc to be converted to a plain IDoc, either use the approach suggested by Michal in this blog (use tunneling and create a by-pass scenario)
/people/michal.krawczyk2/blog/2005/11/21/xi-idoc-to-idoc-tunneling--how-fast-and-easy-can-you-get
or
download the IDoc as a flat Idoc and use PI to transfer the content from one directory to another.
PS: All this is applicable if you want the Flat version of XMLized Idoc
Regards
Suraj

Memory problem with loading a csv file and displaying 2 xy graphs

Hi there, i'm having some memory issues with this little program.
What i'm trying to do is reading a .csv file of 215 mb (6 million lines more or less), extracting the x-y values as 1d array and displaying them in 2 xy graphs (vi attacked).
I've noticed that this process eats from 1.6 to 2 gb of ram and the 2 x-y graphs, as soon as they are loaded (2 minutes more or less) are really realy slow to move with the scrollbar.
My question is: Is there a way for use less memory resources and make the graphs move smoother ?
Thanks in advance,
Ierman Gert
Attachments:
read from file test.vi ‏106 KB

Hi lerman,
how many datapoints do you need to handle? How many do you display on the graphs?
Some notes:
- Each graph has its own data buffer. So all data wired to the graph will be buffered again in memory. When wiring a (big) 1d array to the graph a copy will be made in memory. And you mentioned 2 graphs...
- load the array in parts: read a number of lines, parse them to arrays as before (maybe using "spreadsheet string to array"?), finally append the parts to build the big array (may lead to memory problems too).
- avoid datacopies when handling big arrays. You can show buffer creation using menu->tools->advanced->show buffer allocation
- use SGL instead of DBL when possible...
Message Edited by GerdW on 05-12-2009 10:02 PM
Best regards,
GerdW
CLAD, using 2009SP1 + LV2011SP1 + LV2014SP1 on WinXP+Win7+cRIO
Kudos are welcome

Address Book -- Importing CSV File created by Numbers

I'm trying to import a csv file generated from MS outlook into address book. When I attempt to import the csv file that Outlook created, Address Book is able to recognize the file. However, I need to modify the file before importing. To do that, I opened in Numbers, deleted some columns and renamed some columns. Then, I exported from Numbers as a .CSV file. However, when I try to import the .CSV file created by numbers, I get the error that I have modified with Numbers, I get the error "Text file import failure".
Does anybody know why address book can't import a CSV file created by Numbers?

Many people have run into this problem, including myself. I had worked many hours on a spreadsheet in Numbers, exported to CSV, and received the error message when trying to import into Address Book. The answers to this dilemma are out there, but tidbits of the whole answer are spread around discussion boards. So I hope to consolidate those answers here.
The main problem for me -- and it's reasonable to assume that others receive the "Text File import error" for the same reason -- is that 1) I had commas in my spreadsheet, and 2) I had hidden RETURNs in the spreadsheet. Once I got rid of those two things, I had no problem importing into Address Book.
Deleting Commas>
The problem with commas in your spreadsheet is that the CSV format uses commas to distinguish when a new cell is formed. So having other commas will inevitably confuse Address Book during the attempted import. I didn't have too many commas in my spreadsheet, so I was able to quickly remove them. Yet, if you have dozens or hundreds of cells with commas in them, you'll need an easier solution. I imagine opening Number's Inspector>Cell Inspector>Format will have the solution of removing commas from, say, currency amounts or something else. Yet, how many commas arise in contact information? Perhaps the easiest way is to use the same method of removing those hidden RETURNs. Read on...
Deleting Hidden RETURNs>
You can't see them, and I couldn't find a "show invisibles" option in Numbers. So you need to search and find them. Yet, how do you search for something hidden? Do the Command-F thing to bring up the search field. The search bar should automatically be ready to search what you type. Hitting the Return key will do nothing for you, but hitting Shift-Option-Return will (you'll likely need to hit Return again to actually perform the search). Numbers will point out how many times these hidden returns are in your spreadsheet, which cells contain them, and give you the option to automatically replace them with something else (a space is a safe option). You can do the same search and replace with commas too.
Save your spreadsheet, export it as CSV, and -- as long as the only errors stemmed from commas and hidden RETURNs -- you shouldn't have a problem having Address Book import it. Good luck!
PS- After importing my spreadsheet, I noticed that some of my Smart Groups in Address Book were not showing some people who should have automatically been in there. The problem was that the Smart Group search had straight apostrophes in it, but the imported contact had curved apostrophes. The computer saw these as two unrelated characters. After some copying and pasting, the problem was solved.

How can i import contacts from a csv file to "iCloud Contacts"?

How can I import contacts from a csv file to "iCloud Contacts"?

The only way I know of to import cells from a csv file is by creating a new file. But then you can select the desired cells and copy them to the other file. I just did it to be sure it works.
1. Create the new spreadsheet file via the upload command.
2. Select the cells (table) that you want to move to the other file and press command+C (copy).
3. Close the new file and open the existing file.
4. Select the top/right cell of the area to receive the copied "table" and press commant+V (paste).
I hope this is what you're trying to do!

Download a UNIX based CGI page into a CSV file using Powershell

Hello,
I have been struggling to get a powershell script to work.
I have a web-based page which ends in .cgi and I want to copy the text from that page to a CSV file.
Normally the below script works for .html or .aspx types of pages but this time I have been getting the 'The remote server returned an error: (401) Unauthorized' page. I think the error is because the page is Unix based and for some reason I am not
able to automate the authentication process.
Below is the Powershell script I have been using. I have already created the securestring for my password and saved in a temp folder.
$wc=new-object system.net.webclient
$password = get-content 'C:\temp\Scripts\SecureString.txt' | convertto-securestring
$wc.Credentials = new-object -typename System.Management.Automation.PSCredential -argumentlist "Domain\username", $password
$wc.downloadfile("https://website.domain.com/cgi-bin/text_status_csv.cgi", "C:\temp\Scripts\output.csv")
Currently I am able to successfully copy the data from the page over if I use
$wc.Credentials = Get-Credential
And then type in the username and password manually. I could really use some help to get this automated.
Thanks in advance for your help

A little searching and you will find hundreds of examples.
https://www.google.com/#newwindow=1&q=powershell+persist+credentials
¯\_(ツ)_/¯

Copy CSV file to IDOC in XI

Similar Messages

Maybe you are looking for