Unable to Load CSV file with comma inside the column(Sql Server 2008)

Hi,
I am not able load an CSV file with Comma inside a column. 
Here is sample File:(First row contain the column names)
_id,qp,c
"1","[ ""0"", ""0"", ""0"" ]","helloworld"
"1","[ ""0"", ""0"", ""0"" ]","helloworld"
When i specify the Text Qualifier as "(Double quotes) it work in SQL Server 2012, where as fail in the SQL Server 2008, complaining with error:
TITLE: Microsoft Visual Studio
The preview sample contains embedded text qualifiers ("). The flat file parser does not support embedding text qualifiers in data. Parsing columns that contain data with text qualifiers will fail at run time.
BUTTONS:
OK
I need to do this in sql server 2008 R2 with Service pack 2, my build version is 10.50.1600.1.
Can someone let me know it is possible in do the same way it is handle in SQL Server 2012?
Or
It got resolved in any successive Cumulative update after 10.50.1600.1?
Regards Harsh

Hello,
If you have CSV with double quotes inside double quotes and with SSIS 2008, I suggest:
in your data flow you use a script transformation component as Source, then define the ouput columns id signed int, Gp unicode string and C unicode string. e.g. Ouput 0 output colmuns
Id - four-byte signed
gp - unicode string
cc - unicode string
Do not use a flat file connection, but use a user variable in which you store the name of the flat file (this could be inside a for each file loop).
The user variable is supplied as a a readonly variable argument to the script component in your dataflow.
In the script component inMain.CreateNewOutputRows use a System.IO.Streamreader with the user variable name to read the file and use the outputbuffer addrow method to add each line to the output of the script component.
Between the ReadLine instraction and the addrow instruction you have to add your code to extract the 3 column values.
public override void CreateNewOutputRows()
Add rows by calling the AddRow method on the member variable named "<Output Name>Buffer".
For example, call MyOutputBuffer.AddRow() if your output was named "MyOutput".
string line;
System.IO.StreamReader file = new System.IO.StreamReader( Variables.CsvFilename);
while ((line = file.ReadLine()) != null)
System.Windows.Forms.MessageBox.Show(line);
if (line.StartsWith("_id,qp,c") != true) //skip header line
Output0Buffer.AddRow();
string[] mydata = Yourlineconversionher(line);
Output0Buffer.Id = Convert.ToInt32(mydata[0]);
Output0Buffer.gp = mydata[1];
Output0Buffer.cc = mydata[2];
file.Close();
Jan D'Hondt - SQL server BI development

Similar Messages

  • Is there a way to open CSV files with more than 255 columns?

    I have a CSV file with more than 255 columns of data.  It's a fairly standard export of social media data that shows volume of posts by day for the past year, from which I can analyze the data and publish customized charts. Very easy in Excel but I'm hitting the Numbers limit of 255 columns per table. Is there a way to work around the limitation? Perhaps splitting the CSV in two? The data shows up in the CSV file when I open via TextEdit, so it's there. Just can't access it in Numbers. And it's not very usable/useful for me in TextEdit.
    Regards,
    Tim

    You might be better off with Excel. Even if you could find a way to easily split the CSV file into two tables, it would be two tables when you want only one.  You said you want to make charts from this data.  While a series on a chart can be constructed from data in two different tables, to do so takes a few extra steps for each series on the chart.
    For a test to see if you want to proceed, make two small tables with data spanning the tables and make a chart from that data.  Make the chart the normal way using the data in the first table then repeat the following steps for each series
    Select the series in the chart
    Go to Format sidebar
    Click in the "Value" box
    Add a comma then select the data for this series from the second chart
    Press Return
    If there is an easier way to do this, maybe someone else will chime in with that info.

  • Do we need to format data and log files with 64k cluster size for sql server 2012?

    Do we need to format data and log files with 64k cluster size for sql server 2012?
    Does this best practice still applies to sql server 2012 & 2014?

    Yes.  The extent size of SQL Server data files, and the max log block size have not changed with the new versions, so the guidance should remain the same.
    Microsoft SQL Server Storage Engine PM

  • Unable to connect BO XI3.1 SP3 on linux to sql server 2008

    I am looking for some assistance with trying to connect to sql server 2008 db. We have BO XI 3.1 SP3 installed in linux environment. Was able to set up oracle and db2 variable to connect to oracle and db2 universes, after loading the db2 client and oracle 32b client. trying to connect to sql server 2008, but getting error Unable to Bind Configuration Object -(WIS 10901). Thank You.

    Hi Ravi,
    Depending on the version you may find useful some KB articles:
    [1463263 - Error: Unable to bind to Configuration object (WIS 10901) - SQL driver for UNIX server |https://service.sap.com/sap/support/notes/1463263]   (XI3.1)
    [1607125 - How to configure SQL Server connectivity for WebIntelligence from a BI4.0 unix environment? |https://service.sap.com/sap/support/notes/1607125]   (BI4.0)
    [1607125 - How to configure SQL Server connectivity for WebIntelligence from a BI4.0 unix environment? |https://service.sap.com/sap/support/notes/1607125]  (BI4.0)
    If this doesn't help can you please specify version and patch level, as well as the exact error message you get.
    Thanks and regards
    Simone

  • What is the best way to deal with memory leak issue in sql server 2008 R2

    What is the best way to deal with memory leak issue in sql server 2008 R2.

    What is the best way to deal with memory leak issue in sql server 2008 R2.
    I have heard of memory leak in OS that too because of some external application or rouge drivers SQL server 2008 R2 if patched to latest SP and CU ( may be if required) does not leaks memory.
    Are you in opinion that since SQL is taking lot of memory and then not releasing it is a memory leak.If so this is not a memory leak but default behavior .You need to set proper value for max server memory in sp_configure to limit buffer pool usage.However
    sql can take more memory from outside buffer pool if linked server ,CLR,extended stored procs XML are heavily utilized
    Any specific issue you are facing
    Please mark this reply as the answer or vote as helpful, as appropriate, to make it useful for other readers

  • Whic files are necessary to run MS SQL Server 2008 Express?

    I am looking for information about which files/folder form a installation for SQL Server 2008 R2 Express is really needed.
    I have a test installation, but think that many files may never be used.
    Therefore, can I find a place to which is a must to have in the installation to run an application running on SQL (or someone tell me so)
    Thanks in advance.
    Svend Nielsen

    Hello,
    all files of a SQL Server installation are required, cause of direct or indirect dependencies; why should MS install a not required file?
    Olaf Helper
    [ Blog] [ Xing] [ MVP]

  • Can you open CSV-files with commas?

    It seems to me that we have something of a regression in Numbers 2.0 (part of iWork '09) - it (like Excel before it) no longer opens regular CSV-files.
    When I use the new Numbers to try to make a CSV-file, it'll actually use semicolons instead of commas, and any of my old CSV-files (like this one: http://d.ooh.dk/misc/postnumre.csv ) (with commas) will load all the values (and the commas) in a single column.
    I wonder if it's just me, or perhaps only a problem when using European notation (with comma as the decimal separator)?

    At last, Apple adopted the same behavior than Bento 1.
    When the decimal separator is the period, it works with standard CSV files.
    When the decimal separator is comma, it works with CSV using the semi-colon as item delimiters.
    To open your old CSV, set temporarily your system to a region whose decimal separator is period. Given that, Numbers will open them flawlessly.
    The ability to chose the separator (as it is now in Bento 2) would have been fine .
    Yvan KOENIG (from FRANCE dimanche 11 janvier 2009 20:30:03)

  • Reading csv file how to get the Column name

    Hi,
    I am trying to read a csv file and then save the data to Oracle.
    Class.forName("sun.jdbc.odbc.JdbcOdbcDriver");
    Connection c = DriverManager.getConnection("jdbc:odbc:;Driver={Microsoft Text Driver (*.txt; *.csv)};Dbq=.;Extensions=csv,txn");
    Statement stmt = c.createStatement();
    ResultSet rs = stmt.executeQuery("select * from filename.csv");
    while(rs.next())
       System.out.println(rs.getString("Num"));
    My csv file looks like this:
    "CHAM-23","COMPANY NAME","Test","12",20031213,15,16
    Number,Environ,Envel,Date,Time
    "1","2",3,"4",5
    "6","7",8,"9",9
    Now is there anyway using the above code I start processing the file from the second row that holds the names of the columns and skip the first row. And also can I get the name of the column using ResultSet something like:
    if columnName.equals("Number")
    Because I may have a csv file that could have more columns:
    "CHAM-24","COMPANY NAME","Test","12",20031213,16,76
    Number,Environ,Envel,Date,Time,Total,Count
    "1","2","3","4","5",3,9
    "6","7","8","9",9",,2
    So I want to get the column name and then based on that column I do some other processing.
    Once I read the value of each row I want to save the data to an Oracle Table. How do I connect to Oracle from my Application. As the database is on the server. Any help is really appreciated. Thanks

    The only thing I could think of (and this is a cluj) would be to attempt to parse the first element of each row as a number. If it fails, you do not have a column name row. You are counting on the fact that you will never have a column name that is a number in the first position.
    However, I agree that not always placing the headers in the same location is asking for trouble. If you have control over the file, format it how you want. If someone else has control over the file, find out why they are doing it that way. Maybe there is a "magic" number in the first row telling you where to jump.
    Also, I would not use ODBC to simply parse a CSV file. If the file is formatted identically to Microsoft's format (headers in first row, all subsequent rows have same number of columns), then it's fine to take a shortcut and not write your own parser. But if the file is not adhering to that format, don't both using the M$ ODBC driver.
    - Saish
    "My karma ran over your dogma." - Anon

  • Isse with using Grouping Sets in SQL Server 2008 R2

    Hi,
    I created a query in SQL Server 2012 using grouping sets that works fine.  When I try to use the same query in SQL Server 2008 I get an error ("Inccorrect syntax near SETS").
    I researched using grouping sets in 2008 and didn't see any issue with my query.  What is different in grouping sets 2008 vs 2012?
    SELECT tl.ClientRegionCd as [Client Region Code] , tl.ClientRegionDesc as [Region Name], count(tl.CompleteICN) as [Trauma Letters Sent]
    from TORT_Trauma_Letters tl
    Where CONVERT(VARCHAR(26), tl.SecondNoticeSent, 23) between '2014-06-12' and '2014-06-12'
    GROUP BY GROUPING SETS((tl.ClientRegionCd, tl.ClientRegionDesc), ())
    Stacie

    Check this blog post as how to deal with date ranges
    http://sqlblog.com/blogs/aaron_bertrand/archive/2009/10/16/bad-habits-to-kick-mishandling-date-range-queries.aspx
    For your result you can simply achieve your result with UNION ALL, e.g.
    select Client, Region, Letters
    from myTable 
    UNION ALL
    select NULL, NULL, SUM(letters) as Letters
    For every expert, there is an equal and opposite expert. - Becker's Law
    My blog
    My TechNet articles

  • Using upgrade advisor on a database with compatibility level 80 on SQL Server 2008

    Hi All,
    My DB compatibility level is 80 which is on SqL server 2008, however I want to upgrade the database server to sql 2014. 
    Now in my case the upgrade advisor of SQL 2014 is not detecting the DB which is in 80 compatibility level and hence I cannot proceed with the upgrade (as I do not want to rollback). 
    So what exactly  can I do in this situation? any help would be appreciated
    aa

    What Olaf said Or if possible change compatibility level of database to 100 and try. I guess since you want to upgrade there wont be any issue in changing. Well if anything happens you can safely change it back to 80
    Please mark this reply as answer if it solved your issue or vote as helpful if it helped so that other forum members can benefit from it
    My Technet Wiki Article
    MVP

  • How do I import a CSV file into numbers so the columns show

    Thanks for the help.  I have a CSV file generated by the Google Keyword tool; when I open it in numbers all the values go into the same row.  I get the correct number of rows but only one column with five values in each one.  I read on here to check and see if the extension is correct and it is csv.  I've also dragged the file onto the number icon in the dock and get the same result.  Google gives you the option of exporting as a tsv file. Did that and I got the exact same result.  I've looked at the CSV file in textedit and it doesn't show any commas, but seems to have tabs.  But the tsv file doesn't open correctly either.  I'm at a loss. 

    If the file is created as TSV one, its name extension must be .txt.
    Numbers open such files flawlessly.
    Yvan KOENIG (VALLAURIS, France) mardi 4 octobre 2011 11:53:27
    iMac 21”5, i7, 2.8 GHz, 4 Gbytes, 1 Tbytes, mac OS X 10.6.8 and 10.7.0
    My iDisk is : <http://public.me.com/koenigyvan>
    Please : Search for questions similar to your own before submitting them to the community

  • Unable to open Office files on SharePoint 2010 sites on Windows Server 2008 R2

    Setup: Windows SharePoint Server 2010 sp1 on Windows Server 2008 R2 (hosted on VMWare VSphere ESXi 5.1)
    Secured using Domain SSL from GlobalSign.
    Client OS: Windows 7 Professional 32bit & Windows XP SP3 32 bit
    Browsers: IE10, IE9,IE8
    Office: Office 2010, Office 2010 sp2 (both 32bit), Office 2007
    Problem Description: Only recently, users started having issues with opening Office files (Word, Excel) on SharePoint sites. When you click on an Office file e.g. Word Document, it comes up with the option to select "Ready only" or "Edit"
    and choosing either options and re-entering the login credentials results in the
    application hanging. If you leave it for a while (about 5 minutes) sometimes the file opens and other times it gives the error "Could not open https://portal.com/XXX/XXX/Document.docx". Even when it opens, it's extremely
    sluggish and can't do anything other than end it via Task Manager. Same with excel files. 
    This happens to all the users all different OS and browsers.
    On XP Pro SP3 w/ IE8, when you click on the file, the login prompt keeps popping back up repeatedly and if you click cancel, the file loads up anyway but no way of saving back onto the SharePoint site.
    IMPORTANT: One exception is when logged in as a Site Collection Administrator, it
    works like a dream bizarrely!!!! Opens, Edits and saves back fine on the SharePoint site.
    Have tried following:
    1. Repaired Office 2010 Pro; Uninstalled/Re-installed Office 2010 Pro
    2. Reset Browser, unticked automatically detect proxy settings.
    3. Emptied OfficeOfflineCache on Client
    4. Removed WebDAV from the SharePoint server
    5. Added Antivirus folder exclusion (We use SEP 11) on the SharePoint Server
    Note: PDF files opens fine for all users.
    I would immensely appreciate if anybody has got any advice/pointers/suggestions. Desperately out of ideas at the moment.
    regards,
    Compter Sez No!

    Hi,
    According to your description, When you try to open a Microsoft Office document from a SharePoint 2010 site in a client application, the document opens slowly and you are prompted to enter your credentials multiple times.
    This issue may be because of SharePoint Web Front End (WFE) server(s) has Antivirus software installed and the exceptions are not set correctly.
    I recommend to remove the antivirus software or set the correct permissions for the antivirus software to resolve the issue.
    To configure the antivirus exceptions, please follow the guidelines in the below articles:
    KB943620:
    Folders to exclude when you perform a file-level antivirus scan on a server operating system.
    KB952167:
    Certain folders may have to be excluded from antivirus scanning when you use a file-level antivirus program in SharePoint.
    KB943556:
    Recommended file and folder exclusions for Microsoft Forefront Client Security or Forefront Endpoint Protection 2010.
    Best regards.
    Thanks

  • Require help with Pivot table query in SQL Server 2008

    Hi,
    I have a query regarding converting columns to rows in SQL Server 2008. Please look at the table below. 
    I need the output to look something like this :
    The columns for the children can be dynamic or fixed ( max of 6 children) based on the Family_ID.  For Example: A family can have 1 child or more than 1 child.
    Not sure how to go about it. Would appreciate your help :)

    Looks like you need dynamic pivot on multiple columns. I have two articles on this topic, start from this one
    T-SQL:
    Dynamic Pivot on Multiple Columns
    It has reference to my other blog post.
    For every expert, there is an equal and opposite expert. - Becker's Law
    My blog
    My TechNet articles

  • Load .csv file data with OWb Process flow using Web

    Hi,
    I Have a file in my local machine( Machines on multiple user's), need to load data through Web user interface.
    Let's say have a web page with multiple radio buttons respective to different sources, by clicking on each button will pass the path of .csv file to through Application, (API or Java programming interface) execute owb Process flow as a accepting file path as a input parameter to execute for loading purpose.
    Should facilitate view data, Update data through web based on user requests.
    Need your guidence how can i implement this with OWb 11g R2.
    Assuming with Web browser functionality. Please confirm it and if yes, please throw some light how could be the steps to implement.
    Thanks

    Hi David,
    Thanks for your reply.
    Undersatnd your proposed solution.But my requirement should be as follows.
    1. Currently under consideration using web page likely to be implement with Java, allowing users to load .csv file data into staging area.(Loading flat file into Data abse table)
    Case 1, Assuming OWB software is not installed on user machine. I think no.
    Is it possible through web page (this case Java page) to trigger java procedure/Pl/SQl procedure or integration of both to laod data into staging area.If yes, how it could effect performance of data load with 1 GB file.
    Case 2, OWb client software installed on User machine, while runtime passing parameters means passing manually?
    In case it is automated, how should i pass machine name & Path to owb runtime web browser.
    Could you please show me guidence how should I acheive this functionality with APEX customization part?
    Thanks agin for your support.
    Anil

  • How to load CSV files into HANA

    1.- Create an CSV file with your data
    2.- Copy the file to dropbox/yourname inside the HANA box
    3.- Create a table in HANA with the structure of your file
    4.- create the control file BBB with the following information:
    import data
    into table XXX."YYY"
    from 'ZZZ.csv'
    record delimited by '\n'  
    fields delimited by ','
    optionally enclosed by '"'
    error log 'Text_Tables.err'
    Where XXX is your schema, YYY is your HANA Table and ZZZ is your file
    5.- Open an Script File and write the following:
    LOAD FROM '/filer/dropbox/yourname/BBB.ctl';
    where BBB is the name of your control file
    Greetings,
    Blag.

    With the recent upgrade to the HANA SP3...the method to upload files to HANA has been slightly changed...so here's the updated tutorial:
    1.- Create an CSV file with your data
    2.- Copy the file to the HANA Dev Center
    3.- Open a FTP connection to ftp.sapdevcenter.com
    (You can find the user name and password by doing a select to SYSTEM.FTP_SERVER)
    and....*Don't paste the Username or password here or any other place!*
    4.- Create a folder with your name on the FTP site and copy the file there
    4.- Create a table in HANA with the structure of your file
    5.- create the control file BBB with the following information:
    import data
    into table XXX."YYY"
    from 'ZZZ.csv'
    record delimited by '\n'
    fields delimited by ','
    optionally enclosed by '"'
    error log 'Text_Tables.err'
    Where XXX is your schema, YYY is your HANA Table and ZZZ is your file
    5.- Open an Script File and write the following:
    IMPORT FROM '/dropbox/yourname/BBB.ctl';
    where BBB is the name of your control file
    I just test it and it works...if it doesn't work for you, please let me know
    Greetings,
    Blag.

Maybe you are looking for