Using sql bulk copy throwing exception -The given value of type String from the data source cannot be converted to type int of the specified target column

Hi All,
I am reading notepads files and inserting data in sql tables from the notepad-
while performing sql bulk copy on this line it throws exception - "bulkcopy.WriteToServer(dt); -"data type related(mentioned in subject )".
Please go through my  logic and tell me what to change to avoid this error -
public void Main()
Dts.TaskResult = (int)ScriptResults.Success;
string[] filePaths = Directory.GetFiles(@"C:\Users\jainruc\Desktop\Sudhanshu\master_db\Archive\test\content_insert\");
for (int k = 0; k < filePaths.Length; k++)
string[] lines = System.IO.File.ReadAllLines(filePaths[k]);
//table name needs to extract after = sign
string[] pathArr = filePaths[0].Split('\\');
string tablename = pathArr[9].Split('.')[0];
DataTable dt = new DataTable(tablename);
|
string[] arrColumns = lines[1].Split(new char[] { '|' });
foreach (string col in arrColumns)
dt.Columns.Add(col);
for (int i = 2; i < lines.Length; i++)
string[] columnsvals = lines[i].Split(new char[] { '|' });
DataRow dr = dt.NewRow();
for (int j = 0; j < columnsvals.Length; j++)
//Console.Write(columnsvals[j]);
if (string.IsNullOrEmpty(columnsvals[j]))
dr[j] = DBNull.Value;
else
dr[j] = columnsvals[j];
dt.Rows.Add(dr);
SqlConnection conn = new SqlConnection();
conn.ConnectionString = "Data Source=UI3DATS009X;" + "Initial Catalog=BHI_CSP_DB;" + "User Id=sa;" + "Password=3pp$erv1ce$4";
conn.Open();
SqlBulkCopy bulkcopy = new SqlBulkCopy(conn);
bulkcopy.DestinationTableName = dt.TableName;
bulkcopy.WriteToServer(dt);
conn.Close();
Issue 1:-
I am reading notepad: getting all column and values in my data table now while inserting for date and time or integer field i need to do explicit conversion how to write for specific column before bulkcopy.WriteToServer(dt);
Issue 2:- Notepad does not contains all columns nor in specific sequence in that case i can add few column ehich i am doing now but the issue is now data table will add my columns + notepad columns and while inserting how to assign in perticular colums?
sudhanshu sharma Do good and cast it into river :)

Hi,
I think you'll have to do an explicit column mapping if they are not in exact sequence in both source and destination.
Have a look at this link:
https://msdn.microsoft.com/en-us/library/system.data.sqlclient.sqlbulkcopycolumnmapping(v=vs.110).aspx
Good Luck!
Kaur.
Please mark as answer if this resolves your issue.

Similar Messages

  • OLAP cubes from heterogeneous data sources using XML DB

    hi.
    Q:1
    How we can create an OLAP cube (XML cubes) from XML data sources if using XML DB.
    Q:2
    How we can create an OLAP cubes from warehouses and flat files and covert these cubes into XML cubes.
    Q:3
    Is there any other tool (Except Analytical workspace manager AWM) which supports the construction of OLAP cubes in XML format from heterogeneous data sources?
    Edited by: user11236392 on Aug 21, 2009 3:50 AM

    Hi Stuart!
    Your undersatnding is partially correct. i am working for providing an architecture for XOLAP, XML is one of my data source.
    The idea is to generate uniform cubes from heterogeneous sources that can be integrated into a global cube. instead of building Oracle OLAP cubes from all the sources (this work is already done) i want to generate an XCube from XML data sources using XQuery.
    On the other hand if we have generated the Oracle OLAP cube from other sources like warehouses or flat files. i have to convert these Oracle OLAP cube into XML cube for uniformity. in an research paper i find that there is an operator Xcube embedded in XQuery which converts the multidimensional data (cube) into XML cube. im looking for the implementation of this operator in Query that how this operator works.
    hope u understand my architecture. but if u still have some confusion, kindly give me ur mail id. i will mail the diagram of my architecture.
    thanks.
    saqib

  • Inserting in the UUT_Results table a value that we read from our data base from a local variable

    We would like to include in the UUT_Results table a value that we read from our data base into a local variable during the execution of our sequence file. We found that by modifying the configure database options we were able to add a column for this variable, but the local variable was not available to be placed into an expression for that column from the local variables or parameters. Is it possible to do this, and if so, how? Station Globals were available to be included in the expression, however the sequence file may be executed on more than one system which makes the Global unavailable on systems other than the one where the sequence file originated.

    You can use the TestStand API to programmatically create global variables at runtime, thus ensuring their existence. For example, you could call Engine.Globals.SetValString("GlobalStringVariableName", PropOption_InsertIfMissing, "variable value")
    Of course, if you need to test multiple UUTs in parallel, a single global is not sufficient. In that case you might consider adding the field you need to the UUT datatype in the process model. You could then access the field in your sequence via RunState.Root.Locals.UUT.YourNewField = YourNewValue.
    If you also want your sequence to run without using a process model, you must check for the existence of the UUT before accessing it. You could use the expression function: PropertyExists("RunState.Root.Locals.UUT.YourNewFi
    eld")

  • How to use a property from other data source

    Scenario: we have 2 datasources, that each have their own set of records having different properties. What I need to do is grab 1 property that exists in all records of 1 of the datasources and display it in the records of the other datasource. There is 1 "common" property (called by different names) in the records of both Data Sources, that have the same value; so this property could be the link between those records even though they belong to 2 different Data Sources.
    Here's a simple example:
    Data Source " Department" has records that have the properties: 'Name', 'Age' and 'Nationality'.
    Data Source "Staff" has records that have the properties:'Staff_name', 'Position'.
    The 'Name' and 'Staff_name' have the same values, and can be used to link the 2 records.
    I'd like to get the records in the 'Department' data souce to also contain the property 'Position" from the "Staff" datasource .
    Is there a manipulator that can do this?
    Thanks!
    J

    Thanks Pravin,
    But I don't want to join all data from the 2 datasources - I just want one property from one data source to appear in the records of the other data source.

  • SQL bulk copy from csv file - Encoding

    Hi Experts
    This is the first time I am creating a PowerShell script and it is almost working. I just have some problems with the actual bulk import to SQL encoding from the text file since it replaces
    special characters with a question mark. I have set the encoding when creating the csv file but that does not seem to reflect on the actual bulk import. I have tried difference scenarios with the encoding part but I cannot find the proper solution for that.
    To shortly outline what the script does:
    Connect to Active Directory fetching all user - but excluding users in specific OU's
    Export all users to a csv in unicode encoding
    Strip double quote text identifiers (if there is another way of handling that it will be much appreciated)
    Clear all records temporary SQL table
    Import records from csv file to temporary SQL table (this is where the encoding is wrong)
    Update existing records in another table based on the records in the temporary table and insert new record if not found.
    The script looks as the following (any suggestions for optimizing the script are very welcome):
    # CSV file variables
    $path = Split-Path -parent "C:\Temp\ExportADUsers\*.*"
    $filename = "AD_Users.csv"
    $csvfile = $path + "\" + $filename
    $csvdelimiter = ";"
    $firstRowColumns = $true
    # Active Directory variables
    $searchbase = "OU=Users,DC=fabrikam,DC=com"
    $ADServer = 'DC01'
    # Database variables
    $sqlserver = "DB02"
    $database = "My Database"
    $table = "tblADimport"
    $tableEmployee = "tblEmployees"
    # Initialize
    Write-Host "Script started..."
    $elapsed = [System.Diagnostics.Stopwatch]::StartNew()
    # GET DATA FROM ACTIVE DIRECTORY
    # Import the ActiveDirectory Module
    Import-Module ActiveDirectory
    # Get all AD users not in specified OU's
    Write-Host "Retrieving users from Active Directory..."
    $AllADUsers = Get-ADUser -server $ADServer `
    -searchbase $searchbase -Filter * -Properties * |
    ?{$_.DistinguishedName -notmatch 'OU=MeetingRooms,OU=Users,DC=fabrikam,DC=com' `
    -and $_.DistinguishedName -notmatch 'OU=FunctionalMailbox,OU=Users,DC=fabrikam,DC=com'}
    Write-Host "Users retrieved in $($elapsed.Elapsed.ToString())."
    # Define labels and get specific user fields
    Write-Host "Generating CSV file..."
    $AllADUsers |
    Select-Object @{Label = "UNID";Expression = {$_.objectGuid}},
    @{Label = "FirstName";Expression = {$_.GivenName}},
    @{Label = "LastName";Expression = {$_.sn}},
    @{Label = "EmployeeNo";Expression = {$_.EmployeeID}} |
    # Export CSV file and remove text qualifiers
    Export-Csv -NoTypeInformation $csvfile -Encoding Unicode -Delimiter $csvdelimiter
    Write-Host "Removing text qualifiers..."
    (Get-Content $csvfile) | foreach {$_ -replace '"'} | Set-Content $csvfile
    Write-Host "CSV file created in $($elapsed.Elapsed.ToString())."
    # DATABASE IMPORT
    [void][Reflection.Assembly]::LoadWithPartialName("System.Data")
    [void][Reflection.Assembly]::LoadWithPartialName("System.Data.SqlClient")
    $batchsize = 50000
    # Delete all records in AD import table
    Write-Host "Clearing records in AD import table..."
    Invoke-Sqlcmd -Query "DELETE FROM $table" -Database $database -ServerInstance $sqlserver
    # Build the sqlbulkcopy connection, and set the timeout to infinite
    $connectionstring = "Data Source=$sqlserver;Integrated Security=true;Initial Catalog=$database;"
    $bulkcopy = New-Object Data.SqlClient.SqlBulkCopy($connectionstring, [System.Data.SqlClient.SqlBulkCopyOptions]::TableLock)
    $bulkcopy.DestinationTableName = $table
    $bulkcopy.bulkcopyTimeout = 0
    $bulkcopy.batchsize = $batchsize
    # Create the datatable and autogenerate the columns
    $datatable = New-Object System.Data.DataTable
    # Open the text file from disk
    $reader = New-Object System.IO.StreamReader($csvfile)
    $columns = (Get-Content $csvfile -First 1).Split($csvdelimiter)
    if ($firstRowColumns -eq $true) { $null = $reader.readLine()}
    Write-Host "Importing to database..."
    foreach ($column in $columns) {
    $null = $datatable.Columns.Add()
    # Read in the data, line by line
    while (($line = $reader.ReadLine()) -ne $null) {
    $null = $datatable.Rows.Add($line.Split($csvdelimiter))
    $i++; if (($i % $batchsize) -eq 0) {
    $bulkcopy.WriteToServer($datatable)
    Write-Host "$i rows have been inserted in $($elapsed.Elapsed.ToString())."
    $datatable.Clear()
    # Add in all the remaining rows since the last clear
    if($datatable.Rows.Count -gt 0) {
    $bulkcopy.WriteToServer($datatable)
    $datatable.Clear()
    # Clean Up
    Write-Host "CSV file imported in $($elapsed.Elapsed.ToString())."
    $reader.Close(); $reader.Dispose()
    $bulkcopy.Close(); $bulkcopy.Dispose()
    $datatable.Dispose()
    # Sometimes the Garbage Collector takes too long to clear the huge datatable.
    [System.GC]::Collect()
    # Update tblEmployee with imported data
    Write-Host "Updating employee data..."
    $queryUpdateUsers = "UPDATE $($tableEmployee)
    SET $($tableEmployee).EmployeeNumber = $($table).EmployeeNo,
    $($tableEmployee).FirstName = $($table).FirstName,
    $($tableEmployee).LastName = $($table).LastName,
    FROM $($tableEmployee) INNER JOIN $($table) ON $($tableEmployee).UniqueNumber = $($table).UNID
    IF @@ROWCOUNT=0
    INSERT INTO $($tableEmployee) (EmployeeNumber, FirstName, LastName, UniqueNumber)
    SELECT EmployeeNo, FirstName, LastName, UNID
    FROM $($table)"
    try
    Invoke-Sqlcmd -ServerInstance $sqlserver -Database $database -Query $queryUpdateUsers
    Write-Host "Table $($tableEmployee) updated in $($elapsed.Elapsed.ToString())."
    catch
    Write-Host "An error occured when updating $($tableEmployee) $($elapsed.Elapsed.ToString())."
    Write-Host "Script completed in $($elapsed.Elapsed.ToString())."

    I can see that the Export-CSV exports into ANSI though the encoding has been set to UNICODE. Thanks for leading me in the right direction.
    No - it exports as Unicode if set to.
    Your export was wrong and is exporting nothing. Look closely at your code:
    THis line exports nothing in Unicode"
    Export-Csv -NoTypeInformation $csvfile -Encoding Unicode -Delimiter $csvdelimiter
    There is no input object.
    This line converts any file to ansi
    (Get-Content $csvfile) | foreach {$_ -replace '"'} | Set-Content $csvfile
    Set-Content defaults to ANSI so the output file is converted.
    Since you are just dumping into a table by manually building a recorset why not just go direct.  You do not need a CSV.  Just dump theresults of the query to a datatable.
    https://gallery.technet.microsoft.com/scriptcenter/4208a159-a52e-4b99-83d4-8048468d29dd
    This script dumps to a datatable object which can now be used directly in a bulkcopy.
    Here is an example of how easy this is using your script:
    $AllADUsers = Get-ADUser -server $ADServer -searchbase $searchbase -Filter * -Properties GivenName,SN,EmployeeID,objectGUID |
    Where{
    $_.DistinguishedName -notmatch 'OU=MeetingRooms,OU=Users,DC=fabrikam,DC=com'
    -and $_.DistinguishedName -notmatch 'OU=FunctionalMailbox,OU=Users,DC=fabrikam,DC=com'
    } |
    Select-Object @{N='UNID';E={$_.objectGuid}},
    @{N='FirstName';Expression = {$_.GivenName}},
    @{N='LastName';Expression = {$_.sn}},
    @{N=/EmployeeNo;Expression = {$_.EmployeeID}} |
    Out-DataTable
    $AllDUsers is now a datatable.  You can just upload it.
    ¯\_(ツ)_/¯

  • Using SQL Statements in Error Exceptions

    Is there any way to include SQL statements within my catch routines (i.e.
    catch (java.io.IOException ex) {
    System.out.println("An error occurred whilst writing to the message buffer: " + ex);
    #sql { INSERT INTO DEBUG_TBL(CURRENT_DT,PROGRAM_ID,DEBUG__MESG) VALUES (SYSDATE,'TEST',:ex) };
    #sql { COMMIT };
    As it stands I get the following error upon compile:
    Error: Unsupported Java type for host item (at position#1): java.io.IOException
    Any help would be greatly appreciated.
    Thx,
    Boybles
    null

    You can use SQL statements in your catch blocks, just not SQL statements that use Java types which do not map to SQL.
    You cannot pass exceptions from Java to SQL. You can catch a SQL exceptions as a SQLException in Java. However, any exception that is thrown by a Java Stored Procedure (including SQLException) is rendered as an "Uncaught Java exception" in SQL.
    When you receive a SQLException there is information about the originator, the error message, etc. in the exception object. See:
    http://technet.oracle.com/docs/products/oracle8i/doc_library/817_doc/java.817/a83723/keyprog5.htm#1004462
    While you cannot insert a Java exception per se into a table, you can extract the message string(s)/code and insert that in a table.

  • Using a dropdown box to change the specified value of a radio button?

    I have a form that generates the body text for an email.
    First there is a dropdown box in which the choices are either "red" or "blue"
    Further down the page, there is a radiobutton list with 2 items "available" and "unavailable".
    The specified values for these items (as defined on the binding tab) are "have blue hats" or "do not have blue hats".
    This value is concatenated in the email body textfield which is to be copied by the user to an email body:
    Concat("Dear", NameField, "," We ", RadioButtonList, " today. ")  // Dear Persons Name, We have blue hats today.
    Is there any way to change the value of the selected radio button based on the selected dropdown choice?
    So if the user selected "red" in the dropdown field, the radio button values could become  "have red hats" or "do not have red hats"?
    Or perhaps there is a better way from the start?
    Thanks!

    Ok, so now I've done that...
    But is there a way to change that value with another field (ie when a user chooses "red" or "blue" in a dropdown box) so that the concatenated message reflects both the users choice of color and the availibility?
    I'm guessing I should probably use an intermediary text box and put
    InterText.rawValue = Concat ("We have ', dropdown1, " hats today") in the change event of the available radio button and:
    InterText.rawValue = Concat ("We do not have ', dropdown1, " hats today") in the change event of the unavailable radio button.
    then concatenate that box in the final textbox.

  • Adding a field/column in sql table/database and adding the same field /column in biztalk schema

    I have a biztalk application that has a source schema and destination schema inside a map.
    A new field/column in the sql database has been created. Now I have to add this field/record to my source schema.
    My question is, do I manually add this field/record to my source schema and update my source map or do I have to generate a new schema using add generated items>consume adapter service>sql and update my source map ?

    You would need to add the fields to the message schema your SQL data is mapped onto.
    Also update the SQL statement which is executed inside your sql adapter settings to return the new columns.
    Rachit
    Please mark as answer or vote as helpful if my reply does

  • Using Terminal to copy all files (hidden and not hidden) and folders from one place to another

    How -using Terminal- can I copy all files (hidden and not hidden) and folders (and their contents) from one place to another

    ditto
    DITTO(1)                  BSD General Commands Manual                 DITTO(1)
    NAME
         ditto -- copy directory hierarchies, create and extract archives
    SYNOPSIS
         ditto [-v] [-V] [-X] [<options>] src ... dst_directory
         ditto [-v] [-V] [<options>] src_file dst_file
         ditto -c [-z | -j | -k] [-v] [-V] [-X] [<options>] src dst_archive
         ditto -x [-z | -j | -k] [-v] [-V] [<options>] src_archive ...
               dst_directory
         ditto -h | --help
    DESCRIPTION
         In its first form, ditto copies one or more source files or directories
         to a destination directory.  If the destination directory does not exist
         it will be created before the first source is copied.  If the destination
         directory already exists then the source directories are merged with the
         previous contents of the destination.

  • How to insert data to the specified row column of the multi column list box

    Hi All
    How do i insert data into the specified column of the multi cplumn list box?
    I have a table that containsall station nos and name.Then another table contains the data the various stations having at  for 24 hrs.That is 12 am to 11 pm.
    And i want to display each stations details as follows using a multi column list box/table
    My stationinfo table
    stnno   stnname......................
    s1           stn1
    s2            stn2
    s3             stn3
    The other table
    stnno      sysdatetime       data
    s1             12am                   1
    s2              12am                   4
    s1               1  am                 2
    So the station s1,s2.... will have data for 24 hrs.
    And i want to display it as follows using a multicolumn listbox
    stnname        12am   1 am ......................................11pm
    s1                   ...................
    s2                 ........................
    What i have in my  mind is to get all station nos
    and in a for loop get the station's data from 12 am to 11 pm
    or
    select every statios data for each hor.But in this case i have to query the database 24 times.So i dont think its a good way.
    Or any other better query available?
    Can anybody suggest me a good idea?
    One more thing...how to insert data into the specified field row or column of a multi column list box?
    Thanks in advance

    hi
    i want to know,,can u say ur need clearly...and i attached two image u see that one
    Indrajit
    | [email protected] | [email protected] .
    Attachments:
    station.JPG ‏35 KB
    station2.JPG ‏79 KB

  • DirectoryEntry.Parent.Name throw exception: The network path was not found.

    I have very simple code to get AD domain name
    Public Function GetDomainName() As String
    Dim sDomainName As String = String.Empty
    Try
    Dim currentUser As String = WindowsIdentity.GetCurrent().Name.Replace("\", "/")
    Dim strPath As String = String.Format("WinNT://{0}", currentUser)
    Dim dirEntry As DirectoryEntry = New DirectoryEntry(strPath)
    sDomainName = dirEntry.Parent.Name
    Catch ex As Exception
    Console.WriteLine(ex.Message)
    Console.WriteLine(ex.StackTrace)
    End Try
    Return sDomainName
    End Function
    Here is exception that it gives:
    The network path was not found.
    , ,    at System.DirectoryServices.DirectoryEntry.Bind(Boolean throwIfFail)
       at System.DirectoryServices.DirectoryEntry.Bind()
       at System.DirectoryServices.DirectoryEntry.get_Parent()
       at Kofax.PrivateProfileData.clsNetSecurity.GetDomainName() in clsNetSecurity.vb:line 46
    Do anyone help me what is the problem?
    Is that my environment problem?
    Thank you

    This is not a recommended approach to get the domain name - also just to make sure do you intend to get the domain name of the computer or the user, if the later use DsCrackNames:
    http://msdn.microsoft.com/en-us/library/ms675970(v=vs.85).aspx (p/invoke .NET version is available)
    The DS_NAME_RESULT item struct contains the domain name the user belongs to in pDomain
    Enfo Zipper
    Christoffer Andersson – Principal Advisor
    http://blogs.chrisse.se - Directory Services Blog

  • Using SQL instances - how to configure the db connection

    WCI 10.3.0
    We changed the DB server, and instead of a regular SQL server using the 1433 port, we are using an instance
    SQLSERVER1\_SQLINSTANCE1
    The problem is that, in this case, the port is dynamic allocated.
    The question is:
    How I configure this in Webcenter Interaction? Anyone experienced this kind of Db migration?
    Thanks!
    Val

    WCI requires use of a static port. You can assign a static port to your named instance.
    Make sure TCP/IP is enabled, as sometimes only named pipes is turned on.
    Then for WCI use the hostname and port you specify. You can do this in the configuration manager.

  • How to use sql:query/ tag to retrive the system date?

    hi all,
    i want to know if i can use the <sql:query/> tag in jstl to get the database server time. please send me a code snippet to retrive n display the result. i'm new to jstl tags. please help.
    thank you,
    AM

    Check out the first hits: http://www.google.com/search?q=jstl+sql+tutorial+site:sun.com

  • Using AGG/CALC DIM function in the BSO target cube of Transparent Partition

    Hi,
    Is it possible to use AGG("dimension") on a dimnesion in the BSO cube that is a target cube for a BSO-BSO Transparent Partition/
    I have created the partition area with 3 dimensions, one being 'Source'. 90% of the source members are being refrenced from source to target via that transparent partition. But I still have the 10% Source members in the target cube that need to roll up to the 'Source"(along with the 90%) for correct aggregation.
    When I try to use AGG("Source") in a FIX statement that's within the partition area, I get the following error:
    Error: 1042013 Network error [10054]: Cannot Receive Data
    Error: 1042012 Network error [10054]: Cannot Send Data
    Can I not aggregate the dimension in the Target cube?
    Thanks

    Hi,
    We have the same problem.
    I try to create transparent pertition between BSO (as a source) and ASO (target) and have the same warning message.
    Just to eliminate possible effect of outline complexity I created very simple DBs with the following outlines:
    __For ASO:__
    mes1
    mes2
    Measures
    prd1
    prd2
    Products
    Actual
    Adjustment
    Scenarios
    __For BSO:__
    mes1
    mes2
    Measures
    prd1
    prd2
    Products
    Adjustment
    Scenarios
    In addition to the message I have the following behavior - ASO has data in "Actual" scenario, BSO has data in "Adjustment" scenario. Before partition definition I can see the data in the right places, but when the partition created, Actual data disapeared from report of ASO. It is still in the database, and when the partition removed - appears again.
    Any ideas?
    Thank you in advance,
    Alexey

  • Can TCP/IP communication be used for 2 Labview processes on the same target?

    I am a newbie to all this LV stuff so bear with me.  We are trying to create a simulation tool that will run on the same machine as the client GUI [also LV].  Can we use TCP/IP to communicate between these 2 LV processes on the same machine?  If so, how?  Thanks for the help.

    Hi nathand,
    I have another issue while communicating with TCP/IP protocol.
    I am sending data from RT target to Host computer(Main UI ) using TCP . Also I have created EXE file of main main UI which one is running on Host computer. and ussing this exe file in three different PC  and want to send the control signal from all four PC (three exe installed and one host PC). I created four communication loop in RT with four different port for communication with four PC.
    When I am running the all VIs ,Its try to connect to the port of the listener and and showing keep waiting for a connection. Another problem is whatever value we sending from four PC , Target vi receiving random value with actual value. Also every time its getting reset.
    So please help me here.
    Thanks!

Maybe you are looking for