Using sql bulk copy throwing exception -The given value of type String from the data source cannot be converted to type int of the specified target column
Hi All,
I am reading notepads files and inserting data in sql tables from the notepad-
while performing sql bulk copy on this line it throws exception - "bulkcopy.WriteToServer(dt); -"data type related(mentioned in subject )".
Please go through my logic and tell me what to change to avoid this error -
public void Main()
Dts.TaskResult = (int)ScriptResults.Success;
string[] filePaths = Directory.GetFiles(@"C:\Users\jainruc\Desktop\Sudhanshu\master_db\Archive\test\content_insert\");
for (int k = 0; k < filePaths.Length; k++)
string[] lines = System.IO.File.ReadAllLines(filePaths[k]);
//table name needs to extract after = sign
string[] pathArr = filePaths[0].Split('\\');
string tablename = pathArr[9].Split('.')[0];
DataTable dt = new DataTable(tablename);
|
string[] arrColumns = lines[1].Split(new char[] { '|' });
foreach (string col in arrColumns)
dt.Columns.Add(col);
for (int i = 2; i < lines.Length; i++)
string[] columnsvals = lines[i].Split(new char[] { '|' });
DataRow dr = dt.NewRow();
for (int j = 0; j < columnsvals.Length; j++)
//Console.Write(columnsvals[j]);
if (string.IsNullOrEmpty(columnsvals[j]))
dr[j] = DBNull.Value;
else
dr[j] = columnsvals[j];
dt.Rows.Add(dr);
SqlConnection conn = new SqlConnection();
conn.ConnectionString = "Data Source=UI3DATS009X;" + "Initial Catalog=BHI_CSP_DB;" + "User Id=sa;" + "Password=3pp$erv1ce$4";
conn.Open();
SqlBulkCopy bulkcopy = new SqlBulkCopy(conn);
bulkcopy.DestinationTableName = dt.TableName;
bulkcopy.WriteToServer(dt);
conn.Close();
Issue 1:-
I am reading notepad: getting all column and values in my data table now while inserting for date and time or integer field i need to do explicit conversion how to write for specific column before bulkcopy.WriteToServer(dt);
Issue 2:- Notepad does not contains all columns nor in specific sequence in that case i can add few column ehich i am doing now but the issue is now data table will add my columns + notepad columns and while inserting how to assign in perticular colums?
sudhanshu sharma Do good and cast it into river :)
Hi,
I think you'll have to do an explicit column mapping if they are not in exact sequence in both source and destination.
Have a look at this link:
https://msdn.microsoft.com/en-us/library/system.data.sqlclient.sqlbulkcopycolumnmapping(v=vs.110).aspx
Good Luck!
Kaur.
Please mark as answer if this resolves your issue.
Similar Messages
-
OLAP cubes from heterogeneous data sources using XML DB
hi.
Q:1
How we can create an OLAP cube (XML cubes) from XML data sources if using XML DB.
Q:2
How we can create an OLAP cubes from warehouses and flat files and covert these cubes into XML cubes.
Q:3
Is there any other tool (Except Analytical workspace manager AWM) which supports the construction of OLAP cubes in XML format from heterogeneous data sources?
Edited by: user11236392 on Aug 21, 2009 3:50 AMHi Stuart!
Your undersatnding is partially correct. i am working for providing an architecture for XOLAP, XML is one of my data source.
The idea is to generate uniform cubes from heterogeneous sources that can be integrated into a global cube. instead of building Oracle OLAP cubes from all the sources (this work is already done) i want to generate an XCube from XML data sources using XQuery.
On the other hand if we have generated the Oracle OLAP cube from other sources like warehouses or flat files. i have to convert these Oracle OLAP cube into XML cube for uniformity. in an research paper i find that there is an operator Xcube embedded in XQuery which converts the multidimensional data (cube) into XML cube. im looking for the implementation of this operator in Query that how this operator works.
hope u understand my architecture. but if u still have some confusion, kindly give me ur mail id. i will mail the diagram of my architecture.
thanks.
saqib -
We would like to include in the UUT_Results table a value that we read from our data base into a local variable during the execution of our sequence file. We found that by modifying the configure database options we were able to add a column for this variable, but the local variable was not available to be placed into an expression for that column from the local variables or parameters. Is it possible to do this, and if so, how? Station Globals were available to be included in the expression, however the sequence file may be executed on more than one system which makes the Global unavailable on systems other than the one where the sequence file originated.
You can use the TestStand API to programmatically create global variables at runtime, thus ensuring their existence. For example, you could call Engine.Globals.SetValString("GlobalStringVariableName", PropOption_InsertIfMissing, "variable value")
Of course, if you need to test multiple UUTs in parallel, a single global is not sufficient. In that case you might consider adding the field you need to the UUT datatype in the process model. You could then access the field in your sequence via RunState.Root.Locals.UUT.YourNewField = YourNewValue.
If you also want your sequence to run without using a process model, you must check for the existence of the UUT before accessing it. You could use the expression function: PropertyExists("RunState.Root.Locals.UUT.YourNewFi
eld") -
How to use a property from other data source
Scenario: we have 2 datasources, that each have their own set of records having different properties. What I need to do is grab 1 property that exists in all records of 1 of the datasources and display it in the records of the other datasource. There is 1 "common" property (called by different names) in the records of both Data Sources, that have the same value; so this property could be the link between those records even though they belong to 2 different Data Sources.
Here's a simple example:
Data Source " Department" has records that have the properties: 'Name', 'Age' and 'Nationality'.
Data Source "Staff" has records that have the properties:'Staff_name', 'Position'.
The 'Name' and 'Staff_name' have the same values, and can be used to link the 2 records.
I'd like to get the records in the 'Department' data souce to also contain the property 'Position" from the "Staff" datasource .
Is there a manipulator that can do this?
Thanks!
JThanks Pravin,
But I don't want to join all data from the 2 datasources - I just want one property from one data source to appear in the records of the other data source. -
SQL bulk copy from csv file - Encoding
Hi Experts
This is the first time I am creating a PowerShell script and it is almost working. I just have some problems with the actual bulk import to SQL encoding from the text file since it replaces
special characters with a question mark. I have set the encoding when creating the csv file but that does not seem to reflect on the actual bulk import. I have tried difference scenarios with the encoding part but I cannot find the proper solution for that.
To shortly outline what the script does:
Connect to Active Directory fetching all user - but excluding users in specific OU's
Export all users to a csv in unicode encoding
Strip double quote text identifiers (if there is another way of handling that it will be much appreciated)
Clear all records temporary SQL table
Import records from csv file to temporary SQL table (this is where the encoding is wrong)
Update existing records in another table based on the records in the temporary table and insert new record if not found.
The script looks as the following (any suggestions for optimizing the script are very welcome):
# CSV file variables
$path = Split-Path -parent "C:\Temp\ExportADUsers\*.*"
$filename = "AD_Users.csv"
$csvfile = $path + "\" + $filename
$csvdelimiter = ";"
$firstRowColumns = $true
# Active Directory variables
$searchbase = "OU=Users,DC=fabrikam,DC=com"
$ADServer = 'DC01'
# Database variables
$sqlserver = "DB02"
$database = "My Database"
$table = "tblADimport"
$tableEmployee = "tblEmployees"
# Initialize
Write-Host "Script started..."
$elapsed = [System.Diagnostics.Stopwatch]::StartNew()
# GET DATA FROM ACTIVE DIRECTORY
# Import the ActiveDirectory Module
Import-Module ActiveDirectory
# Get all AD users not in specified OU's
Write-Host "Retrieving users from Active Directory..."
$AllADUsers = Get-ADUser -server $ADServer `
-searchbase $searchbase -Filter * -Properties * |
?{$_.DistinguishedName -notmatch 'OU=MeetingRooms,OU=Users,DC=fabrikam,DC=com' `
-and $_.DistinguishedName -notmatch 'OU=FunctionalMailbox,OU=Users,DC=fabrikam,DC=com'}
Write-Host "Users retrieved in $($elapsed.Elapsed.ToString())."
# Define labels and get specific user fields
Write-Host "Generating CSV file..."
$AllADUsers |
Select-Object @{Label = "UNID";Expression = {$_.objectGuid}},
@{Label = "FirstName";Expression = {$_.GivenName}},
@{Label = "LastName";Expression = {$_.sn}},
@{Label = "EmployeeNo";Expression = {$_.EmployeeID}} |
# Export CSV file and remove text qualifiers
Export-Csv -NoTypeInformation $csvfile -Encoding Unicode -Delimiter $csvdelimiter
Write-Host "Removing text qualifiers..."
(Get-Content $csvfile) | foreach {$_ -replace '"'} | Set-Content $csvfile
Write-Host "CSV file created in $($elapsed.Elapsed.ToString())."
# DATABASE IMPORT
[void][Reflection.Assembly]::LoadWithPartialName("System.Data")
[void][Reflection.Assembly]::LoadWithPartialName("System.Data.SqlClient")
$batchsize = 50000
# Delete all records in AD import table
Write-Host "Clearing records in AD import table..."
Invoke-Sqlcmd -Query "DELETE FROM $table" -Database $database -ServerInstance $sqlserver
# Build the sqlbulkcopy connection, and set the timeout to infinite
$connectionstring = "Data Source=$sqlserver;Integrated Security=true;Initial Catalog=$database;"
$bulkcopy = New-Object Data.SqlClient.SqlBulkCopy($connectionstring, [System.Data.SqlClient.SqlBulkCopyOptions]::TableLock)
$bulkcopy.DestinationTableName = $table
$bulkcopy.bulkcopyTimeout = 0
$bulkcopy.batchsize = $batchsize
# Create the datatable and autogenerate the columns
$datatable = New-Object System.Data.DataTable
# Open the text file from disk
$reader = New-Object System.IO.StreamReader($csvfile)
$columns = (Get-Content $csvfile -First 1).Split($csvdelimiter)
if ($firstRowColumns -eq $true) { $null = $reader.readLine()}
Write-Host "Importing to database..."
foreach ($column in $columns) {
$null = $datatable.Columns.Add()
# Read in the data, line by line
while (($line = $reader.ReadLine()) -ne $null) {
$null = $datatable.Rows.Add($line.Split($csvdelimiter))
$i++; if (($i % $batchsize) -eq 0) {
$bulkcopy.WriteToServer($datatable)
Write-Host "$i rows have been inserted in $($elapsed.Elapsed.ToString())."
$datatable.Clear()
# Add in all the remaining rows since the last clear
if($datatable.Rows.Count -gt 0) {
$bulkcopy.WriteToServer($datatable)
$datatable.Clear()
# Clean Up
Write-Host "CSV file imported in $($elapsed.Elapsed.ToString())."
$reader.Close(); $reader.Dispose()
$bulkcopy.Close(); $bulkcopy.Dispose()
$datatable.Dispose()
# Sometimes the Garbage Collector takes too long to clear the huge datatable.
[System.GC]::Collect()
# Update tblEmployee with imported data
Write-Host "Updating employee data..."
$queryUpdateUsers = "UPDATE $($tableEmployee)
SET $($tableEmployee).EmployeeNumber = $($table).EmployeeNo,
$($tableEmployee).FirstName = $($table).FirstName,
$($tableEmployee).LastName = $($table).LastName,
FROM $($tableEmployee) INNER JOIN $($table) ON $($tableEmployee).UniqueNumber = $($table).UNID
IF @@ROWCOUNT=0
INSERT INTO $($tableEmployee) (EmployeeNumber, FirstName, LastName, UniqueNumber)
SELECT EmployeeNo, FirstName, LastName, UNID
FROM $($table)"
try
Invoke-Sqlcmd -ServerInstance $sqlserver -Database $database -Query $queryUpdateUsers
Write-Host "Table $($tableEmployee) updated in $($elapsed.Elapsed.ToString())."
catch
Write-Host "An error occured when updating $($tableEmployee) $($elapsed.Elapsed.ToString())."
Write-Host "Script completed in $($elapsed.Elapsed.ToString())."I can see that the Export-CSV exports into ANSI though the encoding has been set to UNICODE. Thanks for leading me in the right direction.
No - it exports as Unicode if set to.
Your export was wrong and is exporting nothing. Look closely at your code:
THis line exports nothing in Unicode"
Export-Csv -NoTypeInformation $csvfile -Encoding Unicode -Delimiter $csvdelimiter
There is no input object.
This line converts any file to ansi
(Get-Content $csvfile) | foreach {$_ -replace '"'} | Set-Content $csvfile
Set-Content defaults to ANSI so the output file is converted.
Since you are just dumping into a table by manually building a recorset why not just go direct. You do not need a CSV. Just dump theresults of the query to a datatable.
https://gallery.technet.microsoft.com/scriptcenter/4208a159-a52e-4b99-83d4-8048468d29dd
This script dumps to a datatable object which can now be used directly in a bulkcopy.
Here is an example of how easy this is using your script:
$AllADUsers = Get-ADUser -server $ADServer -searchbase $searchbase -Filter * -Properties GivenName,SN,EmployeeID,objectGUID |
Where{
$_.DistinguishedName -notmatch 'OU=MeetingRooms,OU=Users,DC=fabrikam,DC=com'
-and $_.DistinguishedName -notmatch 'OU=FunctionalMailbox,OU=Users,DC=fabrikam,DC=com'
} |
Select-Object @{N='UNID';E={$_.objectGuid}},
@{N='FirstName';Expression = {$_.GivenName}},
@{N='LastName';Expression = {$_.sn}},
@{N=/EmployeeNo;Expression = {$_.EmployeeID}} |
Out-DataTable
$AllDUsers is now a datatable. You can just upload it.
¯\_(ツ)_/¯ -
Using SQL Statements in Error Exceptions
Is there any way to include SQL statements within my catch routines (i.e.
catch (java.io.IOException ex) {
System.out.println("An error occurred whilst writing to the message buffer: " + ex);
#sql { INSERT INTO DEBUG_TBL(CURRENT_DT,PROGRAM_ID,DEBUG__MESG) VALUES (SYSDATE,'TEST',:ex) };
#sql { COMMIT };
As it stands I get the following error upon compile:
Error: Unsupported Java type for host item (at position#1): java.io.IOException
Any help would be greatly appreciated.
Thx,
Boybles
nullYou can use SQL statements in your catch blocks, just not SQL statements that use Java types which do not map to SQL.
You cannot pass exceptions from Java to SQL. You can catch a SQL exceptions as a SQLException in Java. However, any exception that is thrown by a Java Stored Procedure (including SQLException) is rendered as an "Uncaught Java exception" in SQL.
When you receive a SQLException there is information about the originator, the error message, etc. in the exception object. See:
http://technet.oracle.com/docs/products/oracle8i/doc_library/817_doc/java.817/a83723/keyprog5.htm#1004462
While you cannot insert a Java exception per se into a table, you can extract the message string(s)/code and insert that in a table. -
Using a dropdown box to change the specified value of a radio button?
I have a form that generates the body text for an email.
First there is a dropdown box in which the choices are either "red" or "blue"
Further down the page, there is a radiobutton list with 2 items "available" and "unavailable".
The specified values for these items (as defined on the binding tab) are "have blue hats" or "do not have blue hats".
This value is concatenated in the email body textfield which is to be copied by the user to an email body:
Concat("Dear", NameField, "," We ", RadioButtonList, " today. ") // Dear Persons Name, We have blue hats today.
Is there any way to change the value of the selected radio button based on the selected dropdown choice?
So if the user selected "red" in the dropdown field, the radio button values could become "have red hats" or "do not have red hats"?
Or perhaps there is a better way from the start?
Thanks!Ok, so now I've done that...
But is there a way to change that value with another field (ie when a user chooses "red" or "blue" in a dropdown box) so that the concatenated message reflects both the users choice of color and the availibility?
I'm guessing I should probably use an intermediary text box and put
InterText.rawValue = Concat ("We have ', dropdown1, " hats today") in the change event of the available radio button and:
InterText.rawValue = Concat ("We do not have ', dropdown1, " hats today") in the change event of the unavailable radio button.
then concatenate that box in the final textbox. -
I have a biztalk application that has a source schema and destination schema inside a map.
A new field/column in the sql database has been created. Now I have to add this field/record to my source schema.
My question is, do I manually add this field/record to my source schema and update my source map or do I have to generate a new schema using add generated items>consume adapter service>sql and update my source map ?You would need to add the fields to the message schema your SQL data is mapped onto.
Also update the SQL statement which is executed inside your sql adapter settings to return the new columns.
Rachit
Please mark as answer or vote as helpful if my reply does -
How -using Terminal- can I copy all files (hidden and not hidden) and folders (and their contents) from one place to another
ditto
DITTO(1) BSD General Commands Manual DITTO(1)
NAME
ditto -- copy directory hierarchies, create and extract archives
SYNOPSIS
ditto [-v] [-V] [-X] [<options>] src ... dst_directory
ditto [-v] [-V] [<options>] src_file dst_file
ditto -c [-z | -j | -k] [-v] [-V] [-X] [<options>] src dst_archive
ditto -x [-z | -j | -k] [-v] [-V] [<options>] src_archive ...
dst_directory
ditto -h | --help
DESCRIPTION
In its first form, ditto copies one or more source files or directories
to a destination directory. If the destination directory does not exist
it will be created before the first source is copied. If the destination
directory already exists then the source directories are merged with the
previous contents of the destination. -
How to insert data to the specified row column of the multi column list box
Hi All
How do i insert data into the specified column of the multi cplumn list box?
I have a table that containsall station nos and name.Then another table contains the data the various stations having at for 24 hrs.That is 12 am to 11 pm.
And i want to display each stations details as follows using a multi column list box/table
My stationinfo table
stnno stnname......................
s1 stn1
s2 stn2
s3 stn3
The other table
stnno sysdatetime data
s1 12am 1
s2 12am 4
s1 1 am 2
So the station s1,s2.... will have data for 24 hrs.
And i want to display it as follows using a multicolumn listbox
stnname 12am 1 am ......................................11pm
s1 ...................
s2 ........................
What i have in my mind is to get all station nos
and in a for loop get the station's data from 12 am to 11 pm
or
select every statios data for each hor.But in this case i have to query the database 24 times.So i dont think its a good way.
Or any other better query available?
Can anybody suggest me a good idea?
One more thing...how to insert data into the specified field row or column of a multi column list box?
Thanks in advancehi
i want to know,,can u say ur need clearly...and i attached two image u see that one
Indrajit
| [email protected] | [email protected] .
Attachments:
station.JPG 35 KB
station2.JPG 79 KB -
I have very simple code to get AD domain name
Public Function GetDomainName() As String
Dim sDomainName As String = String.Empty
Try
Dim currentUser As String = WindowsIdentity.GetCurrent().Name.Replace("\", "/")
Dim strPath As String = String.Format("WinNT://{0}", currentUser)
Dim dirEntry As DirectoryEntry = New DirectoryEntry(strPath)
sDomainName = dirEntry.Parent.Name
Catch ex As Exception
Console.WriteLine(ex.Message)
Console.WriteLine(ex.StackTrace)
End Try
Return sDomainName
End Function
Here is exception that it gives:
The network path was not found.
, , at System.DirectoryServices.DirectoryEntry.Bind(Boolean throwIfFail)
at System.DirectoryServices.DirectoryEntry.Bind()
at System.DirectoryServices.DirectoryEntry.get_Parent()
at Kofax.PrivateProfileData.clsNetSecurity.GetDomainName() in clsNetSecurity.vb:line 46
Do anyone help me what is the problem?
Is that my environment problem?
Thank youThis is not a recommended approach to get the domain name - also just to make sure do you intend to get the domain name of the computer or the user, if the later use DsCrackNames:
http://msdn.microsoft.com/en-us/library/ms675970(v=vs.85).aspx (p/invoke .NET version is available)
The DS_NAME_RESULT item struct contains the domain name the user belongs to in pDomain
Enfo Zipper
Christoffer Andersson – Principal Advisor
http://blogs.chrisse.se - Directory Services Blog -
Using SQL instances - how to configure the db connection
WCI 10.3.0
We changed the DB server, and instead of a regular SQL server using the 1433 port, we are using an instance
SQLSERVER1\_SQLINSTANCE1
The problem is that, in this case, the port is dynamic allocated.
The question is:
How I configure this in Webcenter Interaction? Anyone experienced this kind of Db migration?
Thanks!
ValWCI requires use of a static port. You can assign a static port to your named instance.
Make sure TCP/IP is enabled, as sometimes only named pipes is turned on.
Then for WCI use the hostname and port you specify. You can do this in the configuration manager. -
How to use sql:query/ tag to retrive the system date?
hi all,
i want to know if i can use the <sql:query/> tag in jstl to get the database server time. please send me a code snippet to retrive n display the result. i'm new to jstl tags. please help.
thank you,
AMCheck out the first hits: http://www.google.com/search?q=jstl+sql+tutorial+site:sun.com
-
Using AGG/CALC DIM function in the BSO target cube of Transparent Partition
Hi,
Is it possible to use AGG("dimension") on a dimnesion in the BSO cube that is a target cube for a BSO-BSO Transparent Partition/
I have created the partition area with 3 dimensions, one being 'Source'. 90% of the source members are being refrenced from source to target via that transparent partition. But I still have the 10% Source members in the target cube that need to roll up to the 'Source"(along with the 90%) for correct aggregation.
When I try to use AGG("Source") in a FIX statement that's within the partition area, I get the following error:
Error: 1042013 Network error [10054]: Cannot Receive Data
Error: 1042012 Network error [10054]: Cannot Send Data
Can I not aggregate the dimension in the Target cube?
ThanksHi,
We have the same problem.
I try to create transparent pertition between BSO (as a source) and ASO (target) and have the same warning message.
Just to eliminate possible effect of outline complexity I created very simple DBs with the following outlines:
__For ASO:__
mes1
mes2
Measures
prd1
prd2
Products
Actual
Adjustment
Scenarios
__For BSO:__
mes1
mes2
Measures
prd1
prd2
Products
Adjustment
Scenarios
In addition to the message I have the following behavior - ASO has data in "Actual" scenario, BSO has data in "Adjustment" scenario. Before partition definition I can see the data in the right places, but when the partition created, Actual data disapeared from report of ASO. It is still in the database, and when the partition removed - appears again.
Any ideas?
Thank you in advance,
Alexey -
Can TCP/IP communication be used for 2 Labview processes on the same target?
I am a newbie to all this LV stuff so bear with me. We are trying to create a simulation tool that will run on the same machine as the client GUI [also LV]. Can we use TCP/IP to communicate between these 2 LV processes on the same machine? If so, how? Thanks for the help.
Hi nathand,
I have another issue while communicating with TCP/IP protocol.
I am sending data from RT target to Host computer(Main UI ) using TCP . Also I have created EXE file of main main UI which one is running on Host computer. and ussing this exe file in three different PC and want to send the control signal from all four PC (three exe installed and one host PC). I created four communication loop in RT with four different port for communication with four PC.
When I am running the all VIs ,Its try to connect to the port of the listener and and showing keep waiting for a connection. Another problem is whatever value we sending from four PC , Target vi receiving random value with actual value. Also every time its getting reset.
So please help me here.
Thanks!
Maybe you are looking for
-
How can I share my iTunes with another computer who is in another state?
I just want to send my sister one song from my iTunes because its a recording of her daughter when she was 2. She no longer has the file and I only have it saved in my iTunes from a long ago. The only known file is the one I have saved on iTunes. I t
-
When I press on a land line number, the phone does not call the number, but comes up with a screen to send a text to it. How do I get my phone to default to phoneing a landline? When I press a mobile number in contacts, the phone automatically phon
-
Crash furing Garageband3 Export to iTunes
I wanted to export a track to itunes from GB3. My itunes resides on external drive. When given the command to do so, GB3 displayed itunes as if it had never run. It cleared my whole list. (The files survived.) I then copied them to the "new " itunes,
-
How can I recover my licence Acrobat 9 Pro from my HDD?
Hi, I want to change my HDD for a SSD. Unfortunately, I have not my licence for Acrobat Pro 9. I still have access to the software. Is it possible to recover it directly from the HDD? Thanks a lot for your help.
-
How can I format pl-sql code with my own settings
Are there any tools for formatting pl-sql in SD 1.1 excepting Ctrl-B? How can I adopt Ctrl-B for my own needs?