Bulk copy from a temp table

My input is from a file. Since I do not have an ETL tool, I am using a stored proc to do the ETL (which also gives me an advantage, I do not have to unload the target table to do the join). So, I dump the file contents into a temp table and use it in proc.
The query is like
Insert into <target table1> (Select fields and some transformation from <temp table> where <key> not in target table and <some joins with other tables in database>
Like this I have four queries for four target tables.
The inserts from the temp table into the target table is very slow because the target has a lot of index and RI. I cannot drop & create the index since the application requirements does not give me that liberty.
My only option is to insert in a temp table similar to the target but without any index/RI/PK and then dump it into a file and then use SQL loader to load the file contents into target table. This is relatively faster but is a very cumbersome route to me.
Is there any other way to do bulk insert from one table to another table like SQL loader without using a file? Is there anyway to bypass the index update operation without dropping the index?
My source will be almost 500,000 rows and target is having 9 million rows.

Posts like this one are better avoided.
Because
- You don't post a version
- You don't post the SQL
- You don't post the EXPLAIN PLAN
It is your assertion the INSERT is to blame, yet it can equally be the SELECT statement involved.
Basically your post boils down to
'It doesn't work. Please help', without any relevant information.
I'm saying this because INSERT SELECT is the fastest method available. OK, you could try the APPEND hint, but in that case you would have to rebuild all indices. Something you state you can not do.
BULK INSERTs will be slower, SQL*Loader will be slower too, as it involves SQLnet. INSERT SELECT is a server side operation.
And the 'solution' to do this by means of a file... Ahem, let's not talk about it. It just doesn't work.
Sybrand Bakker
Senior Oracle DBA

Similar Messages

MS SQL server 2008 - Bulk copy from XML to DB table throws bcp_init error

I have MS SQL server 2008 installed ,
Windows version - Windows 7 Professional with SP1
Doing a bulk copy process using the SQL library function bcp_init function in c++ throws xml error and its not inserting the data into the tables.
Error received ,
XML Datatransfer error: XML data or another error occurred while reading file 'd:\temp\scripts\dbtoolscripts\table_data.xml':
But this works in others machines with the same windows version and SQL version.

We are using the same SQL lib function bcp_init , we have written a separate class to load the ODBC32.dll , sqlncli.dll and use the bcp_init and other SQL functions from that. As i mentioned earlier we all use the same application build , but
its not working in my machine only other machines its working.
There is no provision to load XML files when you use the BCP interface in sqlncli.dll. You can of course load XML files, but the BCP API does not know that it is XML, it only sees a number of bytes.
So that error message is not coming from the BCP API, but somewhere else. Maybe your own code in reaction to some error from the BCP API. But without any clue of that error message, we can't help you.
I think you will need to do some debugging or by some other means improve your diagnostics.
Erland Sommarskog, SQL Server MVP, [email protected]

Bulk-copy from Sybase View into a File using Java Runtime...help !!!

The following is a bulk-copy operation, from a table in Sybase database into a file in Unix machine.
bcp view_name out file_name -c -U user_id -P password -S server_name
This bulk-copy utility is working fine when run from command line of a Unix machine, for any number of records.
But when I try to do it from a Java application for more than 1.84 lakh records in the view, the following problem occurs.
The file is created and from the view, 1.84 lakh records are copied into the file.
Then the program hangs.
The application waits for the bulk-copy process to be completed, which does not resume after copy of 1.84 lakh records.
No exceptions are thrown.
Even for just one record more than 1.84 lakh records, this happens.
The number of bytes corresponding to 1.84 lakh records many records are 11040060.
This is the bulk-copy portion of the application.
String strCommand = "bcp view_name out file_name -c -U user_id -P password -S server_name";
Runtime run = Runtime.getRuntime();
Process proc = run.exec(strCommand);
int i = proc.waitFor();
I am capturing InterruptedException, IOException and the superset Exception also.
Pls throw some light on this.

Sorry to bother you, but I saw this topic you posted on the Sun Java Forum and was wondering if you were ever able to resolve it.
I am experiencing the exact same problem and was hoping to get it resolved soon.
Thanks in advance!

SQL bulk copy from csv file - Encoding

Hi Experts
This is the first time I am creating a PowerShell script and it is almost working. I just have some problems with the actual bulk import to SQL encoding from the text file since it replaces
special characters with a question mark. I have set the encoding when creating the csv file but that does not seem to reflect on the actual bulk import. I have tried difference scenarios with the encoding part but I cannot find the proper solution for that.
To shortly outline what the script does:
Connect to Active Directory fetching all user - but excluding users in specific OU's
Export all users to a csv in unicode encoding
Strip double quote text identifiers (if there is another way of handling that it will be much appreciated)
Clear all records temporary SQL table
Import records from csv file to temporary SQL table (this is where the encoding is wrong)
Update existing records in another table based on the records in the temporary table and insert new record if not found.
The script looks as the following (any suggestions for optimizing the script are very welcome):
# CSV file variables
$path = Split-Path -parent "C:\Temp\ExportADUsers\*.*"
$filename = "AD_Users.csv"
$csvfile = $path + "\" + $filename
$csvdelimiter = ";"
$firstRowColumns = $true
# Active Directory variables
$searchbase = "OU=Users,DC=fabrikam,DC=com"
$ADServer = 'DC01'
# Database variables
$sqlserver = "DB02"
$database = "My Database"
$table = "tblADimport"
$tableEmployee = "tblEmployees"
# Initialize
Write-Host "Script started..."
$elapsed = [System.Diagnostics.Stopwatch]::StartNew()
# GET DATA FROM ACTIVE DIRECTORY
# Import the ActiveDirectory Module
Import-Module ActiveDirectory
# Get all AD users not in specified OU's
Write-Host "Retrieving users from Active Directory..."
$AllADUsers = Get-ADUser -server $ADServer `
-searchbase $searchbase -Filter * -Properties * |
?{$_.DistinguishedName -notmatch 'OU=MeetingRooms,OU=Users,DC=fabrikam,DC=com' `
-and $_.DistinguishedName -notmatch 'OU=FunctionalMailbox,OU=Users,DC=fabrikam,DC=com'}
Write-Host "Users retrieved in $($elapsed.Elapsed.ToString())."
# Define labels and get specific user fields
Write-Host "Generating CSV file..."
$AllADUsers |
Select-Object @{Label = "UNID";Expression = {$_.objectGuid}},
@{Label = "FirstName";Expression = {$_.GivenName}},
@{Label = "LastName";Expression = {$_.sn}},
@{Label = "EmployeeNo";Expression = {$_.EmployeeID}} |
# Export CSV file and remove text qualifiers
Export-Csv -NoTypeInformation $csvfile -Encoding Unicode -Delimiter $csvdelimiter
Write-Host "Removing text qualifiers..."
(Get-Content $csvfile) | foreach {$_ -replace '"'} | Set-Content $csvfile
Write-Host "CSV file created in $($elapsed.Elapsed.ToString())."
# DATABASE IMPORT
[void][Reflection.Assembly]::LoadWithPartialName("System.Data")
[void][Reflection.Assembly]::LoadWithPartialName("System.Data.SqlClient")
$batchsize = 50000
# Delete all records in AD import table
Write-Host "Clearing records in AD import table..."
Invoke-Sqlcmd -Query "DELETE FROM $table" -Database $database -ServerInstance $sqlserver
# Build the sqlbulkcopy connection, and set the timeout to infinite
$connectionstring = "Data Source=$sqlserver;Integrated Security=true;Initial Catalog=$database;"
$bulkcopy = New-Object Data.SqlClient.SqlBulkCopy($connectionstring, [System.Data.SqlClient.SqlBulkCopyOptions]::TableLock)
$bulkcopy.DestinationTableName = $table
$bulkcopy.bulkcopyTimeout = 0
$bulkcopy.batchsize = $batchsize
# Create the datatable and autogenerate the columns
$datatable = New-Object System.Data.DataTable
# Open the text file from disk
$reader = New-Object System.IO.StreamReader($csvfile)
$columns = (Get-Content $csvfile -First 1).Split($csvdelimiter)
if ($firstRowColumns -eq $true) { $null = $reader.readLine()}
Write-Host "Importing to database..."
foreach ($column in $columns) {
$null = $datatable.Columns.Add()
# Read in the data, line by line
while (($line = $reader.ReadLine()) -ne $null) {
$null = $datatable.Rows.Add($line.Split($csvdelimiter))
$i++; if (($i % $batchsize) -eq 0) {
$bulkcopy.WriteToServer($datatable)
Write-Host "$i rows have been inserted in $($elapsed.Elapsed.ToString())."
$datatable.Clear()
# Add in all the remaining rows since the last clear
if($datatable.Rows.Count -gt 0) {
$bulkcopy.WriteToServer($datatable)
$datatable.Clear()
# Clean Up
Write-Host "CSV file imported in $($elapsed.Elapsed.ToString())."
$reader.Close(); $reader.Dispose()
$bulkcopy.Close(); $bulkcopy.Dispose()
$datatable.Dispose()
# Sometimes the Garbage Collector takes too long to clear the huge datatable.
[System.GC]::Collect()
# Update tblEmployee with imported data
Write-Host "Updating employee data..."
$queryUpdateUsers = "UPDATE $($tableEmployee)
SET $($tableEmployee).EmployeeNumber = $($table).EmployeeNo,
$($tableEmployee).FirstName = $($table).FirstName,
$($tableEmployee).LastName = $($table).LastName,
FROM $($tableEmployee) INNER JOIN $($table) ON $($tableEmployee).UniqueNumber = $($table).UNID
IF @@ROWCOUNT=0
INSERT INTO $($tableEmployee) (EmployeeNumber, FirstName, LastName, UniqueNumber)
SELECT EmployeeNo, FirstName, LastName, UNID
FROM $($table)"
try
Invoke-Sqlcmd -ServerInstance $sqlserver -Database $database -Query $queryUpdateUsers
Write-Host "Table $($tableEmployee) updated in $($elapsed.Elapsed.ToString())."
catch
Write-Host "An error occured when updating $($tableEmployee) $($elapsed.Elapsed.ToString())."
Write-Host "Script completed in $($elapsed.Elapsed.ToString())."

I can see that the Export-CSV exports into ANSI though the encoding has been set to UNICODE. Thanks for leading me in the right direction.
No - it exports as Unicode if set to.
Your export was wrong and is exporting nothing. Look closely at your code:
THis line exports nothing in Unicode"
Export-Csv -NoTypeInformation $csvfile -Encoding Unicode -Delimiter $csvdelimiter
There is no input object.
This line converts any file to ansi
(Get-Content $csvfile) | foreach {$_ -replace '"'} | Set-Content $csvfile
Set-Content defaults to ANSI so the output file is converted.
Since you are just dumping into a table by manually building a recorset why not just go direct. You do not need a CSV. Just dump theresults of the query to a datatable.
https://gallery.technet.microsoft.com/scriptcenter/4208a159-a52e-4b99-83d4-8048468d29dd
This script dumps to a datatable object which can now be used directly in a bulkcopy.
Here is an example of how easy this is using your script:
$AllADUsers = Get-ADUser -server $ADServer -searchbase $searchbase -Filter * -Properties GivenName,SN,EmployeeID,objectGUID |
Where{
$_.DistinguishedName -notmatch 'OU=MeetingRooms,OU=Users,DC=fabrikam,DC=com'
-and $_.DistinguishedName -notmatch 'OU=FunctionalMailbox,OU=Users,DC=fabrikam,DC=com'
} |
Select-Object @{N='UNID';E={$_.objectGuid}},
@{N='FirstName';Expression = {$_.GivenName}},
@{N='LastName';Expression = {$_.sn}},
@{N=/EmployeeNo;Expression = {$_.EmployeeID}} |
Out-DataTable
$AllDUsers is now a datatable. You can just upload it.
¯\_(ツ)_/¯

UCCX DB Get Step - Retrieving data from a Temp Table

Hello All,
I have a situaion where I am running a stored procedure from a DB Read step and this stored procedure creates a temporary table that holds the data from SQL. I need assistance in determining how to retrieve this data as the table is dropped once the SP has finished. I am sure there will be additional information requested for you to help me out, but I appeciate everyone's help in advance.
Chris G

Hi Chris,
I am trying to achieve the same. Did you manage to figure this out?
Thanks.

Doubling Summed Numbers from Temp Table

I have a crystal report that uses an Oracle Stored Procedure. The stored procedure sums (using Oracle sum function) some of the data from another table into a temp table, does some additional data manipulation and then selects from the temp table. My problem is any of the data that was summed in the stored procedure is displaying as double in the Crystal Report, (e.g. a field that sums to 10 is displaying as 20 in the Crystal Report.) When I run this Stored Procedure from Oracle SQL Developer I get the correct data.
Could anyone tell me why this is happening and what to do to correct they problem? I am using Version 11.0.0.1282 of Crystal and Oracle 10g.
Thanks,
Debbie

Hi Debbie,
You also did not mention which DB driver you are using, ODBC, OLE DB or our native Oracle driver. Try one of the other ones to see they work. Could be an issue in the specific driver you are using.
Testing in Oracle will use the native client engine. Try using a third party SQL, ODBC or OLE DB test tool to verify running the SP works.
Thanks again
Don

Insert data from an tabular to a temp table and fetching a columns.

Hi guys ,
I am working in apex 3.2 in which in a page i have a data's fom various tables and displays it in tabular form. Then i have to insert the tabular form data to a temp table and fetch the data from the temp table and insert into my main table. I think that i have to use a cursor to fetch the data from the temp table and insert into the main table but i didnt get the perfect example for doing this. Can any one help me to sort it out.
Thanks With regards
Balaji

Hi,
Follow this scenario.
Your Query:
SELECT t1.col1, t1.col2, t2.col1, t2.col2, t3.col1
FROM table1 t1, table2 t2, table3 t3
(where some join conditions);On insert button click call this process
DECLARE
temp1 VARCHAR2(100);
temp2 VARCHAR2(100);
temp3 VARCHAR2(100);
temp4 VARCHAR2(100);
temp5 VARCHAR2(100);
BEGIN
     FOR i IN 1..apex_application.g_f01.COUNT
     LOOP
          temp1    := apex_application.g_f01(i);
          temp2    := apex_application.g_f02(i);
          temp3    := apex_application.g_f03(i);
          temp4    := apex_application.g_f04(i);
          temp5    := apex_application.g_f05(i);
          INSERT INTO table1(col1, col2) VALUES(temp1, temp2);
          INSERT INTO table2(col1, col2) VALUES(temp3, temp4);
          INSERT INTO table3(col1) VALUES(temp5);
     END LOOP;
END;You don't even need temp tables and cursor to make an insert into different tables.
Thanks,
Ramesh P.
*(If you know you got the correct answer or helpful answer, please mark as corresponding.)*

Global Temp table Vs Regular table

Hi,
I need a temp table to store few IDs (which I select from a complex query). I would be deleting the data from the temp table at the end of the session.
From a performance perspective, I was wondering if I should be using a GLOBAL temporary table or just a regular table in which I will manually delete rows at the end.
The tkprof showed using the global temp table expensive. If yes, I just want to confirm it and know why it is the case.
Thanks!
Anand

Assuming that the cursor you open to get the transaction_id actually has a where clause, and refraining from comment on the need for dynamic column lists, I would likely do it something like:
DECLARE
   lv_query   VARCHAR2 (32767);
   l_col_list VARCHAR2(4000);
BEGIN
   l_col_list := control_module_common_pkg.get_columns('EPOS_TRANSACTION_HEADER',
                                                       FALSE);
   lv_query := 'INSERT INTO AK_TRANSACTION_HEADER ('||l_col_list||
               ' ) SELECT '||l_col_list||' FROM EPOS_TRANSACTION_HEADER
               WHERE ETH_TRANSACTION_ID IN (SELECT eth_transaction_id
                                            FROM epos_transaction_header
                                            WHERE <whatever would be in the cursor>)';
   EXECUTE IMMEDIATE lv_query;
   COMMIT;
END;If there really is no where clause ion the cursor, then it would simply be:
DECLARE
   lv_query   VARCHAR2 (32767);
   l_col_list VARCHAR2(4000);
BEGIN
   l_col_list := control_module_common_pkg.get_columns('EPOS_TRANSACTION_HEADER',
                                                       FALSE);
   lv_query := 'INSERT INTO AK_TRANSACTION_HEADER ('||l_col_list||
               ' ) SELECT '||l_col_list||' FROM EPOS_TRANSACTION_HEADER';
   EXECUTE IMMEDIATE lv_query;
   COMMIT;
END;In either case, I would also re-consider the commit in the procedure. It should perhaps be the caller of this procedure that issues the commit, not the procedure itself.
John

Usage of Temp tables in SSIS 2012

Hello,
We have many SSIS packages (2008 R2) which imports data to a temp table and process it from there.
We are upgrading to SQL server 2012 and facing the issue with temp table as working table and our ssis packages fail in 2012. While investigating found that SQL Server 2012 deprecates FMTONLY option
and instead uses
sp_describe_first_result_set , which does not support using of temp tables as import table. SSIS works fine in our workstations but not in the DEV box. With SQL 2012, I can execute from my workstation, which has (11.0.2100.60) where as DEV server
has SQL Server version 11.0.3000.0
Also when I ran profile with that of the DEV box, it gives two different statements
from workstation (11.0.2100.60)
CREATE TABLE #temp (
Id varchar(255) NULL,
Name varchar(255) NULL )
go
declare @p1 int
set @p1=NULL
declare @p3 int
set @p3=229378
declare @p4 int
set @p4=294916
declare @p5 int
set @p5=NULL
exec sp_cursoropen @p1 output,N'select * from #temp',@p3 output,@p4 output,@p5 output
select @p1, @p3, @p4, @p5
go
it works fine
But with the DEV server (version 11.0.3000.0), it executes the below sql and it fails to get the meta data
CREATE TABLE #temp (
Id varchar(255) NULL,
Name varchar(255) NULL )
exec [sys].sp_describe_first_result_set N'select * from [dbo].[#temp]'
On checking the assembly difference between the versions, I could only see Microsoft.SqlServer.ManagedDTS.dll being 11.0.3000.0, which I replace by 11.0.2100.60 version. but still getting the same result.
The other different I found is with ,Net framework libraries.
Could you advise whats the assembly causing this issue between our workstation and DEV server i.e 11.0.2100.60 and 11.0.3000.0
Many thanks

Scripts are taken from profiler.
The error message is
The metadata could not be determined because statement 'Select * from #branchscan' uses a temp table.
I could see the work around saying use of table variable and global temp tables. We are having around 100+ ssis packages which uses temp table for loading the data from a flat file and respective SP to process the data from the temp table. above
error is thrown during the pre-execute phase of the OLE db Destination, when trying to get the meta data of the table.
At this stage, it would be difficult for us to change the logic to global temp or TVP
Thanks

Select from (too many) tables

Hi all,
I'm a proud Oracle Apex developer. We have developed an Interactive Report that is generated from many joined tables in a remote system. I've read that to improve performances we can do the following:
1) Create a temporary table on our system that stores the app_user id and the colmun as a result of the query
2) Create a procedure that does:
declare
param1:= :PXX_item
param2:= :PXY_item.
param3:= :V('APP_USER')
insert into <our_table>
(select param3, <query from remore system>)
commit;
3) Rediresct to a query page where IR reads from this temp table
On "Exit" button there's a procedure that purge table data of that user (delete from temp where user=V('app_user'), so the temp table is only filled with necessary data.
Do you see any inconvenience? Application will be used from about 500 users, about 50 concurrent users at a time.
Thank you!

1) We don't have a control on source syste, we can only perform query on itI was referring to a materialized view on the system where Apex is installed, not on the source database.
2) There are many tables involvedI don't understand why this is a problem. Too much data I can see, but too many tables... not so much.
3) Data has to be in real time, with no delayThis would a problem for MV or collections. The collections would store the data as of the initial query. Any IRs using the collection after the fact would be using stale data. If you absolutely have to have the data as of right now every time, then the full query must run on the remote system every time. Tuning that query is the only option to make it faster.
4) There are many transactions on the source tables (they are the core of the source system) and so MV could not be refreshed so fastProbably could be with fast refresh enabled, but not necessarily practical to do so. As I indicated in 3, you have painted yourself into a corner here. You have indicated a need for a real-time query and that eliminates a number of possibilities for query-once use-many performance solutions.

Temp table, and gather table stats

One of my developers is generating a report from Oracle. He loads a subset of the data he needs into a temp table, then creates an index on the temp table, and then runs his report from the temp table (which is a lot smaller than the original table).
My question is: Is it necessary to gather table statistics for the temp table, and the index on the temp table, before querying it ?

It depends yesterday I have very bad experience with stats one of my table has NUM_ROWS 300 and count(*)-7million and database version is 9206(bad every with optimizer bugs) so queries starts breaking and lot of buffer busy and latch free it took while to figure out but I have deleted the stats and every thing came under control - my mean to say statistics are good and bad. Once you start collecting you should keep an eye.
Thanks.

PLSQL script not collecting temp table fields - variables problem?

I've 'written' a script to extract data from a temp table and load it directly into the associated Oracle tables via HRMS's
API packages.. but when I put the DBMS_OUTPUT.PUT_LINE's in I see that although it seems to read first API OK it doesn't collect the information from their relevant fields in the temp table.. Can anyone help please please..?
======================== code ==========================
SET serveroutput ON SIZE 1000000
SET verify OFF
SET feedback OFF
DECLARE
-- Debugging/error handling
v_err_seq NUMBER := 0;
v_err_num VARCHAR2 (30);
v_err_msg VARCHAR2 (250);
v_err_line VARCHAR2 (350);
-- Work variables
p_hire_date DATE;
p_business_group_id NUMBER := 0;
p_person_id NUMBER := 0;
p_address_line1 VARCHAR2 (240);
p_date_of_birth VARCHAR2 (35);
p_address_line2 VARCHAR2 (240);
employee_number VARCHAR2 (14);
p_employee_number VARCHAR2 (14);
emp_number VARCHAR2 (14);
p_email_address VARCHAR2 (240);
p_address_line3 VARCHAR2 (240);
p_first_name VARCHAR2 (150);
p_address_line4 VARCHAR2 (240);
p_middle_names VARCHAR2 (30);
p_post_code VARCHAR2 (30);
p_last_name VARCHAR2 (150);
p_nationality VARCHAR2 (30);
p_sex VARCHAR2 (30);
p_national_identifier VARCHAR2 (30);
p_title VARCHAR2 (30);
v_rec_cnt NUMBER := 0;
insert_flag VARCHAR2 (8);
-- ip_p_address_id NUMBER;
ip_p_address_id per_addresses.address_id%TYPE;
ip_p_object_version_number NUMBER;
ip_p_party_id per_addresses.party_id%TYPE;
l_person_id per_all_people_f.person_id%TYPE;
l_employee_number VARCHAR2 (35);
l_validate BOOLEAN DEFAULT FALSE;
l_assignment_id NUMBER;
l_per_object_version_number NUMBER;
l_asg_object_version_number NUMBER;
l_per_effective_start_date DATE;
l_per_effective_end_date DATE;
l_full_name VARCHAR2 (240);
l_per_comment_id NUMBER;
l_assignment_sequence NUMBER;
l_assignment_number VARCHAR2 (100);
l_name_combination_warning BOOLEAN;
l_assign_payroll_warning BOOLEAN;
l_address_id NUMBER;
l_object_version_number NUMBER;
return_code NUMBER;
return_message VARCHAR2 (2000);
command_prin VARCHAR2 (4000);
-- Get employee details info from work table
CURSOR get_employee_details
IS
SELECT p_person_id, p_validate, p_hire_date, p_business_group_id,
p_last_name, p_sex, p_date_of_birth, p_email_address,
p_employee_number, p_first_name, p_marital_status,
p_middle_names, p_nationality, p_title, p_national_identifier,
p_address_line1, p_address_line2, p_address_line3,
p_address_line4, p_post_code
FROM SU_TEMPLOYEE_DETAILS;
-- checks employee details info from PER_ALL_PEOPLE_F table
-- v_err_seq := 1;
CURSOR c_check_employee (emp_number VARCHAR2)
IS
SELECT per.person_id, per.business_group_id, per.last_name,
per.start_date, per.date_of_birth, per.email_address,
per.employee_number, per.first_name, per.marital_status,
per.middle_names, per.nationality, per.national_identifier,
per.sex, per.title, padd.address_id, padd.primary_flag,
padd.address_line1, padd.address_line2, padd.address_line3,
padd.town_or_city, padd.postal_code, padd.telephone_number_1,
padd.object_version_number
FROM per_all_people_f per, per_addresses padd
WHERE per.employee_number = emp_number
AND per.person_id = padd.person_id;
emp_rec c_check_employee%ROWTYPE;
BEGIN
--v_err_seq := 2;
command_prin := SQLERRM;
LOOP
-- Process each record in the work table
FOR v_emp IN get_employee_details
LOOP
v_rec_cnt := v_rec_cnt + 1;
-- determine whether customer already exists
OPEN c_check_employee (v_emp.p_employee_number);
FETCH c_check_employee
INTO emp_rec;
IF c_check_employee%NOTFOUND
THEN
insert_flag := 'I';
ELSE
insert_flag := 'X';
END IF;
IF insert_flag = 'I'
THEN
-- RETURN 'Employee does not exist, continue import..';
DBMS_OUTPUT.PUT_LINE ('Employee does not exist, continue import..');
ELSE
DBMS_OUTPUT.PUT_LINE ('Employee found - record cannot be imported.');
END IF;
CLOSE c_check_employee;
-- v_err_seq := 3;
-- Create new PER_ALL_PEOPLE_F and PER_ADDRESSES record from
-- info in table record
IF insert_flag = 'I'
THEN
BEGIN -- Importing Employee Procedure --
DBMS_OUTPUT.PUT_LINE ('          ');
DBMS_OUTPUT.PUT_LINE ('Importing employees....Hold On.......!     ');
     DBMS_OUTPUT.PUT_LINE ('          ');
BEGIN
Hr_Employee_Api.create_gb_employee
(p_validate => l_validate, --FALSE,
p_hire_date => p_hire_date,
p_business_group_id => p_business_group_id,
p_date_of_birth => p_date_of_birth,
p_email_address => p_email_address,
p_first_name => p_first_name,
p_middle_names => p_middle_names,
p_last_name => p_last_name,
p_sex => p_sex,
p_ni_number => p_national_identifier,
p_employee_number => l_employee_number,
p_person_id => l_person_id,
p_title => p_title,
p_assignment_id => l_assignment_id,
p_per_object_version_number => l_per_object_version_number,
p_asg_object_version_number => l_asg_object_version_number,
p_per_effective_start_date => l_per_effective_start_date,
p_per_effective_end_date => l_per_effective_end_date,
p_full_name => l_full_name,
p_per_comment_id => l_per_comment_id,
p_assignment_sequence => l_assignment_sequence,
p_assignment_number => l_assignment_number,
p_name_combination_warning => l_name_combination_warning,
p_assign_payroll_warning => l_assign_payroll_warning
DBMS_OUTPUT.PUT_LINE
('..employee record updated succesfully..');
DBMS_OUTPUT.PUT_LINE ('          ');
DBMS_OUTPUT.PUT_LINE ('          ');
EXCEPTION
WHEN OTHERS
THEN
DBMS_OUTPUT.PUT_LINE ('..SQLCodeErrors:- ' || SQLCODE);
     DBMS_OUTPUT.PUT_LINE (' ');
DBMS_OUTPUT.PUT_LINE ('Person ID:-' || p_person_id || l_person_id);
DBMS_OUTPUT.PUT_LINE ('Assignmnt Seq - '|| l_assignment_sequence);
DBMS_OUTPUT.PUT_LINE ('l_ass_no - ' ||l_assignment_number);
-- DBMS_OUTPUT.PUT_LINE ('Record failed to load.. ' || SQLERRM);
DBMS_OUTPUT.PUT_LINE (SUBSTR (command_prin, 1, 250));
END;
BEGIN -- Importing Associated Address Procedure --
DBMS_OUTPUT.PUT_LINE ('          ');
-- ('..and the associated employee address....');
Hr_Person_Address_Api.create_person_address
(p_validate => l_validate,
-- p_effective_date => p_hire_date,
p_effective_date => SYSDATE,
p_pradd_ovlapval_override => NULL,
p_validate_county => NULL,
p_person_id => l_person_id,
p_primary_flag => 'Y',
p_style => 'GB_GLB',
-- p_date_from => p_hire_date,
p_date_from => SYSDATE,
p_date_to => NULL,
p_address_type => NULL,
p_comments => NULL,
p_address_line1 => p_address_line1,
p_address_line2 => p_address_line2,
p_address_line3 => p_address_line3,
p_town_or_city => p_address_line4,
p_region_1 => NULL,
p_region_2 => NULL,
p_region_3 => NULL,
p_postal_code => p_post_code,
p_country => p_nationality,
p_telephone_number_1 => NULL,
p_telephone_number_2 => NULL,
p_telephone_number_3 => NULL,
p_addr_attribute_category => NULL,
p_addr_attribute1 => NULL,
p_addr_attribute2 => NULL,
p_addr_attribute3 => NULL,
p_addr_attribute4 => NULL,
p_addr_attribute5 => NULL,
p_addr_attribute6 => NULL,
p_addr_attribute7 => NULL,
p_addr_attribute8 => NULL,
p_addr_attribute9 => NULL,
p_addr_attribute10 => NULL,
p_addr_attribute11 => NULL,
p_addr_attribute12 => NULL,
p_addr_attribute13 => NULL,
p_addr_attribute14 => NULL,
p_addr_attribute15 => NULL,
p_addr_attribute16 => NULL,
p_addr_attribute17 => NULL,
p_addr_attribute18 => NULL,
p_addr_attribute19 => NULL,
p_addr_attribute20 => NULL,
p_add_information13 => NULL,
p_add_information14 => NULL,
p_add_information15 => NULL,
p_add_information16 => NULL,
p_add_information17 => NULL,
p_add_information18 => NULL,
p_add_information19 => NULL,
p_add_information20 => NULL,
-- p_party_id => NULL,
p_party_id => ip_p_party_id,
p_address_id => ip_p_address_id,
p_object_version_number => ip_p_object_version_number
DBMS_OUTPUT.PUT_LINE ('Address Updation/Insertion has been successful!');
EXIT WHEN command_prin IS NULL;
command_prin := SUBSTR (command_prin, 251);
END;
END;
-- v_err_seq := 4;
-- End of customer related details
END IF;
END LOOP;
-- DBMS_OUTPUT.PUT_LINE ('Records read : ' || v_rec_cnt);
-- v_err_seq := 5;
--EXCEPTION
-- WHEN OTHERS THEN
-- ROLLBACK;
-- Output Error Message
-- v_err_num := TO_CHAR(SQLCODE);
-- v_err_msg := SUBSTR(SQLERRM,1,250);
-- v_err_line := 'Oracle error (seqno=' || v_err_seq || ') ' ||
-- v_err_num ||' occurred processing record '||
-- TO_CHAR(v_rec_cnt + 1) ||' : '||v_err_msg;
-- DBMS_OUTPUT.PUT_LINE(v_err_line);
END LOOP;
COMMIT;
END;
--END;
EXIT;
======================================================
many thanks to all...
Steven

Ive just sussed it - I had'nt put the 'v_emp' at the front of the fields from the temp table to pick then up! we continue..

How to get current month from filename and bulk insert from text file into table?

I set up some dynamic SQL to help my bulk copy data from a text file to a table. This works fine for files that come in every day; I get the previous day’s data, based on the file name that’s placed
in the folder. That’s why I’m using the ‘-1’. The dates will look like this: '20140131', so I'm using type 112.
declare @fullpath1 varchar(1000)
select @fullpath1 = '''\\system.local\ms\london\FTP\' + convert(varchar, getdate()-1, 112) + '_INDEXPRICES_EOM.SPC'''
declare @cmd1 nvarchar(1000)
print (@cmd1)
select @cmd1 = 'bulk insert [dbo].[SB_Monthly] from ' + @fullpath1 + ' with (FIELDTERMINATOR = ''\t'', FIRSTROW = 5, LASTROW = 675, ROWTERMINATOR=''0x0a'')'
print(@cmd1)
exec (@cmd1)
I think the syntax will be somewhat similar to this:
YEAR(date_column)=YEAR(getdate()) AND MONTH(date_column)=MONTH(getdate())
I’m not totally sure how to incorporate that into my current syntax.
Knowledge is the only thing that I can give you, and still retain, and we are both better off for it.

I tried a couple versions of this.
Declare @StartDate Date, @EndDate Date
Select @StartDate = convert(varchar, getdate()-28, 112), @EndDate = convert(varchar, getdate()-1, 112)
BEGIN
declare @fullpath1 varchar(1000)
select @fullpath1 = '''\\ms\london\FTP\' + ''' between ''' + Convert(Varchar(10), @StartDate, 101) + ''' and ''' + Convert(Varchar(10), @EndDate, 101) + '''_SP.SPC'''
declare @cmd1 nvarchar(1000)
print (@cmd1)
select @cmd1 = 'bulk insert [dbo].[SPBMI_Monthly] from ' + @fullpath1 + ' with (FIELDTERMINATOR = ''\t'', FIRSTROW = 5, LASTROW = 675, ROWTERMINATOR=''0x0a'')'
print(@cmd1)
exec (@cmd1)
END
Here’s the string:
bulk insert [dbo].[SPBMI_Monthly] from '\\ms\london\FTP\' between '02/03/2014' and '03/02/2014'_SP.SPC' with (FIELDTERMINATOR = '\t', FIRSTROW = 5, LASTROW = 675, ROWTERMINATOR='0x0a')
The error message I keep getting is:
Msg 156, Level 15, State 1, Line 1
Incorrect syntax near the keyword 'between'.
Msg 319, Level 15, State 1, Line 1
Incorrect syntax near the keyword 'with'. If this statement is a common table expression, an xmlnamespaces clause or a change tracking context clause, the previous statement must be terminated with a semicolon.
I feel like I’m already pushing this thing to the limit.
Maybe this last part isn’t possible.
Knowledge is the only thing that I can give you, and still retain, and we are both better off for it.

Using sql bulk copy throwing exception -The given value of type String from the data source cannot be converted to type int of the specified target column

Hi All,
I am reading notepads files and inserting data in sql tables from the notepad-
while performing sql bulk copy on this line it throws exception - "bulkcopy.WriteToServer(dt); -"data type related(mentioned in subject )".
Please go through my logic and tell me what to change to avoid this error -
public void Main()
Dts.TaskResult = (int)ScriptResults.Success;
string[] filePaths = Directory.GetFiles(@"C:\Users\jainruc\Desktop\Sudhanshu\master_db\Archive\test\content_insert\");
for (int k = 0; k < filePaths.Length; k++)
string[] lines = System.IO.File.ReadAllLines(filePaths[k]);
//table name needs to extract after = sign
string[] pathArr = filePaths[0].Split('\\');
string tablename = pathArr[9].Split('.')[0];
DataTable dt = new DataTable(tablename);
|
string[] arrColumns = lines[1].Split(new char[] { '|' });
foreach (string col in arrColumns)
dt.Columns.Add(col);
for (int i = 2; i < lines.Length; i++)
string[] columnsvals = lines[i].Split(new char[] { '|' });
DataRow dr = dt.NewRow();
for (int j = 0; j < columnsvals.Length; j++)
//Console.Write(columnsvals[j]);
if (string.IsNullOrEmpty(columnsvals[j]))
dr[j] = DBNull.Value;
else
dr[j] = columnsvals[j];
dt.Rows.Add(dr);
SqlConnection conn = new SqlConnection();
conn.ConnectionString = "Data Source=UI3DATS009X;" + "Initial Catalog=BHI_CSP_DB;" + "User Id=sa;" + "Password=3pp$erv1ce$4";
conn.Open();
SqlBulkCopy bulkcopy = new SqlBulkCopy(conn);
bulkcopy.DestinationTableName = dt.TableName;
bulkcopy.WriteToServer(dt);
conn.Close();
Issue 1:-
I am reading notepad: getting all column and values in my data table now while inserting for date and time or integer field i need to do explicit conversion how to write for specific column before bulkcopy.WriteToServer(dt);
Issue 2:- Notepad does not contains all columns nor in specific sequence in that case i can add few column ehich i am doing now but the issue is now data table will add my columns + notepad columns and while inserting how to assign in perticular colums?
sudhanshu sharma Do good and cast it into river :)

Hi,
I think you'll have to do an explicit column mapping if they are not in exact sequence in both source and destination.
Have a look at this link:
https://msdn.microsoft.com/en-us/library/system.data.sqlclient.sqlbulkcopycolumnmapping(v=vs.110).aspx
Good Luck!
Kaur.
Please mark as answer if this resolves your issue.

Based on input file i need to update default values to some columns of database table using bulk copy process

Hi Team,
Am using BULK INSERT Format file option to load data into table from .txt file here am facing an issue i.e ibased on input file i need to insert default values to some columns of table so we can not declare it on table level, Can we give default values in
format file ? if we can give how it is ? or Any alternate possibilities to this scenario instead of BULK INSERT ?
Thanks,
Sudhakar

Thanks for your response, here i don't have any rights to change table structure the table is created by different team, my work is to load data from file to table. Is there any chance to supply default values by*XML* format file instead of *.fmt* file
please let me know the possibility.
Again, no. If you want to supply default values that are not present in DEFAULT constraints in the table definition, you will need to write your own code. There are plentyfull of options, and I have mentioned some already.
Here are some more:
* Table-valued parameters, see here for examples:
http://www.sommarskog.se/arrays-in-sql-2008.html
* Use BULK INSERT to load data to a staging table, and then apply the default values when you copy from staging to target.
Erland Sommarskog, SQL Server MVP, [email protected]

Bulk copy from a temp table

Similar Messages

Maybe you are looking for