Sqlldr multiple files/tables

Hello.
I have sqlldr loading data from 5 different external files to 5 different tables working just fine. I'm using 5 control files, since each file has a different structure and different target table. So, I run Sqlldr 5 times using the proper control file.
File1.dat ---> Table 1 using controlfile1.ctl
File2.dat ---> Table 2 using controlfile2.ctl
FileN.dat ---> Table N using controlfileN.ctl
I'd like to run sqlldr only one with only ONE control file which includes 5 loads.
Any ideas?
Thanks.
I'm trying to create a control file which includes several source files and load data into different tables.

This isn't really a spatial question, you might have better luck getting an answer in a different topic area.
I think the question you are going to get is... why do you have sqlldr files?
Can't the source just provide an export file, or even a sql script to run? That's the way most people move data.
Bryan

Similar Messages

SQL Loader to Load Multiple Tables from Multiple Files

Hi
I wish to create a control file to load multiple tables from multiple files
viz.Emp.dat into emp table and Dept.dat into Dept table and so on
How could I do it?
Can I create a control file like this:
OPTIONS(DIRECT=TRUE,
SKIP_UNUSABLE_INDEXES=TRUE,
SKIP_INDEX_MAINTENANCE=TRUE)
UNRECOVERABLE
LOAD DATA
INFILE 'EMP.dat'
INFILE 'DEPT.dat'
INTO TABLE emp TRUNCATE
FIELDS TERMINATED BY "|" OPTIONALLY ENCLOSED BY '"'
(empno,
ename,
deptno)
INTO TABLE dept TRUNCATE
FIELDS TERMINATED BY "|" OPTIONALLY ENCLOSED BY '"'
(deptno,
dname,
dloc)
Appreciate a Quick Reply
mailto:[email protected]

Which operating system? ("Command Prompt" sounds like Windows)
UNIX/Linux: a shell script with multiple calls to sqlldr run in the background with "&" (and possibly nohup)
Windows: A batch file using "start" to launch multiple copies of sqlldr.
http://www.pctools.com/forum/showthread.php?42285-background-a-process-in-batch-%28W2K%29
http://www.microsoft.com/resources/documentation/windows/xp/all/proddocs/en-us/start.mspx?mfr=true
Edited by: Brian Bontrager on May 31, 2013 4:04 PM

Collect data from a dynamic XML file into multiple internal tables

I need to convert the XML file into multiple internal tables. I tried many links and posts in SDN but still was facing difficulty in achieving this. Can some one tell me where I am going wrong.
My XML file is of the following type.It is very complex and the dynamice.
The following tags occur more than once in the XML file. The "I" and "L" tags and its child tags can occur ones or more than once for each XML file and it is not constant. i.e in one file they can occur 1 time and in another they can occur 100 times.
"I" and "L" are child tags of <C>

 <J>10</J>
 <K>EN</K>

<L>
 <J>20</J>
 <N>BB</N>
</L>
Tags <C> and <F> occur only ones for each XML file. <C> is the child tag of "A" and "F" is the child tag of <C>.
I need to collect <D>, <E> in one internal table ITAB.
I need to collect <G>, <H> in one internal table JTAB.
I need to collect <J>, <K> in one internal table KTAB.
I need to collect <J>, <N> in one internal table PTAB.
Below is the complete XML file.
?xml version="1.0" encoding="iso-8859-1" ?>
<A>
 
 <C>
 <D>RED</D>
 <E>999</E>
 <F>
 <G>TRACK</G>
 <H>PACK</H>
 </F>
 
 <J>10</J>
 <K>EN</K>
 
 
 <J>20</J>
 <K>TN</K>
 
 
 <J>30</J>
 <K>KN</K>
 
 <L>
 <J>10</J>
 <N>AA</N>
 </L>
 <L>
 <J>20</J>
 <N>BB</N>
 </L>
 <L>
 <J>30</J>
 <N>CC</N>
 </L>
 </C>
</A>
With the help of SDN I am able to gather the values of <D> <E> in one internal table.
Now if I need to gather
<G>, <H> in one internal table JTAB.
<J>, <K> in one internal table KTAB.
<J>, <N> in one internal table PTAB.
I am unable to do. I am following XSLT transformation method. If some one has some suggestions. Please help.
Here is my ABAP program
TYPE-POOLS abap.
CONSTANTS gs_file TYPE string VALUE 'C:\TEMP\ABCD.xml'.
* This is the structure for the data from the XML file
TYPES: BEGIN OF ITAB,
 D(10) TYPE C,
 E(10) TYPE C,
 END OF ITAB.
* Table for the XML content
DATA: gt_itab TYPE STANDARD TABLE OF char2048.
* Table and work ares for the data from the XML file
DATA: gt_ITAB TYPE STANDARD TABLE OF ts_ITAB,
 gs_ITAB TYPE ts_ITAB.
* Result table that contains references
* of the internal tables to be filled
DATA: gt_result_xml TYPE abap_trans_resbind_tab,
 gs_result_xml TYPE abap_trans_resbind.
* For error handling
DATA: gs_rif_ex TYPE REF TO cx_root,
 gs_var_text TYPE string.
* Get the XML file from your client
CALL METHOD cl_gui_frontend_services=>gui_upload
EXPORTING
 filename = gs_file
CHANGING
 data_tab = gt_itab1
EXCEPTIONS
 file_open_error = 1
 file_read_error = 2
 no_batch = 3
 gui_refuse_filetransfer = 4
 invalid_type = 5
 no_authority = 6
 unknown_error = 7
 bad_data_format = 8
 header_not_allowed = 9
 separator_not_allowed = 10
 header_too_long = 11
 unknown_dp_error = 12
 access_denied = 13
 dp_out_of_memory = 14
 disk_full = 15
 dp_timeout = 16
 not_supported_by_gui = 17
 error_no_gui = 18
 OTHERS = 19.
IF sy-subrc <> 0.
MESSAGE ID sy-msgid TYPE sy-msgty NUMBER sy-msgno
 WITH sy-msgv1 sy-msgv2 sy-msgv3 sy-msgv4.
ENDIF.
* Fill the result table with a reference to the data table.
* Within the XSLT stylesheet, the data table can be accessed with
* "IITAB".
GET REFERENCE OF gt_shipment INTO gs_result_xml-value.
gs_result_xml-name = 'IITAB'.
APPEND gs_result_xml TO gt_result_xml.
* Perform the XSLT stylesheet
TRY.
 CALL TRANSFORMATION zxslt
 SOURCE XML gt_itab1
 RESULT (gt_result_xml).
CATCH cx_root INTO gs_rif_ex.
 gs_var_text = gs_rif_ex->get_text( ).
 MESSAGE gs_var_text TYPE 'E'.
ENDTRY.
* Now let's see what we got from the file
LOOP AT gt_ITAB INTO gs_ITAB.
WRITE: / 'D:', gs_ITAB-D.
WRITE: / 'E :', gs_ITAB-E.
ENDLOOP.
Transformation
<xsl:transform xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">
<xsl:output encoding="iso-8859-1" indent="yes" method="xml" version="1.0"/>
<xsl:strip-space elements="*"/>
<xsl:template match="/">
 <asx:abap xmlns:asx="http://www.sap.com/abapxml" version="1.0">
 <asx:values>
 <IITAB>
 <xsl:apply-templates select="//C"/>
 </IITAB>
 </asx:values>
 </asx:abap>
</xsl:template>
<item>
 <D>
 <xsl:value-of select="D"/>
 </D>
 <E>
 <xsl:value-of select="E"/>
 </E>
 </item>
</xsl:template>
</xsl:transform>
Now the above pgm and transformation work well and I am able to extract data into the ITAB. Now what changes should I make in transformation and in pgm to collect
<G>, <H> in one internal table JTAB.
<J>, <K> in one internal table KTAB.
<J>, <N> in one internal table PTAB.
Please help..i am really tring hard to figure this out. I am found lot of threads addressing this issue but not my problem.
Kindly help.
Regards,
VS

Hi Rammohan,
Thanks for the effort!
But I don't need to use GUI upload because my functionality does not require to fetch data from presentation server.
Moreover, the split command advised by you contains separate fields...f1, f2, f3... and I cannot use it because I have 164 fields. I will have to split into 164 fields and assign the values back to 164 fields in the work area/header line.
Moreover I have about 10 such work areas. so the effort would be ten times the above effort! I want to avoid this! Please help!
I would be very grateful if you could provide an alternative solution.
Thanks once again,
Best Regards,
Vinod.V

Multiple flat file in and multiple target tables

Hi,
How can we have multiple flat file into multiple targets.
I am trying to load data from multiple flat files into respective tables. But it gives error like
VLD-2411: Cannot handle two file structures
Make sure that only one file structure is used in a SQL*Loader map
Can anyone help.
Regards
Rakesh Kumar

I donot thing in one mapping you can take multiple sqlloader file.
If want to load data form multiple file use External table.

Sql*loader map multiple files to multiple tables

Can a single control file map multiple files to multiple different tables? If so, what does the syntax look like? I've tried variations of the following, but haven't hit the jackpot yet.
Also, I understand that a direct load will automatically turn off most constraint checking. I'd like to turn this back on when I'm done loading all tables. How/when do I do that? I can find multiple references to 'REENABLE DISABLED CONSTRAINTS', but I don't know where to say that.
TIA.
LOAD DATA
INFILE 'first.csv'
TRUNCATE
INTO TABLE first_table
FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"'
(a,b,c)
INFILE 'second.csv'
TRUNCATE
INTO TABLE second_table
FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"'
(x,y,z,xx,yy,zz)
etc.

Here you go what you want..
http://www.psoug.org/reference/sqlloader.html
LOAD DATA
INFILE 'c:\temp\demo09a.dat'
INFILE 'c:\temp\demo09b.dat'
APPEND
INTO TABLE denver_prj
WHEN projno = '101' (
projno position(1:3) CHAR,
empno position(4:8) INTEGER EXTERNAL,
projhrs position(9:10) INTEGER EXTERNAL)
INTO TABLE orlando_prj
WHEN projno = '202' (
projno position(1:3) CHAR,
empno position(4:8) INTEGER EXTERNAL,
projhrs position(9:10) INTEGER EXTERNAL)
INTO TABLE misc_prj
WHEN projno != '101' AND projno != '202' (
projno position(1:3) CHAR,
empno position(4:8) INTEGER EXTERNAL,
projhrs position(9:10) INTEGER EXTERNAL)
Thanks
Aravindh

Sliding Window Table Partitioning Problems with RANGE RIGHT, SPLIT, MERGE using Multiple File Groups

There is misleading information in two system views (sys.data_spaces & sys.destination_data_spaces) about the physical location of data after a partitioning MERGE and before an INDEX REBUILD operation on a partitioned table. In SQL Server 2012 SP1 CU6,
the script below (SQLCMD mode, set DataDrive & LogDrive variables for the runtime environment) will create a test database with file groups and files to support a partitioned table. The partition function and scheme spread the test data across
4 files groups, an empty partition, file group and file are maintained at the start and end of the range. A problem occurs after the SWITCH and MERGE RANGE operations, the views sys.data_spaces & sys.destination_data_spaces show the logical, not the physical,
location of data.
--=================================================================================
-- PartitionLabSetup_RangeRight.sql
-- 001. Create test database
-- 002. Add file groups and files
-- 003. Create partition function and schema
-- 004. Create and populate a test table
--=================================================================================
USE [master]
GO
-- 001 - Create Test Database
:SETVAR DataDrive "D:\SQL\Data\"
:SETVAR LogDrive "D:\SQL\Logs\"
:SETVAR DatabaseName "workspace"
:SETVAR TableName "TestTable"
-- Drop if exists and create Database
IF DATABASEPROPERTYEX(N'$(databasename)','Status') IS NOT NULL
BEGIN
ALTER DATABASE $(DatabaseName) SET SINGLE_USER WITH ROLLBACK IMMEDIATE
DROP DATABASE $(DatabaseName)
END
CREATE DATABASE $(DatabaseName)
ON
( NAME = $(DatabaseName)_data,
FILENAME = N'$(DataDrive)$(DatabaseName)_data.mdf',
SIZE = 10,
MAXSIZE = 500,
FILEGROWTH = 5 )
LOG ON
( NAME = $(DatabaseName)_log,
FILENAME = N'$(LogDrive)$(DatabaseName).ldf',
SIZE = 5MB,
MAXSIZE = 5000MB,
FILEGROWTH = 5MB ) ;
GO
-- 002. Add file groups and files
--:SETVAR DatabaseName "workspace"
--:SETVAR TableName "TestTable"
--:SETVAR DataDrive "D:\SQL\Data\"
--:SETVAR LogDrive "D:\SQL\Logs\"
DECLARE @nSQL NVARCHAR(2000) ;
DECLARE @x INT = 1;
WHILE @x <= 6
BEGIN
SELECT @nSQL =
'ALTER DATABASE $(DatabaseName)
ADD FILEGROUP $(TableName)_fg' + RTRIM(CAST(@x AS CHAR(5))) + ';
ALTER DATABASE $(DatabaseName)
ADD FILE
NAME= ''$(TableName)_f' + CAST(@x AS CHAR(5)) + ''',
FILENAME = ''$(DataDrive)\$(TableName)_f' + RTRIM(CAST(@x AS CHAR(5))) + '.ndf''
TO FILEGROUP $(TableName)_fg' + RTRIM(CAST(@x AS CHAR(5))) + ';'
EXEC sp_executeSQL @nSQL;
SET @x = @x + 1;
END
-- 003. Create partition function and schema
--:SETVAR TableName "TestTable"
--:SETVAR DatabaseName "workspace"
USE $(DatabaseName);
CREATE PARTITION FUNCTION $(TableName)_func (int)
AS RANGE RIGHT FOR VALUES
0,
15,
30,
45,
60
CREATE PARTITION SCHEME $(TableName)_scheme
AS
PARTITION $(TableName)_func
TO
$(TableName)_fg1,
$(TableName)_fg2,
$(TableName)_fg3,
$(TableName)_fg4,
$(TableName)_fg5,
$(TableName)_fg6
-- Create TestTable
--:SETVAR TableName "TestTable"
--:SETVAR BackupDrive "D:\SQL\Backups\"
--:SETVAR DatabaseName "workspace"
CREATE TABLE [dbo].$(TableName)(
[Partition_PK] [int] NOT NULL,
[GUID_PK] [uniqueidentifier] NOT NULL,
[CreateDate] [datetime] NULL,
[CreateServer] [nvarchar](50) NULL,
[RandomNbr] [int] NULL,
CONSTRAINT [PK_$(TableName)] PRIMARY KEY CLUSTERED
[Partition_PK] ASC,
[GUID_PK] ASC
) ON $(TableName)_scheme(Partition_PK)
) ON $(TableName)_scheme(Partition_PK)
ALTER TABLE [dbo].$(TableName) ADD CONSTRAINT [DF_$(TableName)_GUID_PK] DEFAULT (newid()) FOR [GUID_PK]
ALTER TABLE [dbo].$(TableName) ADD CONSTRAINT [DF_$(TableName)_CreateDate] DEFAULT (getdate()) FOR [CreateDate]
ALTER TABLE [dbo].$(TableName) ADD CONSTRAINT [DF_$(TableName)_CreateServer] DEFAULT (@@servername) FOR [CreateServer]
-- 004. Create and populate a test table
-- Load TestTable Data - Seconds 0-59 are used as the Partitoning Key
--:SETVAR TableName "TestTable"
SET NOCOUNT ON;
DECLARE @Now DATETIME = GETDATE()
WHILE @Now > DATEADD(minute,-1,GETDATE())
BEGIN
INSERT INTO [dbo].$(TableName)
([Partition_PK]
,[RandomNbr])
VALUES
DATEPART(second,GETDATE())
,ROUND((RAND() * 100),0)
END
-- Confirm table partitioning - http://lextonr.wordpress.com/tag/sys-destination_data_spaces/
SELECT
N'DatabaseName' = DB_NAME()
, N'SchemaName' = s.name
, N'TableName' = o.name
, N'IndexName' = i.name
, N'IndexType' = i.type_desc
, N'PartitionScheme' = ps.name
, N'DataSpaceName' = ds.name
, N'DataSpaceType' = ds.type_desc
, N'PartitionFunction' = pf.name
, N'PartitionNumber' = dds.destination_id
, N'BoundaryValue' = prv.value
, N'RightBoundary' = pf.boundary_value_on_right
, N'PartitionFileGroup' = ds2.name
, N'RowsOfData' = p.[rows]
FROM
sys.objects AS o
INNER JOIN sys.schemas AS s
ON o.[schema_id] = s.[schema_id]
INNER JOIN sys.partitions AS p
ON o.[object_id] = p.[object_id]
INNER JOIN sys.indexes AS i
ON p.[object_id] = i.[object_id]
AND p.index_id = i.index_id
INNER JOIN sys.data_spaces AS ds
ON i.data_space_id = ds.data_space_id
INNER JOIN sys.partition_schemes AS ps
ON ds.data_space_id = ps.data_space_id
INNER JOIN sys.partition_functions AS pf
ON ps.function_id = pf.function_id
LEFT OUTER JOIN sys.partition_range_values AS prv
ON pf.function_id = prv.function_id
AND p.partition_number = prv.boundary_id
LEFT OUTER JOIN sys.destination_data_spaces AS dds
ON ps.data_space_id = dds.partition_scheme_id
AND p.partition_number = dds.destination_id
LEFT OUTER JOIN sys.data_spaces AS ds2
ON dds.data_space_id = ds2.data_space_id
ORDER BY
DatabaseName
,SchemaName
,TableName
,IndexName
,PartitionNumber
--=================================================================================
-- SECTION 2 - SWITCH OUT
-- 001 - Create TestTableOut
-- 002 - Switch out partition in range 0-14
-- 003 - Merge range 0 -29
-- 001. TestTableOut
:SETVAR TableName "TestTable"
IF OBJECT_ID('dbo.$(TableName)Out') IS NOT NULL
DROP TABLE [dbo].[$(TableName)Out]
CREATE TABLE [dbo].[$(TableName)Out](
[Partition_PK] [int] NOT NULL,
[GUID_PK] [uniqueidentifier] NOT NULL,
[CreateDate] [datetime] NULL,
[CreateServer] [nvarchar](50) NULL,
[RandomNbr] [int] NULL,
CONSTRAINT [PK_$(TableName)Out] PRIMARY KEY CLUSTERED
[Partition_PK] ASC,
[GUID_PK] ASC
) ON $(TableName)_fg2;
GO
-- 002 - Switch out partition in range 0-14
--:SETVAR TableName "TestTable"
ALTER TABLE dbo.$(TableName)
SWITCH PARTITION 2 TO dbo.$(TableName)Out;
-- 003 - Merge range 0 - 29
--:SETVAR TableName "TestTable"
ALTER PARTITION FUNCTION $(TableName)_func()
MERGE RANGE (15);
-- Confirm table partitioning
-- Original source of this query - http://lextonr.wordpress.com/tag/sys-destination_data_spaces/
SELECT
N'DatabaseName' = DB_NAME()
, N'SchemaName' = s.name
, N'TableName' = o.name
, N'IndexName' = i.name
, N'IndexType' = i.type_desc
, N'PartitionScheme' = ps.name
, N'DataSpaceName' = ds.name
, N'DataSpaceType' = ds.type_desc
, N'PartitionFunction' = pf.name
, N'PartitionNumber' = dds.destination_id
, N'BoundaryValue' = prv.value
, N'RightBoundary' = pf.boundary_value_on_right
, N'PartitionFileGroup' = ds2.name
, N'RowsOfData' = p.[rows]
FROM
sys.objects AS o
INNER JOIN sys.schemas AS s
ON o.[schema_id] = s.[schema_id]
INNER JOIN sys.partitions AS p
ON o.[object_id] = p.[object_id]
INNER JOIN sys.indexes AS i
ON p.[object_id] = i.[object_id]
AND p.index_id = i.index_id
INNER JOIN sys.data_spaces AS ds
ON i.data_space_id = ds.data_space_id
INNER JOIN sys.partition_schemes AS ps
ON ds.data_space_id = ps.data_space_id
INNER JOIN sys.partition_functions AS pf
ON ps.function_id = pf.function_id
LEFT OUTER JOIN sys.partition_range_values AS prv
ON pf.function_id = prv.function_id
AND p.partition_number = prv.boundary_id
LEFT OUTER JOIN sys.destination_data_spaces AS dds
ON ps.data_space_id = dds.partition_scheme_id
AND p.partition_number = dds.destination_id
LEFT OUTER JOIN sys.data_spaces AS ds2
ON dds.data_space_id = ds2.data_space_id
ORDER BY
DatabaseName
,SchemaName
,TableName
,IndexName
,PartitionNumber
The table below shows the results of the ‘Confirm Table Partitioning’ query, before and after the MERGE.
The T-SQL code below illustrates the problem.
-- PartitionLab_RangeRight
USE workspace;
DROP TABLE dbo.TestTableOut;
USE master;
ALTER DATABASE workspace
REMOVE FILE TestTable_f3 ;
-- ERROR
--Msg 5042, Level 16, State 1, Line 1
--The file 'TestTable_f3 ' cannot be removed because it is not empty.
ALTER DATABASE workspace
REMOVE FILE TestTable_f2 ;
-- Works surprisingly!!
use workspace;
ALTER INDEX [PK_TestTable] ON [dbo].[TestTable] REBUILD PARTITION = 2;
--Msg 622, Level 16, State 3, Line 2
--The filegroup "TestTable_fg2" has no files assigned to it. Tables, indexes, text columns, ntext columns, and image columns cannot be populated on this filegroup until a file is added.
--The statement has been terminated.
If you run ALTER INDEX REBUILD before trying to remove files from File Group 3, it works. Rerun the database setup script then the code below.
-- RANGE RIGHT
-- Rerun PartitionLabSetup_RangeRight.sql before the code below
USE workspace;
DROP TABLE dbo.TestTableOut;
ALTER INDEX [PK_TestTable] ON [dbo].[TestTable] REBUILD PARTITION = 2;
USE master;
ALTER DATABASE workspace
REMOVE FILE TestTable_f3;
-- Works as expected!!
The file in File Group 2 appears to contain data but it can be dropped. Although the system views are reporting the data in File Group 2, it still physically resides in File Group 3 and isn’t moved until the index is rebuilt. The RANGE RIGHT function means
the left file group (File Group 2) is retained when splitting ranges.
RANGE LEFT would have retained the data in File Group 3 where it already resided, no INDEX REBUILD is necessary to effectively complete the MERGE operation. The script below implements the same partitioning strategy (data distribution between partitions)
on the test table but uses different boundary definitions and RANGE LEFT.
--=================================================================================
-- PartitionLabSetup_RangeLeft.sql
-- 001. Create test database
-- 002. Add file groups and files
-- 003. Create partition function and schema
-- 004. Create and populate a test table
--=================================================================================
USE [master]
GO
-- 001 - Create Test Database
:SETVAR DataDrive "D:\SQL\Data\"
:SETVAR LogDrive "D:\SQL\Logs\"
:SETVAR DatabaseName "workspace"
:SETVAR TableName "TestTable"
-- Drop if exists and create Database
IF DATABASEPROPERTYEX(N'$(databasename)','Status') IS NOT NULL
BEGIN
ALTER DATABASE $(DatabaseName) SET SINGLE_USER WITH ROLLBACK IMMEDIATE
DROP DATABASE $(DatabaseName)
END
CREATE DATABASE $(DatabaseName)
ON
( NAME = $(DatabaseName)_data,
FILENAME = N'$(DataDrive)$(DatabaseName)_data.mdf',
SIZE = 10,
MAXSIZE = 500,
FILEGROWTH = 5 )
LOG ON
( NAME = $(DatabaseName)_log,
FILENAME = N'$(LogDrive)$(DatabaseName).ldf',
SIZE = 5MB,
MAXSIZE = 5000MB,
FILEGROWTH = 5MB ) ;
GO
-- 002. Add file groups and files
--:SETVAR DatabaseName "workspace"
--:SETVAR TableName "TestTable"
--:SETVAR DataDrive "D:\SQL\Data\"
--:SETVAR LogDrive "D:\SQL\Logs\"
DECLARE @nSQL NVARCHAR(2000) ;
DECLARE @x INT = 1;
WHILE @x <= 6
BEGIN
SELECT @nSQL =
'ALTER DATABASE $(DatabaseName)
ADD FILEGROUP $(TableName)_fg' + RTRIM(CAST(@x AS CHAR(5))) + ';
ALTER DATABASE $(DatabaseName)
ADD FILE
NAME= ''$(TableName)_f' + CAST(@x AS CHAR(5)) + ''',
FILENAME = ''$(DataDrive)\$(TableName)_f' + RTRIM(CAST(@x AS CHAR(5))) + '.ndf''
TO FILEGROUP $(TableName)_fg' + RTRIM(CAST(@x AS CHAR(5))) + ';'
EXEC sp_executeSQL @nSQL;
SET @x = @x + 1;
END
-- 003. Create partition function and schema
--:SETVAR TableName "TestTable"
--:SETVAR DatabaseName "workspace"
USE $(DatabaseName);
CREATE PARTITION FUNCTION $(TableName)_func (int)
AS RANGE LEFT FOR VALUES
-1,
14,
29,
44,
59
CREATE PARTITION SCHEME $(TableName)_scheme
AS
PARTITION $(TableName)_func
TO
$(TableName)_fg1,
$(TableName)_fg2,
$(TableName)_fg3,
$(TableName)_fg4,
$(TableName)_fg5,
$(TableName)_fg6
-- Create TestTable
--:SETVAR TableName "TestTable"
--:SETVAR BackupDrive "D:\SQL\Backups\"
--:SETVAR DatabaseName "workspace"
CREATE TABLE [dbo].$(TableName)(
[Partition_PK] [int] NOT NULL,
[GUID_PK] [uniqueidentifier] NOT NULL,
[CreateDate] [datetime] NULL,
[CreateServer] [nvarchar](50) NULL,
[RandomNbr] [int] NULL,
CONSTRAINT [PK_$(TableName)] PRIMARY KEY CLUSTERED
[Partition_PK] ASC,
[GUID_PK] ASC
) ON $(TableName)_scheme(Partition_PK)
) ON $(TableName)_scheme(Partition_PK)
ALTER TABLE [dbo].$(TableName) ADD CONSTRAINT [DF_$(TableName)_GUID_PK] DEFAULT (newid()) FOR [GUID_PK]
ALTER TABLE [dbo].$(TableName) ADD CONSTRAINT [DF_$(TableName)_CreateDate] DEFAULT (getdate()) FOR [CreateDate]
ALTER TABLE [dbo].$(TableName) ADD CONSTRAINT [DF_$(TableName)_CreateServer] DEFAULT (@@servername) FOR [CreateServer]
-- 004. Create and populate a test table
-- Load TestTable Data - Seconds 0-59 are used as the Partitoning Key
--:SETVAR TableName "TestTable"
SET NOCOUNT ON;
DECLARE @Now DATETIME = GETDATE()
WHILE @Now > DATEADD(minute,-1,GETDATE())
BEGIN
INSERT INTO [dbo].$(TableName)
([Partition_PK]
,[RandomNbr])
VALUES
DATEPART(second,GETDATE())
,ROUND((RAND() * 100),0)
END
-- Confirm table partitioning - http://lextonr.wordpress.com/tag/sys-destination_data_spaces/
SELECT
N'DatabaseName' = DB_NAME()
, N'SchemaName' = s.name
, N'TableName' = o.name
, N'IndexName' = i.name
, N'IndexType' = i.type_desc
, N'PartitionScheme' = ps.name
, N'DataSpaceName' = ds.name
, N'DataSpaceType' = ds.type_desc
, N'PartitionFunction' = pf.name
, N'PartitionNumber' = dds.destination_id
, N'BoundaryValue' = prv.value
, N'RightBoundary' = pf.boundary_value_on_right
, N'PartitionFileGroup' = ds2.name
, N'RowsOfData' = p.[rows]
FROM
sys.objects AS o
INNER JOIN sys.schemas AS s
ON o.[schema_id] = s.[schema_id]
INNER JOIN sys.partitions AS p
ON o.[object_id] = p.[object_id]
INNER JOIN sys.indexes AS i
ON p.[object_id] = i.[object_id]
AND p.index_id = i.index_id
INNER JOIN sys.data_spaces AS ds
ON i.data_space_id = ds.data_space_id
INNER JOIN sys.partition_schemes AS ps
ON ds.data_space_id = ps.data_space_id
INNER JOIN sys.partition_functions AS pf
ON ps.function_id = pf.function_id
LEFT OUTER JOIN sys.partition_range_values AS prv
ON pf.function_id = prv.function_id
AND p.partition_number = prv.boundary_id
LEFT OUTER JOIN sys.destination_data_spaces AS dds
ON ps.data_space_id = dds.partition_scheme_id
AND p.partition_number = dds.destination_id
LEFT OUTER JOIN sys.data_spaces AS ds2
ON dds.data_space_id = ds2.data_space_id
ORDER BY
DatabaseName
,SchemaName
,TableName
,IndexName
,PartitionNumber
--=================================================================================
-- SECTION 2 - SWITCH OUT
-- 001 - Create TestTableOut
-- 002 - Switch out partition in range 0-14
-- 003 - Merge range 0 -29
-- 001. TestTableOut
:SETVAR TableName "TestTable"
IF OBJECT_ID('dbo.$(TableName)Out') IS NOT NULL
DROP TABLE [dbo].[$(TableName)Out]
CREATE TABLE [dbo].[$(TableName)Out](
[Partition_PK] [int] NOT NULL,
[GUID_PK] [uniqueidentifier] NOT NULL,
[CreateDate] [datetime] NULL,
[CreateServer] [nvarchar](50) NULL,
[RandomNbr] [int] NULL,
CONSTRAINT [PK_$(TableName)Out] PRIMARY KEY CLUSTERED
[Partition_PK] ASC,
[GUID_PK] ASC
) ON $(TableName)_fg2;
GO
-- 002 - Switch out partition in range 0-14
--:SETVAR TableName "TestTable"
ALTER TABLE dbo.$(TableName)
SWITCH PARTITION 2 TO dbo.$(TableName)Out;
-- 003 - Merge range 0 - 29
:SETVAR TableName "TestTable"
ALTER PARTITION FUNCTION $(TableName)_func()
MERGE RANGE (14);
-- Confirm table partitioning
-- Original source of this query - http://lextonr.wordpress.com/tag/sys-destination_data_spaces/
SELECT
N'DatabaseName' = DB_NAME()
, N'SchemaName' = s.name
, N'TableName' = o.name
, N'IndexName' = i.name
, N'IndexType' = i.type_desc
, N'PartitionScheme' = ps.name
, N'DataSpaceName' = ds.name
, N'DataSpaceType' = ds.type_desc
, N'PartitionFunction' = pf.name
, N'PartitionNumber' = dds.destination_id
, N'BoundaryValue' = prv.value
, N'RightBoundary' = pf.boundary_value_on_right
, N'PartitionFileGroup' = ds2.name
, N'RowsOfData' = p.[rows]
FROM
sys.objects AS o
INNER JOIN sys.schemas AS s
ON o.[schema_id] = s.[schema_id]
INNER JOIN sys.partitions AS p
ON o.[object_id] = p.[object_id]
INNER JOIN sys.indexes AS i
ON p.[object_id] = i.[object_id]
AND p.index_id = i.index_id
INNER JOIN sys.data_spaces AS ds
ON i.data_space_id = ds.data_space_id
INNER JOIN sys.partition_schemes AS ps
ON ds.data_space_id = ps.data_space_id
INNER JOIN sys.partition_functions AS pf
ON ps.function_id = pf.function_id
LEFT OUTER JOIN sys.partition_range_values AS prv
ON pf.function_id = prv.function_id
AND p.partition_number = prv.boundary_id
LEFT OUTER JOIN sys.destination_data_spaces AS dds
ON ps.data_space_id = dds.partition_scheme_id
AND p.partition_number = dds.destination_id
LEFT OUTER JOIN sys.data_spaces AS ds2
ON dds.data_space_id = ds2.data_space_id
ORDER BY
DatabaseName
,SchemaName
,TableName
,IndexName
,PartitionNumber
The table below shows the results of the ‘Confirm Table Partitioning’ query, before and after the MERGE.
The data in the File and File Group to be dropped (File Group 2) has already been switched out; File Group 3 contains the data so no index rebuild is needed to move data and complete the MERGE.
RANGE RIGHT would not be a problem in a ‘Sliding Window’ if the same file group is used for all partitions, when they are created and dropped it introduces a dependency on full index rebuilds. Larger tables are typically partitioned and a full index rebuild
might be an expensive operation. I’m not sure how a RANGE RIGHT partitioning strategy could be implemented, with an ascending partitioning key, using multiple file groups without having to move data. Using a single file group (multiple files) for all partitions
within a table would avoid physically moving data between file groups; no index rebuild would be necessary to complete a MERGE and system views would accurately reflect the physical location of data.
If a RANGE RIGHT partition function is used, the data is physically in the wrong file group after the MERGE assuming a typical ascending partitioning key, and the 'Data Spaces' system views might be misleading. Thanks to Manuj and Chris for a lot of help
investigating this.
NOTE 10/03/2014 - The solution
The solution is so easy it's embarrassing, I was using the wrong boundary points for the MERGE (both RANGE LEFT & RANGE RIGHT) to get rid of historic data.
-- Wrong Boundary Point Range Right
--ALTER PARTITION FUNCTION $(TableName)_func()
--MERGE RANGE (15);
-- Wrong Boundary Point Range Left
--ALTER PARTITION FUNCTION $(TableName)_func()
--MERGE RANGE (14);
-- Correct Boundary Pounts for MERGE
ALTER PARTITION FUNCTION $(TableName)_func()
MERGE RANGE (0); -- or -1 for RANGE LEFT
The empty, switched out partition (on File Group 2) is then MERGED with the empty partition maintained at the start of the range and no data movement is necessary. I retract the suggestion that a problem exists with RANGE RIGHT Sliding Windows using multiple
file groups and apologize :-)

Hi Paul Brewer,
Thanks for your post and glad to hear that the issue is resolved. It is kind of you post a reply to share your solution. That way, other community members could benefit from your sharing.
Regards.
Sofiya Li
Sofiya Li
TechNet Community Support

Need to Load data in XML file into multiple Oracle tables

Experts
I want to load data that is stored at XML file into multiple Oracle table using plsql.
I am novice in this area.Kindly explain in depth or step by step so that i can achive this.
Thanks in advnace.

Hi,
extract your xml and then you can use insert all clause.
here's very small example on 10.2.0.1.0
SQL> create table table1(id number,val varchar2(10));
Table created.
SQL> create table table2(id number,val varchar2(10));
Table created.
SQL> insert all
2 into table1 values(id,val)
3 into table2 values(id2,val2)
4 select extractValue(x.col,'/a/id1') id
5 ,extractValue(x.col,'/a/value') val
6 ,extractValue(x.col,'/a/value2') val2
7 ,extractValue(x.col,'/a/id2') id2
8 from (select xmltype('<a><id1>1</id1><value>a</value><id2>2</id2><value2>b</value2></a>') col from dual) x;
2 rows created.
SQL> select * from table1;
 ID VAL
 1 a
SQL> select * from table2;
 ID VAL
 2 b Ants

Create multiple external tables from same flat file ?

using oracle 10g
currently create many external tables like so..
CREATE TABLE "XT_UNITS"
"Q1_2012" VARCHAR2(25 BYTE),
"Q2_2012" VARCHAR2(25 BYTE),
"Q3_2012" VARCHAR2(25 BYTE),
"Q4_2012" VARCHAR2(25 BYTE)
ORGANIZATION EXTERNAL
TYPE ORACLE_LOADER DEFAULT DIRECTORY "XE_FILES" ACCESS PARAMETERS ( records delimited BY newline
skip 1 fields terminated BY ',' OPTIONALLY ENCLOSED BY '"' MISSING FIELD VALUES
ARE NULL ) LOCATION ( 'xtunits.csv' ))
is there any way I can use 1 flat file (csv) to populate many external tables ?
maybe by section ?

Jay wrote:
using oracle 10g
currently create many external tables like so..
CREATE TABLE "XT_UNITS"
"Q1_2012" VARCHAR2(25 BYTE),
"Q2_2012" VARCHAR2(25 BYTE),
"Q3_2012" VARCHAR2(25 BYTE),
"Q4_2012" VARCHAR2(25 BYTE)
ORGANIZATION EXTERNAL
TYPE ORACLE_LOADER DEFAULT DIRECTORY "XE_FILES" ACCESS PARAMETERS ( records delimited BY newline
skip 1 fields terminated BY ',' OPTIONALLY ENCLOSED BY '"' MISSING FIELD VALUES
ARE NULL ) LOCATION ( 'xtunits.csv' ))
is there any way I can use 1 flat file (csv) to populate many external tables ?
maybe by section ?Not a logical question.
An external table definition relates to one or more files.
You could probably create multiple externable table definitions to access the same file(s), but you may end up with some operating system file locking issues if you're not careful, and to be honest why would you need to do that?

File loaded successfully by SQLldr but external table failed

Hi ,
When i tried to create an external table to load data from file it failed with the below error
"KUP-04018: partial record at end of file",
When i query on the external table for rows with rownum<2500 i can see the records being properly loaded but when i tried to fetch all records it throws the above error.
But the same file when i loaded with sqlldr it loaded all the records successfully.
Can you please let me know the reason for this?
Regards

Can you post us your SQLLDR control file, your external table definition, a sample of your data (preferably including the last lines of data which you believe are erroring) and also let us know your database version.

Loading data from multiple files to multiple tables

How should I approach on creating SSIS package to load data from multiple files to multiple tables. Also, Files will have data which might overlap so I might have to create stored procedure for it. Ex. 1st day file -data from au.1 - aug 10 and 2nd day
file might have data from aug.5 to aug 15. So I might have to look for max and min date and truncate table with in that date range.

thats ok. ForEachLoop would be able to iterate through the files. You can declare a variable inside loop to capture the filenames. Choose fully qualified as the option in loop
Then inside loop
1. Add execute sql task to delete overlapping data from the table. One question here is where will you get date from? Does it come inside filename?
2. Add a data flow task with file source pointing to file .For this add a suitable connection manager (Excel/Flat file etc) and map the connection string property to filename variable using expressions
3. Add a OLEDB Destination to point to table. You can use table or view from variable - fast load option and map to variable to make tablename dynamic and just set corresponding value for the variable to get correct tablename
Please Mark This As Answer if it helps to solve the issue Visakh ---------------------------- http://visakhm.blogspot.com/ https://www.facebook.com/VmBlogs

Can we create single External Table for multiple files?

HI,
Can we create External table for multiple files? Could anyone please explain it.
Thanks and regards
Gowtham Sen.

to merge 16 files having same structureWell, if files have the same structure, as per the reading of the example from the following documentation, you can create one external table for all your files :
http://download-uk.oracle.com/docs/cd/B19306_01/server.102/b14231/tables.htm#i1007480
Nicolas.

SSIS 2012 - Importing multiple files with different columns (where columns are stored in a table)

Hi everyone,
I have been tasked to upgrade an existing DTS package into an SSIS package.
The old package uses 3 tables.
Table 1) Has a list of file names
Table 2) Has a list of columns per file
Table 3) Is the data destination.
So what is supposed to happen is:
Step 1) the package should load the files into a resultset
Step 2) for each file in the resultset, the package should get the related columns
Step 3) the package should now map the columns to the destination table (variable column numbers here)
Step 4) the import should take place.
I have gotten to step 2 but am not sure how to dynamically map the columns to the destination table.
Any pointers anyone?
Thanks in advance!

Hi Tonyhektic,
What’s the nature of Table 2, does it only store the column names for each file? I don’t know how it is implemented in DTS package, however, in SSIS package, dynamic columns mapping is not supported. To work around this issue, we need to use Script component
to do the columns mapping. You can refer to Sorna’s script above or have a look at the following blogs:
http://munishbansal.wordpress.com/2009/06/09/dynamic-columns-mapping-%E2%80%93-script-component-as-destination-ssis/
http://blog.quasarinc.com/ssis/best-solution-to-load-dynamically-change-csv-file-in-ssis-etl-package/
Regards,
Mike Yin
TechNet Community Support

Importing data from Microsoft excel file to Oracle Database with Multiple Data Tables. Need expert advice and guidance

I posted a query on Importing data from Microsoft Excel to Oracle Database (Multiple Data Tables). I got some answer and reference from the forum.
I presented to my Oracle consultant and representative from Oracle Malaysia. They said impossible. I do not believe what they said. I do believe can be done.
Can someone help or direct me to an expert that can help me on this

e90f478a-c529-4c48-b189-51eebeaed477 wrote:
I posted a query on Importing data from Microsoft Excel to Oracle Database (Multiple Data Tables). I got some answer and reference from the forum.
I presented to my Oracle consultant and representative from Oracle Malaysia. They said impossible. I do not believe what they said. I do believe can be done.
Can someone help or direct me to an expert that can help me on this
We don't know the "query on Importing data from Microsoft Excel to Oracle Database (Multiple Data Tables). "
We don't know where you posted said query.
We don't know what "some answer and reference" you received "from the forum."
We don't know what it was that your "Oracle consultant and representative from Oracle Malaysia" said was "impossible".
So on what basis are we supposed to "help or direct" to "to an expert that can help "?

Loading multiple files with SQL Loader

Hello.
I will appreciate your recommendation about the way to load multiple files (to multiple tables) using SQL Loader with only one Control file.
file1 to load to Table1, file2 to load to Table2 etc.
How the Control file should look like?
I was looking on Web, but didn't find exactly what I need.
Thanks!

Ctl File : myctl.ctl
---------- Start ---------
LOAD DATA
INFILE 'F:\sqlldr\abc1.dat'
INFILE 'F:\sqlldr\abc2.dat'
INTO TABLE hdfc1
(TRANS_DATE CHAR,
NARRATION CHAR,
VALUE_DATE CHAR,
DEBIT_AMOUNT INTEGER,
CREDIT_AMOUNT INTEGER,
CHQ_REF_NUMBER CHAR,
CLOSING_BALANCE CHAR)
INTO TABLE hdfc2
(TRANS_DATE CHAR,
NARRATION CHAR,
VALUE_DATE CHAR,
DEBIT_AMOUNT INTEGER,
CREDIT_AMOUNT INTEGER,
CHQ_REF_NUMBER CHAR,
CLOSING_BALANCE CHAR)
-----------End-----------
Sqlldr Command
sqlldr scott/tiger@dbtalk control=F:\sqlldr\myctl.ctl log=F:\sqlldr\ddl_file1.txt
Regards,
Abu

Error in loading the multiple file loading

hi,
i am following this blog to load the multiple files into rdbms table.
http://www.odigurus.com/2011/05/multiple-files-single-target-table.html
the problem is that when i assign the source Flat file Datastore
Model Resource Name #Project_Name.FILE_NAME
But when i point the Single File in the Resource Name:EMP.DMP. its working fine. loaded suceessfully.... But problem in Multiple loading???
it gives me error
SQL*Loader-500: Unable to open file (E:/File/#Project_Name.FILE_NAME)
SQL*Loader-553: file not found
SQL*Loader-509: System error: The system cannot find the file specified.
SQL*Loader-2026: the load was aborted because SQL Loader cannot continue.
********************Task Error Code In Loading**********************************************
import java.lang.String
import java.lang.Runtime as Runtime
from jarray import array
import java.io.File
import os
import re
ctlfile = r"""E:/File/Med_CDR_File.ctl"""
logfile = r"""E:/File/Med_CDR_File.log"""
outfile = r"""E:/File/Med_CDR_File.out"""
oracle_sid=''
if len('')>0: oracle_sid = '@'+''
loadcmd = r"""sqlldr 'test/<@=snpRef.getInfo("DEST_PASS") @>%s' control='%s' log='%s' > "%s" """ % (oracle_sid,ctlfile, logfile, outfile)
rc = os.system(loadcmd)
if rc <> 0 and rc <> 2:
 raise "Load Error", "See %s for details" % logfile
# Init Vars
nbIns = 0
nbRej = 0
nbNull = 0
strprt = ""
maxAllowedError = r"""0"""
c = 0
flag = 0
# Open log file
f = open(logfile, "r")
try:
 lines = f.readlines()
 for line in lines:
 if line.rstrip().upper().endswith(r"""TEST.TC$_0MSC_SMS:""".upper()):
 flag = 1
 c = 0
 if flag == 1:
 if c > 0 and c <= 4:
 if c == 1 :
 nbIns = int(re.findall("\d+", line)[0])
 elif c == 2:
 nbRej = int(re.findall("\d+", line)[0])
 elif c == 4:
 nbNull = int(re.findall("\d+", line)[0])
 break
 c+=1
 strprt = "\n\tIns:\t%s\n\tReject:\t%s\n\tNullField:\t%s" % (nbIns, nbRej, nbNull)
finally:
 f.close()
# if some rows has been rejected due to invalide data, check KM option LOA_ERRORS
if rc == 2:
 if nbRej > int(maxAllowedError):
 raise strprt
 break
waiting
Regards,
Edited by: AMSI on Nov 10, 2012 1:40 AM

Hi AMSI,
Did you try to run only the interface? It might not work.
Try executing your interface from within a package. A previous step should refresh / set the variable.
Regards,
Jerome Fr

Sqlldr multiple files/tables

Similar Messages

Maybe you are looking for