Data misplaced during Flat file data load
Hi all,
i have to load the data from the flat file(txt format) but i found some record are not populating in PSA.
the voluem of the data is very huge. how to resolve this and data load seems to be green in info package monitoring.
Hi,
How did you conclude that the records are missing the PSA? Is it because the count is less?
There are the chances that the records are duplicated in the flat file.
Perhaps, you can increase the number of records per datapackage in the settings and can check the count.
If you feel, a particular record is not getting loaded, the load that record selectively through filter.
Split the load according to your convenience and load so that it will be easy to keep track of the records.
Thanks.
Similar Messages
-
Hi,
I have to implement the following scenario in SSIS but don't know how to do since I never worked with SSIS before. Please help me.
I have 20 different text files in a single folder and 20 different tables corresponding to each text file in SQL Server 2008 R2 Database. I need to extract the data from each text file and
load the data into corresponding table in Sql Server Database. Please guide me in how many ways I can do this and which is the best way to implement this job. Actually I have to automate this job. Few files are in same format(with same column names
and datatypes) where others are not.
1. Do I need to create 20 different projects ?
or
Can I implement this in only one project by having 20 packages?
or
Can I do this in one project with only one package?
Thanks in advance.As I said I don't know how to use object data type, I just given a shot as below. I know the following code has errors can you please correct it for me.
Public
Sub Main()
' Add your code here
Dim f1
As FileStream
Dim s1
As StreamReader
Dim date1
As
Object
Dim rline
As
String
Dim Filelist(1)
As
String
Dim FileName
As
String
Dim i
As
Integer
i = 1
date1 =
Filelist(0) =
"XYZ"
Filelist(1) =
"123"
For
Each FileName
In Filelist
f1 = File.OpenRead(FileName)
s1 = File.OpenText(FileName)
rline = s1.ReadLine
While
Not rline
Is
Nothing
If Left(rline, 4) =
"DATE"
Then
date1 (i)= Mid(rline, 7, 8)
i = i + 1
Exit
While
End
If
rline = s1.ReadLine
End
While
Next
Dts.Variables(
"date").Value = date1(1)
Dts.Variables(
"date1").Value = date1(2)
Dts.TaskResult = ScriptResults.Success
End
Sub -
Getting error RSTSODSPART during flat file data load to datasource
Hi SAP Gurus,
In BI, I dont know why i am getting this error while loading data from flat file to data source using infopackage. Always i am deleting data from PSA before running it even have this error. Could u pls help me out in this issue.... Thanks....
Error when inserting in PSA table RSTSODSPART
Message no. RSAODS206
Diagnosis
An error occurred with the insert into PSA table RSTSODSPART for InfoSource .
System Response
Your data was not updated in the PSA table.
It is possible that data already exists for this request and data packet .Hello,
Do you have an MS SQL Database, if yes then the patch in the note 1340371 should resolve the problem. If you have a different DB release please let me know your DB release and your BW release and support package level.
Best Regards,
Des -
Flat file data load - ODS - Look up data in Startroutine
Hi All,
There is a requirement that I have two ODS say 1) ABC and XYZ.
For Both ODS , We load flat file data.
First we load data to ABC for current fiscal period
During the data load to XYZ, We lookup part nos data in ABC for current fiscal period , if data there , for those part nos we load XYZ Flat file data.
My requirement is that , I need to fetch part nos data from ABC for Fiscal period < current fisacl period. Then I need to extract already loaded data for those part nos from XYZ and Mark the flag field as Non reportable .
How can we achieve this ?
Please advice
Thanks
AjayHi ,
Thanks for your reply.
I have done so. When I add lines of My Internal table data to DATA_PACKAGE, It gives the syntax error that both structures are not unique. We have several fields in XYZ ods but they are not in Communication structure.
Thats the problem. Also , if I write the code , will it be executed for each datapackage. I mean , for each datapacket process , my code will fetch whole data from ABC and then from XYZ. Repetion will be there?
Any other logic can i use ?
Thanks
Ajay -
What is the best way to load and convert data from a flat file?
Hi,
I want to load data from a flat file, convert dates, numbers and some fields with custom logic (e.g. 0,1 into N,Y) to the correct format.
The rows where all to_number, to_date and custom conversions succeed should go into table STG_OK. If some conversion fails (due to an illegal format in the flat file), those rows (where the conversion raises some exception) should go into table STG_ERR.
What is the best and easiest way to archive this?
Thanks,
Carsten.Hi,
thanks for your answers so far!
I gave them a thought and came up with two different alternatives:
Alternative 1
I load the data from the flat file into a staging table using sqlldr. I convert the data to the target format using sqlldr expressions.
The columns of the staging table have the target format (date, number).
The rows that cannot be loaded go into a bad file. I manually load the data from the bad file (without any conversion) into the error table.
Alternative 2
The columns of the staging table are all of type varchar2 regardless of the target format.
I define data rules for all columns that require a later conversion.
I load the data from the flat file into the staging table using external table or sqlldr without any data conversion.
The rows that cannot be loaded go automatically into the error table.
When I read the data from the staging table, I can safely convert it since it is already checked by the rules.
What I dislike in alternative 1 is that I manually have to create a second file and a second mapping (ok, I can automate this using OMB*Plus).
Further, I would prefer using expressions in the mapping for converting the data.
What I dislike in alternative 2 is that I have to create a data rule and a conversion expression and then keep the data rule and the conversion expression in sync (in case of changes of the file format).
I also would prefer to have the data in the staging table in the target format. Well, I might load it into a second staging table with columns having the target format. But that's another mapping and a lot of i/o.
As far as I know I need the data quality option for using data rules, is that true?
Is there another alternative without any of these drawbacks?
Otherwise I think I will go for alternative 1.
Thanks,
Carsten. -
How to load data from a flat file which is there in the application server
HI All,
how to load data from a flat file which is there in the application server..Hi,
Firstly you will need to place the file(s) in the AL11 path. Then in your infopackage in "Extraction" tab you need to select "Application Server" option. Then you need to specify the path as well as the exact file you want to load by using the browsing button.
If your file name keeps changing on a daily basis i.e. name_ddmmyyyy.csv, then in the Extraction tab you have the option to write an ABAP routine that generates the file name. Here you will need to append sy-datum to "name" and then append ".csv" to generate complete filename.
Please let me know if this is helpful or if you need any more inputs.
Thanks & Regards,
Nishant Tatkar. -
Loading CLOB data to a flat file in owb 11.2.0.3
Hi,
Also,I have one more question.I want to load a table with clob data into a flat file using my owb client 11.2.0.3.But when I run this simple mapping ,I am getting this error:
ORA-06502:pl/sql:NUMERIC OR VALUE ERROR:CHARACTER STRING BUFFER TOO SMALL.
Please suggest what needs to be done.I was up all night as I had a dead line doing this....
Turned out my Mapping was corrupted.
If you do a series of synchronizations by object name, position, id the mapping would end up corrupted.
I had to delete my staging table operator within the mapping, drag and drop and reconnect and now my loads work just fine..
Thanks -
Error Loading Data into a flat file
I am recieving the following error when loading data into a flat file from a flat file. SQL 2005 is my back end DB. If I cut the file iin half approx 500K rows my ODI interface works fine. Not sure what to look at.. I rebuit the interface which before was just dying giving no error and now I am getting this.
Thanks
java.lang.Exception
at com.sunopsis.dwg.dbobj.SnpSessTaskSql.treatTask(SnpSessTaskSql.java)
at com.sunopsis.dwg.dbobj.SnpSessStep.treatSessStep(SnpSessStep.java)
at com.sunopsis.dwg.dbobj.SnpSession.treatSession(SnpSession.java)
at com.sunopsis.dwg.cmd.DwgCommandSession.treatCommand(DwgCommandSession.java)
at com.sunopsis.dwg.cmd.DwgCommandBase.execute(DwgCommandBase.java)
at com.sunopsis.dwg.cmd.e.i(e.java)
at com.sunopsis.dwg.cmd.g.y(g.java)
at com.sunopsis.dwg.cmd.e.run(e.java)
at java.lang.Thread.run(Unknown Source)Figured it out, found similar post that stated changing the HEAP size
Increase the page size in odiparams.bat in the bin folder and restart Designer.
For eg:
set ODI_INIT_HEAP=128m
set ODI_MAX_HEAP=1024m -
Loading Master Data from a Flat File
Hi Guru, I have flat files of different SAP tables (Transactional data tables and Master Data ), I want to upload this flat files to a DSO, my question is in the case of Master Data Files example SAP table USR03 ( User Address Data) in a Flat file format which is better to create a Master data text and attributes Datasource or create it as a transactional data Datasource to upload it to a DSO.
Hi
Master data datasource should be used to load master data like customer details. customer data is the master data which does not change frequently. customer infoobject may have some attributes like customer name, customer contact number, customer address, etc. we have to maintain these attributes and text data.
Transaction data is something which keeps changing very frequently like price of the material, sales, etc. for loading such data we go for transaction datasources.
As mentioned above too,the changes in the address are very rare, therefore youshould not use transactional datasource.
Regards, Rahul
Edited by: Rahul Pant on Feb 7, 2011 12:24 PM -
Steps to load the data by using flat file for hierarchies in BI 7.0
Hi Gurus,
steps to load the data by using flat file for hierarchies in BI 7.0hi ,
u will get the steps int he following blog by Prakash Bagali
Hierarchy Upload from Flat files
regards,
Rathy -
Step by Step details on how to load data from a flat file
hi can anyone explain how to load data from a flat file. Pls giv me step by step details. thnx
hi sonam.
it is very easy to load data from flat file. whn compared with other extrations methods...
here r the step to load transation data from a flat file.......
step:1 create a Flat File.
step:2 log on to sap bw (t.code : rsa1 or rsa13).
and observe the flat file source system icon. i.e pc icon.
step:3 create required info objects.
3.1: create infoarea
(infoObjects Under Modeling > infoObjects (root node)-> context menu -
> create infoarea).
3.2: create char /keyfig infoObject Catalog.(select infoArea ---.context menu --->create infoObject catalog).
3.3: create char.. infoObj and keyFig infoObjects accourding to ur requirement and activate them.
step:4 create infoSource for transaction data and create transfer structure and maintain communication structure...
4.1: first create a application component.(select InfoSources Under modeling-->infosources<root node>>context menu-->create applic...component)
4.2: create infoSource for transation data(select appl..comp--.context menu-->create infosource)
>select O flexible update and give info source name..
>continue..
4.4: *IMp* ASSIGN DATASOURCE..
(EXPAND APPLIC ..COMP..>EXPAND YOUR INFOSOURCE>CONTEXT MENU>ASSIGN DATASOURCE.)
>* DATASOURCE *
>O SOURCE SYSTEM: <BROWSE AND CHOOSE YOUR FLAT FILE SOURCE SYSTEM>.(EX:PC ICON).
>CONTINUE.
4.5: DEFINE DATASOURCE/TRANSFER STRUCTURE FOR IN FOSOURCE..
> SELECT TRANSFER STRUCTURE TAB.
>FILL THE INFOOBJECT FILLED WITH THE NECESSARY INFOOBJ IN THE ORDER OR SEQUENCE OF FLAT FILE STRUCTURE.
4.6: ASSIGN TRANSFER RULES.
---> NOW SELECT TRANSFER RULES TAB. AND SELECT PROPOSE TRANSFER RULES SPINDLE LIKE ICON.
(IF DATA TARGET IS ODS -
INCLUDE 0RECORDMODE IN COMMUNICATION STRUCTURE.)
--->ACTIVATE...
STEP:5 CREATE DATATARGET.(INFOCUBE/ODS OBJECT).
5.1: CREATE INFO CUBE.
-->SELECT YOUR INFOAREA>CONTEXT MENU>CREATE INFOCUBE.
5.2: CREATE INFOCUBE STRUCTURE.
---> FILL THE STRUCTURE PANE WILL REQUIRE INFOOBJECTS...(SELECT INFOSOURCE ICON>FIND UR INFOSOURCE >DOUBLE CLICK > SELECT "YES" FOR INFOOBJECT ASSIGNMENT ).
>MAINTAIN ATLEAST ON TIME CHAR.......(EX; 0CALDAY).
5.3:DEFINE AND ASSIGN DIMENSIONS FOR YOUR CHARACTERISTICS..
>ACTIVATE..
STEP:6 CREATE UPDATE RULES FOR INFOCUDE USING INFOSOURCE .
>SELECT UR INFOCUBE >CONTEXT MENU> CREATE UPDATE RULES.
> DATASOURCE
> O INFOSOURCE : _________(U R INFOSOURCE). AND PRESS ENTER KEY.......
>ACTIVATE.....UR UPDATE RULES....
>>>>SEE THE DATA FLOW <<<<<<<<----
STEP:7 SCHEDULE / LOAD DATA..
7.1 CREATE INFOPACKAGE.
--->SELECT INFOSOURCE UNDER MODELING> EXPAND UR APPLIC.. COMP..> EXPAND UR INFOSOURCE..> SELECT UR DATASOURCE ASSIGN MENT ICON....>CONTEXT MENU> CREAE INFOPACKAGE..
>GIVE INFOPACKAGE DISCREPTION............_________
>SELECT YOUR DATA SOURCE.-------> AND PRESS CONTINUE .....
>SELECT EXTERNAL DATA TAB...
> SELECT *CLIENT WORKSTATION oR APPLI SERVER >GIVE FILE NAME > FILE TYPE> DATA SAPARATER>
>SELECT PROCESSING TAB
> PSA AND THEN INTO DATATARGETS....
>DATATARGET TAB.
>O SELECT DATA TARGETS
[ ] UPDATE DATATARGET CHECK BOX.....
--->UPDATE TAB.
O FULL UPDATE...
>SCHEDULE TAB..
>SELECT O START DATA LOAD IMMEDIATELY...
AND SELECT "START" BUTTON........
>>>>>>>>>>
STEP:8 MONITOR DATA
> CHECK DATA IN PSA
CHECK DATA IN DATA TARGETS.....
>>>>>>>>>>> <<<<<<<<<----
I HOPE THIS LL HELP YOU..... -
Errors when loading flat file data
We just test to load a very simple flat file data with only two lines and the two lines of data in preview of InfoSource is correct. But when run InfoPackage to load data, the monitor of the InfoPackage shows the following errors (see in between two dashed lines below):
Error getting SID for ODS object ZDM_SUBS
Activation of data records from ODS object ZDM_SUBS terminated
Error when assigning SID (details in long text)
Error when assigning SID (details in long text)
Error when assigning SID (details in long text)
Error when assigning SID (details in long text)
Error when assigning SID (details in long text)
Error when assigning SID (details in long text)
Error when assigning SID (details in long text)
Error when assigning SID (details in long text)
Value 'Bottom' (hex. '0042006F00740074006F006D') of characteristic ZRATEPLN contains invalid characters
Value 'Dealer' (hex. '004400650061006C00650072') of characteristic ZCHANNEL contains invalid characters
Value 'Bottom' (hex. '0042006F00740074006F006D') of characteristic ZRATEPLN contains invalid characters
Value '19884/' of characteristic 0DATE is not a number with 000008 spaces
Value '/19812' of characteristic 0DATE is not a number with 000008 spaces
Value '19884/' of characteristic 0DATE is not a number with 000008 spaces
In the flat file (excel sheet saved as a CSV file), for each row of the data, there are two fields which are start_date and end_date and the date format is MM/DD/YYYY and in the Transfer Rule, we transfer the date format from MM/DD/YYYY to YYYYMMDD which is required by DATS InfoObject type in BW. If you need the excel sheet of data in order to answer our questions about the above errors, you can give us your e-mail address and we can send the simple two rows of data excel sheet file to you.
Thanks!Hi Kevin,
1.You can use lowercase letters in the values for your characteristics provided you have checked the lowercase checkbox in the general tab page of Create characteristic screen.But when you do so no masterdata tables,text tables, or another level of attributes underneath are allowed.
OR
Use only upper case letters in your characteristic unchecking the above mentioned box.
2.The date format in the CSV file should be yyyymmdd.It should have 8 characters . I guess there is something strange in your "calendardays" since I could not find 8 characters irrespective of the order.Do not forget to use zeroes.
Hope this works.
Reward if it is helpful.
Regards,
Balaji -
Flat File Data Loads to BI 7.0
Hi Experts,
Please update me what is the best approach i have to follow for the below scenario of Flat File Data Loads
I will get data in Excel ....with Two worksheets....from the user
My requirment is to place the file in Central location avaliable to the user and BW to update if any changes necessary and want to load data(full0 to bw from file if there are any changes
Please update me how to deal with this scenarion of Two work sheets,In a central location...
ThanksEasiest thing would be to use a DSO with change log to handle the changes to pass onto to any cubes and load a full every night
Then let the change log worry about any changes to the workbook
You have to be careful about the DSO keys though for this to work properly
Now to automate the loads - just how are you planning to create the infopackage as it will only read a csv and not the binary xls
Well it will read the binary xls if you maybe use a dbconnect with a jdbc driver to read the xls (that's on my next thing to do - but if you are as your user id suggests a "bw learner" then that may be a bit complicated)
The only other thign to do is to write a macro that automatically creates the csv file on the app server when the user quits the xls
Or off course you can just dump the csv each night - but then that is a manual task and in systems I design I hate manual tasks as staff go on holiday and peopel change jobs and it's not really very SoX compliant -
Conversion_Exit_Cunit_error occured while loading the Flat file data
Hi
Iam tryign to load Flat file data into an ODS, i am getting error like Error Conversion Cunit.
Also we are using 0unit in the ODS for which CUNIT is a conversion rule
Can you please suggest me why iam getting this errorHi Sunil
Hope you can check whether you are loadig the flat file data from application server or Client workstation.
May be if you are loading from Client work station you will face problem of this type.
Try to check if any change in format in the file.
at the end of the file delete the spaces. -
Flat file data loading error using process chain
Hi SAP Experts,
I am having a problem loading the flat file data to the cube using process chain. The issue is that when i run the process chain it fails giving the message " Date format error, please enter the date in the format _.yyyy" . I am using " 0calmonth in the datasource" . Strange is that when i manually execute the infopackage, i dont get any errors and am able to load the same file successfully to the dso and the cube. Is there any special setting for the process chain that i am missing?
The date format in the flat file is mm/yyyy. I have tried all the options i could including recreating the datasource and invain dont see any success so far. please help me solving this problem as we r in the middle of testing cycle.
Thanking you all for your quick response and support all the time.
Kind Regards,
SanjeevHi Sanjeev,
I believe you are opening the .csv file again after saving it. In this case the initial 0 in single digit month (say 02/2010) is getting changed to 2/2010 and the resulting file is not readable to the system. Just do not open the file after you have entered data in the .csv file and saved and closed it. If required, open the file in notepad, but not in excel in case you want to re-check the data.
Hope this helps.
regards,
biplab
Maybe you are looking for
-
Hi guys, When I try to update my iphone4s to the latest software on itunes via my MacBook a message displays a message: 'This iPhone "Seans iphone" cannot be updated at this time because the iphone software update server could not be contacted or is
-
Y computer-----/keyboard is typing hyphens spontaneously and it's difficul-to -control.-
Hi, When-ever I want to type, -----------hyphens keep appearing -and they ar-e difficult to co-ntrol. ---------like this! ----------------------------- Has anyone seen -this problem? thanks Kerensa S-heen
-
Setting property for multiple objects?
Adobe Pro X; ES2 I need to set the access property for multiple objects based on the selection of a radio button group. I assume I have to use resolveNodes(), but cannot for the life of me get it to work. Here's what I've played with : if (this.rawVa
-
Firefox Freezes with Script Error
Firefox keeps freeing/hanging. First you will see the errors that pop up for me. Then the troubleshooting I have done to solve this problem. Script Error: Script: http://z-ecx.images-amazon.com/images/G/01/browser-scripts/imageBlock/imageBlock-319070
-
NEED HELP ASAP-populate contents from 5 tables into one master table
Scenario: Assign a new_customer_id to each existing customer of our three business. A master lookup table (M) was given to me with pre-populated (SEQ_NOs), (NEW_CUSTIDs), and NULL in (OLD_CUSTIDS). My duty is to (1) Fill the M.OLD_CUSTID column with