Data misplaced during Flat file data load

Hi all,
    i have to load the data from the flat file(txt format) but i found some record are not populating in PSA.
the voluem of the data is very huge. how to resolve this and data load seems to be green in info package monitoring.

Hi,
How did you conclude that the records are missing the PSA?  Is it because the count is less?
There are the chances that the records are duplicated in the flat file.
Perhaps, you can increase the number of records per datapackage in the settings and can check the count.
If you feel, a particular record is not getting loaded, the load that record selectively through filter.
Split the load according to your convenience and load so that it will be easy to keep track of the records.
Thanks.

Similar Messages

  • How to extract data from multiple flat files to load into corresponding tables in SQL Server 2008 R2 ?

    Hi,
              I have to implement the following scenario in SSIS but don't know how to do since I never worked with SSIS before. Please help me.
              I have 20 different text files in a single folder and 20 different tables corresponding to each text file in SQL Server 2008 R2 Database. I need to extract the data from each text file and
    load the data into corresponding table in Sql Server Database. Please guide me in how many ways I can do this and which is the best way to implement this job.  Actually I have to automate this job. Few files are in same format(with same column names
    and datatypes) where others are not.
    1. Do I need to create 20 different projects ?
                   or
        Can I implement this in only one project by having 20 packages?
                 or
        Can I do this in one project with only one package?
    Thanks in advance.

    As I said I don't know how to use object data type, I just given a shot as below. I know the following code has errors can you please correct it for me.
    Public
    Sub Main()
    ' Add your code here 
    Dim f1
    As FileStream
    Dim s1
    As StreamReader
    Dim date1
    As
    Object
    Dim rline
    As
    String
    Dim Filelist(1)
    As
    String
    Dim FileName
    As
    String
    Dim i
    As
    Integer
    i = 1
    date1 =
    Filelist(0) =
    "XYZ"
    Filelist(1) =
    "123"
    For
    Each FileName
    In Filelist
    f1 = File.OpenRead(FileName)
    s1 = File.OpenText(FileName)
    rline = s1.ReadLine
    While
    Not rline
    Is
    Nothing
    If Left(rline, 4) =
    "DATE"
    Then
    date1 (i)= Mid(rline, 7, 8)
     i = i + 1
    Exit
    While
    End
    If
    rline = s1.ReadLine
    End
    While
    Next
    Dts.Variables(
    "date").Value = date1(1)
    Dts.Variables(
    "date1").Value = date1(2)
    Dts.TaskResult = ScriptResults.Success
    End
    Sub

  • Getting error RSTSODSPART during flat file data load to datasource

    Hi SAP Gurus,
    In BI, I dont know why i am getting this error while loading data from flat file to data source using infopackage. Always i am deleting data from PSA before running it even have this error. Could u pls help me out in this issue.... Thanks....
    Error when inserting in PSA table RSTSODSPART
    Message no. RSAODS206
    Diagnosis
    An error occurred with the insert into PSA table RSTSODSPART for InfoSource .
    System Response
    Your data was not updated in the PSA table.
    It is possible that data already exists for this request  and data packet .

    Hello,
    Do you have an MS SQL Database, if yes then the patch in the note 1340371 should resolve the problem. If you have a different DB release please let me know your DB release and your BW release and support package level.
    Best Regards,
    Des

  • Flat file data load - ODS - Look up data in Startroutine

    Hi All,
    There is a requirement that I have two ODS say 1) ABC and XYZ.
    For Both ODS , We load flat file data.
    First we load data to ABC for current fiscal period
    During the data load to XYZ, We lookup part nos data in ABC for current fiscal period , if data there , for those part nos we load XYZ Flat file data.
    My requirement is that , I need to fetch part nos data from ABC for Fiscal period < current fisacl period.  Then I need to extract already loaded data for those part nos from XYZ and Mark the flag field as Non reportable .
    How can we achieve this ?
    Please advice
    Thanks
    Ajay

    Hi ,
    Thanks for your reply.
    I have done so. When I add lines of My Internal table data to DATA_PACKAGE, It gives the syntax error that both structures are not unique. We have several fields in XYZ ods but they are not in Communication structure.
    Thats the problem. Also , if I write the code , will it be executed for each datapackage. I mean , for each datapacket process , my code will fetch whole data from ABC and then from XYZ. Repetion will be there?
    Any other logic can i use ?
    Thanks
    Ajay

  • What is the best way to load and convert data from a flat file?

    Hi,
    I want to load data from a flat file, convert dates, numbers and some fields with custom logic (e.g. 0,1 into N,Y) to the correct format.
    The rows where all to_number, to_date and custom conversions succeed should go into table STG_OK. If some conversion fails (due to an illegal format in the flat file), those rows (where the conversion raises some exception) should go into table STG_ERR.
    What is the best and easiest way to archive this?
    Thanks,
    Carsten.

    Hi,
    thanks for your answers so far!
    I gave them a thought and came up with two different alternatives:
    Alternative 1
    I load the data from the flat file into a staging table using sqlldr. I convert the data to the target format using sqlldr expressions.
    The columns of the staging table have the target format (date, number).
    The rows that cannot be loaded go into a bad file. I manually load the data from the bad file (without any conversion) into the error table.
    Alternative 2
    The columns of the staging table are all of type varchar2 regardless of the target format.
    I define data rules for all columns that require a later conversion.
    I load the data from the flat file into the staging table using external table or sqlldr without any data conversion.
    The rows that cannot be loaded go automatically into the error table.
    When I read the data from the staging table, I can safely convert it since it is already checked by the rules.
    What I dislike in alternative 1 is that I manually have to create a second file and a second mapping (ok, I can automate this using OMB*Plus).
    Further, I would prefer using expressions in the mapping for converting the data.
    What I dislike in alternative 2 is that I have to create a data rule and a conversion expression and then keep the data rule and the conversion expression in sync (in case of changes of the file format).
    I also would prefer to have the data in the staging table in the target format. Well, I might load it into a second staging table with columns having the target format. But that's another mapping and a lot of i/o.
    As far as I know I need the data quality option for using data rules, is that true?
    Is there another alternative without any of these drawbacks?
    Otherwise I think I will go for alternative 1.
    Thanks,
    Carsten.

  • How to load data from a  flat file which is there in the application server

    HI All,
              how to load data from a  flat file which is there in the application server..

    Hi,
    Firstly you will need to place the file(s) in the AL11 path. Then in your infopackage in "Extraction" tab you need to select "Application Server" option. Then you need to specify the path as well as the exact file you want to load by using the browsing button.
    If your file name keeps changing on a daily basis i.e. name_ddmmyyyy.csv, then in the Extraction tab you have the option to write an ABAP routine that generates the file name. Here you will need to append sy-datum to "name" and then append ".csv" to generate complete filename.
    Please let me know if this is helpful or if you need any more inputs.
    Thanks & Regards,
    Nishant Tatkar.

  • Loading CLOB data to a flat file in owb 11.2.0.3

    Hi,
    Also,I have one more question.I want to load a table with clob data into a flat file using my owb client 11.2.0.3.But when I run this simple mapping ,I am getting this error:
    ORA-06502:pl/sql:NUMERIC OR VALUE ERROR:CHARACTER STRING BUFFER TOO SMALL.
    Please suggest what needs to be done.

    I was up all night as I had a dead line doing this....
    Turned out my Mapping was corrupted.
    If you do a series of synchronizations by object name, position, id the mapping would end up corrupted.
    I had to delete my staging table operator within the mapping, drag and drop and reconnect and now my loads work just fine..
    Thanks

  • Error Loading Data into a flat file

    I am recieving the following error when loading data into a flat file from a flat file. SQL 2005 is my back end DB. If I cut the file iin half approx 500K rows my ODI interface works fine. Not sure what to look at.. I rebuit the interface which before was just dying giving no error and now I am getting this.
    Thanks
    java.lang.Exception
         at com.sunopsis.dwg.dbobj.SnpSessTaskSql.treatTask(SnpSessTaskSql.java)
         at com.sunopsis.dwg.dbobj.SnpSessStep.treatSessStep(SnpSessStep.java)
         at com.sunopsis.dwg.dbobj.SnpSession.treatSession(SnpSession.java)
         at com.sunopsis.dwg.cmd.DwgCommandSession.treatCommand(DwgCommandSession.java)
         at com.sunopsis.dwg.cmd.DwgCommandBase.execute(DwgCommandBase.java)
         at com.sunopsis.dwg.cmd.e.i(e.java)
         at com.sunopsis.dwg.cmd.g.y(g.java)
         at com.sunopsis.dwg.cmd.e.run(e.java)
         at java.lang.Thread.run(Unknown Source)

    Figured it out, found similar post that stated changing the HEAP size
    Increase the page size in odiparams.bat in the bin folder and restart Designer.
    For eg:
    set ODI_INIT_HEAP=128m
    set ODI_MAX_HEAP=1024m

  • Loading Master Data from a Flat File

    Hi Guru, I have flat files of different SAP tables (Transactional data tables and Master Data ), I want to upload this flat files to a DSO, my question is in the case of Master Data Files example SAP table USR03 ( User Address Data) in a Flat file format which is better to create a Master data text and attributes Datasource or create it as a transactional data Datasource to upload it to a DSO.

    Hi
    Master data datasource should be used to load master data like customer details. customer data is the master data which does not change frequently. customer infoobject may have some attributes like customer name, customer contact number, customer address, etc. we have to maintain these attributes and text data.
    Transaction data is something which keeps changing very frequently like price of the material, sales, etc. for loading such data we go for transaction datasources.
    As mentioned above too,the changes in the address are very rare, therefore youshould not use transactional datasource.
    Regards, Rahul
    Edited by: Rahul Pant on Feb 7, 2011 12:24 PM

  • Steps to load the data by using flat file for hierarchies in BI 7.0

    Hi Gurus,
    steps to load the data by using flat file for hierarchies in BI 7.0

    hi ,
    u will get the steps int he following blog by Prakash Bagali
    Hierarchy Upload from Flat files
    regards,
    Rathy

  • Step by Step details on how to load data from a flat file

    hi can anyone explain how to load data from a flat file. Pls giv me step by step details. thnx

    hi sonam.
    it is very easy to load data from flat file. whn compared with other extrations methods...
    here r the step to load transation data from a flat file.......
    step:1 create a Flat File.
    step:2 log on to sap bw (t.code : rsa1 or rsa13).
    and observe the flat file source system icon. i.e pc icon.
    step:3 create required info objects.
    3.1: create infoarea
         (infoObjects Under Modeling > infoObjects (root node)-> context menu -
    > create infoarea).
    3.2:  create char /keyfig infoObject Catalog.(select infoArea ---.context menu --->create infoObject catalog).
    3.3:   create char.. infoObj and keyFig infoObjects accourding to ur requirement and activate them.
    step:4 create infoSource for transaction data and create transfer structure and maintain communication structure...
        4.1: first create a application component.(select InfoSources Under modeling-->infosources<root node>>context menu-->create  applic...component)
       4.2: create infoSource  for transation data(select appl..comp--.context menu-->create infosource)
    >select O flexible update and give info source name..
    >continue..
    4.4: *IMp* ASSIGN DATASOURCE..
      (EXPAND APPLIC ..COMP..>EXPAND YOUR INFOSOURCE>CONTEXT MENU>ASSIGN DATASOURCE.)
    >* DATASOURCE *
    >O SOURCE SYSTEM: <BROWSE AND CHOOSE YOUR FLAT FILE SOURCE SYSTEM>.(EX:PC ICON).
    >CONTINUE.
    4.5: DEFINE DATASOURCE/TRANSFER STRUCTURE  FOR IN FOSOURCE..
    > SELECT TRANSFER STRUCTURE TAB.
    >FILL THE INFOOBJECT FILLED WITH THE NECESSARY  INFOOBJ IN THE ORDER OR SEQUENCE OF FLAT FILE STRUCTURE.
    4.6: ASSIGN TRANSFER RULES.
    ---> NOW SELECT TRANSFER RULES TAB. AND SELECT PROPOSE TRANSFER RULES SPINDLE LIKE ICON.
    (IF DATA TARGET IS ODS -
    INCLUDE 0RECORDMODE IN COMMUNICATION STRUCTURE.)
    --->ACTIVATE...
    STEP:5  CREATE DATATARGET.(INFOCUBE/ODS OBJECT).
    5.1: CREATE INFO CUBE.
    -->SELECT YOUR INFOAREA>CONTEXT MENU>CREATE INFOCUBE.
    5.2: CREATE INFOCUBE STRUCTURE.
    ---> FILL THE STRUCTURE PANE WILL REQUIRE INFOOBJECTS...(SELECT INFOSOURCE ICON>FIND UR INFOSOURCE >DOUBLE CLICK > SELECT "YES" FOR INFOOBJECT ASSIGNMENT ).
    >MAINTAIN ATLEAST  ON TIME CHAR.......(EX; 0CALDAY).
    5.3:DEFINE AND ASSIGN DIMENSIONS FOR YOUR CHARACTERISTICS..
    >ACTIVATE..
    STEP:6 CREATE UPDATE RULES FOR INFOCUDE USING INFOSOURCE .
    >SELECT UR INFOCUBE >CONTEXT MENU> CREATE UPDATE RULES.
    > DATASOURCE
    > O INFOSOURCE : _________(U R INFOSOURCE). AND PRESS ENTER KEY.......
    >ACTIVATE.....UR UPDATE RULES....
    >>>>SEE THE DATA FLOW <<<<<<<<----
    STEP:7  SCHEDULE / LOAD DATA..
    7.1 CREATE INFOPACKAGE.
    --->SELECT INFOSOURCE UNDER MODELING> EXPAND UR APPLIC.. COMP..> EXPAND UR INFOSOURCE..> SELECT UR DATASOURCE ASSIGN MENT ICON....>CONTEXT MENU> CREAE INFOPACKAGE..
    >GIVE INFOPACKAGE DISCREPTION............_________
    >SELECT YOUR DATA SOURCE.-------> AND PRESS CONTINUE .....
    >SELECT EXTERNAL DATA TAB...
    > SELECT *CLIENT WORKSTATION oR APPLI SERVER  >GIVE FILE NAME > FILE TYPE> DATA SAPARATER>
    >SELECT PROCESSING TAB
    > PSA AND THEN INTO DATATARGETS....
    >DATATARGET TAB.
    >O SELECT DATA TARGETS
    [ ] UPDATE DATATARGET CHECK BOX.....
    --->UPDATE TAB.
    O FULL UPDATE...
    >SCHEDULE TAB..
    >SELECT O START DATA LOAD IMMEDIATELY...
    AND SELECT  "START" BUTTON........
    >>>>>>>>>>
    STEP:8 MONITOR DATA
    > CHECK DATA IN PSA
    CHECK DATA IN DATA TARGETS.....
    >>>>>>>>>>> <<<<<<<<<----
    I HOPE THIS LL HELP YOU.....

  • Errors when loading flat file data

    We just test to load a very simple flat file data with only two lines and the two lines of data in preview of InfoSource is correct.  But when run InfoPackage to load data, the monitor of the InfoPackage shows the following errors (see in between two dashed lines below):
    Error getting SID for ODS object ZDM_SUBS
    Activation of data records from ODS object ZDM_SUBS terminated
    Error when assigning SID (details in long text)
    Error when assigning SID (details in long text)
    Error when assigning SID (details in long text)
    Error when assigning SID (details in long text)
    Error when assigning SID (details in long text)
    Error when assigning SID (details in long text)
    Error when assigning SID (details in long text)
    Error when assigning SID (details in long text)
    Value 'Bottom' (hex. '0042006F00740074006F006D') of characteristic ZRATEPLN contains invalid characters
    Value 'Dealer' (hex. '004400650061006C00650072') of characteristic ZCHANNEL contains invalid characters
    Value 'Bottom' (hex. '0042006F00740074006F006D') of characteristic ZRATEPLN contains invalid characters
    Value '19884/' of characteristic 0DATE is not a number with 000008 spaces
    Value '/19812' of characteristic 0DATE is not a number with 000008 spaces
    Value '19884/' of characteristic 0DATE is not a number with 000008 spaces
    In the flat file (excel sheet saved as a CSV file), for each row of the data, there are two fields which are start_date and end_date and the date format is MM/DD/YYYY and in the Transfer Rule, we transfer the date format from MM/DD/YYYY to YYYYMMDD which is required by DATS InfoObject type in BW.  If you need the excel sheet of data in order to answer our questions about the above errors, you can give us your e-mail address and we can send the simple two rows of data excel sheet file to you.
    Thanks!

    Hi Kevin,
    1.You can use lowercase letters in the values for your characteristics provided you have checked the lowercase checkbox in the general tab page of Create characteristic screen.But when you do so no masterdata tables,text tables, or another level of attributes underneath are allowed.
                            OR
    Use only upper case letters in your characteristic unchecking the above mentioned box.
    2.The date format in the CSV file should be yyyymmdd.It should have 8 characters . I guess there is something strange in your "calendardays" since I could not find 8 characters irrespective of the order.Do not forget to use zeroes.
    Hope this works.
    Reward if it is helpful.
    Regards,
    Balaji

  • Flat File Data Loads to BI 7.0

    Hi Experts,
    Please update me what is the best approach i have to follow for the below scenario of Flat File Data Loads
    I will get data in Excel ....with Two worksheets....from the user
    My requirment is to place the file in Central location avaliable to the user and BW to update if any changes necessary  and want to load data(full0  to bw from file if there are any changes
    Please update me how to deal with this scenarion of Two work sheets,In a central location...
    Thanks

    Easiest thing would be to use a DSO with change log to handle the changes to pass onto to any cubes and load a full every night
    Then let the change log worry about any changes to the workbook
    You have to be careful about the DSO keys though for this to work properly
    Now to automate the loads - just how are you planning to create the infopackage as it will only read a csv and not the binary xls
    Well it will read the binary xls if you maybe use a dbconnect with a jdbc driver to read the xls (that's on my next thing to do - but if you are as your user id suggests a "bw learner" then that may be a bit complicated)
    The only other thign to do is to write a macro that automatically creates the csv file on the app server when the user quits the xls
    Or off course you can just dump the csv each night - but then that is a manual task and in systems I design I hate manual tasks as staff go on holiday and peopel change jobs and it's not really very SoX compliant

  • Conversion_Exit_Cunit_error occured while loading the Flat file data

    Hi
    Iam tryign to load Flat file data into an ODS, i am getting error like Error Conversion Cunit.
    Also we are using 0unit in the ODS for which CUNIT is a conversion rule
    Can you please suggest me why iam getting this error

    Hi Sunil
    Hope you can check whether you are loadig the flat file data from application server or Client workstation.
    May be if you are loading from Client work station you will face problem of this type.
    Try to check if any change in format in the file.
    at the end of the file delete the spaces.

  • Flat file data loading error using process chain

    Hi SAP Experts,
    I am having a problem loading the flat file data to the cube using process chain. The issue is that when i run the process chain it fails giving the message " Date format error, please enter the date in the format  _.yyyy" . I am using " 0calmonth in the datasource" . Strange is that when i manually execute the infopackage, i dont get any errors and am able to load the same file successfully to the dso and the cube. Is there any special setting for the process chain that i am missing?
    The date format in the flat file is mm/yyyy. I have tried all the options i could including recreating the datasource and invain dont see any success so far. please help me solving this problem as we r in the middle of testing cycle.
    Thanking you all for your quick response and support all the time.
    Kind Regards,
    Sanjeev

    Hi Sanjeev,
    I believe you are opening the .csv file again after saving it. In this case the initial 0 in single digit month (say 02/2010) is getting changed to 2/2010 and the resulting file is not readable to the system. Just do not open the file after you have entered data in the .csv file and saved and closed it. If required, open the file in notepad, but not in excel in case you want to re-check the data.
    Hope this helps.
    regards,
    biplab

Maybe you are looking for