Huge Data upload

Dear Experts,
I got a requirement from my client, having 40,00,000 of customers need to upload into SAP.
What is the best way to do?
According to my knowledge...
LSMW is slow,
BAPI is slow,
Call Traction is slow
Session method
I do not know about API
Could you please suggest me which is the best method to upload 40,00,000 records.

Maybe Batch-Input?
[Managing Batch-Input Sessions|http://help.sap.com/saphelp_nw04/helpdata/en/69/c250274ba111d189750000e8322d00/frameset.htm]
[BDC or Batch Input Program tips and tricks|http://www.sap-img.com/bdc.htm]
Hope it helps!
Regards.

Similar Messages

Huge data upload to custom table

I created a custom table and the requirement is to load huge data to this table around 20million records. I have input file with 1million records. After loading few files, I got error max extents 300 is reached. We talked to basis and they increased to 1000. Still I'm getting this error.
The way my porgram written was
reading all input data to internal table i_zmstdcost_sum_final.
INSERT zmstdcost_sum FROM TABLE i_zmstdcost_sum_final.
Can any one suggest me the best solution or how to alter my program?
Thanks,
PV

Hi
If you need to load a very large number of records u shouldn't load the record in an internal table and use the commit statament after a certain number of record and close a file after loading all its records.
LOOP AT T_FILE.
OPEN DATASET T_FILE.
IF SY-SUBRC = 0.
 DO.
 READ DATASET T_FILE INTO WA.
 IF SY-SUBRC <> 0.
 EXIT.
 ENDIF.
 INSERT ZTABLE FROM WA.
 IF SY-SUBRC = 0.
 COUNT = COUNT + 1.
 ENDIF.
 IF COUNT = 1000.
 COUNT = 0.
 COMMIT WORK.
 ENDIF.
 ENDDO.
 IF COUNT > 0.
 COMMIT WORK.
 COUNT = 0.
 ENDIF.
 CLOSE DATASET T_FILE.
ENDIF.
ENDLOOP.
Max

Huge data uploads

I have the requirement to upload customers ,Materials .Each of them have around half a million records. Is there any method or procedure which will considerably reduce the time of run( I am using BAPI ).

Hi!
BAPI or IDOC - in general they need the same time (no discussion about +/- 10%).
With IDOC you will get an easy way to create the data in SAP parallel -> best way to get use all resource and reduce total runtime.
Deactivate change documents and application log. If change documents can't be deactivated by an import parameter, then make an extension in the xxx_document_write function module. This might give you about 20% better performance (depending on the total size of change docs / appl log tables).
Make a fresh runstat before starting -> all necessary check tables should have a good index access. Also a new version after half the run can help.
Regards,
Christian

Optimization for bulk data upload

Hi everyone!
I've got the following issue:
I have to do a bulk data upload using JMS deploy in a glassfish 2.1, to process and validate data in a Oracle 10g DB System before it is insert.
I have my web interface that loads a file and then delegates the process to a Stateless Session Bean which read a N number of lines and after that send a message to a JMS Queue. The JMS has to parse the each line, validate them with the data already in the DB, and finally persist the new data.
This process is high process consuming, and I need to improve the performance time. I tried to change glassfish default JMS and JDBC pool size, but I have no a huge difference.
Do you have any advice that could help me?
Thanks in advance!

Hi! thank you for you answer!
High process consuming is in the MDB
I'm grouping each N number of read lines in the EJB and then send the message to the JMS. The MDB process the persists each line as info in different related tables.
Thanks again!

Enhancement_CIN_Capture Incoming Excise Invoice-J1IEX bulk data upload

Dear All,
Sub:CIN_Capture Incoming Excise Invoice-J1IEX bulk data upload option requirement
We are capturing the Incoming excise invoices manually in the
transaction J1IEX with huge datau2019s and according to the volume of data
it is very difficult for us to enter manually and now we required for
the option of bulk data processing to upload the data from the Excel
file(received the softcopy from the supplier).
As per our observations we found the BAPI named
BAPI_EXCINV_CREATE_FROMDATA but the update level of this BAPI is not
available in our system because as per the Indian Government norms one
ofthe current Excise Duty Tariff is as below
1. Basic Excise Duty (BED 8%).
2. Education Cess (ECess 2%)
3. Secondary Education Cess (SECess 1%)
and we observed the SECess (1%) is not covered in the above mentioned
BAPI so we are not in a position to proceed further.
So Kindly update us is any other relevant option will solve the purpose.
We are in a quite difficult situation to uplaod the datas to our system
so please do the needful.
Regards,
PrabuM

Please note that CIN uses the 'MB_MIGO_BADI' definition and 'CIN_PLUG_IN_TO_MIGO' is the implementation. You can create multiple implementations of this BADI. The same BADI is used for single step Capture & Post of excise invoice in MIGO. Kindly use this BADI as per your needs. SAP std does not support any BAPIs for Goods Receipts with Excise updates

Reg:Efficient solution for a data upload scenario

Hi All,
I have the following task.
Required data from a legacy system(generate data only in the form of flat files)to SAP R3 as FB01 journals and the output file should be generated periodically(daily,weekly,fortnightly etc)
Solution Approaches:
1)Write a BDC program to extract the data.
2) Write a ABAP Progam to populate IDoc (if standard IDOc is available) or generate a Outbound proxy (If standard IDoc is not available) to push the data into SAP XI.
Could anyone tell me which would be the the best and efficient approach for this task and need your recommendations.
Thanks in Advance.
B.Lavanya
Edited by: Lavanya Balanandham on Mar 31, 2008 2:23 PM

Hi Lavanya,
Required data from a legacy system(generate data only in the form of flat files)to SAP R3 as FB01 journals - use BDC for this thing because it will be better for large source files.
the output file should be generated periodically(daily,weekly,fortnightly etc) - if this output file contains acknowledgment for the data uploaded by the above process, create a ABAP report for it and schedule it..........but if this output contains some other IDOC data which you need to send as a file to a third-party system, then go for SAP XI provided the IDOC data is not too large... but if the IDOC size is huge, then just create a ABAP report for outputting data to a file on application server and FTP the file to third-party system.
Regards,
Rajeev Gupta

Load Huge data into oracle table

Hi,
I am using oracle 11g Express Edition, I have a file of .csv forma, Which has a data of size 500MB which needs to be uploaded into oracle table.
Please suggest which would be the best method to upload the data into table. Data is employee ticket history which is of huge data.
How to do the mass upload of data into oracle table need experts suggestion on this requirement.
Thanks
Sudhir

Sudhir_Meru wrote:
Hi,
I am using oracle 11g Express Edition, I have a file of .csv forma, Which has a data of size 500MB which needs to be uploaded into oracle table.
Please suggest which would be the best method to upload the data into table. Data is employee ticket history which is of huge data.
How to do the mass upload of data into oracle table need experts suggestion on this requirement.
Thanks
SudhirOne method is to use SQL Loader (sqlldr)
Another method is to define an external table in Oracle which is allowing you to view your big file as a table in database.
You may want to have a look at this guide: Choosing the Right Export/Import Utility and this Managing External Tables.
Regards.
Al
Edited by: Alberto Faenza on Nov 6, 2012 10:24 AM

Data upload for vendor balances using BDC

hi abap experts,
I have a requirement on data uploading using BDC.
For the vendor balances ie. for transaction FBL1N ( I was given a template for vendor balance upload and need to write a BDC program for that ) I need upload the exsisting transaction data to the system. is recording necessary for this?
can u pls help me with step by step process for vendor balance uploading.
Thanks,
Hema.

Hi
Please follow the following Steps:
Steps for recording:
Step1: Goto TCODE SHDB
Step2: Click on New Recording
Step3: Give the necessary Details such as TOCDE, Desc, ...
Step4: Do the screen by screen recording.(Please avoid extra screen to appear)
Step5: Save the recording.
Step6. Select the recording and click on Program button on toolbar.
Step7: Give the Program name and click on radio button Transfer from recording.
Step 8: It will open a new session with SE38 and a program with the recoding.
Step 9: then just add the basic code for BDC.
Regards,
Lokesh

Basic Data Upload to MATERIAL Using LSMW is not working

HI All,
we are using LSMW    /SAPDMC/SAP_LSMW_IMPORT_TEXTS program to upload the basic text of the material, all steps are executed correctly and shows records are transfered correctly , but the in MM03 the text is not uploading..
EPROC_PILOT - MASTER - TEXT_UPLOAD Basic long text 1line
Field Mapping and Rule
        /SAPDMC/LTXTH                  Long Texts: Header
            Fields
                OBJECT                       Texts: Application Object
                                    Rule :   Constant
                                    Code:    /SAPDMC/LTXTH-OBJECT = 'MATERIAL'.
                NAME                         Name
                                    Source: LONGTEXT-NAME (Name)
                                    Rule :   Transfer (MOVE)
                                    Code:    /SAPDMC/LTXTH-NAME = LONGTEXT-NAME.
                ID                           Text ID
                                    Source: LONGTEXT-ID (Text ID)
                                    Rule :   Transfer (MOVE)
                                    Code:    /SAPDMC/LTXTH-ID = LONGTEXT-ID.
                SPRAS                        Language Key
                                    Source: LONGTEXT-SPRAS (Language Key)
                                    Rule :   Transfer (MOVE)
                                    Code:    /SAPDMC/LTXTH-SPRAS = LONGTEXT-SPRAS.
                                             * Caution: Source field is longer than target field
            /SAPDMC/LTXTL                  Long Texts: Row
            /SAPDMC/LTXTL                  Long Texts: Row
                Fields
                    TEXTFORMAT                   Tag column
                                        Rule :   Constant
                                        Code:    /SAPDMC/LTXTL-TEXTFORMAT = 'L'.
                    TEXTLINE                     Text Line
                                        Source: LONGTEXT-TEXTLINE (Text Line)
                                        Rule :   Transfer (MOVE)
                                        Code:    /SAPDMC/LTXTL-TEXTLINE = LONGTEXT-TEXTLINE.
and at last it displaying as follws
LSM Workbench: Convert Data For EPROC_PILOT, MASTER, TEXT_UPLOAD
2010/02/01 - 10:14:25
File Read:          EPROC_PILOT_MASTER_TEXT_UPLOAD.lsmw.read
File Written:       EPROC_PILOT_MASTER_TEXT_UPLOAD.lsmw.conv
Transactions Read:                    1
Records Read:                         1
Transactions Written:                 1
Records Written:                      2
can any one tell us what could be problem
Regards
Channappa Sajjanar

Hi , thanks for your reply,
i run the all the steps .
when i run the program it gives message as follows
Legacy System Migration Workbench
Project:                              EPROC_PILOT     eProcurement Pilot
Subproject:                           MASTER          Master data Upload / Change
Object:                               TEXT_UPLOAD     Basic long text 1line
File :                                EPROC_PILOT_MASTER_TEXT_UPLOAD.lsmw.conv
Long Texts in Total:                  1
Successfully Transferred Long Texts: 1
Non-Transferred Long Texts:           0

How to schedule Job for data uploading from source to BI

Hi to all,
How to schedule Job for data uploading from source to BI,
Why we required and how we do it.
As I am fresher in BI, I need to know from bottom.
Regards
Pavneet Rana

Hi.
You can create [process chain |http://www.sdn.sap.com/irj/scn/go/portal/prtroot/docs/library/uuid/502b2998-1017-2d10-1c8a-a57a35d52bc8?quicklink=index&overridelayout=true]for data loading pocess and schedule start process to any time/date etc ...
Regadrs.

Legacy asset data upload for multiple line items

Hello
Legacy asset data upload for multiple line items- for example Building is an asset which is having different line items for purchasing of land,constructaion or renovation etc.........now to upload the legacy data what should be consider...only one line item for Building or numbers of line item.........
which one is the proper way to do this exercise?
regards

Hi,
It completely depends on clients requirement. but following are few approaches:
1. Define Building as Asset Class.
2. Create a Building at specific location at asset code and
3. create asset sub numbers for other components.
other approach would be:
1. Define Building as Asset Class.
2. Create a Building at specific location at asset code and
3. create further asset codes for other components where field description will be used for relating each other.
Regards,
Sayujya

Any tools are there in data upload for other than BDC.

Any tools are there in data upload for other than BDC.

Hi Hassan,
LSMW
<a href="http://sapabap.iespana.es/sapabap/manuales/pdf/lsmw.pdf">http://sapabap.iespana.es/sapabap/manuales/pdf/lsmw.pdf</a>
Direct Input
<a href="http://help.sap.com/saphelp_di471/helpdata/EN/fa/097174543b11d1898e0000e8322d00/content.htm">Direct Input</a>
BAPI's
Example :
BAPI_QUOTATION_CREATEFROMDATA2 - Customer Quotation: Create Customer Quotation
BAPI_PO_CREATE1 - Create Purchase Order
Close your previous threads if you have got the answers.
Regards,
AS

TS4605 Hi, I was working in WORD on a file containing huge data. My machine just hung up one day while working and now I seem to have lost the file how do I get it back. Please HELP me.

Hi, I was working in WORD on a file containing huge data. My machine just hung up one day while working and now I seem to have lost the file how do I get it back. Please HELP me.

Well, iCloud has nothing to do with this.
Do you have the built-in backup function Time Machine running on your Mac?
See: http://support.apple.com/kb/ht1427

Master data upload into SAP system

Hello,
I want to know if there is any standard method to upload material master, customer master, vendor master and finance master data into SAP system.
I am not referring to LSMW's, BDC's and using BAPI's. I am aware of standard programs like RMDATIND for material master upload, RFBIDE00 for customer master upload and RFBIKR00 for vendor master upload. But these are using direct input method and SAP recommends this only for testing purpose. I am not sure if this could be really used in actual live scenarios.
From some other posts in the forum, I came to know about some transactions like BDLR, SXDB and BMVO. Can some one tell me how to use these T.Codes?
If some one has any detailed documentation on these T.codes or in general standard master data upload techniques please send it to [email protected]
Thanks in advance,
CMV

Hi,
Define the following attributes, using the F4 input help and F1 field help:
Report
Name of a registered program for this program type
Variant
You can only specify a variant with programs that are started directly.
With direct input, data from the data transfer file undergoes the same checks as with the online transaction and is then transferred directly into the SAP System. The database is updated directly with the transferred data.
For the documentation of other transactions please refer the correponding program documentation..which is more helpful,
Reward points if helpful,
Regards,
jinesh

BW upgrade EHP1, data uploads should stop?

Dear experts,
we have plan for system upgrade. current system is BW 7.0 sp 17, Now planned for EHP1 and SP9.
I know the there some post upgrade activities, which include consistancy check for objects (infoobject, transfer rules, cubes,dso, etc...)
Could some one pls confirm, do we need to stop the data uploads/stop process chains during system upgrade?
Thanks in advance!
Best Regards,
Mannu

Hi Ingo,
RSRT was giving proper results. We have now implemented few SAP notes and the issues got resolved.
The following are the notes:
1499233 - MDX:bXML flattening, unbalanced hierarchy, empty columns
1485648 - MDX: bXML flattening and hierarchies and displaced columns
1446245 - MDX: Error when RSR_MDX_BXML_GET_GZIP_DATA is called
1441767 - MDX: No data for bXML if only ALL member is requested
1438091 - MDX: basXML: Object MEASURE 0 not found
1435844 - MDX:Wrong no. decimal places for basXML flattening interface
1432162 - MDX: Flattening problems when using hierarchies
1420169 - MDX: bXML flattening: Subsequent note
1411491 - MDX: bXML flattening in transac. MDXTEST: Selecting packages
1404328 - MDX: bXML flattening and PROPERTIES: Columns overwritten
Thanks for your inputs.
Regards,
shesha.

Huge Data upload

Similar Messages

Maybe you are looking for