Best data upload protocol.

HI Everyone,
We are managing a JSP based application in which we uploads data using HTTP Protocol. Daily data uploads are in GB's. the problems that we are facing are:-
Memory Heap size.
Request timeout errors of our web server.
No Resume support in case of network error.
Now we want to upgrade our application to get following benefits:-
Get rid of above mention problems.
Reduce data upload time.
Resume support in case of network errors.
I have following questions in this regard:-
What should be the Data upload protocol ? HTTP OR FTP.
What should be the Application interface for data uploading module ? like Java Applet, JSP OR Java Desktop application etc .
To resolve Memory Heap size errors we are using cronjob to restart web server, is it OK ? OR there is some other batter solutions for this.
Your value able advice is required to make correct decision.
Regards.

There is no best. What CAN you use? If FTP is an option (and make it SFTP) then of course go for that, then you don't have to be bothered by the clunky HTTP file upload.
> What should be the Application interface for data uploading module
How about a proper existing SFTP client? There is absolutely no reason at all why that should be anything Java related.
> To resolve Memory Heap size errors we are using cronjob to restart web server, is it OK ? OR there is some other batter solutions for this.
How about finding and solving the source of your memory leaks?

Similar Messages

  • Best Practice for Flat File Data Uploaded by Users

    Hi,
    I have the following scenario:
    1.     Users would like to upload data from flat file and subsequently view their reports.
    2.     SAP BW support team would not be involved in data upload process.
    3.     Users would not go to RSA1 and use InfoPackages & DTPs. Hence, another mechanism for data upload is required.
    4.     Users consists of two group, external and internal users. External users would not have access to SAP system. However, access via a portal is acceptable.
    What are the best practice we should adopt for this scenario?
    Thanks!

    Hi,
    I can share what we do in our project.
    We get the files from the WEB to the Application Server in path which is for this process.The file placed in the server has a naming convention based on ur project,u can name it.Everyday the same name file is placed in the server with different data.The path in the infopackage is fixed to that location in the server.After this the process chain trigers and loads the data from that particular  path which is fixed in the application server.After the load completes,a copy of file is taken as back up and deleted from that path.
    So this happens everyday.
    Rgds
    SVU123
    Edited by: svu123 on Mar 25, 2011 5:46 AM

  • What are the best data and bit rate setting for uploading from final cut express to Youtube?

    Can anyone suggest the best data rate and bit rate presets for uploading footage from final cut express 4 to Youtube? What settings will provide the best resolution, quality, and match the current youtube requirements?
    Thank you in advance for your help,
    Susan Kayne

    It depends on whether you are using aspect ratios of 4:3 or 16:9.
    Below is some simple guidance that will provide good quality with reasonably small file sizes.
    The first part is for 4:3 video:-
    1. File>Export Using QT Conversion.
    2. The "Format" window should say, "QT Movie".
    3. In "Use" select "LAN/Intranet" from the dropdown menu.
    4. Click "Save" and when it has finished encoding, upload it to YouTube.
    If you are making 16:9 video (Standard or High Definition) do steps 1 to 3 above.
    Then when you have selected "LAN/Intranet" press the "Options" button and in the new
    window that opens press the  "Size"  button and change the  "640x480" to  "853x480"
    To do this you will have to click on the  640x480 and a dropdown menu appears.
    Select "Custom" from  the bottom of the menu and in the window that opens
    you will see 2 boxes.
    Put  853  in the first box and  480  in the second.
    Click OK.
    Then Save it.

  • BW upgrade EHP1, data uploads should stop?

    Dear experts,
    we have plan for system upgrade. current system is BW 7.0 sp 17, Now planned for EHP1 and SP9.
    I know the there some post upgrade activities, which include consistancy check for objects (infoobject, transfer rules, cubes,dso, etc...)
    Could some one pls confirm, do we need to stop the data uploads/stop process chains during system upgrade?
    Thanks in advance!
    Best Regards,
    Mannu

    Hi Ingo,
    RSRT was giving proper results. We have now implemented few SAP notes and the issues got resolved.
    The following are the notes:
    1499233 - MDX:bXML flattening, unbalanced hierarchy, empty columns
    1485648 - MDX: bXML flattening and hierarchies and displaced columns
    1446245 - MDX: Error when RSR_MDX_BXML_GET_GZIP_DATA is called
    1441767 - MDX: No data for bXML if only ALL member is requested
    1438091 - MDX: basXML: Object MEASURE 0 not found
    1435844 - MDX:Wrong no. decimal places for basXML flattening interface
    1432162 - MDX: Flattening problems when using hierarchies
    1420169 - MDX: bXML flattening: Subsequent note
    1411491 - MDX: bXML flattening in transac. MDXTEST: Selecting packages
    1404328 - MDX: bXML flattening and PROPERTIES: Columns overwritten
    Thanks for your inputs.
    Regards,
    shesha.

  • Issues in mass data Upload

    Hi All,
            Hope you all are doing fine.
            I have to do master data upload for my next project. I have gone through LSMW, DX-WB, Recording etc. and now I am quite comfortable with all of these.
            As i have tested these tools for maximum 8-10 records, I am interested in hearing from you all, your experiences regarding actual data upload where volume is high and data may be difficult to verify manualy.  Particularly I am interested in how to make upload faster,Error free,consistent(No record being posted twice etc.)
            Your inputs would be higlhly appreciated.
            Thanks a lot for patient reading.
    Bye and Regards.

    Hi navdeep,
    the mass data upload depending on tht data u want to upload, there ara several function module to upload data like create reservation or upload long text inspection method tell me what the data u want to upload
    Best Regard
    Waleed Sadat

  • Data Upload using TDMS

    Hi,
    We are using TDMS for transferring the data and for scrambling as well and we need to explore if TDMS can be used for Data upload from SAP to SAP.  If yes, how this can be acheived using which TDMS functionality.
    Also, how can TDMS be integrated with SAP BODS tool?
    Thanks and best regards,

    Hello Amr,
    TDMS can't be used for data upload.
    Thanks
    Anita

  • Reg:Efficient solution for a data upload scenario

    Hi All,
            I have the following task.
             Required  data from a legacy system(generate  data only in the form of flat files)to SAP R3 as FB01 journals and the output file should be generated periodically(daily,weekly,fortnightly etc…)
    Solution Approaches:
    1)Write a BDC program to extract the data.
    2) Write a ABAP Progam to populate IDoc (if standard IDOc is available) or generate a Outbound proxy (If standard IDoc is not available) to push the data into SAP XI.
    Could anyone  tell me which would be the the best and efficient approach for this task and need your recommendations.
    Thanks in Advance.
    B.Lavanya
    Edited by: Lavanya Balanandham on Mar 31, 2008 2:23 PM

    Hi Lavanya,
    Required data from a legacy system(generate data only in the form of flat files)to SAP R3 as FB01 journals - use BDC for this thing because it will be better for large source files.
    the output file should be generated periodically(daily,weekly,fortnightly etc…)  - if this output file contains acknowledgment for the data uploaded by the above process, create a ABAP report for it and schedule it..........but if this output contains some other IDOC data which you need to send as a file to a third-party system, then go for SAP XI provided the IDOC data is not too large... but if the IDOC size is huge, then just create a ABAP report for outputting data to a file on application server and FTP the file to third-party system.
    Regards,
    Rajeev Gupta

  • More detailed documentation of the data upload page set ?

    Hallo,
    Is there more detailed documentation available of the Upload Wizard and Data Set definitions?
    In particular:
    Is there a way to skip the Data Mapping part by providing a locally pre-recorded definition of the selected columns and their format, so this step can be skipped for the end user.
    The predefined regions: "Data / Table Mapping - Data Upload Column Mapping" is not very transparent.
    The bottom line is that I would like to skip this step completely, still leveraging the built in APEX capabilities.
    Regards,
    - Thomas
    Edited by: t-o-b on 17.04.2012 08:29

    Hi Thomas,
    The data upload was designed leaving the mapping step clear and accessible to the end user, in order to make sure, that the uploading of data does not end up by upload incorrect data.
    The best way, I would advise you to match the column names of the files to be uploaded same as the column names of your table, that way the matching is done for you.
    Otherwise, in case you are trying to skip the second step, you probably dont need the wizard, instead you can just write the code that read the file and upload directly to the table.
    Regards,
    Patrick

  • Graphic & Data Upload

    Good afternoon,
    Just looking for some high level ideas. Any help you can provide will be greatly appreciated.
    I've been asked to develop a unique Apex application. Well, unique for me. :-)
    I work for a manufacturer and they need a quality inspection app created. I've been developing Apex apps for about 3 years and I know how to code up the main aspects of this application. I know what to do concerning the main data entry, etc., but I don't know how to best go about one key aspect.
    The key aspect - this app will be used on the plant floor, so they won't always be connected to the network. The quality inspectors want to have an application on a laptop that has a graphic of the component being inspected. For example, let's say it is a stove. There will be several graphics. Front, back, sides, top, interior. Then, over each graphic there will be a grid. Numbers along the top, letters down the side. If the inspector sees a flaw on the side in the E-5 location they will click a LOV that is in that section and select the flaw. The LOV will contains things like - scratch, dent, bad electrical connection, etc. You probably get the idea. Then, at some point they finish the inspection, connect to the network, and upload the information. The information uploaded will be serial number and the inspection grid information.
    I have a few main questions:
    1. Is Apex appropriate for the inspection component, particularly since the user will NOT have a persistent network connection.
    2. If Apex is the best solution for the inspection part, how do I go about doing it?
    3. If not Apex then what? Would maybe some XML/graphic solution be best? If so, how would I get that data uploaded to the main Apex app?
    I'm just brainstorming at this point. Any questions, comments, suggestions are welcome.
    Thx, Tony

    Here's some thoughts..
    1) You could use a watermark to show the image as a background and then have your controls placed accordingly.... (Layout would be interesting).
    2) XE installed on the laptop and then upload to main database application from XE when you are connected to network.
    3) Flex MIGHT be an option, again with XE as a back-end database is an option, since you want custom layout of the form with controls on the image.
    Thank you,
    Tony Miller
    Webster, TX
    Never Surrender Dreams!
    JMS
    If this question is answered, please mark the thread as closed and assign points where earned..

  • Master data uploading

    Hi friends,
    1.Please send me the prerequisites for HR Master Data Uploading?
    2.Also tell me is it mandatory to create the jobs and positions, on OM?, If no what is the option?
    since we have to use the imp. Infotype Org Assignt 0001>position field.  We have 133 positions and jobs. how to do it?
    Please reply.
    Thanks,
    milinds

    Hi Milind,
    1.Please send me the prerequisites for HR Master Data Uploading?
    >>You need to first understand what all data need to be uploaded i.e. which all infotype you will be maintaining for the master data.
    Then the second step is to identifying the fields for different infotypes, so while doing this you can create the template for each infotype. Along with this you also need to understand the source data means will it get directly uploaded or do any translation logic is required.
    After this you can create LSMW's or BDC program.
    2.Also tell me is it mandatory to create the jobs and positions, on OM?, If no what is the option?
    since we have to use the imp. Infotype Org Assignt 0001>position field. We have 133 positions and jobs. how to do it?
    >>If you want authoraizations and workflows for approvals. also if FI is integrated then its always best to have positions, organizational units, cost centers in place and the rellevant relationships between the objects.
    You can create BDC using TCODE PP02 for creating positions, Jobs and their relationships.
    Regards,
    gusd

  • KEPM Data Upload with BAPI

    Dear sirs:
    I have to build a data upload program for transaction KEPM (Edit Planning Data), using the BAPI BAPI_COPAPLANNING_POSTDATA. Does anyone have a sample.
    Thanks for your help.
    Best regards,
    Dairo

    <b>FYI</b>
    FU BAPI_COPAPLANNING_POSTDATA
    Text
    Write CO-PA Planning data
    Functionality
    You use this method to return planning data from Profitability Analysis back into the system. This overwrites existing planning data in the system (or, more precisely, the delta value between the old and new values is determined and updated as a line item).
    You typically call up this method after reading planning data with the GetData method and making changes to it. To import the changed data back into the system, you fill the parameters SELECTEDFIELDS and SELECTION with data in the same way as you called up GetData. When data is being written, these parameters determine how the delta value between the planning data to be imported and the old planning data in the system is determined: firstly, the old planning data in the system is read in accordance with the values for the parameters SELECTEDFIELDS and SELECTION, and then the delta for the data to be imported is created.
    The data to be posted is imported using the table DATA. Since this table is also used to read data with the GetData method, it should also be populated accordingly when data is imported. See also the documentation for the GetData method and for the parameter DATA for this method.
    Warning: Under certain circumstances, data that is not imported but is already present in the system as an old value are canceled in this process.
    If, for example, products A1 and A2 from plan version 001 are selected in the selection criterion SELECTION but table DATA only contains planning data for product A1, the planning data for A2 that is in version 001 and already in the system is canceled.
    Parameters
    OPERATINGCONCERN
    TYPEOFPROFITANALYSIS
    TESTRUN
    SELECTEDFIELDS
    SELECTION
    DATA
    RETURN
    Exceptions
    Function Group
    KE_BAPI_COPAPLANNING
    Thanks
    SK

  • Time management data upload

    Hi,
    I have to design a data uploading strategy for the HR infotype 2011(time management). Am new to HR.
    1. let me know in which tabe can i see the fields for this infotype.
    2. Which upload technique is best for Time management data .
    3. We are taking the upload excel from a time machine where time-in and time-out is supposed to be maintained. But suppose if a person coomes and checks-in but forgets to check-out, then his entries for chek-out wont be  uploaded into the excel but rest other will be .
      can at this time I can code in to make the check-out to be +8 (eg he comes @ 2 and no check-out time then can we give check-out time to be 10 ), just for the moment . The logic will be told laater .
      Also is it configurable by the functional team ? to induce this logic .
    Thanks in advance.

    Hi abap,
    1. let me know in which tabe can i see the fields for this infotype.
      Table is PA2011.
    2. Which upload technique is best for Time management data .
    Make a z program,
    and use HR_INFOTYPE_OPERATION
    to upload data into this infotype.
    regards,
    amit m.

  • Help on Data upload

    Hi all,
    Today Quality Server has been copied by all the data from Production Server.
    After this copying no Data upload is taking place from Quality server to BW Quality server.
    I think Now quality server has setting of Data load to BW Production server and it needs to be configured for Data load to BW Quality Server.
    Can anyone help me what changes needs to be done.
    Thanks

    Hi Saurabh,
       Check in the Development system for the corresponding entry ... First check in SM59 what are the RFC destinations... If you have authorization then create or else you may need to get your basis team involved.
    Best regards,
    Kazmi

  • Xls. sheet for Master data upload

    Hi Can any body suggest me or send me the sample, how to maintain xls. sheet Template of  particular fields for Infotypes 0,1,2,7,8 etc for the purpose of data upload.
    <removed by Moderator>
    thanks
    S Mishra

    Hi Mishra,
    You can look into the Standard Business Blueprint Templates - Data Transfer Tool of SAP....and get an idea...
    Check this Note 1060029 - SAP Best Practices for HCM US - Variants, Misc, LSMW ...
    You will find files for infotypes and you can set up your templates based on these....
    Other way is to
    Go to SE11 >> Enter the Infotype Number PNNNN (with NNNN being the Infotype)...
    It presents you with the structure.......Copy that structure and you can create your excel sheet with that template....
    Good Luck !!!!
    Kumarpal Jain.

  • Data upload for vendor balances using BDC

    hi abap experts,
    I have a requirement on data uploading using BDC.
    For the vendor balances ie. for transaction FBL1N  ( I was given a template for vendor balance upload and need to write a BDC program for that ) I need upload the exsisting transaction data to the system. is recording necessary for this?
    can u pls help me with step by step process for vendor balance uploading.
    Thanks,
    Hema.

    Hi
    Please follow the following Steps:
    Steps for recording:
    Step1: Goto TCODE SHDB
    Step2: Click on New Recording
    Step3: Give the necessary Details such as TOCDE, Desc, ...
    Step4: Do the screen by screen recording.(Please avoid extra screen to appear)
    Step5: Save the recording.
    Step6. Select the recording and click on Program button on toolbar.
    Step7: Give the Program name and click on radio button Transfer from recording.
    Step 8: It will open a new session with SE38 and a program with the recoding.
    Step 9: then just add the basic code for BDC.
    Regards,
    Lokesh

Maybe you are looking for