Problem when uploading a large file in PI - weird SQL I/O errors

Hi guys,
I'm facing a very difficult problem when uploading a 35 MB with an FTPs adapter. I see in the logs that, after the translation to XML, it's going to 170 MB.
I receive the following error in the CC Monitoring:
Error: com.sap.aii.af.ra.ms.api.DeliveryException: Problem inserting 41827ca7-6b8c-4a87-198d-ad8a81fcb12b(OUTBOUND) into the database: com.sap.engine.services.dbpool.exceptions.BaseSQLException: Connection is invalid.
When I look in the NWA Monitoring, I see the following details:
SQL error occurred on connection affhb201:X11:SAPSR3DB: code=17,002, state="null", message="Io exception: Socket closed";
SQL statement is "INSERT INTO "XI_AF_MSG" ("MSG_ID","DIRECTION","MSG_BYTES","TIMES_FAILED","SENT_RECV_TIME","STATUS","CONN_NAME","MSG_TYPE","REF_TO_MSG_ID","ADDRESS","TRANSPORT","CREDENTIAL","TRAN_HEADER","MSG_PROFILE","CONVERSATION_ID","SCHEDULE_TIME","PERSIST_UNTIL","FROM_P........
I cannot check the Visual Admin Logs 'cause I don't have access to them yet.
I'm pretty convinced that some swap memory, message size or whatever setting on the adapter engine or on the Java stack is preventing this. I do not get any message in CC Monitoring when uploading a smaller, 6 MB version of the same file.
Can you please help me solve this or give me some interesting pointers?
Never before did we experience something like this in the PI system. In addition, I didn't find any useful resource on the SDN and throughout the SAP notes for this.
Let me know if you need more info about this.
Best regards,
George

Hi George
I am facing the same issue, where did you configure the message split in the Communication Channel?
If I do the message split as you said, is it going to create several files or how does it work?
Thanks in advanced
Emmanuel

Similar Messages

How to avoid the split problem when uploading the data from csv file

Dear Friends,
              I have to upload data from the .csv file to my custom table , i have found a problem when uploading the data .
i am using the code as below , please suggest me what i have to do in this regard
      SPLIT wa_raw_csv AT ',' INTO
                wa_empdata_csv-status
                 wa_empdata_csv-userid
                 wa_empdata_csv-username
                 wa_empdata_csv-Title
                 wa_empdata_csv-department.
APPEND wa_empdata_csv TO itab.
in the flat file i can see for one of the record for the field Title as
Director, Finance - NAR............there by through my code the wa_empdata_csv-Title is getting splited data as "Director, and Department field is getting Finance - NAR" , i can see that even though " Director, Finance - NAR" is one word it is getting split .
.......which is the problem iam facing.Please could any body let me know how in this case i should handle in my code that this word
"Director,Finance - NAR"   wil not be split into two words.
Thanks & Records
Madhuri

Hi Madhuri,
Best way to avoid such problem is to use TAB delimited file instead of comma separated data. Generally TAB does not appear in data.
If you are generating the file, use tab instead of comma.
If you cannot modify the format of file and data length in file is fixed character, you will need to define the structure and then move data in fixed length structure.
Regards,
Mohaiyuddin

Upload very large file to SharePoint Online

Hi,
I tried uploading very large files to SharePoint online via the REST API using JAVA. Uploading files with a size of ~160 mb works well, but if the filesize is between 400mb and 1,6 gb I either receive a Socket Exception (Connection reset by peer) or an error
of SharePoint telling me "the security validation for the page is invalid".
So first of all I want to ask you, how to work around the security validation error. As far as I understand the token, which I have added to my X-RequestDigest header of the http post request seems to be expired (by default it expires after 1800 seconds).
Uploading such large files is time consuming, so I can't change the token in my header while uploading the file. How can I extend the expiration time of the token (if it is possible)? Could it help to send post requests with the same token
continuously while uploading the file and therefore prevent the token from expiration? Is there any other strategy to upload such large files which need much more than 1800 seconds for upload?
Additionally any thoughts on the socket exception? It happens quite sporadically. So sometimes it happens after uploading 100 mb, and the other time I have already uploaded 1 gb.
Thanks in advance!

Hi,
Thanks for the reply, the reason I'm looking into this is so users can migrate there files to SharePoint online/Office 365. The max file limit is 2gb, so thought I would try and cover this.
I've looked into that link before, and when I try and use those end points ie StartUpload I get the following response
{"error":{"code":"-1, System.NotImplementedException","message":{"lang":"en-US","value":"The
method or operation is not implemented."}}}
I can only presume is that these endpoints have not been implemented yet, even though the documentation says they only available for Office 365 which is what I am using.
Also, the other strange thing is that I can actually upload a file of 512mb to the server as I can see it in the web UI. The problem I am experiencing is get a response, it hangs on this line until the TimeOut is reached.
WebResponse wresp = wreq.GetResponse();
I've tried flushing and closing the stream, sendchunked but still no success

Itunes 10.6.1.7 problem: when I change the file "media type" from 'Music' to 'Podcast' the file disapears from ITUNES. I do this via (1) right click, (2) select 'Get Info', (3) select 'options' tab, and (4) change media type. What is the problem?

Itunes 10.6.1.7 problem: when I change the file "media type" from 'Music' to 'Podcast' the file disapears from ITUNES. I do this via (1) right click, (2) select 'Get Info', (3) select 'options' tab, and (4) change media type. What is the problem?

Hi Memalyn
Essentially, the bare issue is that you have a 500GB hard drive with only 10GB free. That is not sufficient to run the system properly. The two options you have are to move/remove files to another location, or to install a larger hard drive (eg 2TB). Drive space has nothing to do with SMC firmware, and usually large media files are to blame.
My first recommendation is this: download and run the free OmniDiskSweeper. This will identify the exact size of all your folders - you can drill down into the subfolders and figure out where your largest culprits are. For example, you might find that your Pictures folder contains both an iPhoto Library and copies that you've brought in from a camera but are outside the iPhoto Library structure. Or perhaps you have a lot of purchased video content in iTunes.
If you find files that you KNOW you do not need, you can delete them. Don't delete them just because you have a backup, since if the backup fails, you will lose all your copies.
Don't worry about "cleaners" for now - they don't save much space and can actually cause problems. Deal with the large file situation first and see how you get on.
Let us know what you find out, and if you manage to get your space back.
Matt

Upload a large file to OCS10g workspace

Hi,
I have a workspace on OCS10g and the quota of the workspace is 2GB.
But, when I upload a large file - about 100 MB, it occurs errors.
Undoubtedly, it works well in the case of small files.
Already, I set the parameter IFS.DOMAIN.MEDIA.CONTENTTRANSFER.ContentLimit to 0.
It means an unlimited permission to upload a workspace.
Is there any other prerequisite to upload a large file to a workspace?
Regards,
Kitae

Have you checked the Apache Timeout directive? It might need to be larger. There are no other general tricks or parameters to tweak that our dev team is aware of. Please file a metalink tar for this and detail the error message. I'm sure Oracle Support Services will be able to help you mine the logs to find out what is going on.

When downloading a large file, like a movie Internet Explorer automatically provides one with the choice of where to save it - e.g. "C" drive, but Firefox doesn't. How can it be done?

When downloading a large file, like a movie, Internet Explorer automatically opens a download window offering the choice to "Open" or "Save"the file. This provides one with the choice of where to save it - e.g. "C" drive, in the "My Documents" or "Desktop" folders but Firefox's download window doesn't. This has always frustrated me because I would rather use Firefox exclusively to access the Internet, but when it comes to saving files downloaded off the Internet, sadly, I have to revert to IE!

If you click on Firefox from the upper-left, then Options, a new window should appear. On that window, click on the general tab. In the middle of the window, you'll see options regarding your downloads, one of which says, "Always ask me where to save files." Click on the bubble for this options, then click Ok. From then on, you should always be prompted on where you want your files saved.

Problem when upload legacy data

Hi all,
I've faced a problem when uploading asset legacy data (AS91) for customer. My customer have started depreciation on May 21, 2009, and I set transfer date is November 30, 2010. So SAP doesn't calculate right planned value for December, 2010. I think that problem related to the first month depreciation. It calculated full month for depreciation (instead of from May 21) and change the planned value for December 2010. So I want SAP just calculate 11 days depreciation for May 2009. How can I config to do this?
Thanks in advance!

Hi,
Check in the asset master Opening Start Date for depreciation.. If it is under open mode change that date and see asset explorer for depreciation values.
Thanks,
Padmaja

Problem with saving a pdf file to computer. Continually get an error message " This document could not be saved. There was a problem reading this document (21).

Need advice on a saving file issue. I'm having problem with saving a .pdf file to computer. Continually get an error message " This document could not be saved. There was a problem reading this document (21). This is new as this error message just recently started to pop-up.

More information about this issue can be found here:
https://forums.adobe.com/thread/1672655
A "quick" fix that worked for me was to uninstall Adobe... then download the base install for Adobe Reader 11.0.
Then download each of the individual updates and run them sequentially.
I've installed back up to the last security update which is version 08 and have been able to do normal Save As operations.
You will have to disable automatic updates in order to stay at version 08 until Adobe resolves this issue in a later release.
http://www.adobe.com/support/downloads/product.jsp?product=10&platform=Windows
Adobe Reader 11.0 - Multilingual (MUI) installer    AdbeRdr11000_mui_Std
Adobe Reader 11.0.01 update - Multilingual (MUI) installer    AdbeRdrUpd11001_MUI.msp
Adobe Reader 11.0.02 update - All languages    AdbeRdrSecUpd11002.msp
Adobe Reader 11.0.03 update - Multilingual (MUI) installer    AdbeRdrUpd11003_MUI.msp
Adobe Reader 11.0.04 update - Multilingual (MUI) installer    AdbeRdrUpd11004_MUI.msp
Adobe Reader 11.0.05 security update - All languages    AdbeRdrSecUpd11005.msp
Adobe Reader 11.0.06 update - Multilingual (MUI) installer    AdbeRdrUpd11006_MUI.msp
Adobe Reader 11.0.07 update - Multilingual (MUI) installer    AdbeRdrUpd11007_MUI.msp
Adobe Reader 11.0.08 security update - All languages    AdbeRdrSecUpd11008.msp

I have a problem when ı want to update and restore my ipad for ios5 : an error occurred while restoring this ipad (-10), what can i do??

i have a problem when ı want to update and restore my ipad for ios5 : an error occurred while restoring this ipad (-10), what can i do??

Error -1 is not mentioned in this article about error messages during a restore process: http://support.apple.com/kb/TS3694
This one comes close:
The device could not be restored. An internal error occurred, or Error 3200: This indicates a network-connectivity or traffic issue. If you see this error, wait an hour or more and try again.
You may also check your security software settings, to make sure that iTunes can contact Apple servers during this process: iTunes for Windows: Troubleshooting security software issues

When opening a msword file wit .doc extension i get an error message: import error. The file format is not valid

when opening a msword file with .doc extension i get an error message: import error. The file format is not valid

Looks like. I tried in the TextEdit format tab to set all the combinations while opening the file, like plain text, rtf and html without success. Always I get the same signs.

I deal with a itunes update problem when i try to update i get coneection has timed out error! i had the same problem when i had vista.. yesterday i formated my pc but i still have the same prommblem. i dont deal with such problems on direct downloads.

i deal with a itunes update problem when i try to update i get coneection has timed out error! i had the same problem when i had vista.. yesterday i formated my pc but i still have the same prommblem. i dont deal with such problems on direct downloads.i have this problem only with tunes. maybe my conection freezes for a sec idk

Sorry I gave this a 'this helped me' but after a long struggle I found some ticked box somewhere from some unauthorised chrome extension blaaa....... the link you gave was great, not only did it confirm all the bad things google is supposed to be up to it provided a better browser. Cheers man.
Anyone who find this; I disabled all extension in chrome. I then searched the word proxy in the settings section. A button came up, "change proxy settings" I clicked on it. Earlier it was greyed out as an extension was interefering.. It brought up a box with my own settings from my computer. Then I clicked LAN settings, then unchecked the 'Use a proxy server' button.
P.S. If you're using chrome, bail, and dowload Komodo Dragon. It's virtually identical, but you're not tracked, and you can use chrome extensions, and you can quickily transfer your chrome preferences into it.

A strange problem when uploading a file in Struts

When i upload a file, the other elements in the form of the jsp cannot be got by the ActionForm, instead, the getters show that they are "null"s. But when i upload no file, everthing is ok.
Moreover, the problem happens when i update an article which can have an picture with it, but when i add a new article, the problem does not appear, instead, the file can be uploaded and other element can also be got.
i think i might be a problem of the volume of the request , because i put more information to update than to add.
Hope someboby can help me with the problem.

When i upload a file, the other elements in the form of the jsp cannot be got by the ActionForm, instead, the getters show that they are "null"s. But when i upload no file, everthing is ok.
Moreover, the problem happens when i update an article which can have an picture with it, but when i add a new article, the problem does not appear, instead, the file can be uploaded and other element can also be got.
i think i might be a problem of the volume of the request , because i put more information to update than to add.
Hope someboby can help me with the problem.

Uploading Very Large Files via HTTP

I am developing some classes that must upload files to a web server via HTTP and multipart/form-data. I am using Apache's Tomcat FileUpload library contained within the commons-fileupload-1.0.jar file on the server side. My code fails on large files or large quantities of small files because of the memory restriction of the VM. For example when uploading a 429 MB file I get this exception:
java.lang.OutOfMemoryError
Exception in thread "main"I have never been successful in uploading, regardless of the server-side component, more than ~30 MB.
In a production environment I cannot alter the clients VM memory setting, so I must code my client classes to handle such cases.
How can this be done in Java? This is the method that reads in a selected file and immediately writes it upon the output stream to the web resource as referenced by bufferedOutputStream:
private void write(File file) throws IOException {
byte[] buffer = new byte[bufferSize];
BufferedInputStream fileInputStream = new BufferedInputStream(new FileInputStream(file));
// read in the file
if (file.isFile()) {
    System.out.print("----- " + file.getName() + " -----");
    while (fileInputStream.available() > 0) {
      if (fileInputStream.available() >= 0 &&
          fileInputStream.available() < bufferSize) {
        buffer = new byte[fileInputStream.available()];
      fileInputStream.read(buffer, 0, buffer.length);
      bufferedOutputStream.write(buffer);
      bufferedOutputStream.flush();
    // close the files input stream
    try {
      fileInputStream.close();
    } catch (IOException ignored) {
      fileInputStream = null;
else {
    // do nothing for now
}The problem is, the entire file, and any subsequent files being read in, are all being packed onto the output stream and don't begin actually moving until close() is called. Eventually the VM gives way.
I require my client code to behave no different than the typcial web browser when uploading or downloading a file via HTTP. I know of several commercial applets that can do this, why can't I? Can someone please educate me or at least point me to a useful resource?
Thank you,
Henryiv

Are you guys suggesting that the failures I'm
experiencing in my client code is a direct result of
the web resource's (servlet) caching of my request
(files)? Because the exception that I am catching is
on the client machine and is not generated by the web
server.
trumpetinc, your last statement intrigues me. It
sounds as if you are suggesting having the client code
and the servlet code open sockets and talk directly
with one another. I don't think out customers would
like that too much.Answering your first question:
Your original post made it sound like the server is running out of memory. Is the out of memory error happening in your client code???
If so, then the code you provided is a bit confusing - you don't tell us where you are getting the bufferedOutputStream - I guess I'll just assume that it is a properly configured member variable.
OK - so now, on to what is actually causing your problem:
You are sending the stream in a very odd way. I highly suspect that your call to
buffer = new byte[fileInputStream.available()];is resulting in a massive buffer (fileInputStream.available() probably just returns the size of the file).
This is what is causing your out of memory problem.
The proper way to send a stream is as follows:
     static public void sendStream(InputStream is, OutputStream os, int bufsize)
                 throws IOException {
          byte[] buf = new byte[bufsize];
          int n;
          while ((n = is.read(buf)) > 0) {
               os.write(buf, 0, n);
     static public void sendStream(InputStream is, OutputStream os)
                 throws IOException {
          sendStream(is, os, 2048);
     }the simple implementation with the hard coded 2048 buffer size is fine for almost any situation.
Note that in your code, you are allocating a new buffer every time through your loop. The purpose of a buffer is to have a block of memory allocated that you then move data into and out of.
Answering your second question:
No - actually, I'm suggesting that you use an HTTPUrlConnection to connect to your servlet directly - no need for special sockets or ports, or even custom protocols.
Just emulate what your browser does, but do it in the applet instead.
There's nothing that says that you can't send a large payload to an http servlet without multi-part mime encoding it. It's just that is what browsers do when uploading a file using a standard HTML form tag.
I can't see that a customer would have anything to say on the matter at all - you are using standard ports and standard communication protocols... Unless you are not in control of the server side implementation, and they've already dictated that you will mime-encode the upload. If that is the case, and they are really supporting uploads of huge files like this, then their architect should be encouraged to think of a more efficient upload mechanism (like the one I describe) that does NOT mime encode the file contents.
- K

Problem in uploading an excel file in wd abap

I am trying to upload an excel file in wd abap but on uploading it shows me the data in the form of some codes but when i try to do the same thing using text file it works well.Plese refer the code .
METHOD onactionon_upload .
TYPES : BEGIN OF itab_str1,
   name(10) TYPE c,
   age(10) TYPE c,
   END OF itab_str1.
DATA : t_table1 TYPE STANDARD TABLE OF itab_str1,
        i_data TYPE STANDARD TABLE OF string,
        lo_nd_sflight TYPE REF TO if_wd_context_node,
        lo_el_sflight TYPE REF TO if_wd_context_element,
        l_string TYPE string,
        fs_table TYPE itab_str1,
        l_xstring TYPE xstring,
        fields TYPE string_table,
        lv_field TYPE string.
DATA : t_table TYPE if_main=>elements_data_tab,
        data_table TYPE if_main=>elements_data_tab.
wd_context->get_attribute(
EXPORTING
    name = 'DATASOURCE'
    IMPORTING
      value = l_xstring
CALL FUNCTION 'HR_KR_XSTRING_TO_STRING'
EXPORTING
    in_xstring          = l_xstring
IMPORTING
   out_string          = l_string.
SPLIT l_string AT cl_abap_char_utilities=>newline INTO TABLE i_data.
LOOP AT i_data INTO l_string.
SPLIT l_string AT cl_abap_char_utilities=>horizontal_tab INTO TABLE fields.
READ TABLE fields INTO lv_field index 1.
fs_table-name = lv_field.
READ TABLE fields INTO lv_field index 2.
fs_table-age = lv_field.
APPEND fs_table TO t_table1.
ENDLOOP.
lo_nd_sflight = wd_context->get_child_node( 'DATA_TAB').
lo_nd_sflight->bind_table( t_table1 ).
ENDMETHOD.

Problem is not with the file format nor Upload element.
problem is with conversion function module, still now for conversion of xcel data to string format we dont have perfect function module.
CALL FUNCTION 'HR_KR_XSTRING_TO_STRING'
EXPORTING
in_xstring = l_xstring
IMPORTING
out_string = l_string.
That's a bug with xcel conversion As of now I dont find any fm which converts xcel data to string without any problem.
Regards
Srinvivas
Edited by: sanasrinivas on Dec 23, 2011 7:02 AM

Error when Uploading _fmb.xml files

Hi...
I'm a newbie in Apex. I'm trying to create a migration project (ie migrating Oracle Forms 6i/10g to xml via Apex). I've converted the *.fmb files to _fmb.xml using frmf2xml.bat utility. My problem is when I tried to upload the _fmb.xml, this error appears:
+1 error has occurred+
+•Select the Forms File you want to upload. For Forms Modules, upload the XML file which is generated by the Forms2XML conversion tool, containing the information of your Forms Module. (Go to error)+
the error message is - javascript:apex.jQuery('#P3_FORMS_XML_FILE').focus();void(0)
the url returns: http://localhost:8080/apex/wwv_flow.accept
I've also tried to upload any type of file but it still returns the same error message. Please, anyone can help me...
Thanks.

when i tried the online workspace as in http://apex.oracle.com/pls/apex/, i successfully created the migration project. But still failed to do it locally. Obviosly my locally setup Apex is having problem uploading the _fmb.xml files. Where have i gone wrong? Could anyone please help me?

Problem when uploading a large file in PI - weird SQL I/O errors

Similar Messages

Maybe you are looking for