Unable to process huge files in SFTP BASED PROXY SERVICE

Hi,
I credted osb project which will have sftp based proxy and sftp based business service.After completing the development i tested the fie transfer through WINSCP ,but my sftp based proxy was picking the files but i am unable to see the files in the output directory for huge files,for small file it is coming after aquite long time .please assist me .

If you enable content streaming then you cant access the file/message content within the Proxy service and can not perform actions like Replace or perform transformations on the content. Use streaming for pass through scenarios only. If you want to read a large file and also perform transformations on content I would recommend using JCA Adapter for FTP and try reading records from file in batches, a few records at a time. Just so we know the requirement better, what is the file size and what is the format of the content?

Similar Messages

Trying to create a Photomerge in Bridge, when the files open in Photoshop ( small jegs ) , I get a error message of " Unable to process latte files " / " null is not an object "

Trying to create a Photomerge in Bridge, when the files open in Photoshop ( small jegs ) , I get a error message of " Unable to process latte files " / " null is not an object "
Please help

Wait a second, Photoshop can make lattes now? Personally I prefer my coffee black, but man have I been under-utilizing this program.

I am trying to restore bookmarks to Version 5 from a backup I made from Version 4.+. I get an error message "Unable to process backup file." What's wrong?

I made a backup of my bookmarks when I was running Firefox Version 4+ (stored in a "json" file on a thumb drive). My primary hard disk crashed and when I down loaded Firefox on to a new hard drive, I got version 5.0. When I tried to restore the saved bookmarks, I got an error message that says "Unable to process backup file."

A possible cause is a problem with the file places.sqlite that stores the bookmarks and the history.
* http://kb.mozillazine.org/Bookmarks_history_and_toolbar_buttons_not_working_-_Firefox
* http://kb.mozillazine.org/Unable_to_process_the_backup_file_-_Firefox

Solved "Error message": Unable to process backup file .json of bookmarks saved from different computer

Want to move bookmarks from old laptop running Firefox 3.1.6 and Win XP to new laptop running Win 7 64-bit and FF 3.1.6 which I just installed. On new laptop I browsed from FF to Restore bookmarks from file I had saved from old laptop (edit to add: it is about 4500kb). The file (bookmarks-2010-10-15.json) is NOT write protected. FF reports "Unable to process backup file".
I have shut down FF and rebooting computer. I am able to restore a bookmark file that was created on new laptop but NOT the one saved from old laptop so I do NOT think the file that the knowledge base (http://kb.mozillazine.org/Unable_to_process_the_backup_file_-_Firefox) refers to as potentially problematic is the problem in my case. I do NOT want to proceed with the other suggestions (create new profile and etc.) for fear of creating even more problems.
Appreciate your advice - I have probably a 1000 or more bookmarks to move and so this is vital and probably typical of anyone with a new computer.

PROBLEM SOLVED I did backup from old laptop again both as .json file and as an export to html. This time the .json file worked to restore from it.
I had read on a different thread here about using the html version and was going to give that a try but also made the new .json at the same time to see if I could get that to work - which it did. I expect that either approach gives the same results and so if one approach is not working for you perhaps the other approach will take care of it.

OSB10gR3:High-availbility of file protocol based proxy service

Hi All,
I just want to know the options available for creating two proxy services polling on the same location using File transport/ making a file based proxy service independent of managed servers.
Actually in OSB doc's File Transport Configuration page, it is mentioned that
Managed Server This field is available only in a clustered domain. Select the Managed Server to act as the polling server. All of the Managed Servers can process the message, but only one can poll for the message. ( [Click here|http://download.oracle.com/docs/cd/E13159_01/osb/docs10gr3/consolehelp/proxyservices.html#wp1272642] to check)
So what if the server on which proxy is deployed for polling, goes down? How can we handle such situation?
Your opinion?
Thanks & Regards,
Anuj

File like FTP and Email transports are singleton services. That is, the processing can take place in any node of the cluster, but the polling only takes place in one. The reason for this is simple, duplicate invocations. If you have 4 nodes in your cluster and they are all polling, when a new file is created, your proxy could be invoked 4 times for the same file.
You could make two proxy services and have them poll the same directory but on different nodes. If the nodes share the same file system, your business service would have to be able to deal with duplicate messages. If it's not the same file system, the application that creates the files would need to keep track of the status of the servers in the cluster to decide where to put the files.
I think the best option would be to look into weblogic's cluster features: leasing and server migration. You could configure weblogic to automatically migrate the failed server to another machine (there's also service migration, but I don't think weblogic is aware of osb level services). Of course you would need to use a file system that's available to both the "primary" and the "backup" servers. Have a look here:
http://download.oracle.com/docs/cd/E12840_01/wls/docs103/cluster/migration.html
You could also create a WLST script to modify your proxy service (change the managed server) in case of failure. There should be a way, maybe through JMX or using the WLDF, to get a notification when the server goes down so you can execute your script. Again, you need a shared file system.
Hope that helps. And if you come up with a good solution let us know!

OBPM BPM Studio support for SOAP 1.2 based Proxy services (OSB)

Hi,
I tried to import proxy services from OSB (as external resource). The proxy service is based on the SOAP 1.2 WSDL standard. When BPM studio introspective SOAP1.2 based proxy services, it is givng the following warninigs. Bacause of this i am unable to create a service object. Please help.
Regards,
Pandu
Introspecting...
downloading /sbresource?PROXY/RoutingAndMessagingService/proxy/OnRampService
parsing /sbresource?PROXY/RoutingAndMessagingService/proxy/OnRampService
[Warning] unknown tag 'header' from 'http://schemas.xmlsoap.org/wsdl/soap12/'.
[Warning] unknown tag 'header' from 'http://schemas.xmlsoap.org/wsdl/soap12/'.
checking /sbresource?PROXY/RoutingAndMessagingService/proxy/OnRampService
parsing http://localhost:7002/sbresource?PROXY%2FRoutingAndMessagingService%2Fproxy%2FOnRampService%3ASCHEMA%2FSchemas%2Finternal%2Fwsdlmessages%2Futilityservices%2Froutingandmessagingservice%2FRoute
analyzing http://localhost:7002/sbresource?PROXY%2FRoutingAndMessagingService%2Fproxy%2FOnRampService%3ASCHEMA%2FSchemas%2Finternal%2Fwsdlmessages%2Futilityservices%2Froutingandmessagingservice%2FRoute
parsing http://localhost:7002/sbresource?PROXY%2FRoutingAndMessagingService%2Fproxy%2FOnRampService%3ASCHEMA%2FSchemas%2Finternal%2Fwsdlheaders%2FHeaderSchema
analyzing http://localhost:7002/sbresource?PROXY%2FRoutingAndMessagingService%2Fproxy%2FOnRampService%3ASCHEMA%2FSchemas%2Finternal%2Fwsdlheaders%2FHeaderSchema
parsing http://localhost:7002/sbresource?PROXY%2FRoutingAndMessagingService%2Fproxy%2FOnRampService%3ASCHEMA%2FSchemas%2Finternal%2Fwsdltypes%2Fcommon%2FHeaderTypes
analyzing http://localhost:7002/sbresource?PROXY%2FRoutingAndMessagingService%2Fproxy%2FOnRampService%3ASCHEMA%2FSchemas%2Finternal%2Fwsdltypes%2Fcommon%2FHeaderTypes
parsing http://localhost:7002/sbresource?PROXY%2FRoutingAndMessagingService%2Fproxy%2FOnRampService%3ASCHEMA%2FSchemas%2Finternal%2Fwsdltypes%2Fcommon%2FSimpleTypes
analyzing http://localhost:7002/sbresource?PROXY%2FRoutingAndMessagingService%2Fproxy%2FOnRampService%3ASCHEMA%2FSchemas%2Finternal%2Fwsdltypes%2Fcommon%2FSimpleTypes
http://www.michaelpage.com/wsdltypes/common/SimpleTypesv1.0 compiled
http://www.michaelpage.com/wsdltypes/common/HeaderTypesv1.0 compiled
http://www.michaelpage.com/wsdlheaders/Headerv1.0 compiled
http://www.michaelpage.com/wsdlmessages/utilityservices/Routev1.0 compiled
[Warning] The binding 'OnRampServiceSOAP' with type 's2:OnRampServicePort'is unsupported. Components will not be generaded for this binding..
'http://www.michaelpage.com/wsdl/utilityservices/routingandmessagingservice/OnRampServicev1.0/' was parsed.
Analyzing dependecies for module 'WS.OnRampService'...
Analyzing dependecies for module 'WS.OnRampService'...
Analyzing Components
Storing Instrospected Types
Introspection completed. Errors: 0, Warnings: 3.

Pandu,
OBPM 10g doesn't support SOAP 1.2 binding.
Edited by: 4dyomi on Nov 20, 2009 11:41 AM
Edited by: 4dyomi on Nov 20, 2009 11:42 AM

Unable to process flat-files without delimiters in ODI 11.1.1.7

Hi,
We have a requirement to process a flat file using ODI 11.1.1.7 (installed on Windows and mounted on Oracle 11g Database). The flat-file is a normal ASCII file coming from mainframes machine and has no delimiters (column as well as rows). The first 48 characters is ROW 1, from character 49 to 96 is ROW 2 and so on. I am unable to create a DataStore for this file in ODI. If I reverse engineer using the file RKM, it is creating a datastore with over 2000 columns which is not the case (actual file layout given below). The absence of a row delimiter is making ODI think that the entire file has only one single row.
I am also unable to create a DataStore manually. If I am not providing any value for the "row delimiter" it is throwing me an error. However, I am able to process this file using SQL*Loader (there is a FIX option available where we can specify the length of a row).
Is there a way to do it from within ODI? I tried to modify a KM to create a control file for SQL*Loader and execute it but the control file is not getting generated as expected.
File Layout:
Column 1 - Warrant Number (Position 1 - 9) - Number
Column 2 - Type (Position 10-10) - Number
Column 3 - Warrant Amount (Position 11-18) - Packed Decimal Signed
Column 4 - Issue Fund (Position 19-22) - String
Column 5 - Issue Sub Fund (Position 23-24) - String
Column 6 - Filler (Position 25-48) - String
Thanks
Srivatsan P

Hi LuizFilipe,
I tried also with your method but i am getting below error when i am trying to view data.
See com.borland.dx.dataset.DataSetException error code: BASE+62
com.borland.dx.dataset.DataSetException: Execution of query failed.
Chained exception:
java.sql.SQLException: ODI-40439: Could not read heading rows from file
at com.sunopsis.jdbc.driver.file.FileResultSet.<init>(FileResultSet.java:164)
at com.sunopsis.jdbc.driver.file.impl.commands.CommandSelect.execute(CommandSelect.java:57)
at com.sunopsis.jdbc.driver.file.CommandExecutor.executeCommand(CommandExecutor.java:33)
at com.sunopsis.jdbc.driver.file.FilePreparedStatement.executeQuery(FilePreparedStatement.java:135)
at com.borland.dx.sql.dataset.o.f(Unknown Source)
at com.borland.dx.sql.dataset.QueryProvider.e(Unknown Source)
at com.borland.dx.sql.dataset.JdbcProvider.provideData(Unknown Source)
at com.borland.dx.dataset.StorageDataSet.refresh(Unknown Source)
at com.borland.dx.sql.dataset.QueryDataSet.refresh(Unknown Source)
at com.sunopsis.graphical.frame.edit.AbstractEditFrameGridBorland.initialize(AbstractEditFrameGridBorland.java:628)
at com.sunopsis.graphical.frame.edit.AbstractEditFrameGridBorland.<init>(AbstractEditFrameGridBorland.java:869)
at com.sunopsis.graphical.frame.edit.EditFrameTableData.<init>(EditFrameTableData.java:50)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at oracle.odi.ui.editor.AbstractOdiEditor$1.run(AbstractOdiEditor.java:176)
at oracle.ide.dialogs.ProgressBar.run(ProgressBar.java:656)
at java.lang.Thread.run(Thread.java:662)
Well i created file datastore with selected below things
File format : fixed
Heading(number of lines) : 0
field separator : null
record separator : by default --> MS-DOS
text delimiter: (blank)
decimal separator : (blank)
then when i did reverse engineering, able to see the row of data and then i separate the data with c1,c2,c3,c4
please let me know did i miss something.
Thanks
Himanshu

How to process huge files using File Adapter without ESR and no FCC?

Hi All,
I came to a scenario where i need to pick the file from R/3 sys which of around 500MB and need to transport it to the Third party sys using PI 7.1 . This is just simply a pass through data i.e.., no mapping is required (no ESR). and i don't to make use of FCC or BPM
My question is can the File Adapter can pick the file which is of 500MB size?
If not then what would be best solution to do it?
Do i need to divide the file into smaller sizes say around 100MB a file each in R/3 sys and then pick those files .
can i do without dividing the files in the R/3 sys?
Thanks in advance.
Sai

Hi Bhaskar,
If you zip probably the size would come less which may increase his performance a little. But I feel SAP also dont recommend this much load with PI.
If I am in his shoes then I will do a simple FTP without using PI itself. Else will look into other options. Thats my two cents.
Hope this alexs blog will give an idea to Sai:
/people/alessandro.guarneri/blog/2007/02/21/sap-xi-acting-as-a-huge-file-mover
Regards,
---Satish
Edited by: Satish Reddy on Jul 20, 2011 3:02 PM

Approach needed for processing huge file.using file-jdbc

Hi,
My Scenario is file-jdbc. I need to update the records in database table. The size of file would be 500mb.
1)Will the recordsets per message in the sender side FCC will help me in processing the file? Any other better solution or any configuration needs to be checked on PI system? as processing should be finished in 3 hours time?
2)i need to update other table with no.of records processed per cycle with time. How to acheive this?
TIA

I would suggest you to go for Stored procedures and follow as mentioned below:
1)Use recordsets per message in the sender side FCC (already mentioned by u)
2) Create a SP and and pass source payload as xml input to the SP.
Ur target structure will be something like this:
http://help.sap.com/saphelp_nw04/helpdata/en/2e/96fd3f2d14e869e10000000a155106/content.htm
<StatementName>
<storedProcedureName action=u201D EXECUTEu201D>
<table>realStoredProcedureeName</table>
<param1 type=SQLDatatype>val1</param1>
</storedProcedureName >
</StatementName>
pass ur source data as xml input to param1.
If u r using PI7.1 then chk this:
http://www.sappi.sapag.co.in/flat-file-to-file-senario/convert-the-input-xml-to-string-in-pi-7-1-using-standard-graphical-mapping-2/
on the database side parse the string (xml document) and then insert the data in the table.. Chk with ur DB team regarding the same.

Using relative-URI in creating the REST based proxy service

Hi,
I am facing an issue when i am trying to create REST based proxy. i am looking at the urls given in the forum post Using Business Service that supports XML over HTTP in OSB
I have created a conditional branch with xpath expression like *./ctx:transport/ctx:request/http:relative-URI/text()*
and i am copying this value to $inbound as showin in the blog http://blogs.oracle.com/jeffdavies/2009/06/restful_services_with_oracle_s_1.html. But when i typed the uri in browser its saying xml parser error unable to find <.
Not sure of the reason. If i modify my proxy to use http-method as shown in the blog its working fine and that too only when i copied ./ctx:transport/ctx:request/http:http-metho/text() to $inbound. But when i copied the same to another variable like branchcond, i am getting same parsing error. And Is it that the variable has to be always $inbound? So what should be the variable name for using relativeURI
And my uri is going to be like the following, thats why i am planning to create the conditonal branch on relative-uri.
http:/localhost:8001/proxy/RESTbasedPS/getGreeting?name=23
http:/localhost:8001/proxy/RESTbasedPS/getFullName?firstName=Some&lastName=Thing
Any leads in above error will help me.
Thanks
Siva

When i created the Conditonal branch, its asking for two mandatory properties. One is 'selected path' and the other is 'variable'. My scenario i am trying to base my conditional branch based on relative uri, so i have given './ctx:transport/ctx:request/http:relative-URI/text()' for 'selected path' variable and giving the value 'inbound' for 'variable' property of conditional branch.
i was trying to do this in similar fashion as given in the blog http://blogs.oracle.com/jeffdavies/2009/06/restful_services_with_oracle_s_1.html, only thing is that i am using http:relative-URI rather http:http-method as shown in the blog.
I am geting the following error in exact:
XML Parsing Error: no element found
Location: http://localhost:7001/proxy/RESTBasedPS/getGreeting?name=siva
Line Number 1, Column 1:
Please let me know if i am missing something or doing wrong with inbound variable.
Thanks
Siva

OSB - how to handle poll file error in FTP Proxy Service

I configed a proxy service to poll file from a FTP server.
How can I capture the error when the proxy service cannnot poll? (FTP server is down, folder not found, username invalid, etc )
I want to make it show in the Operations >Message Reports table.

It is not possible. Those errors will only be logged to logfile.
Manoj

Aperture Unable to process RAW files even though finder shows a preview

I have recently upgraded to Aperture 3.0 which decided to delete my library. Using PhotoRec i am slowly recovering all of my files.
The files that have been recovered are .cr2 files and .jpg files. The jpegs display fine without a problem however every 2 out of 3 cr2 RAW files will not import into iPhoto, Aperture or pretty much any other RAW processing software. Aperture just says "Unsupported File Type".
My issue is that Finder is showing a preview of the image in the icon, and is showing a preview in CoverFlow, but everything refuses to open the file. This must surley mean that the data is still intact within the .cr2 file? else Finder would be unable to display the preview of the file?
Does anyone know how to fix this or what could be causing this issue?
Apple please, please fix this issue with Aperture, I fear that I have lost a lot of photos in this process.
Please help!

Aarthek - the Canon PowerShot G2 is not supported centrally on your Mac.
http://www.apple.com/aperture/specs/raw.html
I've just finished importing my old archive produced by this camera. What I did was to produce fresh JPEGs (medium quality, so low file size) in the same directory as the original RAW images. I happen to have Photoshop CS3 and could easily do these in batches, but worst case, you should still have Canon's ImageBrowser suite which could do this. I then imported the JPGs for cataloguing, keywording, etc. Then, if I want to do some serious work on an image from my G2, I'll right-click the image and select +Show in Finder+ and get straight to my corresponding RAW image which I can then process in Photoshop.
but I don't understand why the thumbnail shows up when I'm importing as though there are no issues.
There's a small embedded JPG in the RAW files and that's what you're seeing. I think that's also what you saw on the G2's display when you were out shooting and stopped to review an image in the field.
Regards,
Gary

Lost bookmarks, get msg. 'unable to process backup file. Also, cannot save new bookmarks

Running Firefox on Windows 7 64kb. Bookmarks disappeared. Can find backups but not recognized.

A possible cause is a problem with the file places.sqlite that stores the bookmarks and the history.
*http://kb.mozillazine.org/Bookmarks_history_and_toolbar_buttons_not_working_-_Firefox

When we should use JMS based proxy services and when JMS Adapter in OSB

Hi,
Can somebody explain me when we could go for JMS Adapter and when we should go for JMS based service in OSB?
The one scenerio i can think of is when my provider/consumer is using jms provider other than WLS jms provider we should use jms adapter.
Edited by: user12883209 on Jan 5, 2012 5:29 AM

I don't think there are clear guidelines available, however here is my view.
1) OSB 11gR1 is not certified with Oracle JMS Adapter. It is certified with Oracle FTP, Database, AQ and application adapters.
http://docs.oracle.com/cd/E21764_01/doc.1111/e15867/interop.htm#OSBAG1403
2) In addition of the Weblogic JMS, OSB can also integrate with following provider without using adapter.
- WebSphere MQ
- Tibco EMS
3) if you need to integrate JMS with BPEL Process or Oracle SOA suite composite, it has built in support for generic Oracle JMS adapter. Oracle generic JMS adapter can integrate with AQ JMS (JMS providers OJMS 8.1.7, 9.0.1.4, and 9.2), TIBCO JMS, IBM Websphere MQSeries (IBM MQSeries JMS 6.0), Weblogic JMS, Apache, and Active MQ
http://docs.oracle.com/cd/E14571_01/integration.1111/e10231/adptr_jms.htm
4) If you need to integrate OSB with Weblogic JMS, WebSphere MQ, Tbco EMS it does not need JMS adapter.
Hope this helps !!!
Jayesh Patel
http://jayesh-patel.blogspot.com/
http://www.yagnasys.com/

Processing requests sequentially on an OSB proxy Service

Dear Gurus,
I have a requirement,in which I am polling message in OSB from an MQ,I want to achieve the sequential processing of messages in OSB.is it possible in OSB?

UOO will be useful for sequential processing after the JMS queue. To maintain sequential integrity before JMS Queue (processing in the proxy which polls from MQ and the business service which puts the polled message to local JMS queue) you will need to ensure a single thread model. The MQ Polling proxy will need to be deployed on a single managed server with a work manager which uses a max thread constraint of 1.
You can also check with MQ folks if there is an implementation on MQ side using which MQ will only release messages one by one. By default it will publish two messages together if there are two listener threads.

Unable to process huge files in SFTP BASED PROXY SERVICE

Similar Messages

Maybe you are looking for