How to do Bulk data transfer using Web Service
In my application I have to write various web services but majority of the web service has to
query database and return back bulk data(rows>10K) through web service.
So I would like to ask what is the efficient way of transferring bulk data using web service as presently
Iam returning the dataset as xml String (using StringBuilder) from web service and consuming the same at client end.
Is there a better way to this in web service?
My env:
Front end can be in any other technology ,UI like C#
Back end : Tomcat 6 on Java 6 with Axis2
Thanks in advance
Innova wrote:
But then also I have to mention a return type although its xml that is getting transferred.
Can you provide me a with a small sample code.Like if I have Emp obect with properties
Empname,EmpID,DOJ,DOB,Dpt . In this case what will be the return type.
My major concern is that my resultset is going to be huge in terms of >10,000 rows so
that much time for formation of xml and then the transfer so how can I reduce the transfer
time ,I meant a faster optimised approach for transferring large data as part of web service.
Simply putting , how can I transfer bulk data in minimum time so that I can reduce the
waiting time at client side for the response.
Thanks in advanceI've done this with thousands before now, and not had a performance problem...at least nothing worth writing home about.
Your return type will be an array of Emp objects. You can use a SOAP array for that (you'll need to look that up for the syntax, since I can't remember it off the top of my head), which gets restricted to being an array of Emp.
Should the normal technique prove to be too slow then you should look at alternatives, but I would expect this to work well enough. And you have no faffing about with devising a new transfer system that then has to be explained to your client system...it'll all be standard.
Similar Messages
-
How can i records with date format using web services?
Hello
I can't record date records using web services. I get no message errors.
I can import string values but no dates (YYYY-MM-DD). Do you have any clue about that?
Regards
Arturohello,
That's the code I'm using to update an opportunity. In the date fields (e.g. dFecha_de_entrega_al_cliente) I've tried to put an specific date in the correct format (If i put it in another format i've got an error message due the wrong format). The CRM accepted the code but it didn't update the values that are different of string.
I don´t know if there is something missing in teh program or if the developer environment is not the adequate.
Regards for your comments
Arturo
Private Sub ActualizarOportunidad(ByVal fila As Data.DataRow, ByVal TipoPersona As String)
Dim oLog As New Log()
Dim IdLog As Integer
Dim NumSerie As String = ""
Try
oLog.Insert_Log("Activación Garantía - Crear Oportunidad", oLog.GetLastIdProceso())
IdLog = oLog.GetLastId()
Dim sr_input As Opportunity.OpportunityUpdate_Input
Dim sr_output As Opportunity.OpportunityUpdate_Output
sr_input = New Opportunity.OpportunityUpdate_Input
Dim sr(1) As Opportunity.OpportunityData
sr(0) = New Opportunity.OpportunityData
NumSerie = fila("NumeroSerie").ToString().Trim()
sr(0).ExternalSystemId = NumSerie
sr(0).OpportunityName = fila("NumeroSerie").ToString().Trim()
sr(0).SalesStage = "Deseo" '"Cerrada/Ganada"
sr(0).dFecha_de_entrega_al_cliente = fila("FechaEmision").ToString().Trim()
sr(0).dFecha_de_facturacin_al_cliente = fila("FechaCompra").ToString().Trim()
sr(0).stNro_Factura = fila("NumeroFactura").ToString().Trim()
sr(0).plActividad_Economica = fila("IdActividad").ToString().Trim()
sr(0).plTipo_de_Venta = fila("TipoCompra").ToString().Trim()
sr(0).CustomObject8ExternalSystemId = fila("ApellidoVendedor").ToString.Trim()
'sr(0).CustomObject8ExternalSystemId = IIf(TipoPersona = "J", fila("DocumentoE").ToString().Trim(), fila("Documento").ToString.Trim())
'sr(0).CustomObject7ExternalSystemId = fila("")
Dim lofsr As Opportunity.ListOfOpportunityData
lofsr = New Opportunity.ListOfOpportunityData
lofsr.Opportunity = sr
sr_input.ListOfOpportunity = lofsr
sr_output = oOpportunity.OpportunityUpdate(sr_input)
oLog.Update_Log(IdLog, "Si", NumSerie, "")
Catch ex As SoapException
Me.txtError.Text = ex.Detail.InnerText.ToString()
oLog.Update_Log(IdLog, "No", NumSerie, ex.Detail.InnerText.ToString())
End Try
End Sub
************************************************************************ -
How to integrate Microsoft and SAP using Web Services?
Hi All
How to integrate Microsoft and SAP using Web Services? If any one has document please send it to me. My id is [email protected]... Please
Help me
Best Regards
Ravi ShankarHi Ravi,
This is for Customizing Email and Other Web Services.
Create addresses
The address maintenance of R/3 users is carried out either via the R/3 User Maintenance (Transaction SU01) or the Private office settings (Transaction SO12):
Address --> Other communication...
Selection of the required communication service.
Enter address.
Copy or save.
b) Configuration of SAPconnect (Transaction SCOT)
Set communication method
4.0 + 4.5: Goto --> Customizing --> Communication methods
as of 4.6: Settings --> Communication methods
Set the method of the required communication service to the value 'SAPCONNECT' and save the setting.
Customized the same for Internet mailing.
C) If you want to Transfer Table Structure Values from SAP R/3 to Web Services then
1. Create BAPI and Transfer whichever things you want.(Note: The Structure
of BAPI should be similar to Web Structure (Datatypes).
2. Create a BDC and Schedule it in the Background.
Hope it helps you, Awaiting for the Reward Points.
Thanks
Subrato Chowdhury -
How to consuming reports from OBIEE using web services?
I need know if there are a forum for post about consuming reports from Oracle Business Intelligence Enterprise Edition using web services for deploy on ASP.Net web application.
Kind Regards.
deniscubaI would start with the BI forums http://forums.oracle.com/forums/category.jspa?categoryID=145
If no answer in a reasonable time, I would post an update in the question indicating that I would cross-post in the http://forums.oracle.com/forums/category.jspa?categoryID=44 (.Net related) or the http://forums.oracle.com/forums/category.jspa?categoryID=10 (Technology > Web Services) forums. -
How to add "Prepare Data" from a web service to the form
Hello,
Can any one please advise on how to add the "Prepare Data" process from the existing web service and have the form to pre-populate the data from this "Prepare Data" process instead of using schema xsd. I heard that this is an alternative or may be a better way to pre-populate data in ES2/ES3 to avoiding creating a data source in Form Designer. I try to find a sample on Adobe site but could not find one, most of them are using schema.
Any guidance or URL to the sample would be helpful.
Thanks,
HDThanks
-
How to bind list data to XML Web service request
How do I bind specific columns in a DataGrid to the Web
service request? I'm having trouble finding any documentation that
addresses that specific pattern, i.e. sending a complex list to the
server via a Flex Web service send() command. I'm fairly new to
Flex programming and don't know if what I want to do is possible.
Here what I've been able to do so far.
1. Using a Web service called a service on the server and
retrieved a complex list.
2. Poplulated a DataGrid with the result
3. The user has selected multiple rows from the DataGrid
using a checkbox column
4. The user pressed a button that calls a Web service send().
This Web service should only send data from only two columns and
only for those rows the user has checked.
5. I can loop over the DataGrid and find the selected rows
and put them in another ArrayCollection called 'selectedRows'.
The issue is that I don't know how to bind 'selectedRows' to
the Web service. Right now I'm reading up on "Working with XML" in
the Programming with ActionScript 3.0 chapter. But I'm just fishing
here. No bites yet.Don't bind. Build the request object programatically, as you
are doing with your selectedRows AC, and send(myObject) that.
Tracy -
How to create items and BOMs using web services
Hi All,
I need to create, change and read Items and BOMs through web services (using SOA Gateway, right). I found this one here: "Process Item (convenience wrapper version)" or "Process Item" within
Internal Name EGO_ITEM_PUB
Type PL/SQL
Product Advanced Product Catalog
Status Active
Business Entity Catalog Item
Right now, I don't know how to use it. If somebody has a simple example to create a single item, this would be perfect! Which one would be the right one to read detailed information about an existing item?
The second object I need to maintain (create, change and read) is BOM. Which API can I use here?
btw: I use EBS 12.1.1 on windows.
Many thanks,
KonradHi!
Thanks for your quick answer but unfortunately, I must admit that I'm an absolute beginner with Oracle and EBS and so things like:
Enable trace and generate the tkprof file or open the form using forms builder to get the name of the API used in thta screen.or
Please check e-trm site for your question do not help me to much :-(
Any other advice?
Konrad
Api's in EBS
Re: Api's in EBS
http://sairamgoudmalla.blogspot.com/2009/05/script-to-find-oracle-apis-for-any.html
API
Fixed Asset API
List of API
Re: List of APIs
Regard
Helios -
How to show all data returned from web service in a dashboard
Hi I need create a dashboard with conection with a web service and whitout establish a range in excel, so be displayed on the dashboard n rows returned by the web service.
how can I do this?Hi Ezequiel,
I agree with Alan.
You might find it useful to read the Xcelsius 2008 Best Practices whitepaper: [https://www.sdn.sap.com/irj/scn/go/portal/prtroot/docs/library/uuid/a084a11c-6564-2b10-79ac-cc1eb3f017ac]
Regards,
Matt -
How to create Folder and subFolder using web services
Hi,
Today I've been working with some Share Point develompment but the truth is that I can't just get it.
Can any body help me with some easy links to start Share Point Development?
Here is wath I'm trying: I need to create a new folder (lets say Folder1) inside a Document library. After Folder1 is created, I need to create a new Folder inside Folder1 (lets say Folder1.1) and so on.
Until now, I'm able to create Folder1 but after that I don't know how to continue creating subfolders.
It's important to say that I would like to solve this problem by using webservices and let de Microsoft.Sharepoint.dll as just in case.
I hope somebody help me.
Thanks.Hi Steve,
I need to create a folder structure(e.g folder1/folder2) by the running the program once.I tried calling the UpdateListItemCreateFolder recursively and passed an extra rootFolder parameter.I changed the RootFolder attribute as shown below but the code isnt working.Please let me know the right procedure to create a folder structure in a list.
public
static void UpdateListItemCreateFolder(string folderName, string rootFolder)
ListService.
Lists listProxy = new ListService.Lists();
string location = "http://localhost:414/Documents/one" + rootFolder.ToString();
string xmlconst = "<Batch OnError='Continue' RootFolder='" + location.ToString() + "'><Method ID='1' Cmd='New'><Field Name='ID'>New</Field><Field Name='FSObjType'>1</Field><Field Name='BaseName'>!@foldername</Field></Method></Batch>";
listProxy.Url =
"http://localhost:414/_vti_bin/lists.asmx";
listProxy.UseDefaultCredentials =
true;
XmlDocument doc = new XmlDocument();
xmlconst.Replace(
"!@root", location);
string xmlFolder = xmlconst.Replace("!@foldername", folderName);
doc.LoadXml(xmlFolder);
XmlNode batchNode = doc.SelectSingleNode("//Batch");
XmlNode resultNode = listProxy.UpdateListItems("Documents", batchNode); -
Import Data object using web service adapter
Hello experts,
In DOE, theres a way to import Data Object from a wsdl using webservice adapter. We got to know that there are some guidelines(constraints) on the operations provided in the wsdl to be able to use webservice adapter. Could you please give a sample wsdl which can be used to import Data object for our reference?
Thanks in Advance,
SrilathaHi Srilatha,
SAP Netweaver 7.3 is not yet released to customers. Once, this happens we will post sufficient documentation related to the Import of Data Objects through WSDLs. In case you require the information urgently, kindly get in touch directly with the colleagues from DOE Mobile team.
Best Regards,
Kranti -
How to find out material code using web services?
Hi,
Is there a WSDL to find out material code of a componet if only inventory ID is known?
Regards
KariThanks
-
How to use web services in Excelsius 2008
I am new to this tool. I have trial version on excelsius 2008 and i would like to build a dashboard that should query our corporate database (in oracle). I think web services is a option to do it. Any samples available?.Please help me
Edited by: AshishDatar on Jul 27, 2009 6:29 PMHi AshishDatar:
For you question you want to get data from data base using web service connection. Here I can give you some suggestions
1.Write a web service to retrieve data from data base, then use Xcelsius u201Cweb service connectionu201D under u201CManage Connectionu201D section to consume the service and get the data from data base to spread sheet.
2.Xcelsius also provide you a tool to generate web service with easy way (without coding). (Suggested)
Using flynet, you can get flynet installation file under Xcelsius installation folder u201CXcelsius\Connectivity\Flynet Webservice Generatoru201D for more information about flynet please refer to
http://myxcelsius.com/2008/11/07/generating-web-services-for-xcelsius-using-flynet-web-service-generator/
Here is also a demo on how to use web service conneciton with Xcelsius + Flynet
https://businessobjects.webex.com/ec0600l/eventcenter/enroll/register.do?siteurl=businessobjects&formId=50213992&confId=50213992&formType=1&loadFlag=1&eventType=1&accessType=
Hope it helps with your work! And let me know if you need any further help.
Thanks!
Bill
Edited by: Bill Xu on Jul 28, 2009 8:02 AM -
Urgent! How to create a ebook using web services?
Hi I would need some help on how to creating a e-book using web services. Sample code would be good for me.
I had been finding sources for quite sometime but i doesn't seems to be easy to find as compare to andriod.
Description:
Windows 8.1 SDK Sliverlight.
The e-book will on PDF format.
Once user click e-book icon, i would link to my database to retrieve all the ebook available via web services.
Then user select the e-book, the content will be again retreive from my database to the users.
Is there any solution? I need it quite urgent.Hi Ding Zheng,
>>Once user click e-book icon, i would link to my database to retrieve all the ebook available via web services. Then user select the e-book, the content will be again retreive from my database to the users.
According this approach, I would suggest you make good use of build-in API to render PDF in store app. see more classes in the following namespace.
http://msdn.microsoft.com/en-us/library/windows/apps/windows.data.pdf.aspx. Using this APIs need PDF file in local storage, so I list the following steps
for you to try.
Step 1, Store PDF file in web site folder and save the PDF URL in database.
Step 2, Use Web service/WCF/REST service to public the PDF to store app.
Step 3, Store app uses service to get PDF list and then display them.
Step 4, User choose one PDF, get the PDF download URL from service and then download it on local storage. Store app support download file form server, you can use BackgroundDownloader. See from
http://msdn.microsoft.com/en-us/library/windows/apps/windows.networking.backgroundtransfer.backgrounddownloader.aspx.
Step 5, Use PDF APIs to render it in store app. See code sample
https://code.msdn.microsoft.com/windowsapps/PDF-viewer-sample-85a4bb30#content.
Please feel free to let me know if you have any concerns.
Regards,
We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place. Click HERE to participate
the survey. -
Bulk batch Jax-WS web services and memory concerns
I have a requirement to create a web service client to upload 150,000+ records from our database. My main question is whether others are creating web services to process bulk data and how they handle memory concerns.
I began with the standard API, populating the jax-b annotated beans created from the wsimport. I used JConsole to analyze memory usage. I noticed that there is a sharp spike in the plot of the heap usage (from 200MB to 800MB) right after the invoke method call.
The problem is that the max heap size on our app server is set to 1GB and since this application shares it with other applications I keep getting a out of memory error.
I was asked to look for alternatives, so I tried using the Dispatch API creating the payload by sub-tree marshaling to StAX. The issue I then ran into is that I have a outputstream and the dispatch api is expecting an inputstream. I tried the options found here http://ostermiller.org/convert_java_outputstream_inputstream.html
However after using jconsole to analyze again it appears the writing to a temp file or using a byte array increases the amount of memory being used to almost equivalent of using the standard API. I tried using piped streams in separate threads and the single threaded circular buffer methods, but the application seems to hang after the invoke method call.
Any help would be appreciated.also, you could "page" the data, returning some max number of records in each page of data and a "next page" cookie. this is the standard approach to returning bulk data from a web service.
-
How to Use SOAPArray to Exchanged Data with a Web Service
The method of a prototype Web service I created is defined to take many parameters
and return an object of a user defined class. Furthermore, the user defined class
includes data elements of another user defined class and the Java ArrayList class.
This works with a Java client referencing the WebLogic created client.jar file
but I don't know how well it will work with a non-Java client. In particular,
with Perl which is the language that will be used by the developer who first will
test with the prototype.
In posts to this newsgroup use of "language-specific, generic containers" has
been discouraged and the "language-agnostic" SOAPArray recommended. I have searched
this newgsroup and the Web for examples of how to use a SOAPArray in a Web service
EJB to receive parameters and return results but found none.
Will someone refer me to an example or give an overview of how a Java Web service
EJB running in WebLogic 6.1 would use SOAPArray to get parameter values and return
results?
Also, I would like confirmation that it is best to use SOAPArray to exchange data
with a Web service to achieve the goal of a service accessible by any language.
Thank you.Replies in-line:
How are the structures, e.g. gltrans-workType, defined in the Web service?The structure is made up of nested Java Beans, but this does not mean that the
client for your web service has to be written in Java. The WSDL that I sent contains
everything that a .NET-based (or Perl-based, or Python-based, or VB-based, or
C++ based) Web Service Stack needs to correctly create all the data types in the
web services' signature! That's the beauty of XML Schema! It's programming language
independent :-)
In
other words, what definition in Java resulted in the WSDL statements?The WSDL wasn't produced by WLS 6.1, but it (WLS 6.1) can consume it.
What is the signature of method submitGLTransWorkAsJavaBean() in the
Web service?public void submitGLTransWorkAsJavaBean(GlTransactionsCpyType glTransactionsCpyType)
GlTransactionsCpyType is the outer-most Java Bean. WLS 6.1 does not generate
Java Beans for you, but it will use ones that you defined. See the Java Bean tutorial
on the Javasoft sitem for details on how to create a Java Bean.
Was the WSDL generated using the WL tools for creating a Web service?No.
Conclusion:
You asked for someone to provide you with an example of how to use SOAP array
in a WSDL, which is what the attached file contained :-) What you want to do now
is find a tool that can generate Java Bean code from this WSDL (Apache Axis has
a wsdl2java tool that should work), or create the Java Beans yourself. Afterwards,
create a WLS 6.1 Web Service a expose it for a Perl or .NET client.
Regards,
Mike Wooten
"Jeff Carey" <[email protected]> wrote:
>
Please elaborate.
How are the structures, e.g. gltrans-workType, defined in the Web service?
In
other words, what definition in Java resulted in the WSDL statements?
What is the signature of method submitGLTransWorkAsJavaBean() in the
Web service?
Was the WSDL generated using the WL tools for creating a Web service?
Thank you.
"Michael Wooten" <[email protected]> wrote:
Hi Jeff,
Sounds like a pretty cool prototype :-)
I have attached a WSDL (at the bottom of this post) that contains a<schema>
that
uses a SOAPArray to create an array of a <complexType>.
HTH,
Mike Wooten
"Jeff Carey" <[email protected]> wrote:
The method of a prototype Web service I created is defined to take
many
parameters
and return an object of a user defined class. Furthermore, the user
defined class
includes data elements of another user defined class and the Java ArrayList
class.
This works with a Java client referencing the WebLogic created client.jar
file
but I don't know how well it will work with a non-Java client. Inparticular,
with Perl which is the language that will be used by the developerwho
first will
test with the prototype.
In posts to this newsgroup use of "language-specific, generic containers"
has
been discouraged and the "language-agnostic" SOAPArray recommended.
I have searched
this newgsroup and the Web for examples of how to use a SOAPArray in
a Web service
EJB to receive parameters and return results but found none.
Will someone refer me to an example or give an overview of how a Java
Web service
EJB running in WebLogic 6.1 would use SOAPArray to get parameter values
and return
results?
Also, I would like confirmation that it is best to use SOAPArray toexchange
data
with a Web service to achieve the goal of a service accessible by any
language.
Thank you.
Maybe you are looking for
-
What will i do if my email won't update?
what should i do if my email won't update?
-
The loading bar can't work correctly!
hello, everybody, i creat a loading bar, but it can't work correctly, the code is that: as u see, the"will_load.swf" is in the same file
-
i am trying to download one app and here is my result (WHY?) this app is just free for today and i have to sownload it. "We could not complete your iTunes Store request. an unknown error occurred. (-50) there was an error in the iTunes Store. Please
-
I accidently uploaded ALL my Contacts to iCloud.This removed them ALL from Microsoft Outlook Contacts. How do i get them back to Outlook Contacts?
-
[solved] Lost playback of musepack (.mpc)
Whatever is causing this is very recent, it was working just a few days ago. I can still play musepack files through gstreamer via Quod Libet, but I used to be able to play them through MPD and audacious, too. I remember doing so as recently as a wee