How to do Bulk data transfer  using Web Service

In my application I have to write various web services but majority of the web service has to
query database and return back bulk data(rows>10K) through web service.
So I would like to ask what is the efficient way of transferring bulk data using web service as presently
Iam returning the dataset as xml String (using StringBuilder) from web service and consuming the same at client end.
Is there a better way to this in web service?
My env:
Front end can be in any other technology ,UI like C#
Back end : Tomcat 6 on Java 6 with Axis2
Thanks in advance

Innova wrote:
But then also I have to mention a return type although its xml that is getting transferred.
Can you provide me a with a small sample code.Like if I have Emp obect with properties
Empname,EmpID,DOJ,DOB,Dpt . In this case what will be the return type.
My major concern is that my resultset is going to be huge in terms of >10,000 rows so
that much time for formation of xml and then the transfer so how can I reduce the transfer
time ,I meant a faster optimised approach for transferring large data as part of web service.
Simply putting , how can I transfer bulk data in minimum time so that I can reduce the
waiting time at client side for the response.
Thanks in advanceI've done this with thousands before now, and not had a performance problem...at least nothing worth writing home about.
Your return type will be an array of Emp objects. You can use a SOAP array for that (you'll need to look that up for the syntax, since I can't remember it off the top of my head), which gets restricted to being an array of Emp.
Should the normal technique prove to be too slow then you should look at alternatives, but I would expect this to work well enough. And you have no faffing about with devising a new transfer system that then has to be explained to your client system...it'll all be standard.

Similar Messages

  • How can i records with date format using web services?

    Hello
    I can't record date records using web services. I get no message errors.
    I can import string values but no dates (YYYY-MM-DD). Do you have any clue about that?
    Regards
    Arturo

    hello,
    That's the code I'm using to update an opportunity. In the date fields (e.g. dFecha_de_entrega_al_cliente) I've tried to put an specific date in the correct format (If i put it in another format i've got an error message due the wrong format). The CRM accepted the code but it didn't update the values that are different of string.
    I don´t know if there is something missing in teh program or if the developer environment is not the adequate.
    Regards for your comments
    Arturo
    Private Sub ActualizarOportunidad(ByVal fila As Data.DataRow, ByVal TipoPersona As String)
    Dim oLog As New Log()
    Dim IdLog As Integer
    Dim NumSerie As String = ""
    Try
    oLog.Insert_Log("Activación Garantía - Crear Oportunidad", oLog.GetLastIdProceso())
    IdLog = oLog.GetLastId()
    Dim sr_input As Opportunity.OpportunityUpdate_Input
    Dim sr_output As Opportunity.OpportunityUpdate_Output
    sr_input = New Opportunity.OpportunityUpdate_Input
    Dim sr(1) As Opportunity.OpportunityData
    sr(0) = New Opportunity.OpportunityData
    NumSerie = fila("NumeroSerie").ToString().Trim()
    sr(0).ExternalSystemId = NumSerie
    sr(0).OpportunityName = fila("NumeroSerie").ToString().Trim()
    sr(0).SalesStage = "Deseo" '"Cerrada/Ganada"
    sr(0).dFecha_de_entrega_al_cliente = fila("FechaEmision").ToString().Trim()
    sr(0).dFecha_de_facturacin_al_cliente = fila("FechaCompra").ToString().Trim()
    sr(0).stNro_Factura = fila("NumeroFactura").ToString().Trim()
    sr(0).plActividad_Economica = fila("IdActividad").ToString().Trim()
    sr(0).plTipo_de_Venta = fila("TipoCompra").ToString().Trim()
    sr(0).CustomObject8ExternalSystemId = fila("ApellidoVendedor").ToString.Trim()
    'sr(0).CustomObject8ExternalSystemId = IIf(TipoPersona = "J", fila("DocumentoE").ToString().Trim(), fila("Documento").ToString.Trim())
    'sr(0).CustomObject7ExternalSystemId = fila("")
    Dim lofsr As Opportunity.ListOfOpportunityData
    lofsr = New Opportunity.ListOfOpportunityData
    lofsr.Opportunity = sr
    sr_input.ListOfOpportunity = lofsr
    sr_output = oOpportunity.OpportunityUpdate(sr_input)
    oLog.Update_Log(IdLog, "Si", NumSerie, "")
    Catch ex As SoapException
    Me.txtError.Text = ex.Detail.InnerText.ToString()
    oLog.Update_Log(IdLog, "No", NumSerie, ex.Detail.InnerText.ToString())
    End Try
    End Sub
    ************************************************************************

  • How to integrate Microsoft and SAP using Web Services?

    Hi All
    How to integrate Microsoft and SAP using Web Services? If any one has document please send it to me. My id is [email protected]... Please
    Help me
    Best Regards
    Ravi Shankar

    Hi Ravi,
    This is for Customizing Email and Other Web Services.
    Create addresses
    The address maintenance of R/3 users is carried out either via the R/3 User Maintenance (Transaction SU01) or the Private office settings (Transaction SO12):
    Address --> Other communication...
    Selection of the required communication service.
    Enter address.
    Copy or save.
    b) Configuration of SAPconnect (Transaction SCOT)
    Set communication method
    4.0 + 4.5: Goto --> Customizing --> Communication methods
    as of 4.6: Settings --> Communication methods
    Set the method of the required communication service to the value 'SAPCONNECT' and save the setting.
    Customized the same for Internet mailing.
    C) If you want to Transfer Table Structure Values from SAP R/3 to Web Services then
           1. Create BAPI and Transfer whichever things you want.(Note: The Structure
               of BAPI should be similar to Web Structure (Datatypes).
           2. Create a BDC and Schedule it in the Background.
    Hope it helps you, Awaiting for the Reward Points.
    Thanks
    Subrato Chowdhury

  • How to consuming reports from OBIEE using web services?

    I need know if there are a forum for post about consuming reports from Oracle Business Intelligence Enterprise Edition using web services for deploy on ASP.Net web application.
    Kind Regards.
    deniscuba

    I would start with the BI forums http://forums.oracle.com/forums/category.jspa?categoryID=145
    If no answer in a reasonable time, I would post an update in the question indicating that I would cross-post in the http://forums.oracle.com/forums/category.jspa?categoryID=44 (.Net related) or the http://forums.oracle.com/forums/category.jspa?categoryID=10 (Technology > Web Services) forums.

  • How to add "Prepare Data" from a web service to the form

    Hello,
    Can any one please advise on how to add the "Prepare Data" process from the existing web service and have the form to pre-populate the data from this "Prepare Data" process instead of using schema xsd. I heard that this is an alternative or may be a better way to pre-populate data in ES2/ES3 to avoiding creating a data source in Form Designer. I try to find a sample on Adobe site but could not find one, most of them are using schema.
    Any guidance or URL to the sample would be helpful.
    Thanks,
    HD

    Thanks

  • How to bind list data to XML Web service request

    How do I bind specific columns in a DataGrid to the Web
    service request? I'm having trouble finding any documentation that
    addresses that specific pattern, i.e. sending a complex list to the
    server via a Flex Web service send() command. I'm fairly new to
    Flex programming and don't know if what I want to do is possible.
    Here what I've been able to do so far.
    1. Using a Web service called a service on the server and
    retrieved a complex list.
    2. Poplulated a DataGrid with the result
    3. The user has selected multiple rows from the DataGrid
    using a checkbox column
    4. The user pressed a button that calls a Web service send().
    This Web service should only send data from only two columns and
    only for those rows the user has checked.
    5. I can loop over the DataGrid and find the selected rows
    and put them in another ArrayCollection called 'selectedRows'.
    The issue is that I don't know how to bind 'selectedRows' to
    the Web service. Right now I'm reading up on "Working with XML" in
    the Programming with ActionScript 3.0 chapter. But I'm just fishing
    here. No bites yet.

    Don't bind. Build the request object programatically, as you
    are doing with your selectedRows AC, and send(myObject) that.
    Tracy

  • How to create items and BOMs using web services

    Hi All,
    I need to create, change and read Items and BOMs through web services (using SOA Gateway, right). I found this one here: "Process Item (convenience wrapper version)" or "Process Item" within
    Internal Name     EGO_ITEM_PUB
    Type               PL/SQL
    Product          Advanced Product Catalog
    Status          Active
    Business Entity     Catalog Item
    Right now, I don't know how to use it. If somebody has a simple example to create a single item, this would be perfect! Which one would be the right one to read detailed information about an existing item?
    The second object I need to maintain (create, change and read) is BOM. Which API can I use here?
    btw: I use EBS 12.1.1 on windows.
    Many thanks,
    Konrad

    Hi!
    Thanks for your quick answer but unfortunately, I must admit that I'm an absolute beginner with Oracle and EBS and so things like:
    Enable trace and generate the tkprof file or open the form using forms builder to get the name of the API used in thta screen.or
    Please check e-trm site for your question do not help me to much :-(
    Any other advice?
    Konrad
    Api's in EBS
    Re: Api's in EBS
    http://sairamgoudmalla.blogspot.com/2009/05/script-to-find-oracle-apis-for-any.html
    API
    Fixed Asset API
    List of API
    Re: List of APIs
    Regard
    Helios

  • How to show all data returned from web service in a dashboard

    Hi I need  create a dashboard with conection with a web service  and  whitout  establish a range in excel,  so be displayed on the dashboard n rows returned by the web service.
    how can I do this?

    Hi Ezequiel,
    I agree with Alan.
    You might find it useful to read the Xcelsius 2008 Best Practices whitepaper: [https://www.sdn.sap.com/irj/scn/go/portal/prtroot/docs/library/uuid/a084a11c-6564-2b10-79ac-cc1eb3f017ac]
    Regards,
    Matt

  • How to create Folder and subFolder using web services

    Hi,
    Today I've been working with some Share Point develompment but the truth is that I can't just get it.
    Can any body help me with some easy links to start Share Point Development?
    Here is wath I'm trying: I need to create a new folder (lets say Folder1) inside a Document library. After Folder1 is created, I need to create a new Folder inside Folder1 (lets say Folder1.1) and so on.
    Until now, I'm able to create Folder1 but after that I don't know how to continue creating subfolders.
    It's important to say that I would like to solve this problem by using webservices and let de Microsoft.Sharepoint.dll as just in case.
    I hope somebody help me.
    Thanks.

    Hi Steve,
    I need to create a folder structure(e.g folder1/folder2) by the running the program once.I tried calling the UpdateListItemCreateFolder recursively and passed an extra rootFolder parameter.I changed the RootFolder attribute as shown below but the code isnt working.Please let me know the right procedure to create a folder structure in a list.
    public
    static void UpdateListItemCreateFolder(string folderName, string rootFolder)
    ListService.
    Lists listProxy = new ListService.Lists();
    string location = "http://localhost:414/Documents/one" + rootFolder.ToString();
    string xmlconst = "<Batch OnError='Continue' RootFolder='" + location.ToString() + "'><Method ID='1' Cmd='New'><Field Name='ID'>New</Field><Field Name='FSObjType'>1</Field><Field Name='BaseName'>!@foldername</Field></Method></Batch>";
    listProxy.Url =
    "http://localhost:414/_vti_bin/lists.asmx";
    listProxy.UseDefaultCredentials =
    true;
    XmlDocument doc = new XmlDocument();
    xmlconst.Replace(
    "!@root", location);
    string xmlFolder = xmlconst.Replace("!@foldername", folderName);
    doc.LoadXml(xmlFolder);
    XmlNode batchNode = doc.SelectSingleNode("//Batch");
    XmlNode resultNode = listProxy.UpdateListItems("Documents", batchNode);

  • Import Data object using web service adapter

    Hello experts,
    In DOE, theres a way to import Data Object from a wsdl using webservice adapter. We got to know that there are some guidelines(constraints) on the operations provided in the wsdl to be able to use webservice adapter. Could you please give a sample wsdl which can be used to import Data object for our reference?
    Thanks in Advance,
    Srilatha

    Hi Srilatha,
    SAP Netweaver 7.3 is not yet released to customers.  Once, this happens we will post sufficient documentation related to the Import of Data Objects through WSDLs.  In case you require the information urgently, kindly get in touch directly with the colleagues from DOE Mobile team.
    Best Regards,
    Kranti

  • How to find out material code using web services?

    Hi,
    Is there a WSDL to find out material code of a componet if only inventory ID is known?
    Regards
    Kari

    Thanks

  • How to use web services in Excelsius 2008

    I am new  to this tool. I have trial version on excelsius 2008 and i would like to build a dashboard that should query our corporate database (in oracle). I think web services is a option to do it.  Any samples available?.Please help me
    Edited by: AshishDatar on Jul 27, 2009 6:29 PM

    Hi AshishDatar:
        For you question you want to get data from data base using web service connection. Here I can give you some suggestions
    1.Write a web service to retrieve data from data base, then use Xcelsius u201Cweb service connectionu201D under u201CManage Connectionu201D section to consume the service and get the data from data base to spread sheet.
    2.Xcelsius also provide you a tool to generate web service with easy way (without coding). (Suggested)
    Using flynet, you can get flynet installation file under Xcelsius installation folder u201CXcelsius\Connectivity\Flynet Webservice Generatoru201D for more information about flynet please refer to
    http://myxcelsius.com/2008/11/07/generating-web-services-for-xcelsius-using-flynet-web-service-generator/
    Here is also a demo on how to use  web service conneciton with Xcelsius + Flynet
    https://businessobjects.webex.com/ec0600l/eventcenter/enroll/register.do?siteurl=businessobjects&formId=50213992&confId=50213992&formType=1&loadFlag=1&eventType=1&accessType=
    Hope it helps with your work! And let me know if you need any further help.
    Thanks!
    Bill
    Edited by: Bill Xu on Jul 28, 2009 8:02 AM

  • Urgent! How to create a ebook using web services?

    Hi I would need some help on how to creating a e-book using web services. Sample code would be good for me.
    I had been finding sources for quite sometime but i doesn't seems to be easy to find as compare to andriod.
    Description:
    Windows 8.1 SDK Sliverlight.
    The e-book will on PDF format.
    Once user click e-book icon, i would link to my database to retrieve all the ebook available via web services.
    Then user select the e-book, the content will be again retreive from my database to the users.
    Is there any solution? I need it quite urgent.

    Hi Ding Zheng,
    >>Once user click e-book icon, i would link to my database to retrieve all the ebook available via web services. Then user select the e-book, the content will be again retreive from my database to the users.
    According this approach, I would suggest you make good use of build-in API to render PDF in store app. see more classes in the following namespace.
     http://msdn.microsoft.com/en-us/library/windows/apps/windows.data.pdf.aspx. Using this APIs need PDF file in local storage, so I list the following steps
    for you to try.
    Step 1, Store PDF file in web site folder and save the PDF URL in database.
    Step 2, Use Web service/WCF/REST service to public the PDF to store app.
    Step 3, Store app uses service to get PDF list and then display them.
    Step 4, User choose one PDF, get the PDF download URL from service and then download it on local storage. Store app support download file form server, you can use BackgroundDownloader. See from
    http://msdn.microsoft.com/en-us/library/windows/apps/windows.networking.backgroundtransfer.backgrounddownloader.aspx.
    Step 5, Use PDF APIs to render it in store app. See code sample
    https://code.msdn.microsoft.com/windowsapps/PDF-viewer-sample-85a4bb30#content.
    Please feel free to let me know if you have any concerns.
    Regards,
    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place. Click HERE to participate
    the survey.

  • Bulk batch Jax-WS web services and memory concerns

    I have a requirement to create a web service client to upload 150,000+ records from our database.  My main question is whether others are creating web services to process bulk data and how they handle memory concerns.
    I began with the standard API, populating the jax-b annotated beans created from the wsimport.  I used JConsole to analyze memory usage.  I noticed that there is a sharp spike in the plot of the heap usage (from 200MB to 800MB) right after the invoke method call.
    The problem is that the max heap size on our app server is set to 1GB and since this application shares it with other applications I keep getting a out of memory error.
    I was asked to look for alternatives, so I tried using the Dispatch API creating the payload by sub-tree marshaling to StAX.  The issue I then ran into is that I have a outputstream and the dispatch api is expecting an inputstream.  I tried the options found here http://ostermiller.org/convert_java_outputstream_inputstream.html
    However after using jconsole to analyze again it appears the writing to a temp file or using a byte array increases the amount of memory being used to almost equivalent of using the standard API.  I tried using piped streams in separate threads and the single threaded circular buffer methods, but the application seems to hang after the invoke method call.
    Any help would be appreciated.

    also, you could "page" the data, returning some max number of records in each page of data and a "next page" cookie. this is the standard approach to returning bulk data from a web service.

  • How to Use SOAPArray to Exchanged Data with a Web Service

    The method of a prototype Web service I created is defined to take many parameters
    and return an object of a user defined class. Furthermore, the user defined class
    includes data elements of another user defined class and the Java ArrayList class.
    This works with a Java client referencing the WebLogic created client.jar file
    but I don't know how well it will work with a non-Java client. In particular,
    with Perl which is the language that will be used by the developer who first will
    test with the prototype.
    In posts to this newsgroup use of "language-specific, generic containers" has
    been discouraged and the "language-agnostic" SOAPArray recommended. I have searched
    this newgsroup and the Web for examples of how to use a SOAPArray in a Web service
    EJB to receive parameters and return results but found none.
    Will someone refer me to an example or give an overview of how a Java Web service
    EJB running in WebLogic 6.1 would use SOAPArray to get parameter values and return
    results?
    Also, I would like confirmation that it is best to use SOAPArray to exchange data
    with a Web service to achieve the goal of a service accessible by any language.
    Thank you.

    Replies in-line:
    How are the structures, e.g. gltrans-workType, defined in the Web service?The structure is made up of nested Java Beans, but this does not mean that the
    client for your web service has to be written in Java. The WSDL that I sent contains
    everything that a .NET-based (or Perl-based, or Python-based, or VB-based, or
    C++ based) Web Service Stack needs to correctly create all the data types in the
    web services' signature! That's the beauty of XML Schema! It's programming language
    independent :-)
    In
    other words, what definition in Java resulted in the WSDL statements?The WSDL wasn't produced by WLS 6.1, but it (WLS 6.1) can consume it.
    What is the signature of method submitGLTransWorkAsJavaBean() in the
    Web service?public void submitGLTransWorkAsJavaBean(GlTransactionsCpyType glTransactionsCpyType)
    GlTransactionsCpyType is the outer-most Java Bean. WLS 6.1 does not generate
    Java Beans for you, but it will use ones that you defined. See the Java Bean tutorial
    on the Javasoft sitem for details on how to create a Java Bean.
    Was the WSDL generated using the WL tools for creating a Web service?No.
    Conclusion:
    You asked for someone to provide you with an example of how to use SOAP array
    in a WSDL, which is what the attached file contained :-) What you want to do now
    is find a tool that can generate Java Bean code from this WSDL (Apache Axis has
    a wsdl2java tool that should work), or create the Java Beans yourself. Afterwards,
    create a WLS 6.1 Web Service a expose it for a Perl or .NET client.
    Regards,
    Mike Wooten
    "Jeff Carey" <[email protected]> wrote:
    >
    Please elaborate.
    How are the structures, e.g. gltrans-workType, defined in the Web service?
    In
    other words, what definition in Java resulted in the WSDL statements?
    What is the signature of method submitGLTransWorkAsJavaBean() in the
    Web service?
    Was the WSDL generated using the WL tools for creating a Web service?
    Thank you.
    "Michael Wooten" <[email protected]> wrote:
    Hi Jeff,
    Sounds like a pretty cool prototype :-)
    I have attached a WSDL (at the bottom of this post) that contains a<schema>
    that
    uses a SOAPArray to create an array of a <complexType>.
    HTH,
    Mike Wooten
    "Jeff Carey" <[email protected]> wrote:
    The method of a prototype Web service I created is defined to take
    many
    parameters
    and return an object of a user defined class. Furthermore, the user
    defined class
    includes data elements of another user defined class and the Java ArrayList
    class.
    This works with a Java client referencing the WebLogic created client.jar
    file
    but I don't know how well it will work with a non-Java client. Inparticular,
    with Perl which is the language that will be used by the developerwho
    first will
    test with the prototype.
    In posts to this newsgroup use of "language-specific, generic containers"
    has
    been discouraged and the "language-agnostic" SOAPArray recommended.
    I have searched
    this newgsroup and the Web for examples of how to use a SOAPArray in
    a Web service
    EJB to receive parameters and return results but found none.
    Will someone refer me to an example or give an overview of how a Java
    Web service
    EJB running in WebLogic 6.1 would use SOAPArray to get parameter values
    and return
    results?
    Also, I would like confirmation that it is best to use SOAPArray toexchange
    data
    with a Web service to achieve the goal of a service accessible by any
    language.
    Thank you.

Maybe you are looking for