Transferin Huge data via Java sockets! Problematic!

Hello!
I tried to write a server app in Java to get huge data which contains a meta data XML format about file information and then binary data of file...
I get data from socket with DataInputStream and then I read like this:
ByteArrayOutputStream out = new ByteArrayOutputStream();
while((got = in.read(res)) != -1)
out.write(res, 0, got);
byte[] recieved = out.toByteArray();
I saw that if I store all receiving data in array it takes alot of memory and I cannot even transfer files which have 650MB of data.
So I decided to write what I get from socket directly to a file...
I used FileOutputStream to do it...
But problem is -> THIS SERVER IS SO SLOW!
I wrote similar code in VB6... And it receives data from client SO SO SO faster than Java...
What's wrong with my code which makes works slow?
how can I solve it?
You experts have any server code example for receiving huge amount of data?
Please advice...
Thanks alot

How to use buffered streams: go to Java tutorials http://java.sun.com/docs/books/tutorial/index.html - look for "basic input/output", and "buffered streams" under it. You'll want BufferedInputStream and BufferedOutputStream.
Alternatively:
        byte buf = new byte[8192];
        while (true) {
            int count = in.read(buf);
            if (count == -1)
                break;
            out.write(buf, 0, count);
        }If at all possible, don't read the whole file into memory at once. Read with a loop like the above and do whatever you are supposed to do with the data a chunk at a time. But if you must have it in memory then I guess you got to do what you got to do.

Similar Messages

Problem while reading data on java socket

Hi All,
I am in big problem based on java socket programming. I run my application and start a ServerSocket on 10000(suppose) port no.As soon as request is coming i create a new thread with Socket assigned to it and process the request. Now i got the response and written on same Socket(Listen to ServerSocket on 10000 port). Now what i want to read the response written earlier on Socket.The written response i print on SOP it is visible. But when i establish InputStream and try to read the data it gives me -1 means no data is available. But i have seen the SOP and data is written to socket. How can solve the problem. I have tried after close the Socket as well as not close the socket.
Please help me out on this.see below code
<CODE>
void upperClassMethod() {
istener = new ServerSocket(port);
while(true) {
clientSocket = listener.accept();
doComms conn_c= new doComms(clientSocket);
Thread t = new Thread(conn_c);
t.start();
//doComms is a inner class of upper level class
class doComms implements Runnable {
          private Socket server;
          doComms(Socket server) {
               //pp = server;
               this.server=server;
               //server=server1;
void processResponse() {
          try {
               BufferedInputStream in = new BufferedInputStream (clientSocket.getInputStream());
               byte [] inBuff = new byte [4096] ;
               int len = in.read (inBuff, 0, inBuff.length) ;
               String output = new String (inBuff) ;
               System.out.println("the output is :::: "+output);
          } catch(Exception e) {
               e.printStackTrace();
</CODE>
in processResponse() method i am not able to get output. Plz help me guys.....
Thanks in advance for ant assistance
Regards,
Pradeep

please see mu rum nethos of doComms class
<CODE>
public void run () {
               input="";
               boolean done = false ;
               try {
                    BufferedInputStream in = new BufferedInputStream (server.getInputStream());
                    out = new PrintStream(server.getOutputStream());
                    while (!done) {
                         byte [] inBuff = new byte [4096] ;
                         try {
                              int len = in.read (inBuff, 0, inBuff.length) ;
                              if (len > 0) {
                                   input = new String (inBuff) ;
                                   Runnable r = new RequestProcessThread();
                                   Thread t = new Thread(r);
                                   t.start();
                              } else if (len == -1) {
                                   done = true ;
                                   //server.close () ;
                         catch (InterruptedIOException iioe) {
               } catch (IOException ioe) {
                    System.out.println("IOException " + ioe + " on socket in thread: " + Thread.currentThread().getName());
                    ioe.printStackTrace();
                    done = true ;
</CODE>

Wsimport, mapping of xs:date to java.util.Date via ext file, and -B option

Summary:
JDK 1.7.0_09 and wsimport and xjc that comes with it.
Global JAXB binding to map xs:date to java.util.Date
I have the following external bindings file:
<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
     xmlns:jaxb="http://java.sun.com/xml/ns/jaxb" xmlns:xjc="http://java.sun.com/xml/ns/jaxb/xjc"
     elementFormDefault="qualified" attributeFormDefault="unqualified"
     jaxb:extensionBindingPrefixes="xjc" jaxb:version="2.1">
     <xs:annotation>
          <xs:appinfo>
               <jaxb:globalBindings>
                    <xjc:serializable />
                    <jaxb:javaType name="java.util.Date" xmlType="xs:date" parseMethod="au.com.xxx.jaxb.DateAdapter.parseDate" printMethod="au.com.xxx.jaxb.DateAdapter.printDate" />
               </jaxb:globalBindings>
          </xs:appinfo>
     </xs:annotation>
</xs:schema>The au.com.xxx.jaxb.DateAdapter code is as follows:
package au.com.xxx.jaxb;
import java.util.Calendar;
import java.util.Date;
import java.util.GregorianCalendar;
import javax.xml.bind.DatatypeConverter;
public class DateAdapter {
     public static Date parseDate(String s) {
          return DatatypeConverter.parseDate(s).getTime();
     public static String printDate(Date dt) {
          Calendar cal = new GregorianCalendar();
          cal.setTime(dt);
          return DatatypeConverter.printDate(cal);
}When I run the following wsimport from the command line, I get:
salvojo@AUD20901BL /cygdrive/c/workspace/JSF/insurance
$ /cygdrive/c/java/jdk1.7.0_09/x64/bin/wsimport -keep -s gen-src -b external/wsdl/jaxb-bindings.xml -wsdllocation /wsdl/Member.wsdl -d WebContent/WEB-INF/classes external/wsdl/Member.wsdl
parsing WSDL...
Generating code...
Compiling code...
C:\workspace\JSF\insurance\gen-src\org\w3\_2001\xmlschema\Adapter1.java:13: error: package au.com.xxx.jaxb does not exist
        return (au.com.xxx.jaxb.DateAdapter.parseDate(value));
                               ^
C:\workspace\JSF\insurance\gen-src\org\w3\_2001\xmlschema\Adapter1.java:17: error: package au.com.xxx.jaxb does not exist
        return (au.com.xxx.jaxb.DateAdapter.printDate(value));
                               ^
2 errors
compilation failed, errors should have been reportedWhich means that wsimport or xjc needs to know the classpath to find au.com.xxx.jaxb.DateAdapter.
But how do I pass the classpath from wsimport to the JAXB compiler ?
There is the -B option in wsimport, but I could not get it to work.
If I read it correctly, I should be able to pass the -classpath option to the JAXB compiler from wsimport via -B.
I tried:
salvojo@AUD20901BL /cygdrive/c/workspace/JSF/insurance
$ /cygdrive/c/java/jdk1.7.0_09/x64/bin/wsimport -keep -s gen-src -B"-classpath WebContent/WEB-INF/classes" -b external/wsdl/jaxb-bindings.xml -wsdllocation /wsdl/Member.wsdl -d WebContent/WEB-INF/classes external/wsdl/Member.wsdl
no such JAXB option: -classpath WebContent/WEB-INF/classes
Usage: wsimport [options] <WSDL_URI>
where [options] include:
-b <path>                 specify jaxws/jaxb binding files or additional schemas
                            (Each <path> must have its own -b)
-B<jaxbOption>            Pass this option to JAXB schema compiler
-catalog <file>           specify catalog file to resolve external entity references
                            supports TR9401, XCatalog, and OASIS XML Catalog format.
-d <directory>            specify where to place generated output files
<...snipped...>... where WebContent/WEB-INF/classes is the classpath where au.com.xxx.jaxb.DateAdapter.class could be found. Obviously it did not like it.
Also, why is wsimport generate org.w3._2001.xmlschema.Adapter1.java ? All it is doing is wrapping up the exact same call that I have specified in my DateAdapter. How can I tell wsimport or xjc to NOT create that extra Adapter1.java and simply directly use my DateAdapter ??

create additional column of type LONG to represent date.
dateFormat is of type java.util.Date:
long newLongDate = dateFormat.getTime();
select object(b) from MyEntity b where b.MYLONGDATE > ?1 and b.MYLOGDATE <= ?2

Java socket not handling the data packets properly

I have created a small webserver application where i am receiving http requests from a siebel client. I am using content-length parameter to read the complete data in a buffer string using the command inputstreamreader.read. It works fine in all the cases, except when its used with a firewall. Some times the firewall splits a message into 2 and my logic fails as it doesn't find as many characters in the first packet as mentioned in the content length.
My question is , how can i take care of these scenarios where a firewall is splitting the messages into multiple packets. why the java socket class is not handling the merging of these packets?
any pointers are welcome.
thanks

Because that's the way TCP/IP works. read() gives you the data it can, and tells you how much was actually read. It's up to the application to reconstruct the application-level object. You could do something like this:    byte[] content = new byte[contentLen];
    int offset = 0;
    int remaining = contentLen;
    int currBytesRead = 0;
    while (remaining > 0 && currBytesRead != -1) {
      int currBytesRead = mySocketInputStream.read(content, offset, remaining);
      remaining -= currBytesRead;
      offset += currBytesRead;
    } (Warning: that's off the top of my head, uncompiled and untested. Use for demonstration purposes only!)
Grant

Java socket: Pending written data

HI
If I open socket a connection to a server then write some data to it's outputstream then close it immediately after, am I guaranteed the data will be delivered before the underlying connection is closed?
String data = "hello!";
Socket s = new Socket("yahoo.com",80);
s.getOutputStream().write(data.getBytes());
s.close()Is the server guaranteed to receive hello?
Thanks,
Mike

am I guaranteed the data will be delivered before the underlying connection is closed?No.
Is the server guaranteed to receive hello?If it can be delivered at all it will be delivered. But socket writes and closes are asynchronous.

Java socket - unsure road to take!

Hi all,
We have an application requiring data transfer from a JSP Web application (socket client) and a stand-alone java application (socket server).
This java stand-alone app communicates to an industrial microcontroller connected to the RS-232 serial port.
When this java apps received a command from the JSP web app, it cannot service any other command from other uses, since there is just one RS-232 serial port on the computer. We would like to rely on the java & network environment to buffer additional commands receive simultaneously, as opposed to write code on the java socket server to handle this. In other works, our java stand-alone operates in an assynchronous, single-task mode.
We are unsure which approach to take as far as java socket goes:
1) simple socket server/client;
2) multithread socket
3) pooled
Can anyone advise us.
Thank you
Trajano

Hi Pete,
1) Yes, we can have more than one user trying to access the micro-controllers, because we have in reality a network of micro-controllers connected to the single RS-232 computer port via a RS-485 to RS-232 converter;
2) If a second user tries to issue a command to micro-controller, I would prefer for the java socket/environment to take care of the buffering, by just delaying the response, until the first user gets its command serviced
3) If there is 1 user, him/her might issue several commands at the same time, by opening several JSP pages;
4) No the controllers can service any instruction coming up at any time. The protocol is master/slave. The java app issues a request and the micro-controlle replies back and ready to accept any other request.
ISSUE: My preference is for the system to take care of commands arriving at the socket server.
This java app has two threads:
1) Thread1 is the java socket server and upon receiving a command it will update three (3) properties:
micro-controller address, command and product code
2) Thread32 will be responsible for polling the 3 properties to check if they've changed. Upon detecting a change, it will build the data string and send to the RS-232 serial port.
Any ideas/suggestions.
Thanks in advance for any assistance.
Regards

Trouble with sending huge arrays via DataSockets.

Hi,
I am having trouble sending huge arrays via Data Sockets from my main vi on the server PC to the vi on the client PC.
To further elaborate, in my main vi program I have created 3 arrays called Array1, Array2 and Array3 on my front Panel. By right clicking the mouse on each set of array and from the pop-up menu I selected Data Operations-> DataSocket Connection and I entered dstp://localhost/Array1 and clicked on Publish to broadcast the data from Array1. Similarly, I did the same for Array2 and Array3.
Now, in my client vi program I have created three arrays on my front Panel to read (Subscribe) the data from the three arrays broadcasted via DataSockets from the server’s main vi program. To subsc
ribe the data I did the similar process above and clicked on Subscribe to read the data (of course the IP address of the client PC will be different then the server PC so enter the hosts IP address). Once the data is received in the client arrays, I am using LV2 globals so that I can use these arrays on other sub-vi’s locally (instead of having each sub-vi get data from the server directly).
I succeeded in doing this with two arrays, however when I added the third array the DataSockets would not work consistently. For example the refresh rate would get slower at times and sometimes 2 out of the 3 arrays would update. I don’t know if I have exceeded the limit on how much data DataSockets can broadcast, but I need to have some mechanism to broadcast 6 arrays (approx. 10000 elements for each array) of double digits precision.
Has anyone come across this issue? Is there another way of broadcasting data more efficiently then DataSockets?
I would appreciate any
help that I can get.
I have attached the files for this program in the zip file.
First run the Server main program, testServeMainVI.vi, and then the client program, testClientMainVI.vi.
Thanks
Nish
Attachments:
beta2.zip ‏70 KB

DataSocket can be a lossy communication. I like the advice to flatten the data to a string, but another option would be to buffer the communcation. The problem might be that the data is being overwritten on the server faster than it is being read. There is an example of buffered datasocket on NI web page: http://venus.ni.com/stage/we/niepd_web_display.DISPLAY_EPD4?p_guid=BA3F9DFED17F62C3E034080020E74861&p_node=DZ52000_US&p_submitted=N&p_rank=&p_answer=&p_source=Internal
Also, I have played with the new built in buffered datasocket in LabVIEW 7.0. It is pretty slick. If buffers the data both on the server and the client side.

How to open-up a pdf file to a particular page via java? (file and page number from another program)

I have created an Access 2003 Form. Data is entered. The data can be viewed once entered in the same form format, with one addition. The addition is a link to another program where we store files. The files stored are pdf documents. Each document is a scanned file (500+ pages). The link can open-up the pdf file. I want the link to open-up the pdf file to a particular page. I will supply the page number from the database.
I am aware of Adobe Acrobat 7.0 PDF Open Parameters using a URL that specifies the file to be opened plus actions to be performed once the file is openend (i.e. http://example.org/doc.pdf#page=3). However, the 3rd party program transfers a file from a server to the client via java. The file does not go via a web browser. The java code transfer the file directly from the file server to the the client.
Any idea how to accomplish the task of viewing a particular page with the process flow described?
I am using Adobe Acrobat 7.0 Professional.

So you are reading this file in a standalone version of Acrobat? If so then use the PDF Open Parameters /a option via the command line.
Sabian
Example:
C:\Program Files\Adobe\Acrobat 7.0\Acrobat>Acrobat.exe /A "zoom=1000=OpenActions" "C:\Program Files\Adobe\Acrobat 7.0\Acrobat\SimpleSubmit.pdf"
Where:
Path to Acrobat: C:\Program Files\Adobe\Acrobat 7.0\Acrobat>Acrobat.exe
Note: The example starts in the Acrobat application directory
Switch: /A
Parameter Syntax: "zoom=1000=OpenActions"
Path to file: "C:\Program Files\Adobe\Acrobat 7.0\Acrobat\SimpleSubmit.pdf"
Note: The quotes are very important for the syntax to work appropriately.

Scheduling webi reports via Java SDK in BI 4.1

Hi,
Has anyone been able to schedule a Webi report containing date prompts via Java SDK in BI 4.1? I have tried doing so but it doesn't work when date prompts are involved, I get the exception raised: java.lang.NullPointerException error. Though the scheduled instance is created and the date prompt appears to be correctly populated the report fails with the above mentioned error message. The code is working for other text prompts but fails when I have a date prompt in the report. I am using the populateWebiPrompts() method to populate the prompts.
CF

Hi Christopher,
Try adding the following line in your code before scheduling:
iWebi.setUserInputLocaleName(documentInstance.getProperties().getProperty(PropertiesType.BASE_LOCALE));
Also as an FYI, please refer below details.
For scheduling webi reports with prompts, the only supported way is to use Restful webservices SDKs.
The feature of scheduling a webi report with prompts were deprected from release BI 4.x and introduced in the new restfull webservices. Restful is the future which SAP is focussing and would be great to have your application on supported terms with SAP.
You can get details about the restful webservices from the documents and blogs available at below forum
http://scn.sap.com/community/restful-sdk
Thanks,
Prithvi

How to create table in interactive form via Java Web Dynpro

Hi,
How to create table in interactive form via Java Web Dynpro ?
Any online tutorial / example ?
Thank you.
Regards,
Eric

Hi Eric,
Just choose the UI element Table from Form Library and drag and drop it on the form. now choose the no. of rows and columns and other settings you want about table from the wizard initiated through this process. This all is what you have to do to create the table. Now to bind it to the fields of the data source bind the individual colums to individual attributes of the node in the datasource.
Hope it will solve your query.
Regards,
Vaibhav Tiwari.

Using plain Java Sockets(not RMI) how..?

hi!
1. Using plain Java Sockets(not RMI) how can the client detect when its server
goes down?
There is a long time interval between client requests and the client wants
to retain a live connection rather than disconnect after every reqest.
Please also cc your reply to [email protected]
Thanks,
\Raghu

If you try to send data when the host is gone, it throws an exception. I don'thow to check if it is alive though. I'm having the same problem right now.

Java socket streams messing up

iv wrote a games server and it uses java socket streams to send the messages back and forth between the game clients and the game server it self
this all works great but when running and over about 5min its send over 1000 packets BUT
it will crash iv found the reason is down to when reading the packets my parser is reading wrong part and grabbing a String part when it should be a INT
now it should NEVER be reading this part of the packet so i put a trace on the packets coming in and ran it every packet was been read in perfect THEN just befor the crash TWO packets were merged together stright out from the socket stream this is y my packet reader was getting confused
what im asking is can this happen in java sockets ? i thought they were bufferered so no matter how fast you send data it would always read it out in the same order ?? not sure how its merging these to packets
is this possible of is my reader got an error in it altho works 95% of the time ??

or do you have multiple buffered streams created from the same socket?

Posting huge data on to JSP page using JSTL tags

Hi,
I have one application where I have to post huge data (approximately 2000 rows of data) into JSP page. I am using JSTL tags and running on Tomcat 5.
It is taking almost 20 to 25 seconds to load the entire page.
Is it the optimal time to load or it could be improved?
Please let me know.
Thanks,
--Subbu.

Hi Evnafets,
Thank you for the response.
Here are the tasks I am doing to display the data on JSP.
0. We are running on Tomcat 5 and the memory size is 1024MB (1GB).
1. Getting the data - I am not performing any database queries. The data is stored in the static cache memory. So the server side response is so quick - less than a milli second.
2. Using Java beans to pass data to the presentation layer (JSP).
3. 10 'if' conditions and 2 'for' loops and 2 'choose' statements are being used in the JSP page while displaying the data.
4. Along with the above, there are 4 javascript files are being used.
5. The jsp file size after rendering the data, is aprox. 160 kb
Hope this information helps you to understand the problem.
Thanks,
--Subbu.

Change mm/dd/yyyy to dd/mm/yyyy when selecting date via dateSelector??

Hi Experts,
i have attribute of date type that has been binded to input field in a web dynpro Java application. when application is executed user can select a date via DateSelector that appears automatically.
Problem is after the date has been selected input field is populated in mm/dd/yyyy format ( ex 03/28/2008). My client wants that to be changed . It must read 28/03/2008 (dd/mm/yyyy).
I tried creating simple type using base type Date and setting representation dd/MM/yyyy however that does NOT work!!
Edited by: Eureka on Mar 28, 2008 5:30 PM

Hi,
Simple type with representation dd/MM/yyyy works for me.
Here is what I have done.
Create a simple type
Name :testDateFormat
package : com.test
Dictionary : Local Dictionary
Built-In-Type : Date
In the Representation Tab :
Format : dd/MM/yyyy
In the WD view created a context of type : com.test.testDateFormat and assinged the context to the inputfield
Now I see date format dd/MM/yyyy, make sure the u specify the correct format.
regards,
AE

How to access calendar rules via java

Hi to all,
We have a process in which we need to set a Timer, but the date must be validated against Calendar Rules. We searched in forums, webs, documentation and we couldn't find a way to do this.
I know it is possible to set an expiration date based on Business Calendar on a Human Task (deadline tab in Human task editor), but this is not what we need.
We actually need a way to set a variable date in the process that can handle only business days. e.g.: 'now' + 2 business days.
Right now we are setting the Timers with this expression from a Business Rule: Duration.from string("PT48H")
Then I associate the business rule output with a Time type variable(deadlineTime) doing: 'now' + deadlineTime
So right after that I can use the Time variable on a timer attached to the human tasks I need. * We need to use business rules because the customer may want to change the times accordingly to their needs through BPM Composer. *
The problem is that this solution does not take into account Saturday and Sundays (holidays are a concern too).
We need to set the expiration time to timers attached to human tasks, but the time should consider working days only. We consulted with experts from Oracle and told us that functionality is available in the PS 6 version. At the moment it is impossible to migrate to that version. I need to know how to access via Java APIs to the calendar rules for checking programmatically.
JDeveloper version: 11.1.1.6
Any suggestion?
Thanks in advance
Marcelo

Hi Yarner,
Once you have all your dll included in a jar and used the nativelib tag, you have to use, at the beginning of you application, the command System.loadLibrary to load all the dll you need. Including the ones called by the others you have direct access.
The dll�s have to be explicitly loaded in the order they are called.
For example: you use lib1.dll and lib2.dll. The lib1.dll needs lib0.dll.
Even if you don't use lib0.dll directly, once its called by lib1.dll you have to put then in the order.
System.loadLibrary("lib0.dll");
System.loadLibrary("lib1.dll");
System.loadLibrary("lib2.dll");
I hope it may help you, good luck.
Mario

Transferin Huge data via Java sockets! Problematic!

Similar Messages

Maybe you are looking for