TCP XML messages parsing time with realtime output.

Question

I am currently coding a project and I have hit a bit of a roadblock. The basic overview is I need to be able to receive around 3,000 lines of XML per second from a JavaScript sending TCP messages. I have to extract the data to two seperate files, output the data to a text pane and also further seperate that data into a table as well. All this data needs to be updated as soon as it is received. I am currently comfortable with receiving around 700 lines per second without data loss but anything over that I start losing data somewhere. As for the basic over view of my code, I have my main program which creates the GUI and starts a listening thread that listens for "clients"(JavaScripts) to connect.
Main:
public static void main(String[] args)
     SplashScreen screen = new SplashScreen();
     screen.showSplashWindow();
     SwingUtilities.invokeLater(new Runnable()
         public void run()
          MainMenu mainmenu = new MainMenu();
          mainmenu.setVisible(true);
          mainmenu.setDefaultCloseOperation(JFrame.DISPOSE_ON_CLOSE);
    }Listener Thread:
final Runnable listener_runner = new Runnable()
          public void run()
              try
               startClientListener();
              catch (Exception e)
         final Thread listener_thread = new Thread(listener_runner,
              "ListenerThread");
         listener_thread.start();Called from listening thread to setup client:
public void startClientListener() throws IOException
     ServerSocket serverSocket = null;
     try
         serverSocket = new ServerSocket(XXXX);
     catch (IOException e)
         System.err.println("Could not listen on port: " + portNumber);
     while (true)
         MultiServerThread new_thread = new MultiServerThread(serverSocket
              .accept(), outputWindow, resultsWindow, prefAttr, runningCount);
         new_thread.start();
    }In my MultiServerThreads runnable I have a BufferedReader to read the input. This reader loops through the input and adds it to a StringBuilder which then passes that data to the output files and the XML parser as seen here:
in = new BufferedReader(new InputStreamReader(socket
              .getInputStream()));
         while ((input_line = in.readLine()) != null)
          build.append(input_line + "\n");
          file_out.append(build);  //Text file
          xml_copy_out.append(build);  //XML file
          loadData(build);  //XML parser
          build = new StringBuilder();
         }My XML parser is configured as follows:
public void loadData(final StringBuilder bufs)
     bufs.insert(0, "<VTR>");
     bufs.append("</VTR>");
     try
         StructuredDocumentHandler par = new StructuredDocumentHandler(
              runningCount);
         SAXParserFactory spf = SAXParserFactory.newInstance();
         spf.setValidating(false);
         javax.xml.parsers.SAXParser sp = spf.newSAXParser();
         final String s = bufs.toString();
         final ByteArrayInputStream file_buf = new ByteArrayInputStream(s
              .getBytes());
         final DataInputStream in_data = new DataInputStream(file_buf);
         org.xml.sax.InputSource input = new InputSource(in_data);
         sp.parse(input, par);
     catch (Exception ex)
         ex.printStackTrace();
    }As the parser goes through each XML message it sends the String to an output window that is straight text in a TextArea. It also stores the data in some arrays which then get built into a row that is added to a JTable. Does anyone have any ideas on how I can get up to 3000 lines per second? I'm not sure if my threading scheme is wrong or what. Is Java not powerful enough to handle 3000 TCP messages per second or can the XML parser not handle that much processing that fast? Any help would be much appreciated because I've kind of run out of ideas. Thanks!

Answer 1

Thanks for the replies. I've trimmed my code down a little bit which has helped. I'm still around 2000 messages a second. Here's where I'm at for an update. Thanks again for all your help. I'm much happier at 2k vs 700 messages per second.
import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.ByteArrayInputStream;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileWriter;
import java.io.InputStreamReader;
import java.net.Socket;
import java.util.ArrayList;
import java.util.Calendar;
import javax.swing.JOptionPane;
import javax.xml.parsers.SAXParser;
import javax.xml.parsers.SAXParserFactory;
import org.xml.sax.Attributes;
import org.xml.sax.InputSource;
import org.xml.sax.helpers.DefaultHandler;
public class MultiServerThread extends Thread
    private final StructuredDocumentHandler par = new StructuredDocumentHandler();
    private final SAXParserFactory spf = SAXParserFactory.newInstance();
    private SAXParser sp;
    public MultiServerThread(final Socket socket,
         final OutputWindow output_window,
         final ResultsWindow results_window,
         final PreferencesDialog.Info attr, final int count)
     try
         spf.setValidating(false);
         sp = spf.newSAXParser();
     catch (Exception ex)
    @Override
    public void run()
     try
         xml_copy_out.append("<VTR>\n");
         in = new BufferedReader(new InputStreamReader(socket
              .getInputStream()));
         while ((input_line = in.readLine()) != null)
          file_out.append(input_line + StaticVariable.NEW_LINE);
          xml_copy_out.append(input_line + StaticVariable.NEW_LINE);
          loadData(input_line);
         xml_copy_out.append("</VTR>");
         xml_copy_out.close();
         file_out.close();
         in.close();
         attributes.parent.setCheckFailed(runningCount, procPassed);
         socket.close();
     catch (FileNotFoundException ex)
     catch (Exception e)
         e.printStackTrace();
   public void killOuts()
    public void loadData(final String xml_str)
     try
         ByteArrayInputStream file_buf = new ByteArrayInputStream(xml_str.getBytes());
         InputSource input = new InputSource(file_buf);
         sp.parse(input, par);
     catch (Exception ex)
         ex.printStackTrace();
    public void addRowData(boolean procedurePassed)
    public void clearArrays()
    public class StructuredDocumentHandler extends DefaultHandler
     boolean procedurePassed = true;
     boolean isTestCase = false;
     boolean isTestCaseInput = false;
     boolean isTestCaseOutput = false;
     boolean isTestStep = false;
     boolean isComment = false;
     boolean isRequirement = false;
     boolean isEnsure = false;
     boolean isVerify = false;
     boolean isResult = false;
     boolean eval = false;
     public StructuredDocumentHandler()
     public void startElement(String uri, String lname, String qname,
          Attributes attributes)
         if (!qname.equals(StaticVariable.XML_COMMAND))
          eval = true;
          if (qname.equals(StaticVariable.XML_TESTCASE))
              isTestCase = true;
          else if (qname.equals(StaticVariable.XML_TESTCASEINPUT))
              isTestCaseInput = true;
          else if (qname.equals(StaticVariable.XML_TESTCASEOUTPUT))
              isTestCaseOutput = true;
          else if (qname.equals(StaticVariable.XML_TESTSTEP))
              isTestStep = true;
          else if (qname.equals(StaticVariable.XML_COMMENT))
              isComment = true;
          else if (qname.equals(StaticVariable.XML_REQUIREMENT))
              isRequirement = true;
          else if (qname.equals(StaticVariable.XML_MEASUREMENT))
              String xml_val = attributes
                   .getValue(StaticVariable.XML_VALUE);
              outputWindow.addData(xml_val + StaticVariable.NEW_LINE);
              measurementArray.add(xml_val);
          else if (qname.equals(StaticVariable.XML_EXPECTED))
              String xml_val = attributes
                   .getValue(StaticVariable.XML_VALUE);
              outputWindow.addData(xml_val + StaticVariable.NEW_LINE);
              expectedArray.add(xml_val);
          else if (qname.equals(StaticVariable.XML_RESULT))
              isResult = true;
          else if (qname.equals(StaticVariable.XML_VERIFY))
              isVerify = true;
              verifyArray.add(new String(attributes
                   .getValue(StaticVariable.XML_RESULT)));
          else if (qname.equals(StaticVariable.XML_ENSURE))
              isEnsure = true;
     public void characters(char[] chars, int start, int length)
         if (eval)
          if (isTestCase)
              outputWindow.addData(new String(chars, start, length)
                   + StaticVariable.NEW_LINE);
              testCaseArray.add(new String(chars, start, length));
          else if (isTestCaseInput)
              outputWindow.addData(new String(chars, start, length)
                   + StaticVariable.NEW_LINE);
              testCaseInputArray.add(new String(chars, start, length));
          else if (isTestCaseOutput)
              outputWindow.addData(new String(chars, start, length)
                   + StaticVariable.NEW_LINE);
              testCaseOutputArray.add(new String(chars, start, length));
          else if (isTestStep)
              outputWindow.addData(new String(chars, start, length)
                   + StaticVariable.NEW_LINE);
              testStepArray.add(new String(chars, start, length));
          else if (isComment)
              outputWindow.addData(new String(chars, start, length)
                   + StaticVariable.NEW_LINE);
              commentArray.add(new String(chars, start, length));
          else if (isRequirement)
              outputWindow.addData(new String(chars, start, length)
                   + StaticVariable.NEW_LINE);
              requirementArray.add(new String(chars, start, length));
          else if (isResult)
              outputWindow.addData(new String(chars, start, length)
                   + StaticVariable.NEW_LINE);
              resultArray.add(new String(chars, start, length));
          else if (isVerify)
              outputWindow.addData(new String(chars, start, length)
                   + StaticVariable.NEW_LINE);
              verifyDescriptionArray
                   .add(new String(chars, start, length));
          else if (isEnsure)
              outputWindow.addData(new String(chars, start, length)
                   + StaticVariable.NEW_LINE);
              ensureArray.add(new String(chars, start, length));
     public void endElement(String uri, String lname, String qname)
         if (!qname.equals(StaticVariable.XML_COMMAND))
          if (qname.equals(StaticVariable.XML_VERIFY))
              addRowData(procedurePassed);
              clearArrays();
          else if (qname.equals(StaticVariable.XML_END))
              clearArrays();
          isTestCase = false;
          isTestCaseInput = false;
          isTestCaseOutput = false;
          isTestStep = false;
          isComment = false;
          isRequirement = false;
          isResult = false;
          isVerify = false;
          isEnsure = false;
          eval = false;
}Any other suggestions on trimming down that I'm not seeing would be much appreciated. Thanks again for all your help.