Using sockets to relay video stream

Question

Hi there,
I've been working with java for 4 years now, and I never had troubles making the applications I wanted with it...since my last project :(
I hope you, gurus from the java community forum, can help me sort this out, but let me explain it to you:
I'd like to stream video content (let's say from a webcam) to several spectators. But my upload bandwidth (ADSL line) is limited to 512kbps, which is far from enough to send a video streaming (with a bitrate of ~400kbps) to 10 or more viewers.
That's why I decided to rent a distant server (with a bandwidth of 5Mbps download/upload), and make a java application run on it, which would be able to broadcast the stream to all the viewers.
So my computer (connected to adsl) captures the video from the webcam, sends it via socket to the rented server, which broadcasts the video to all the spectators. If no video is sent to the rented server, then it should broadcast local files to the spectators.
Here is a schema of the project so you understand easily (my english is not as good as I'd like it to be, so you may not understand with words only):
http://www.bf-stream.com/schema1.gif
My application runs quite well with 5 spectators, but as soon as more viewers come to see the video, it starts hanging, and all the viewers suffer lags and buffering, which is not exactly what I expected...
Of course, I checked the administration interface on the rented server to see if this was due to a cpu/ram/bandwitdh problem. With no success: with 10 viewers connected, the app uses less than 1% of the cpu power, something like 200kb of RAM, and a bandwidth of 3-4Mbps (which is still under the actual bandwidth capacity of the server).
That's why I came to the conclusion that my java app was not as good as I thought, and decided to show it to you, so you can help me make it better.
So here is a schema showing the classes and their interactions:
http://www.bf-stream.com/schema2.gif
And here are the sources of the classes:
StreamRelay.java:
package com.RR4;
import java.net.Socket;
import java.util.Vector;
import java.util.Observable;
import java.util.Observer;
* StreamRelay:
*  Main class controlling every aspects of the stream
*  (live content, or files read from hard disk)
public class StreamRelay extends Observable implements Observer  {
     static public String VERSION = "0.5";
     static public int BUFFER_SIZE = 16;
     private StreamServer streamServer;
     private LiveServer liveServer;
     LocalFile localFile;
     Vector clients;
     boolean streamFromSocket;
     public StreamRelay(int livePort, int relayPort) {
          streamFromSocket = false;
          // clients: vector containing every StreamClient connected
          clients = new Vector();
          // localFile: class reading local files when no live is sent to the relay
          localFile = new LocalFile(this);
          new Thread(localFile).start();
          // streamServer: server listening for new StreamClient
          streamServer = new StreamServer(this, relayPort);
          new Thread(streamServer).start();
          // liveServer: server listening for new sources for live content
          liveServer = new LiveServer(this, livePort);
          new Thread(liveServer).start();
     * addClient: adds new StreamClient to 'clients' vector (a StreamClient is a 'spectator')
     public void addClient(Socket clientSocket) {
         StreamClient client = new StreamClient(this, clientSocket);
     * removeClient: removes a StreamClient from 'clients' vector
     public void removeClient(int index) {
          clients.remove(index);
     * update: updates every StreamClient (which are all Observers of StreamRelay)
     public void update(Observable observable, Object obj) {
          if ((observable.getClass().toString().indexOf("LocalFile")!=-1&&!streamFromSocket)||(observable.getClass().toString().indexOf("LiveStream")!=-1)) {
               this.notifyObservers( (byte[]) obj);
               this.setChanged();
     public static void main(String[] args) {
          new StreamRelay(8000, 8080);
LocalFile.java:
package com.RR4;
import java.util.Observable;
import java.io.*;
* LocalFile: class reading local file when no live content is sent to the relay
public class LocalFile extends Observable implements Runnable {
     // filenames is an array containing a list of files to be read when no live content is sent
     private String[] filenames = {
          "file1",
          "file2"
     private InputStream stream;
     boolean streamOpened;
     private StreamRelay parent;
     int fileIndex;
     // Constructor: initialises LocalFile
     //  sets fileIndex to 0, and sets main StreamRelay as Observer
     public LocalFile(StreamRelay parent) {
          this.parent = parent;
          fileIndex = 0;
          this.addObserver(parent);
          initStream();
      * initStream: initialises stream to read the next file in 'filenames' array
     public void initStream() {
          try {
               stream = new BufferedInputStream(new FileInputStream(filenames[fileIndex]), StreamRelay.BUFFER_SIZE);
               streamOpened = true;
               fileIndex++;
               if (fileIndex==filenames.length)
                    fileIndex = 0;
          } catch (Exception exp) {
               exp.printStackTrace();
               streamOpened = false;
      * run: main loop of the class.
      *     the file is actually read: a buffer of BUFFER_SIZE is filled and sent to
      *     the observer (StreamRelay)
     public void run() {
          byte[] buffer = new byte[StreamRelay.BUFFER_SIZE];
          byte[] bufferToSend;
          boolean quit = false;
          int actualBufferSize = 0;
          while (!quit) {
               try {
                    this.setChanged();
                    // Bytes are read until the buffer is actually filled with BUFFER_SIZE bytes
                    // Only then is it sent to the observer...
                    while (streamOpened && ((actualBufferSize = stream.read(buffer, 0, StreamRelay.BUFFER_SIZE)) > 0)) {
                         if (parent.clients.size()>0 && (!parent.streamFromSocket)) {
                              if (actualBufferSize<StreamRelay.BUFFER_SIZE) {
                                   bufferToSend = new byte[actualBufferSize];
                                   System.arraycopy(buffer, 0, bufferToSend, 0, actualBufferSize);
                                   this.notifyObservers(bufferToSend);
                              } else
                                   this.notifyObservers(buffer);
                              this.setChanged();
                         } else {
                              try { Thread.sleep(100); } catch (Exception exp) {}
               } catch (Exception exp) {
                    exp.printStackTrace();
               try { stream.close(); } catch (Exception exp) {}
               initStream();
StreamServer.java:
package com.RR4;
import java.net.ServerSocket;
public class StreamServer extends Thread {
     private ServerSocket serverSocket;
     private boolean socketOpened;
     private StreamRelay parent;
     * StreamServer: server listening for new StreamClient
     public StreamServer(StreamRelay parent, int relayPort) {
          this.parent = parent;
          try {
               serverSocket = new ServerSocket(relayPort);
               socketOpened = true;
          } catch (Exception exp) {
               exp.printStackTrace();
               socketOpened = false;
     public void run() {
          try {
               while (socketOpened) {
                    parent.addClient(serverSocket.accept());
          } catch (Exception exp) {
               exp.printStackTrace();
StreamClient.java:
package com.RR4;
import java.net.Socket;
import java.util.Observer;
import java.util.Observable;
import java.io.*;
* StreamClient: class representing spectators connected to view the video content
*     whether it is live content, or local files
public class StreamClient implements Observer {
     private Socket socket;
     private OutputStream outStream;
     private boolean connected = true;
     private StreamRelay parent;
     public StreamClient(StreamRelay parent, Socket socket) {
          this.parent = parent;
          this.socket = socket;
          try {
               // initialises OutputStream from socket
               outStream = socket.getOutputStream();
          } catch (Exception exp) {
               try { outStream.close(); } catch (Exception e) {}
               try { socket.close(); } catch (Exception e) {}
               exp.printStackTrace();
               connected = false;
          if (connected) {
               // if initializing of OutputStream didn't fail
               // add this class to StreamBroadcast 'clients' vector
               // and add this class to StreamRelay observers
               parent.clients.add(this);
               parent.addObserver(this);
      * update: actually send read bytes to the client
     public void update(Observable observable, Object obj) {
         try {
               outStream.write( (byte[]) obj);
               outStream.flush();
          } catch (Exception exp) {
               // if read bytes couldn't be sent
               // remove this client from clients list and observers of StreamRelay
               // and try to close OutputStream and Socket
               connected = false;
               try { parent.deleteObserver(this); } catch (Exception e) {}
               try { parent.clients.remove(this); } catch (Exception e) {}
               try { outStream.close(); } catch (Exception e) {}
               try { socket.close(); } catch (Exception e) {}
LiveServer.java:
package com.RR4;
import java.net.ServerSocket;
* LiveServer:
*     SocketServer listening to new 'Live streams'
public class LiveServer extends Thread {
     private ServerSocket liveSocket;
     private boolean liveServerOpened;
     private StreamRelay parent;
     public LiveServer(StreamRelay parent, int livePort) {
          this.parent = parent;
          try {
               liveSocket = new ServerSocket(livePort);
               liveServerOpened = true;
          } catch (Exception exp) {
               exp.printStackTrace();
               liveServerOpened = false;
     public void run() {
          LiveStream liveStream;
          try {
               while (liveServerOpened) {
                    liveStream = new LiveStream(parent, liveSocket.accept());
                    new Thread(liveStream).start();
          } catch (Exception exp) {
               exp.printStackTrace();
LiveStream.java:
package com.RR4;
import java.util.Observable;
import java.io.*;
import java.net.Socket;
*     LiveStream:
*          Socket receiving live content from another distant computer
*          to display it instead of the local files.
public class LiveStream extends Observable implements Runnable {
     private InputStream stream;
     boolean streamOpened;
     private StreamRelay parent;
     public LiveStream(StreamRelay parent, Socket socket) {
          this.parent = parent;
          this.addObserver(parent);
          try {
               stream = new BufferedInputStream(socket.getInputStream(), StreamRelay.BUFFER_SIZE);
               streamOpened = true;
          } catch (Exception exp) {
               exp.printStackTrace();
               streamOpened = false;
     public void run() {
          byte[] buffer = new byte[StreamRelay.BUFFER_SIZE];
          byte[] bufferToSend;
          int actualBufferSize = 0;
          try {
               this.setChanged();
               // Bytes are read until the buffer is actually filled with BUFFER_SIZE bytes
               // Only then is it sent to the observer...
               while (streamOpened && ((actualBufferSize = stream.read(buffer, 0, StreamRelay.BUFFER_SIZE)) > 0)) {
                    if (!parent.streamFromSocket)
                         parent.streamFromSocket = true;
                    if (parent.clients.size() > 0) {
                         if (actualBufferSize<StreamRelay.BUFFER_SIZE) {
                              bufferToSend = new byte[actualBufferSize];
                              System.arraycopy(buffer, 0, bufferToSend, 0, actualBufferSize);
                              this.notifyObservers(bufferToSend);
                         } else
                              this.notifyObservers(buffer);
                         this.setChanged();
                    } else
                         try { Thread.sleep(100); } catch (Exception exp) {}
          } catch (Exception exp) {
               exp.printStackTrace();
          } finally {
               try { stream.close(); } catch (Exception exp) {}
               this.deleteObserver(parent);
               parent.streamFromSocket = false;
}For your information, I uses NSV/VP6 vbr as video codec (but this should have no incidence on it, since the app only takes the video stream from a socket and broadcasts it to other sockets, without analysing it or modifying it). And the java app is hosted on a Celeron 2.6 GHz, 128MB RAM.
I really hope you'll be able to help me with this project, as it is really important to me...
I've been trying several Stream types available with the JDK but I hadn't any success... I've also been playing with the BUFFER_SIZE parameter, unsuccessfully too...
Anyway, thanks in advance for reading me so far... and I hope, for helping me... I know the java community is strong, and I hope I won't have to make it with C or C++ :(

Answer 1

Hi again :)
I've been focusing on the local part of the stream (no live video, just playing local files and sending them to clients) using NIO.
NIOStreamRelay.java:
package com.RR4;
import java.io.*;
import java.nio.*;
import java.nio.channels.*;
import java.nio.channels.spi.*;
import java.net.*;
import java.util.*;
public class NIOStreamRelay {
     static int BUFFER_SIZE = 1024;
     LocalFile localFile;
     Vector clients;
     public NIOStreamRelay() throws IOException {
          localFile = new LocalFile(this);
          new Thread(localFile).start();
          clients = new Vector();
          acceptConnections();
     public void acceptConnections() throws IOException {          
          // Selector for incoming requests
          Selector acceptSelector = SelectorProvider.provider().openSelector();
          // Create a new server socket and set to non blocking mode
          ServerSocketChannel ssc = ServerSocketChannel.open();
          ssc.configureBlocking(false);
          // Bind the server socket to the local host and port
          InetAddress lh = InetAddress.getLocalHost();
          InetSocketAddress isa = new InetSocketAddress(lh, 8080);
          ssc.socket().bind(isa);
          // Register accepts on the server socket with the selector. This
          // step tells the selector that the socket wants to be put on the
          // ready list when accept operations occur, so allowing multiplexed
          // non-blocking I/O to take place.
          SelectionKey acceptKey = ssc.register(acceptSelector, SelectionKey.OP_ACCEPT);
          int keysAdded = 0;
          // Here's where everything happens. The select method will
          // return when any operations registered above have occurred, the
          // thread has been interrupted, etc.
          while ((keysAdded = acceptSelector.select()) > 0) {
              // Someone is ready for I/O, get the ready keys
              Set readyKeys = acceptSelector.selectedKeys();
              Iterator i = readyKeys.iterator();
              // Walk through the ready keys collection and process date requests.
              while (i.hasNext()) {
                    SelectionKey sk = (SelectionKey)i.next();
                    i.remove();
                    // The key indexes into the selector so you
                    // can retrieve the socket that's ready for I/O
                    ServerSocketChannel nextReady = (ServerSocketChannel)sk.channel();
                    // Accept the date request and send back the date string
                    Socket s = nextReady.accept().socket();
                    OutputStream socketStream = s.getOutputStream();
                    StreamClient newClient = new StreamClient(socketStream, this);
                    localFile.addObserver(newClient);
                    clients.add(newClient);
     public static void main(String[] args) {
          try {
               NIOStreamRelay streamRelay = new NIOStreamRelay();
          } catch (Exception e) {
               e.printStackTrace();
LocalFile.java:
package com.RR4;
import java.util.Observable;
import java.io.*;
* LocalFile: class reading local file when no live content is sent to the relay
public class LocalFile extends Observable implements Runnable {
     // filenames is an array containing a list of files to be read when no live content is sent
     private String[] filenames = {
          "test.nsv",
          "test2.nsv"
     private InputStream stream;
     boolean streamOpened;
     int fileIndex;
     NIOStreamRelay parent;
     // Constructor: initialises LocalFile
     //  sets fileIndex to 0, and sets main StreamRelay as Observer
     public LocalFile(NIOStreamRelay parent) {
          this.parent = parent;
          fileIndex = 0;
          initStream();
      * initStream: initialises stream to read the next file in 'filenames' array
     public void initStream() {
          try {
               stream = new BufferedInputStream(new FileInputStream(filenames[fileIndex]), NIOStreamRelay.BUFFER_SIZE);
               streamOpened = true;
               fileIndex++;
               if (fileIndex==filenames.length)
                    fileIndex = 0;
          } catch (Exception exp) {
               exp.printStackTrace();
               streamOpened = false;
      * run: main loop of the class.
      *     the file is actually read: a buffer of BUFFER_SIZE is filled and sent to
      *     the observer (StreamRelay)
     public void run() {
          byte[] buffer = new byte[NIOStreamRelay.BUFFER_SIZE];
          byte[] bufferToSend;
          boolean quit = false;
          int actualBufferSize = 0;
          while (!quit) {
               try {
                    this.setChanged();
                    // Bytes are read until the buffer is actually filled with BUFFER_SIZE bytes
                    // Only then is it sent to the observer...
                    while (streamOpened && ((actualBufferSize = stream.read(buffer, 0, NIOStreamRelay.BUFFER_SIZE)) > 0)) {
                         if (parent.clients.size() > 0) {
                              if (actualBufferSize<NIOStreamRelay.BUFFER_SIZE) {
                                   bufferToSend = new byte[actualBufferSize];
                                   System.arraycopy(buffer, 0, bufferToSend, 0, actualBufferSize);
                                   this.notifyObservers(bufferToSend);
                              } else
                                   this.notifyObservers(buffer);
                              this.setChanged();
                         } else {
                              try { Thread.sleep(100); } catch (Exception exp) {}
               } catch (Exception exp) {
                    exp.printStackTrace();
               try { stream.close(); } catch (Exception exp) {}
               initStream();
StreamClient.java:
package com.RR4;
import java.io.*;
import java.net.*;
import java.util.*;
public class StreamClient implements Observer {
     OutputStream out;
     NIOStreamRelay parent;
     public StreamClient(OutputStream out, NIOStreamRelay parent) {
          this.out = out;
          this.parent = parent;
     public void update(Observable observable, Object obj) {
         try {
               out.write( (byte[]) obj);
               out.flush();
          } catch (Exception exp) {
               // if read bytes couldn't be sent
               // remove this client from clients list and observers of StreamRelay
               // and try to close OutputStream and Socket
               try { parent.localFile.deleteObserver(this); } catch (Exception e) {}
               try { parent.clients.remove(this); } catch (Exception e) {}
               try { out.close(); } catch (Exception e) {}
}Does it look better to you?
I know I'm still using single threading, but as the IO should be non-blocking, I guess it should be better.
Furthermore, I tried it locally, and was able to launch 30+ clients without buffering problems (huh, ok, my cpu here is only a 1.6GHz, so the display was a bit lagguy, but it didn't seem to buffer at all)