Writing a conference server for RTP streams

Question

Hello,
I'm trying to write a conference server which accepts multiple RTP streams (one for each participant), creates a mixed RTP stream of all other participants and sends that stream back to each participant.
For 2 participants, I was able to correctly receive and send the stream of the other participant to each party.
For 3 participants, creating the merging data source does not seem to work - i.e. no data is received by the participants.
I tried creating a cloneable data sources instead, thinking that this may be the root cause, but when creating cloneable data sources from incoming RTP sources, I am unable to get the Processor into Configured state, it seems to deadlock. Here's the code outline :
    Iterator pIt = participants.iterator();
    List dataSources = new ArrayList();
    while(pIt.hasNext()) {
      Party p = (Party) pIt.next();
      if(p!=dest) {
        DataSource ds = p.getDataSource();
        DataSource cds = Manager.createCloneableDataSource(ds);
        DataSource clone= ((SourceCloneable)cds).createClone();
        dataSources.add(clone);
    Object[] sources = dataSources.toArray(new DataSource[0]);
    DataSource dataSource =   Manager.createMergingDataSource((DataSource[])sources);
    Processor p = Manager.createProcessor(dataSource);
    MixControllerListener cl = new MixControllerListener();
    p.addControllerListener(cl);
    // Put the Processor into configured state.
    p.configure();
    if (!cl.waitForState(p, p.Configured)) {
        System.err.println("Failed to configure the processor.");
        assert false;
    }Here are couple of stack traces :
"RTPEventHandler" daemon prio=1 tid=0x081d6828 nid=0x3ea6 in Object.wait() [98246000..98247238]
        at java.lang.Object.wait(Native Method)
        - waiting on <0x9f37e4a8> (a java.lang.Object)
        at java.lang.Object.wait(Object.java:429)
        at demo.Mixer$MixControllerListener.waitForState(Mixer.java:248)
        - locked <0x9f37e4a8> (a java.lang.Object)
        at demo.Mixer.createMergedDataSource(Mixer.java:202)
        at demo.Mixer.createSendStreams(Mixer.java:165)
        at demo.Mixer.createSendStreamsWhenAllJoined(Mixer.java:157)
        - locked <0x9f481840> (a demo.Mixer)
        at demo.Mixer.update(Mixer.java:123)
        at com.sun.media.rtp.RTPEventHandler.processEvent(RTPEventHandler.java:62)
        at com.sun.media.rtp.RTPEventHandler.dispatchEvents(RTPEventHandler.java:96)
        at com.sun.media.rtp.RTPEventHandler.run(RTPEventHandler.java:115)
"JMF thread: com.sun.media.ProcessEngine@a3c5b6[ com.sun.media.ProcessEngine@a3c5b6 ] ( configureThread)" daemon prio=1 tid=0x082fe3c8 nid=0x3ea6 in Object.wait() [977e0000..977e1238]
        at java.lang.Object.wait(Native Method)
        - waiting on <0x9f387560> (a java.lang.Object)
        at java.lang.Object.wait(Object.java:429)
        at com.sun.media.parser.RawBufferParser$FrameTrack.parse(RawBufferParser.java:247)
        - locked <0x9f387560> (a java.lang.Object)
        at com.sun.media.parser.RawBufferParser.getTracks(RawBufferParser.java:112)
        at com.sun.media.BasicSourceModule.doRealize(BasicSourceModule.java:180)
        at com.sun.media.PlaybackEngine.doConfigure1(PlaybackEngine.java:229)
        at com.sun.media.ProcessEngine.doConfigure(ProcessEngine.java:43)
        at com.sun.media.ConfigureWorkThread.process(BasicController.java:1370)
        at com.sun.media.StateTransitionWorkThread.run(BasicController.java:1339)
"JMF thread" daemon prio=1 tid=0x080db410 nid=0x3ea6 in Object.wait() [97f41000..97f41238]
        at java.lang.Object.wait(Native Method)
        - waiting on <0x9f480578> (a com.ibm.media.protocol.CloneableSourceStreamAdapter$PushBufferStreamSlave)
        at java.lang.Object.wait(Object.java:429)
        at com.ibm.media.protocol.CloneableSourceStreamAdapter$PushBufferStreamSlave.run(CloneableSourceStreamAdapter.java:375)
        - locked <0x9f480578> (a com.ibm.media.protocol.CloneableSourceStreamAdapter$PushBufferStreamSlave)
        at java.lang.Thread.run(Thread.java:534)Any ideas ?
Thanks,
Jarek

Answer 1

bgl,
I was able to get past the cloning issue by following the Clone.java example to the letter :)
Turns out that the cloneable data source must be added as a send stream first, and then the clonet data source. Now for each party in the call the conf. server does the following :
Party(RTPManager mgr,DataSource ds) {
      this.mgr=mgr;
      this.ds=Manager.createCloneableDataSource(ds);
   synchronized DataSource cloneDataSource() {
      DataSource retVal;
      if(getNeedsCloning()) {
        retVal = ((SourceCloneable) ds).createClone();
      } else {
        retVal = ds;
        setNeedsCloning();
      return retVal;
    private void setNeedsCloning() {
      needsCloning=true;
    private boolean getNeedsCloning() {
      return needsCloning;
     private synchronized void addSendStreamFromNewParticipant(Party newOne) throws UnsupportedFormatException, IOException {
    debug("*** - New one joined. Creating the send streams. Curr count :" + participants.size());
    Iterator pIt = participants.iterator();
    while(pIt.hasNext()) {
      Party p = (Party)pIt.next();
      assert p!=newOne;
      // update existing participant
      SendStream sendStream = p.getMgr().createSendStream(newOne.cloneDataSource(),0);
      sendStream.start();
      // send data from existing participant to the new one
      sendStream = newOne.getMgr().createSendStream(p.cloneDataSource(),0);
      sendStream.start();
    debug("*** - Done creating the streams.");So I made some progress, but I'm still not quite there.
The RTP manager JavaDoc for createSendStream states the following :
* This method is used to create a sending stream within the RTP
* session. For each time the call is made, a new sending stream
* will be created. This stream will use the SDES items as entered
* in the initialize() call for all its RTCP messages. Each stream
* is sent out with a new SSRC (Synchronisation SouRCe
* identifier), but from the same participant i.e. local
* participant. <BR>
For 3 participants, my conf. server creates 2 send streams to every one of them, so I'd expect 2 SSRCs on the wire. Examining the RTP packets in Ethereal, I only see 1 SSRC, as if the 2nd createSendStream call failed. Consequently, each participany in the conference is able to receive voice from only 1 other participant, even though I create RTPManager instance for each participany, and add 2 send streams.
Any ideas ?
Thanks,
Jarek