Single queue: concurrent processing of messages in multiple consumers

Hi,
I am new to jms . The goal is to  process messages concurrently from a queue in an asynchronous listener's onMessage method  by attaching a listener instance to multiple consumer's with each consumer using its own session and running in a separate thread, that way the messages are passed on to the different consumers for concurrent processing. 
1) Is it ossible to process messsages concurrently from a single queue by creating multiple consumers ?
2)  I came up with the below code, but would like to get your thoughts on whether the below code looks correct for what I want to accomplish.  
import java.io.IOException;
import java.io.InputStreamReader;
import java.util.Properties;
import javax.jms.Connection;
import javax.jms.ConnectionFactory;
import javax.jms.Destination;
import javax.jms.JMSException;
import javax.jms.Message;
import javax.jms.MessageConsumer;
import javax.jms.MessageListener;
import javax.jms.Session;
import javax.jms.TextMessage;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import com.walmart.platform.jms.client.JMSConnectionFactory;
public class QueueConsumer implements Runnable, MessageListener {
  public static void main(String[] args) {
   // Create an instance of the client
    QueueConsumer consumer1 = new QueueConsumer();
    QueueConsumer consumer2 = new QueueConsumer();
    try {
    consumer1.init("oms","US.Q.CHECKOUT-ORDER.1.0.JSON");   //US.Q.CHECKOUT-ORDER.1.0.JSON   is the queue name
    consumer2.init("oms","US.Q.CHECKOUT-ORDER.1.0.JSON");
    }catch( JMSException ex ){
    ex.printStackTrace();
    System.exit(-1);
    // Start the client running
    Thread newThread1 = new Thread(consumer1);
    Thread newThread2 = new Thread(consumer1);
    newThread1.start();newThread2.start();
    InputStreamReader aISR = new InputStreamReader(System.in);
          char aAnswer = ' ';
          do {
              try {
  aAnswer = (char) aISR.read();
catch (IOException e)
  // TODO Auto-generated catch block
  e.printStackTrace();
} while ((aAnswer != 'q') && (aAnswer != 'Q'));
          newThread1.interrupt();
          newThread2.interrupt();
          try {
  newThread1.join();newThread2.join();
  } catch (InterruptedException e) {
  // TODO Auto-generated catch block
  e.printStackTrace();
          System.out
  .println("--------------------exiting main thread------------------------"+Thread.currentThread().getId());
        System.exit(0);
// values will be read from a resource properties file
private static String connectionFactoryName = null;
private static String queueName = null;
// thread safe object ref
  private static ConnectionFactory qcf = null;
  private static Connection queueConnection = null;
// not thread safe
  private Session ses = null;
  private Destination queue = null;
  private MessageConsumer msgConsumer = null;
  public static final Logger logger = LoggerFactory
  .getLogger(QueueConsumer.class);
  public QueueConsumer() {
  super();
  public void onMessage(Message msg) {
  if (msg instanceof TextMessage) {
  try {
  System.out
  .println("listener is "+Thread.currentThread().getId()+"--------------------Message recieved from queue is ------------------------"
  + ((TextMessage) msg).getJMSMessageID());
  } catch (JMSException ex) {
  ex.printStackTrace();
  public void run() {
  // Start listening
  try {
  queueConnection.start();
  } catch (JMSException e) {
  e.printStackTrace();
  System.exit(-1);
  while (!Thread.currentThread().isInterrupted()) {
  synchronized (this) {
  try {
  wait();
  } catch (InterruptedException ex) {
  break;
  * This method is called to set up and initialize the necessary Session,
  * destination and message listener
  * @param queue2
  * @param factoryName
  public void init(String factoryName, String queue2) throws JMSException {
  try {
  qcf = new JMSConnectionFactory(factoryName);
  /* create the connection */
  queueConnection = qcf.createConnection();
  * Create a session that is non-transacted and is client
  * acknowledged
  ses = queueConnection.createSession(false,
  Session.CLIENT_ACKNOWLEDGE);
  queue = ses.createQueue(queue2);
  logger.info("Subscribing to destination: " + queue2);
  msgConsumer = ses.createConsumer(queue);
  /* set the listener  */
  msgConsumer.setMessageListener(this);
  System.out.println("Listening on queue " +queue2);
  } catch (Exception e) {
  e.printStackTrace();
  System.exit(-1);
  private static void setConnectionFactoryName(String name) {
  connectionFactoryName = name;
  private static String getQueueName() {
  return queueName;
  private static void setQueueName(String name) {
  queueName = name;

Hi Mark,
if the messages are sent with quality of service EO (=exactly once) you can add queues.
From documentation:
Netweaver XI -> Runtime -> Processing xml messages -> Queues for asynchronous message processing
...For incoming messages you set the parameter EO_INBOUND_PARALLEL from the Tuning category.
If more than one message is sent to the same receiver, use the configuration parameter EO_OUTBOUND_PARALLEL of the Tuning category to process messages simultaneously in different queues.
Transaction: SXMB_ADM
Regards
Holger

Similar Messages

  • Concurrent processing of messages

    Hello,
    We are splitting big messages in the file adapter.
    After the split, we want XI to process the different messages concurrently (at the same time).
    Does anybody have an idea how to arrange this?
    Now all messages are processed after each other.
    Regards,
    Mark

    Hi Mark,
    if the messages are sent with quality of service EO (=exactly once) you can add queues.
    From documentation:
    Netweaver XI -> Runtime -> Processing xml messages -> Queues for asynchronous message processing
    ...For incoming messages you set the parameter EO_INBOUND_PARALLEL from the Tuning category.
    If more than one message is sent to the same receiver, use the configuration parameter EO_OUTBOUND_PARALLEL of the Tuning category to process messages simultaneously in different queues.
    Transaction: SXMB_ADM
    Regards
    Holger

  • Queue Concurrent Processing using PL/SQL notification

    I have a queue that I need to have the processing done concurrently no matter how many messages are in the queue.  Currently one 1 message is processed at a time, but I need at least 10 processed at once.  Each message could take up to 3 minutes to process so doing one at a time is taking too long when the number of messages grows to more than 5 or 10.  How can this be done?
    begin
      DBMS_AQADM.CREATE_QUEUE_TABLE
      ( queue_table => 'z3.health_check_qtab',
        queue_payload_type =>  'z3.sil_queue_msg_typ',
        multiple_consumers => TRUE );
      DBMS_AQADM.CREATE_QUEUE
      ( queue_name => 'HEALTH_CHECK_Q',
        queue_table => 'z3.health_check_qtab');
      DBMS_AQADM.START_QUEUE
      (  queue_name => 'HEALTH_CHECK_Q');
    end;
    begin
      dbms_aqadm.add_subscriber
      ( queue_name => 'Z3.HEALTH_CHECK_Q',
        subscriber => sys.aq$_agent( 'HC_recipient1', null, null ) );
    end;
    BEGIN
      dbms_aq.register
      ( sys.aq$_reg_info_list(
        sys.aq$_reg_info('Z3.HEALTH_CHECK_Q:HC_RECIPIENT1',
                         DBMS_AQ.NAMESPACE_AQ,
                        'plsql://Z3.Z_IMPACT_LIST_PKG.DEQUEUE_HEALTH_CHECK',
                        HEXTORAW('FF')) ) ,
      1 );
    end;
    Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production

    I'm also looking into how to do this (on 10gR2 RAC). I would like to enqueue say 1,000,000 requests and have 10-30 (configurable) worker 'threads' dequeueing and executing a PL/SQL procedure during certain parts of the day.

  • Conditionally sending messages to multiple consumers

    How would I best accomplish this requirement?
    I have one system which produces messages
    Multiple systems may want to consume those messages
    I want the producer to be able to send messages conditionally
    In one instance perhaps I send same message to each consumer
    However in another case maybe I only want to send a message to consumers A and B
    It would be OK if I could flag certain messages to be for certain consumers, and then those consumers only retrieve those messages designated to them. However I dont think this is possible as any such flag the consumer would have to read once it actually consumes the message and reads it, I dont think there is any way for the queue to make that decision for them.
    So it sounds like what I need is a hybrid of point-to-point and publish-subscribe
    I want a universal way to conditionally send messages to one or more consumers, and the conditions are dynamic.
    Any suggestions?
    Must it be a PTP with separate queue for every application?

    lozina wrote:
    Interesting... message filters. I like it.
    So with this approach, I can send some messages to everyone, and other messages I can have tailored to specific subscribers?pretty much.
    But the subscriber does not actually have to "fetch" each message to determine if the message is meant for him or not, right? When using message filters this act at the queue, not at the client? Basically I want to avoid a client downloading 1,000 messages only to find out none of them are for him. unfortunately, that's probably up to the implementation. the implementation may choose to work that way (download every message and check it), or it may have a way of applying the filter at the server. however, the filters work only on the message properties, so the impl may be able to just get message headers and filter w/out getting the entire message. you'll have to test with the jms impl you are planning on using and see how that works.
    Thanks!
    [Edit] these are called Message Selectors right? yes.
    And looks like they are specified at the time the client looks up the queue in JNDI?yes, these are provided at subscription time.
    Now is this limited to publish-subscribe?no you can use selectors when subscribing to a queue as well. but, queues won't scale very well to lots of consumers (and are hard to manage with dynamic numbers of consumers).

  • Inbound queue stopped processing messages

    Inbound queues not processing the messages in XI server. it throwing the syserror.
    please advise.

    Hi Asha,
    Check your RFC destination which is connecting to your external system . ask your basis team to Provide necessary roles for the user ....
    > 837595 and assign the required authentication to the user being used in the RFC ADAPTER
    Also Please go through the notes#1487160 and #1393039.
    Regards,
    Naveen

  • Multiple Processes dequeuing messages from same queue

    Hi,
    I have 2 Processes that are listening on the same queue to dequeu the messages. How can i make sure each process picks the message intended for that process. What setting do I need so that each process picks the right message not all the messages.
    As some messages needs to be processed by Process1 and some messages by Process2.
    Thanks

    I would like answer to this too! The only way I can solve this is by having 2 queues.
    As far as I can figure out, the dequeue just picks up the next available message. Both processes can dequeue the same message if you make the queue multiple consumer.
    The processes that picks up the message can put it back if the it is not meant for itself - but this is an inefficient and potentially ineffective solution.

  • Multiple queue vs single queue - peformance

    Hi All,
    Could pls throw some light on the following design.
    Currenty, We have more than 100 queues and for each there is a different MDB to receive the message.Now, somehow we made a single instance of same MDB listen to all these queues by looking at the input message. The requiement now is, is it possible to club all the queues into single queue as it's asynchronous messaging. The receiver may also try write into the same queue at the end of message processing.If so, how would be performance. Is this performance better than having multiple queues?.
    The message are expecting to be 25 k/day in the queue. And we are using Weblogic server.
    Please let me know if you need more details.
    Thanks
    Srinath

    You could try to use selectors to assign messages to particular MDBs. This way you off load some of the work on the broker but selectors by their nature usually have an impact on broker's performance. It all depends on your application, your expectations, size and the rate of your messages, how long it takes to process them on the client side. There is no simple answer in this case. You would have to test some scenarios to come up with a best solution that fits your requirements. Having said that I personally believe that some degree of partitioning of your queue could be beneficial. On a heavily loaded system it may increase the concurrency of the system and its resilience, particularly if you place the queues in different brokers. Try to imagine your broker as if it was a DB server with just one table (I'm not sure if this is the best analogy but I can't think about anything else at the moment).

  • Single Queue Vs Multiple Queues

    Hi,
    I am in the process of architecting some queues between remote databases. There are different document types being propagated from one queue to another, and they have different priorities. At the end queue there are multiple processes to dequeue and action the different document types.
    My question is, is it better to have one end queue for each process/document type ( ie multiple end point queues), or one single end point queue with all the documents in them regardless of the type?
    I know that it is possible to have the single queue end point, but is this the best option and why?
    Thanks,
    matt

    Hi Peter,
    What do you mean by different document type?Document content only is different. The payload will be the same. All the documents will be passed through as blobs. The payload will have a field to define the document content type.
    Do you intend to store different message types in the same source queue?At this stage we can go either way. If we have one single end point queue we will more than likely have one single source queue.
    If so what is the definition of the payload? The payload definition will be the same in all cases, both source and destination.
    What version of databases are you propagating between?10.2
    How many dequeueing processes will you have at the remote end?One program for each of the different document content type (invoices, remittance, etc.). Initially there will be 4, but this is likely to grow to 12 or more.
    Ultimately we expect around 10,000 documents per day to pass through the system.
    Thanks,
    Matt

  • Log messages from multiple instances in single file.

    Hi!
    I have a requirement that i need to log messages from muliple instances of the same object in a file. The new file will be created every day. Likewise, multiple objects might have various instances each.
    One class
    ->multiple instances
    -> log message stored in single file.
    Note :
    I am using the Message driven bean. I need to log from the bean class. JDK 1.3
    If u could help me out that would be great.

    As long as they are all from the same OS program (a single Java VM), that's OK - you can use Log4j, and use a rotating file logger.
    If you point two different virtual machine processes at the same file, one may have it open when the other is trying to rotate it, and your rotation may fail (at best) and/or you may lose the old log (the worst case).
    If you need to collect log messages from multiple processes (or even multiple machines), use a syslog-based logger (Log4j has a SyslogAppender) or use Log4j's SocketAppender to write to a log4j-builtin log listener (SocketNode).

  • Multiple queue receivers with disjoint message selectors - portable?

    Hi all,
    I was wondering whether an application that uses multiple queue receivers with disjoint message selectors would be portable across different JMS providers.
    I fear it's not - at least the spec clearly states that the implementation of message delivery to multiple receivers is provider specific, and I couldn't find any comment that this would be different with disjoint selectors. But I wanted to check with you - maybe I missed anything?
    What could be an alternative then (in case you want to avoid multiple queues)? Using a topic with durable subscribers?
    Regards,
    Sabine

    Hi Nimo,
    accepted - as long as the JMS spec requires the JMS provider to support multiple receivers. And that's exactly the point where I'm not sure...
    Literally it states:
    "For PTP, JMS does not specify the semantics of concurrent QueueReceivers for the same queue; however, JMS does not prohibit a provider from supporting this."
    The formulation "does not prohibit" makes me think that JMS allows providers also not to support this, for example, to throw an exception when a second consumer is created for the same queue, or not to serve it at all with messages as long as the first consumer is active. Not that I think it would be a very useful implementation - I just want to know if it would be possible. In that case, the application would not be portable.
    Regards,
    Sabine

  • Installing a callback for multiple windows messages and processing EVENT_NEWHANDLE message

    Good afternoon all,  I am a newbie to using the Win32 API.  I have two questions:
    When a single callback is installed to process multiple windows messages, will the install function return the same windows handle for the panel?
    I also saw noted in the help your panel should handle and respond to the EVENT_NEWHANDLE message.  Does the InstallWinMsgCallback function need to be called make sure your panel handles this message as well?
    Thank you for your help.

    Hi hortoxn1,
    We have a KnowledgeBase Article on this topic that you may find useful:
    "How Do CVI Panels Process Windows Messages?"
    http://digital.ni.com/public.nsf/allkb/29C12428E7974EEF862565EF00839871
    Does that explain what you're looking for?
    Steven

  • Queue processing of messages (files / idocs)

    Hi,
    We have 4 communication channels:
    INT A
    1 filesender (file)
    1 filereceiver (idoc)
    INT B
    1 filesender (file)
    1 filereceiver (idoc)
    So, the issue is we need to process the messages in order. First, 2 files from INT A, next 1 file from INT B. The correlationship is the key.
    Is any configuration possible for queue processing with no using of BPM?
    Regards,

    hi,
    yes you can set queue (EOIO) processing in both filesenders
    and if you hava ERP (IDOC) on WAS 6.40 it can be processed
    in this one queue
    how to (IDOC queue processing part) described in my book:
    <a href="/people/michal.krawczyk2/blog/2006/10/11/xi-new-book-mastering-idoc-business-scenarios-with-sap-xi"><b>Mastering IDoc Business Scenarios with SAP XI</b></a>
    Regards,
    michal
    <a href="/people/michal.krawczyk2/blog/2005/06/28/xipi-faq-frequently-asked-questions"><b>XI / PI FAQ - Frequently Asked Questions</b></a>

  • How can a JMS adapter be configured in BPEL to consume messages from multiple queues ?

    How can a JMS adapter be configured in BPEL to consume messages from multiple queues ?

    If you want to use JMS with AQ as datastore then there is some configuration you need to do to enable this. This is outside SOA Suite per sa, e.g. no adapter required.
    If you want to connect to the AQ direct then use the AQ adapter.
    this blog may be of some help understand the configuration
    http://biemond.blogspot.com/2008/01/oracle-jms-with-esb-and-bpel.html
    cheers
    James

  • Multiple consumers of the same Queue

    In the past my use of JMS queues has been limited to a single message producer and a single consumer. I am now looking at having multiple copies of the same consumer running for load balancing and failover purposes, and this leads me to what I think is a basic question. BTW, we're running OpenMQ 4.4u1.
    I have messages being produced and saved in a single queue.
    I then have a consumer program whose logic is something like this:
    1. Create session with no transaction and CLIENT_ACKNOWLEDGE.
    2. Receive message from queue.
    3. Do some processing with message. Repeat processing attempts multiple times, if needed, until successful.
    4. Once processing is successful, acknowledge message.
    Assume that I am running multiple copies of the consumer in separate JVMs (or on separate servers, for that matter).
    Here are my related questions.
    1. If step 3 takes a long time (e.g., several minutes) can I be sure that only one instance of the consumer will receive the message?
    2. If a consumer is in step 3 and a catastrophic failure occurs (e.g., the JVM or server it is running on crashes) the message will never be acknowledged, but the session will not be shut down in an orderly fashion, either. Will the message that was received but never acknowledged eventually be delivered to another copy of the consumer?
    3. If step three takes a very long time (hours or days, rather than minutes), will the broker misinterpret this as a failure of the consumer and deliver the same message to another one of the consumers?
    Thank you for reading and/or responding to this question.
    Bill

    1. If step 3 takes a long time (e.g., several minutes) can I be sure that only one instance of the consumer will receive the message?A queue message will only be delivered and consumed by 1 consumer.
    2. If a consumer is in step 3 and a catastrophic failure occurs (e.g., the JVM or server it is running on crashes) the message will never be acknowledged, but the session will not be shut down in an orderly fashion, either. Will the message that was received but never acknowledged eventually be delivered to another copy of the consumer?Yes.
    3. If step three takes a very long time (hours or days, rather than minutes), will the broker misinterpret this as a failure of the consumer and deliver the same message to another one of the consumers?No, as long as the consumer's Session or Connection are intact

  • Processing JMS messages in sequence one after another

    This is my use case which I wan to implement in OSB.
    1. I have a queue which will receive messages in huge numbers from a single sender.
    2. I have a proxy which will read each message and process the message and send it out.
    In step 1 , I want each message to be read sequentially and when step 2 finishes executing then I want to read the next message from the queue.
    I need to give an acknowledgment to JMS that the message has finished processing and it can make the next message available to the proxy.
    What is the best option to implement this use case ?.
    Regards

    By default Weblogic JMS Queues are FIFO. But that is true only if there is only one consumer of the queue.
    In case of a clustered environment that is not true and in case of OSB even with a single managed server multiple threads are created for a JMS listener Proxy(by default 16). So there will be 16 instances of the Proxy trying to read messages from the Queue and each will be handed out a message and sequential integrity is disturbed.
    There are three ways to overcome this problem:
    1. Unit of Order
    This is suitable only when you want messages of the same group to be processed sequentially while multiple groups can be processed in parallel.
    For e.x. you are getting multiple orders for multiple customers and you want to ensure that the orders of same customer are processed sequentially (in case a customer tries to change the quantity of an order then the latest one should be processed last). In this case you can set UOO as the CustomerID and each customer's order will be processed sequentially but orders of different customers will be still processed in parallel. WLS JMS achieves this by assigning a target queue instance on the cluster for each UOO value. It will assign Q!MS1 to CustID=1, Q!MS2 to CustID=2 and so on. All messages of the same UOO(of the same customer) will go to the same instance on the queue. Furthermore it will ensure that no two messages of the same UOO are released together to the listeners. So if a queue instance has 5 messages each of both Cust=1 and Cust=2, only one message each of Cust1 and Cust2 will be processed by the listeners even if there are 16 listening threads.
    Pros and Cons:
    Overall performance is improved since you can still achieve some parallel processing.
    There will be a slight overhead of processing the UOO headers on the JMS servers.
    Can not ensure sequencing of all the messages. (Although you can achieve that by setting the same UOO on all the messages)
    If the managed server assigned to a UOO is down, if a publisher tried to put the next message for that UOO, it will fail since it will not try to send that message to any other managed server.
    2. Single threaded processing
    If you don't want to process even different groups of messages in parallel and if you want absolute sequencing (i.e. irrespective of the CustomerID you want all the messages to be processed in the sequence they arrive) you will need to process them in a single threaded model. The JMS queue should be deployed on only one managed server of the cluster. The JMS proxy listening to the queue should also be deployed on a single managed server(You will need to change the targeting for the EJB created for this Proxy in the Deployments) and there should be a Work Manager for this Proxy with Maximum Thread Constraint set as 1. In the Connection Factory which the Proxy uses, set the Maximum Messages per session to 1. Another approach for Single threaded could be to set the same value of UOO on all of the messages. This will make all the messages to go to the same Q instance in the cluster and also will make sure that even if Proxy has multiple threads, only one message will be processed at a time.
    Pros and Cons:
    Completely single threaded processing, will take more time since messages will be processed one after the other.
    Load balancing will go awry as all the messages will be processed by only one server.
    3. Using custom implementation
    The most complex way is to create a completely custom implementation. Put all the messages in a DB and then process them one at a time based on timestamp.
    Pros and Cons:
    It will take more effort to implement than the other two approaches.
    It will again need single threaded processing after the messages are put on the DB
    Performance wise there will be more impact because of additional DB calls
    More complex to maintain
    Based on your exact requirements you can choose which approach you want. If you have SOA suite as well them it would be better to move this solution to SOA suite as OESB(Mediator) component of SOA suite has re-sequencing feature.

Maybe you are looking for

  • 6th gen nano not recognized by win7, nor itunes

    i've been using my 6th gen nano for a few months now with no problems. suddenly today, i plugged it in and it wasn't recognized by windows and doesn't show up in itunes. i keep getting errors stating the unknown device is not recognized. i've uninsta

  • Purchase order mass maintenance

    Dear All..                 Im using purchase order mass maintenence through MASS T-code with object BUS2012 i want to change field EKPO-MHDRZ it is wrongly maintained as 2 i want to remove this via mass maintainence. at the time of saving the documen

  • Adding a background image to every page.

    Problem:  I am attempting to add a company logo (.jpg) file to a multipage PDF document.  I know I can add the logo as a stamp but I need the logo to be on every page in the same position and with the same size.  I was hoping that using it as a water

  • Not playing songs i've downloaded off burned Cds

    I had a Cd that had over 150 MP3 songs on it. So I imported it like I would any other Cd, and it worked fine. I put them on my ipod and everything was fine. And then after about a week i was on itunes on my computer and clicked on one of the songs to

  • Last argument in AGO function

    Hi Everyone... Hope everyone is good . So we have this requirement where user want to see selected quarter last month, then previous quarter last month and previous year last month based on Year and Quarter selection. for example. Prompts: Year - 201