MDB cannot loads enitities if more than 10 concurrent messages are sent tog

Question

Hello,
I have seen a weird behavior of the MDB in Jboss or perhaps in transaction management.
Here is a my scenario
I have a session bean sb, an entity bean e and a message driven bean mb
When a request comes to sb , it persists a new entity e and send a message (primary key) to mb
mb then loads e from the database using one of the findByXX method and does something else.
If 10 concurrent request comes to sb, 10 different entities are persisted and 10 different messages are sent to mb
mb are able to load 10 different e from the database
Everything works fine as long as there are 10 concurrent request. If there are 13 concurrent request, then one or two mb
cannot load e from the database. It seems that the session bean has persisted the entities but some of them are still unload able from mb. They have been persisted , (I saw in the logs), but mb cannot load them.
Initially I thought that a transaction mishap might be happening, so instead of persisting the entity and sending the message in one method in sb , I separated these two tasks in two methods a and b, in which both a and b uses RequiresNew transaction attribute. But the problem still remains. Please note if the number of parallel request are less than 10 then everything works fine. Is there something I am missing or is Jboss doing something buggy?
The above situation is reproducible .
Here is some more information about the problem along with some code snapshots. Note that the code is trimmed for clarity purposes.
Request is a simple entity (e as mentioned in the problem before) that is persisted in session bean . Here is a trimmed snapshot
Request.java
package gov.fnal.ms.dm.entity;
import java.io.Serializable;
import javax.persistence.Column;
import javax.persistence.Entity;
@Entity
@NamedQueries({@NamedQuery(name = "Request.findById", query = "SELECT r FROM Request r WHERE r.id = :id"),
@NamedQuery(name = "Request.findAll", query = "SELECT r FROM Request r"),
@Table(name = "cms_dbs_migration.Request")
@SequenceGenerator(name="seq", sequenceName="cms_dbs_migration.SEQ_REQUEST")
public class Request implements Serializable {
private String detail;
@Id
@GeneratedValue(strategy = GenerationType.AUTO, generator="seq")
@Column(nullable = false)
private Long id;
private String notify;
.....The session bean is MSSessionEJBBean.
MSSessionEJBBean.java
@Stateless(name="MSSessionEJB")
@WebService(endpointInterface = "gov.fnal.ms.dm.session.MSSessionEJBWS")
public class MSSessionEJBBean implements MSSessionEJB, MSSessionEJBLocal, MSSessionEJBWS {
@PersistenceContext(unitName="dm")
private EntityManager em;
.......    The methods that is called concurrently in this bean is
@TransactionAttribute(TransactionAttributeType.REQUIRES_NEW)
public Request addRequest(String srcUrl, String dstUrl, String path,
String dn, String withParents, String withForce, String notify) throws
Exception {
Request r = addRequestUnit(srcUrl, dstUrl, path, dn, withParents, withForce, notify);
if (r != null) sendToQueue(r.getId());
return r;
}    It first adds the request entity and then sends a message to the queue in separate methods each using TransactionAttributeType.REQUIRES_NEW (I don't think this TransactionAttributeType is needed . I was just playing to see if this resolves the problem)
Here is the method that actually persist the entity
@TransactionAttribute(TransactionAttributeType.REQUIRES_NEW)
public Request addRequestUnit(String srcUrl, String dstUrl, String path,
String dn, String withParents, String withForce, String notify) throws
Exception {
....// Does something before persisting
Request r = new Request();
r.setNotify(notify);
r.setPath(path);
r.setPerson(p);
r.setWithForce(withForce);
r.setWithParents(withParents);
r.setProgress(new Integer(0));
r.setStatus("Queued");
r.setDetail("Waiting to be Picked up");
em.persist(r);
System.out.println("Request Entity persisted");
System.out.println("ID is " + r.getId());
return r;
}I can see that the log files has messages
*{color:#800000}"Request Entity persisted" "ID is " 12 (or some other number){color}*
. So it is safe to assume that the request entity is persisted properly before the message is sent to the queue. Here is how the message is sent to the queue
@TransactionAttribute(TransactionAttributeType.REQUIRES_NEW)
private void sendToQueue(long rId) throws Exception {
QueueSession session = null;
QueueConnection conn = null;
try {
conn = factory.createQueueConnection();
session = conn.createQueueSession(false, QueueSession.AUTO_ACKNOWLEDGE);
MapMessage mapMsg = session.createMapMessage();
mapMsg.setLong("request", rId);
session.createSender(queueRequest).send(mapMsg);
} finally {
if (session != null) session.close();
if(conn != null) try {
conn.close();
}catch(Exception e) {e.printStackTrace();}
System.out.println("Message sent to queue");
}I can see the message
{color:#800000}*"Message sent to queue"*{color}
in the log file right after I see the
*{color:#800000}
"Request Entity persisted"{color}*
message. So it correct to assume that the message is infact sent to the queue.
Here is the md bean that gets the message from the queue and process it
@MessageDriven(activationConfig =
@ActivationConfigProperty(propertyName="destinationType",
propertyValue="javax.jms.Queue"),
@ActivationConfigProperty(propertyName="destination",
propertyValue="queue/requestmdb")
public class RequestMessageDrivenBean implements MessageListener {
@EJB
private MSSessionEJBLocal ejbObj;
public void onMessage(Message message) {
try{
System.out.println("Message recived ");
if(message instanceof MapMessage) {
System.out.println("This is a instance of MapMessage");
MapMessage mMsg = (MapMessage) message;
long rId = mMsg.getLong("request");
List<Request> rList = ejbObj.getRequestById(new Long(rId));
System.out.println("rList size is " +  rList.size());
for(Request r: rList) {
//Does something and print something in logs
System.out.println("Finsihed onMessage");
}catch(Exception e) {
System.out.println(e.getMessage());
}    One thing to note here is that getRequestById is another method in the same session bean MSSessionEJB. Can that be the reason for not loading the request from the database when many concurrent request are present? Here is what it does
@TransactionAttribute(TransactionAttributeType.REQUIRES_NEW)
public List<Request> getRequestById(Long id) throws Exception {
System.out.println("Inside getRequestById id is " + id);
return em.createNamedQuery("Request.findById")
.setParameter("id", id)
.getResultList();
}    In the logs I do see
{color:#800000}*"This is a instance of MapMessage"*{color}
and I do see the correct Rid that was persisted before in the log
*{color:#800000}
"inside getRequestById id is 12"{color}*
. Finally I do see
*{color:#800000}
"Finsihed onMessage"{color}*
for all the request no matter how many are sent concurrently.
*{color:#ff0000}Here is where the problem is . rList size is 1 for most of the cases but it returns 0 when the number of concurrent message exceeds 10.*
*{color}*
So you see there is no exception per se, just that entity does not get loaded from the database. I would like to mention this again, it does work properly when the number of concurrent request are less than 10. All the invocation of onMessage in MB are able to load the entity properly in that case. Its only when the number of concurrent request increases more than 10, the onMessage is unable to load the entity and the rList is 0.
FYI I am using jboss-4.2.2.GA on RedHat enterprise linux 4 with jdk1.6.0_04
Please let me know if there is more information required on this.
Thank you

Answer 1

Hello,
I have seen a weird behavior of the MDB in Jboss or perhaps in transaction management.
Here is a my scenario
I have a session bean sb, an entity bean e and a message driven bean mb
When a request comes to sb , it persists a new entity e and send a message (primary key) to mb
mb then loads e from the database using one of the findByXX method and does something else.
If 10 concurrent request comes to sb, 10 different entities are persisted and 10 different messages are sent to mb
mb are able to load 10 different e from the database
Everything works fine as long as there are 10 concurrent request. If there are 13 concurrent request, then one or two mb
cannot load e from the database. It seems that the session bean has persisted the entities but some of them are still unload able from mb. They have been persisted , (I saw in the logs), but mb cannot load them.
Initially I thought that a transaction mishap might be happening, so instead of persisting the entity and sending the message in one method in sb , I separated these two tasks in two methods a and b, in which both a and b uses RequiresNew transaction attribute. But the problem still remains. Please note if the number of parallel request are less than 10 then everything works fine. Is there something I am missing or is Jboss doing something buggy?
The above situation is reproducible .
Here is some more information about the problem along with some code snapshots. Note that the code is trimmed for clarity purposes.
Request is a simple entity (e as mentioned in the problem before) that is persisted in session bean . Here is a trimmed snapshot
Request.java
package gov.fnal.ms.dm.entity;
import java.io.Serializable;
import javax.persistence.Column;
import javax.persistence.Entity;
@Entity
@NamedQueries({@NamedQuery(name = "Request.findById", query = "SELECT r FROM Request r WHERE r.id = :id"),
@NamedQuery(name = "Request.findAll", query = "SELECT r FROM Request r"),
@Table(name = "cms_dbs_migration.Request")
@SequenceGenerator(name="seq", sequenceName="cms_dbs_migration.SEQ_REQUEST")
public class Request implements Serializable {
private String detail;
@Id
@GeneratedValue(strategy = GenerationType.AUTO, generator="seq")
@Column(nullable = false)
private Long id;
private String notify;
.....The session bean is MSSessionEJBBean.
MSSessionEJBBean.java
@Stateless(name="MSSessionEJB")
@WebService(endpointInterface = "gov.fnal.ms.dm.session.MSSessionEJBWS")
public class MSSessionEJBBean implements MSSessionEJB, MSSessionEJBLocal, MSSessionEJBWS {
@PersistenceContext(unitName="dm")
private EntityManager em;
.......    The methods that is called concurrently in this bean is
@TransactionAttribute(TransactionAttributeType.REQUIRES_NEW)
public Request addRequest(String srcUrl, String dstUrl, String path,
String dn, String withParents, String withForce, String notify) throws
Exception {
Request r = addRequestUnit(srcUrl, dstUrl, path, dn, withParents, withForce, notify);
if (r != null) sendToQueue(r.getId());
return r;
}    It first adds the request entity and then sends a message to the queue in separate methods each using TransactionAttributeType.REQUIRES_NEW (I don't think this TransactionAttributeType is needed . I was just playing to see if this resolves the problem)
Here is the method that actually persist the entity
@TransactionAttribute(TransactionAttributeType.REQUIRES_NEW)
public Request addRequestUnit(String srcUrl, String dstUrl, String path,
String dn, String withParents, String withForce, String notify) throws
Exception {
....// Does something before persisting
Request r = new Request();
r.setNotify(notify);
r.setPath(path);
r.setPerson(p);
r.setWithForce(withForce);
r.setWithParents(withParents);
r.setProgress(new Integer(0));
r.setStatus("Queued");
r.setDetail("Waiting to be Picked up");
em.persist(r);
System.out.println("Request Entity persisted");
System.out.println("ID is " + r.getId());
return r;
}I can see that the log files has messages
*{color:#800000}"Request Entity persisted" "ID is " 12 (or some other number){color}*
. So it is safe to assume that the request entity is persisted properly before the message is sent to the queue. Here is how the message is sent to the queue
@TransactionAttribute(TransactionAttributeType.REQUIRES_NEW)
private void sendToQueue(long rId) throws Exception {
QueueSession session = null;
QueueConnection conn = null;
try {
conn = factory.createQueueConnection();
session = conn.createQueueSession(false, QueueSession.AUTO_ACKNOWLEDGE);
MapMessage mapMsg = session.createMapMessage();
mapMsg.setLong("request", rId);
session.createSender(queueRequest).send(mapMsg);
} finally {
if (session != null) session.close();
if(conn != null) try {
conn.close();
}catch(Exception e) {e.printStackTrace();}
System.out.println("Message sent to queue");
}I can see the message
{color:#800000}*"Message sent to queue"*{color}
in the log file right after I see the
*{color:#800000}
"Request Entity persisted"{color}*
message. So it correct to assume that the message is infact sent to the queue.
Here is the md bean that gets the message from the queue and process it
@MessageDriven(activationConfig =
@ActivationConfigProperty(propertyName="destinationType",
propertyValue="javax.jms.Queue"),
@ActivationConfigProperty(propertyName="destination",
propertyValue="queue/requestmdb")
public class RequestMessageDrivenBean implements MessageListener {
@EJB
private MSSessionEJBLocal ejbObj;
public void onMessage(Message message) {
try{
System.out.println("Message recived ");
if(message instanceof MapMessage) {
System.out.println("This is a instance of MapMessage");
MapMessage mMsg = (MapMessage) message;
long rId = mMsg.getLong("request");
List<Request> rList = ejbObj.getRequestById(new Long(rId));
System.out.println("rList size is " +  rList.size());
for(Request r: rList) {
//Does something and print something in logs
System.out.println("Finsihed onMessage");
}catch(Exception e) {
System.out.println(e.getMessage());
}    One thing to note here is that getRequestById is another method in the same session bean MSSessionEJB. Can that be the reason for not loading the request from the database when many concurrent request are present? Here is what it does
@TransactionAttribute(TransactionAttributeType.REQUIRES_NEW)
public List<Request> getRequestById(Long id) throws Exception {
System.out.println("Inside getRequestById id is " + id);
return em.createNamedQuery("Request.findById")
.setParameter("id", id)
.getResultList();
}    In the logs I do see
{color:#800000}*"This is a instance of MapMessage"*{color}
and I do see the correct Rid that was persisted before in the log
*{color:#800000}
"inside getRequestById id is 12"{color}*
. Finally I do see
*{color:#800000}
"Finsihed onMessage"{color}*
for all the request no matter how many are sent concurrently.
*{color:#ff0000}Here is where the problem is . rList size is 1 for most of the cases but it returns 0 when the number of concurrent message exceeds 10.*
*{color}*
So you see there is no exception per se, just that entity does not get loaded from the database. I would like to mention this again, it does work properly when the number of concurrent request are less than 10. All the invocation of onMessage in MB are able to load the entity properly in that case. Its only when the number of concurrent request increases more than 10, the onMessage is unable to load the entity and the rList is 0.
FYI I am using jboss-4.2.2.GA on RedHat enterprise linux 4 with jdk1.6.0_04
Please let me know if there is more information required on this.
Thank you