Significance of pipeline in asynchronous queue receiving

Hi
          Just want make sure I am getting the concepts right:
          A pipeline size of 10 [default] indicates that 10 messages will be prefetched for the asynch queue consumer [say an MDB].
          Also lets take that there are 100 msgs in the Queue and the top 10 are error messages - does this pipeline size of 10 indicates that only those top 10 errors will keep on coming back without giving the other messages a chance? Thats the part that is confusing me. No matter what I do - like giving a redelivery delay of 60secs - it still those top 10 that comes back.
          If I set the pipeline size to -1 then everything works great ie I get all the messages. But the problem is with a pipeline size of 1 I get a greeds consumers and in case of multiple instances of MDB [say in a single server/cluster] I am not sure if it would do justice to the proper load distribution.
          I really need some help understanding this - can someone from BEA put some pointers.
          thanks
          Anamitra

Inbound - Tcode: SMQ2 :  Inbound in the context of CRM and this queue shows the business transaction, master data and configuration data that are coming from ECC into CRM .
Naming Convention for Queues
The names of queues for data transfers from an ERP back end to the CRM server are made up as follows:
  R3A (from ERP back end to CRM)
    I for the initial data transfer / for Data Synchronization
    D for the delta data transfer
    U for loading to the ERP back end
    R for Defining Requests
    <Object> Name or part of object name, or the name of the higher-level independent object
    <Key> Only delta queues are subdivided by keys
Outbound - Tcode SMQ1 - Outbound in the context of CRM and this queue shows the transactions and data that are going from CRM to ECC
See more on [Using CRM Queues|http://help.sap.com/saphelp_crm70/helpdata/EN/49/2d9d9707361903e10000000a42189c/content.htm] following the link.
Read the [CRM Adapter Framework|http://help.sap.com/saphelp_crm70/helpdata/EN/49/2e33a2a6c51d24e10000000a421937/content.htm] for advanced learning.

Similar Messages

  • How to manage an Asynchronous Queue???

    Hi all,
    Been searching for quite some time on what the best way is/simply how to manage an Asynchronous Queue.
    I would have thought it was easy, but I'm definately getting the impression I am wrong. Here is some information that I would like to be front-facing on my client (whether Swing/Applet/Web-based hopefully does not make a difference):
    1. The number of messages waiting
    2. The number of messages being processed at the moment
    3. The date/timestamp of each message in waiting
    4. (optional) a user or programatically assigned name for each message in the queue (so that I will know what message I am looking at in-waiting).
    Seems pretty standard, but I don't see a standard way of getting this information! Someone, please show me I'm not insane! =)
    Environment is JDK 1.4.2, J2EE 1.3, JBoss 3.2.2, Message-Driven Beans.
    TIA,
    -D

    1,3 & 4 are possible from the QueueBrowser

  • Significance of Inboun and Outbound queue in sales order replication

    Hi CRM Leads,
    What is the significance of Inbound Queue and Outbound queue in saleas order replication from CRM to ECC.
    What exactly happens in Outbound Queue and Inbound Queue.
    Thanks,
    Sandeep
    What

    Inbound - Tcode: SMQ2 :  Inbound in the context of CRM and this queue shows the business transaction, master data and configuration data that are coming from ECC into CRM .
    Naming Convention for Queues
    The names of queues for data transfers from an ERP back end to the CRM server are made up as follows:
      R3A (from ERP back end to CRM)
        I for the initial data transfer / for Data Synchronization
        D for the delta data transfer
        U for loading to the ERP back end
        R for Defining Requests
        <Object> Name or part of object name, or the name of the higher-level independent object
        <Key> Only delta queues are subdivided by keys
    Outbound - Tcode SMQ1 - Outbound in the context of CRM and this queue shows the transactions and data that are going from CRM to ECC
    See more on [Using CRM Queues|http://help.sap.com/saphelp_crm70/helpdata/EN/49/2d9d9707361903e10000000a42189c/content.htm] following the link.
    Read the [CRM Adapter Framework|http://help.sap.com/saphelp_crm70/helpdata/EN/49/2e33a2a6c51d24e10000000a421937/content.htm] for advanced learning.

  • Asynchronous Queue listener - stale connection.

    Tom,
              I have an asynchronous listener for a remote distributed queue (4 jms servers). The listener process is running which had connected with the queue say 10-12 hours ago. The problem I have is, once message comes to one of the queue, even though listener has not lost the connection (as nothing comes in logs and ExceptionListener is implemented), Listener doesnt pickup the message. why?
              we are using WL 8.1 right now.
              Thanks,
              Malav

    You need to either ensure that the there's a listener on every distributed queue member, or configure the distributed queue to automatically forward messages from idle queues to queues that have consumers ("queue forwarding").
              Tom

  • Sync/Async Bridge without Asynchronous Send / Receive Steps

    Hello together,
    we have a synchronous HTTP <--> RFC Scenario using a sync/async Bridge in BPM with the following characteristics:
    1. HTTP Payload is received (opens Sync/Async bridge)
    2.  lookup is done in an LDAP System (we use an UDF here)
    3. Customzied JCo is called which takes the XML String as input parameter (done by Java Mapping)
    4. Return Value of JCo then is delivered back to HTTP Client (closes Sync/Async Bridge)
    --> No RFC Adapter is used here as the provided JCo does all the parsing
    When testing the sync/async bridge is started but we receive an timeout after 30 sec.
    Could it be that the sync/async required exactly a pair of async send and receive steps (which we do not have in our scenario and which we do not need)?
    This would be a problem, as we have to use BPM in this synchronous scenario. Do you have any idea?
    Thank you very much for your help.

    Thank you for your reply, but I think you misunderstood me. Please let me try to clarify once more:
    1. Synchronous HTTP Sender Interface (Request/Response) opens Sync/Async Bridge
    2. LDAP Lookup is done by UDF using the Request Message in Transformation Step
    3. Switch is done based on result of Lookup
    4. Branch 1 in switch calls customized JCo with Java Mapping in Transformation Step (Request Message as input, Response message as output)
    5. Response message (delivered from JCo Call) should close the Async/Sync Bridge and be send back to HTTP Sender
    --> We have no async send / receive steps in this scenario.
    The customized Jco has been provided by an external partner as there is lots of parsing logic implemented. So we want to connect to the backend SAP system using this JCo and not via a  send step.
    My key qestion is therefore: Is it absolutely necessary to use exactly a pair of async send / receive steps in a sync/async bridge?
    In this case we have to look for another design concept without BPM which would make the solution very difficult I think....

  • Anybody made experiences with JMS Queue Receiver (QPAC)?

    you find my question in this thread:
    http://www.adobeforums.com/cgi-bin/webx/.3bc3bd03
    regards, nico

    Made progress on this issue... reade more hear:
    http://www.adobeforums.com/cgi-bin/webx/.3bc3bd03

  • Issue with custom receive Pipeline component

    I have been facing issue with creating a custom receive pipeline component. The Pipeline is to receive large file, if the file size is large it has to read the incoming stream to a folder and pass only some meta data through the MessageBox. The Execute method
    I am using is,
    #region IComponent Members
    public IBaseMessage Execute(IPipelineContext pContext, IBaseMessage pInMsg)
    if (_largeFileLocation == null || _largeFileLocation.Length == 0)
    _largeFileLocation = Path.GetTempPath();
    if (_thresholdSize == null || _thresholdSize == 0)
    _thresholdSize = 4096;
    if (pInMsg.BodyPart.GetOriginalDataStream().Length > _thresholdSize)
    Stream originalStream = pInMsg.BodyPart.GetOriginalDataStream();
    string largeFilePath = _largeFileLocation + "\\" + pInMsg.MessageID.ToString() + ".zip";
    FileStream fs = new FileStream(largeFilePath, FileMode.Create);
    // Write message to disk
    byte[] buffer = new byte[1];
    int bytesRead = originalStream.Read(buffer, 0, buffer.Length);
    while (bytesRead != 0)
    fs.Flush();
    fs.Write(buffer, 0, buffer.Length);
    bytesRead = originalStream.Read(buffer, 0, buffer.Length);
    fs.Flush();
    fs.Close();
    // Create a small xml file
    string xmlInfo = "<ns0:MsgInfo xmlns:ns0='http://SampleTestPL.SchemaLocation'><LargeFilePath>" + largeFilePath + "</LargeFilePath></ns0:MsgInfo>";
    byte[] byteArray = System.Text.Encoding.UTF8.GetBytes(xmlInfo);
    MemoryStream ms = new MemoryStream(byteArray);
    pInMsg.BodyPart.Data = ms;
    return pInMsg;
    #endregion
    Here I want the xml to be dropped in to the File share Eg: E:\Dropbox\PL\send and and the entire message to be dropped in the folder Eg: E:\Dropbox\sendLarge. so in the ReceivePipeline properties i set like
    And in the send port the destination i give is E:\Dropbox\PL\send.
    The issue is both the xml and the message are getting dropped in to the same folder E:\Dropbox\PL\send and the message is not getting dropped in E:\Dropbox\SendLarge. Any help is greatly appreciated.

    using System;
    using System.Collections.Generic;
    using System.Text;
    using Microsoft.BizTalk.Message.Interop;
    using Microsoft.BizTalk.Component.Interop;
    using System.IO;
    namespace Sample.ReceivePipelineLargeFile
    [ComponentCategory(CategoryTypes.CATID_PipelineComponent)]
    [ComponentCategory(CategoryTypes.CATID_Decoder)]
    [System.Runtime.InteropServices.Guid("53fd04d5-8337-42c2-99eb-32ac96d1105a")]
    public class ReceivePipelineLargeFile : IBaseComponent,
    IComponentUI,
    IComponent,
    IPersistPropertyBag
    #region IBaseComponent Members
    public string Description
    get
    return "Pipeline component used to receive large file and save it ina disk";
    public string Name
    get
    return "ReceivePipelineLargeFile";
    public string Version
    get
    { return "1.0.0.0";
    #endregion
    #region IComponentUI Members
    public IntPtr Icon
    get
    return new System.IntPtr();
    public System.Collections.IEnumerator Validate(object projectSystem)
    return null;
    #endregion
    #region IPersistPropertyBag Members
    private string _largeFileLocation;
    private int _thresholdSize;
    public string LargeFileLocation
    get { return _largeFileLocation; }
    set { _largeFileLocation = value; }
    public int ThresholdSize
    get { return _thresholdSize; }
    set { _thresholdSize = value; }
    public void GetClassID(out Guid classID)
    classID = new Guid("B261C9C2-4143-42A7-95E2-0B5C0D1F9228");
    public void InitNew()
    public void Load(IPropertyBag propertyBag, int errorLog)
    object val1 = null;
    object val2 = null;
    try
    propertyBag.Read("LargeFileLocation", out val1, 0);
    propertyBag.Read("ThresholdSize", out val2, 0);
    catch (ArgumentException)
    catch (Exception ex)
    throw new ApplicationException("Error reading PropertyBag: " + ex.Message);
    if (val1 != null)
    _largeFileLocation = (string)val1;
    if (val2 != null)
    _thresholdSize = (int)val2;
    public void Save(IPropertyBag propertyBag, bool clearDirty, bool saveAllProperties)
    object val1 = (object)_largeFileLocation;
    propertyBag.Write("LargeFileLocation", ref val1);
    object val2 = (object)_thresholdSize;
    propertyBag.Write("ThresholdSize", ref val2);
    #endregion
    #region IComponent Members
    public IBaseMessage Execute(IPipelineContext pContext, IBaseMessage pInMsg)
    if (_largeFileLocation == null || _largeFileLocation.Length == 0)
    _largeFileLocation = Path.GetTempPath();
    if (_thresholdSize == null || _thresholdSize == 0)
    _thresholdSize = 4096;
    if (pInMsg.BodyPart.GetOriginalDataStream().Length > _thresholdSize)
    Stream originalStream = pInMsg.BodyPart.GetOriginalDataStream();
    string largeFilePath = _largeFileLocation + "\\" + pInMsg.MessageID.ToString() + ".zip";
    FileStream fs = new FileStream(largeFilePath, FileMode.Create);
    // Write message to disk
    byte[] buffer = new byte[1];
    int bytesRead = originalStream.Read(buffer, 0, buffer.Length);
    while (bytesRead != 0)
    fs.Flush();
    fs.Write(buffer, 0, buffer.Length);
    bytesRead = originalStream.Read(buffer, 0, buffer.Length);
    fs.Flush();
    fs.Close();
    // Create a small xml file
    string xmlInfo = "<ns0:MsgInfo xmlns:ns0='http://SampleTestPL.SchemaLocation'><LargeFilePath>" + largeFilePath + "</LargeFilePath></ns0:MsgInfo>";
    byte[] byteArray = System.Text.Encoding.UTF8.GetBytes(xmlInfo);
    MemoryStream ms = new MemoryStream(byteArray);
    pInMsg.BodyPart.Data = ms;
    return pInMsg;
    #endregion
    Thanks Osman Hawari, for trying to help me out.

  • Receiver ASynchronous RFC

    I need to develop an Asynchronous RFC Receiver side?   How should I accomplish this?
    appreicate your help.

    Hi Mohan,
    no, you dont call the RFC direcly. You just trigger a message, what ever you like, to XI. The inbound interface - your RFC - will be found by the interface determination of you IB directory. The ABAP call is made by the RFC adapter (you have to configure that RFC communication channel like described in <a href="http://help.sap.com/saphelp_nw04/helpdata/en/33/c6e63b60c25767e10000000a11402f/frameset.htm">SAP library: RFC-Adapter</a>) who translates the XML to DIAG protocol.
    Regards,
    Udo

  • Encrypt xml message in receive pipeline

    Hi,
       I am new to biztalk .
       I have a requirement where i need to encrypt the xml message in the receive pipeline.I know i have to develop a custom pipeline component to do this but i don't know in which stage of receive pipeline should i place this component in?  
    please advice .
    Thanks in advance 

    Hi Rose,
    To decrypt the message in receive pipeline and you will place the custom pipeline component in Decode stage
    of the receive pipeline and encryption will be done at send pipeline and you will place the custom pipeline component in Encode stage of the send pipeline.
    Decryption at receive pipeline:
    Encryption at
    send pipeline:
    BizTalk
    Server 2013: Encrypting and Decrypting a Message
    You can also refer the MSDN articles on how to decrypt and
    encrypt message in BizTalk.
    How to Configure BizTalk Server for Receiving Encrypted Messages
    How to Configure BizTalk Server for Sending Encrypted Messages
    Rachit
    Please mark as answer or vote as helpful if my reply does

  • Passing to multiple receive pipelines.

    Hello,
    I am rather new to BizTalk and have a design question.  Currently we are receive a custom receive pipeline in a receive port with multiple receive locations.  What I would like to do
    is pass a given message (these are HL7 messages coming in through the MLLP adapter) to both the current receive pipeline and another receive pipeline?  Is this possible?  If so how would I go about doing this?  I have thought about using an
    orchestration to call the receive pipelines but at this point the message is already in XML.  I need the native HL7 message to be passed through both pipelines.
    Thanks.
    -Chas

    Why do you want to do by passing same message to multiple pipelines.You associate a receive pipeline to a receive location not to a receive port.
    If you want to do two different process to the received native message, you can think about creating a custom receive pipeline with various component in different stages to process the native message before it being converted to XML.
    If this answers your question please mark it accordingly. If this post is helpful, please vote as helpful.

  • XBTO and XBTS Queues

    I have a simple question.  Why are the XBTO* and XBTS* queues shown in the SMQ2 (Inbound Queue Monitor) when these are actuallly Outbound and should have been shown under SMQ1 (Outbound Queue Monitor) ?
    Thanks,
    Badari

    Dear Badari,
    in context of PI it is important to make clear whether you are talking about qRfc queues or PI queues.
    Let me start with some words on pipeline processing: The pipeline for asynchronous message processing in the Integration Server mainly consists of 2 parts. When a message is received by the Integration Server the inbound handler does some initial checks, creates an XI message object, serializes the same and schedules the message for asynchronous pipeline processing. During part 1of pipeline processing the  receiver determination, interface determination and message split take place. Then the message object is serialized and again it is scheduled for asynchronous processing. During part 2 of pipeline processing the mapping and outbound binding is executed before invoking the adapter for sending the message to the receiver.
    Part 1 of pipeline processing is named the "inbound" while the second part is called "outbound". Consequently PI refers to queues of part 1 as to "PI inbound queues" and all queues used by part 2 of pipeline processing are called "PI outbound queues".
    And here is comes the crucial point: Both PI inbound queues and PI outbound queues are technically implemented as qRfc inbound queues.
    Due to this reason you will find PI queues in SMQ2 only.
    Best regards,
    Harald Keimer

  • IDOCs in TRFC Queue

    Hi,
    We have a ALE scenario where we send IDOCs to other SAP system.
    The IDOCs we send are sitting in TRFC queues, they are not sent to the target system. I checked all the ALE settings, they are fine. The processing mode is "Immediate".
    When I go to SM58 I will find the entry for the transaction with status "Transaction Recorded". If I select the entry and press F6 (Execute LUW) the IDOC is sent to the target system and the queue becomes empty.
    What I am missing? I dont want to go to SM58 everytime I post an IDOC. I am sure there will be some setting to process the IDOCs from the TRFC queue automatically. Can you please let me know that?
    Regards,
    Ravikanth

    HI,
    the problem is caused by immediately processing.
    Pipeline:
    Outbound Processing -> sender IDOC -> TRFC Queue -> receiver IDOC - Inbound Processing
    In case your are using IP (immediately Processing) at sending side your are using 1 LUW to send it to the TRFC Queue and saving IDOC at receving side -STOP.
    In case your are using IP at both sides .. your are using 1 LUW for the complete Pipe.
    That means this LUW is still open if the IDOc is processed in the receiving system.
    ToDO:
    Go to your Receiving System and check the Processing Settings. Try to use Background Jobs or get higher Batch Processing to avaoid TRF Queue Jam
    Regards,
    Gordon

  • Is there any tool monitoring messages in the pipeline?

    I wonder whether there is a tool which can tell the time each queue spends in each step of the pipeline?
    e.g. how much time a queue spent on client determination, on xml conversion, ...
    Thanks!

    I wonder whether there is a tool which can tell the time each queue spends in each step of the pipeline?
    Pipeline step means the Receiver Determination, Interface Determination etc.....so you can find this time in SXMB_MONI itself.
    SXMB_MONI --> Open Message Processing entry --> Performance Header --> Under Performance Header --> SAP:Timestamp --> For every step you will have the respective timestamp......I think the value after decimal indicates miliseconds (not sure on this part)
    Regards,
    Abhishek.

  • AQ Asynchronous Processes Count

    Hi,
    Oracle 9i Version 2
    aq_tm_processes = 2;
    job_queue_processes = 20;
    Does anyone know if it is possible to have more than 10 AQ callback procedures running at the same time?
    I use a PL/SQL callback procedure (asynchronous) that calls a JAVA stored procedure. The procedure simply inserts a row into a table and then sleeps for two minutes. When I look at the rows inserted into my test table it appears that 10 processes are run simultaneously. I would like to increase this number if possible.
    Any ideas or information is appreciated.
    Thanks,
    Keith

    I added a subscriber as you suggested and then registered the pl/sql function.
    When I enqueue messages to the queue, the callback procedure is still not
    notified that the queue received a message....
    Created a subscriber:
    DECLARE
    subscriber sys.aq$_agent;
    BEGIN
    subscriber := sys.aq$_agent('derive_sub','derivations.S3068_QUE',NULL);
    DBMS_AQADM.ADD_SUBSCRIBER(queue_name => 'S3068_que', subscriber => subscriber);
    END;
    registered the pl/sql procedure as such:
    declare
    reginfo1 sys.aq$_reg_info;
    reginfolist sys.aq$_reg_info_list;
    BEGIN
    reginfo1 := sys.aq$_reg_info('derivations.S3068_QUE:derive_sub',dbms_aq.namespace_aq, 'plsql://derivations.notifycb',HEXTORAW('FF'));
    reginfolist := sys.aq$_reg_info_list(reginfo1);
    sys.dbms_aq.register(reginfolist,1);
    END;
    enqueued the queue:
    DECLARE
    enqueue_options dbms_aq.enqueue_options_t;
    message_props dbms_aq.message_properties_t;
    message_handle RAW(16);
    message message_type;
    BEGIN
    message := message_type('TEST','TESTING');
    dbms_aq.enqueue(queue_name => 'S3068_que',
    enqueue_options => enqueue_options,
    message_properties => message_props,
    payload => message,
    msgid => message_handle);
    Commit;
    END;
    Any other ideas would be appreciated.
    Thanks,
    Keith

  • How to find the number of pending messages in a Queue

    All,
    Is there a way to find the number of pending messages in a given QUEUE to be picked by the queue reader?
    My reader is not picking all the messages as many sent by my writer, i am not sure where they are getting lost? or if the JMS implementation is buffering them and does not flush until it does not have enough to throw out ( which any way does not make sense!!).
    Please help.
    Chandra

    You're right in thinking that the behaviour you describe looks wrong.
    Take a look at the QueueBrowser interface which can be used to inspect a queue. You get one using QueueSession.createBrowser(Queue). The QueueBrowser allows you to scan a queue without consuming the messages. However note that the state of the queue may change while you're observing it through the QueueBrowser so what you see might not always be a 100% accurate representation of the queue at any given time.
    Other things to check:
    - message selectors - are you setting a message selector on your queue receiver which is resulting in not all of the messages 'matching'.
    - expired messages - are you setting a time-to-live on the messages that you publish.
    Hope this helps,
    David Ingham
    http://www.arjuna.com

Maybe you are looking for