Singleton pattern, scheduled receive location and suspended orchestrations

Hello,
In our project we have several orchestrations that are triggered every minute with the Scheduler Adapter (business requirement is near real-time).
Each of these orchestration must have only 1 instance running at the same time, so we have implemented the singleton pattern. We followed a design like this one: https://fehlberg.wordpress.com/2008/06/06/biztalk-singleton-orchestration-design/
But we end up having suspended orchestrations with the error "The instance completed without consuming all of its messages. The instance and its unconsumed messages have been suspended."
I know it's a risk with this pattern. I thought it would happen only when the instance would run in more than 1 min (because the scheduler adapter is sending a file every minute).
But it happens even with some instances running in about 15sec, so there is something I don't understand, here is an example:
Here we can see that the TransferRegion orchestration started at 10:26:00 (which is expected) and failed.
When I open the orchestration debugger, the time is different, the orchestration is shown starting at 10:26:44 (44 sec later):
And the orchestration ended at 10:27:00:453.
The message not consumed that caused the error has been triggered at 10:27:00.
So I understand that the message arrived just after the listen shape but before the orchestration ends, but I don't understand why the orchestration started 44 sec after is was supposed to ...
Any idea? I'm kind of clueless for now ...

So, are these 8 Orchestrations actually processing messages or are they doing some other work every minute or so?
If they're not processing actual messages, maybe a Windows Service or job scheduled by the SQL Agent or Windows Scheduler would be more...appropriate.  We all love BizTalk, but some parts of the stack do some things better.
If they are processing messages or doing some genuine BPI type work, calling services, transformations etc., then maybe a Singleton+Scheduled Task is not the best solution.
I'm assuming the actual requirement is the process must run with no less than 1 minute between executions, but if an execution takes >1 min, don't overlap.  An internal 1 min timer would probably work very well, but you still have the problem of
activating and deactivating the Orchestration.  You'd basically have to build some control infrastructure similar to the EDI Batching Service.
Here's a completely different suggestion:
SQL Table that maintains the state of each process, say Active or Complete.
Poll every 15 seconds a Stored Procedure that tests the status and returns an activation message when that process shows Complete.  It would flip it to Active at the same time.
Orchestration runs and the last shape sends a message to change the status to Complete.
Rinse Repeat.  No long running Orchestrations, Singletons, Correlations, etc...
Hi John,
We have 8 independent orchestrations. Each of them is triggered by different scheduled receive location.
The orchestrations then process messages (calling stored procedures, mapping and calling web services).
You are right for the requirement. I tried to deactivate the receive location at the beginning of the process and restart it at the end, but I ran into some weird errors on the SSO DB and others like "Could not retrieve transport type data for Receive
Location 'Trigger_xxxxx_SCHEDULE' from config store. The transaction associated with the current connection has completed but has not been disposed.  The transaction must be disposed before the connection can be used to execute SQL statements."
Thanks for the new design suggestion, I will see the effort required and if the client approves it ;)

Similar Messages

  • Unable to pick up the file from Receive Location

    BizTalk is  unable to pick up the files from receive location.
    My admin console is working so slow.
    It is showing the problem of snap-in as follows:
    I have checked all my receive location and its masking,it is working fine.
    Actually it is working fine in dev server(snap-in problem is not occuring in dev server ) but when I import MSI and binding in UAT server.
    It is not picking up the file from receive location.
    I want to know one more thing :if my message box database is not working fine,will it receive the files from receive location.
    Prakash

    For files not received by receive location. If the Receive Location is still enabled but files are not processed by Receive Location, then check whether the host instance  configured with your receive location is running. If the host instance is in
    running state, restart the host instance and try.
    Regarding the error in the admin console as shown in the image, people started to see this issue with admin console from BizTalk 2009 onwards. This happens in cases like you have done some activity with console and before its been refreshed it you do more
    activities like expanding the node you will see this error. Sometimes WMI is bit slowly to refresh your admin console screen. Just wait for few seconds untill the WMI screen is rendered completely. Ensure you have all the cumulative updates installed for your
    version of BizTalk and also ensure the service pack for your OS is updated.
    Obviously if message box is not working, files will not be received/processed. But if message box is not in healthy state this would affect all other application not just one and in-fact it would affect entire BizTalk not just Receive Locations.
    To start analysing this file not been processed by Receive Location, start with checking the host instance as mentioned.
    If this answers your question please mark it accordingly. If this post is helpful, please vote as helpful by clicking the upward arrow mark next to my reply.

  • Processing Multiple Files for more than 100 Receive Location - File Size - 25 MB each file, file type DML

    Hi Everybody
    Please suggest.
    For one of our BizTalk interface, we have around 120 receive locations.  We are just moving (*.dml) files from the source to destination without doing any processing.  
    We receive lots of files in different receive locations and in few cases the file size will be around 25 MB and so while moving these large files, the CPU usage is varying between 10% to 90+% and the time consuming for this single huge file is around
    10 to 20 minutes.  This solution was already in place and was designed by the previous vendor for moving the files to their clients.  Each client has 2 receive locations and they have around 60 clients.  Is there any best solution for implementing
    this with in BizTalk or outside BizTalk? Please suggest.
    I am also looking for how to control the number of files which gets picked from the BizTalk receive location.  For example, If we have say 1000 files in receive location and we want to pick at a time only 50 files only (batch of 50) then is it possible?
    because currently it is picking all the files available in source location, and one of the process is dropping thousands of files in to the source location, so we want to control  the number of files getting picked (or even if we can control to pick the
    number of KBs).  Please guide us on how we can control the number of files.

    Hi Rajeev,
    25 MB per file, 1000 files. Certainly you got to revisit the reason for choosing BizTalk.
    “the time consuming for this single huge file is around 10 to 20 minutes”
     - This is a problem.
    You could consider other file transfer options like XCopy or RobotCopy etc if you want to transfer to another local/shared drive. Or you can consider using SSIS
    which does comes with many adapters to send to destination system depending on their destination transfer protocol.
    But in your case, you have some of the advantages that you get with BizTalk. For your scenario, you have more source systems (more Receive locations), with BizTalk
    it’s always easier to manage these configurations, you can easily enable and disable them when a need arise. You can easily configure tracking; configure host instances based on load etc. So you can consider following design for your requirement. This design
    would suit you well since you’re not processing the message and just pass it through from source to destination:
    Use a custom pipeline component in the Receive Locations which receives the large file.
    Stores the received file into disk and creates a small XML metadata message that contains the information about where the large file is stored.
    The small XML message is then published into the
    message box db
    instead of the large file. Let the metadata file also contain the same context properties as the received file.
    In the send port, use another custom pipeline component that process the metadata xml file, retrieve the location of the disk where the file is stored, access the file and send it to destination.
    Read the following article on this design..
    http://www.codeproject.com/Articles/180333/Transfer-Large-Files-using-BizTalk-Send-Side
    This way you don’t need to publish the whole message into message box DB which would considerably reduce the processing time and utilises host instance to process
    more files. This way you can still get the advantages of BizTalk and still process large files.
    And regarding your question of restricting the Receive location to handles the number of files from receives location. No it’s not possible.
    Regards,
    M.R.Ashwin Prabhu
    If this answers your question please mark it accordingly. If this post is helpful, please vote as helpful by clicking the upward arrow mark next to my reply.

  • Multiple Receive location but Single Instance of Orchestration

    Hi,
    I have only one Receive Port (POP3 adapter) and multiple receive locations configured to different email IDs.
    ReceivePort1
    - Receive location1 - [email protected]
    - Receive location2 - [email protected]
    - Receive location3 - [email protected]
    If I send a mail with more than 1 recipient then that many orchestration instance is being generated.
    If I send a email where 'To' field has all three email Id also, always orchestration should instantiated only once. If I make my Orchestration as Singleton, does it help?
    Regards,
    Lakshmi

    Hi Laxme,
    Ask the team which manages the mailboxes to create a rule for these different emailIDs. The rule should route the mails (which you want to process in BizTalk, so you can have filter) to these email ID/ mailboxes to another email ID/mailbox something like
    [email protected]
    Then create a Receive port and Receive Location to receive emails from
    [email protected] In the port’s inbound map, apply map to convert the incoming message to the different mailbox to an canonical (common) schema
    Create a orchestration to receive message of type of the canonical (common) schema
    Define correlation set and types and in the correlation properties we will use the “BTS.ReceivePortName” property. This will enable you to have the convoy configured on all the messages arriving on a particular receive port. And define a singleton orchestration,
    if you want to have one instance of the orchestration created. Since you will have one mailbox to receive messages you can process the message in your orchestration even without the singleton orchestration depending on your actual processing requirements.
    If this answers your question please mark it accordingly. If this post is helpful, please vote as helpful by clicking the upward arrow mark next to my reply.

  • Service Locator and session facade pattern

    What are differences between Service Locator and session facade pattern?
    For me it seems one and the same.
    Please explain me in detail. Expecting kind help.
    Thanks,
    Rahul
    Edited by: rahulb1 on Feb 26, 2008 5:07 PM

    rdoekes wrote:
    The ServiceLocator Pattern is a singleton which hold a map of used DataSources. The idea is that you perform a lookup once and use the cache afterwards. The session facade is just one DataSource lookup.huh? Those J2EE patterns are not really reduced to datasources and the SessionFacade has actually nothing to do with datasources.
    Session Facade
    It has Facade in it and a Facade is usually used to give a simple entrypoint by providing a standartized interface. The same is now with the Session Facade. You have a Session bean that represents a high-level business component that interacts and calls lower-level business components.
    Imagine having client a accessing 5 business objects with the remote calls, which is not an efficient way (network latency) or accessing the SessionFace and the session face is doing all the work through local access.
    Service Locator
    As J2EE components are using JDNI to lookup for ejb interfaces,DataSources, JMS components, connections etc. isntead of writing all the lookup in many code piecess across the project, you write a service locator that gives you a centralized place to handle the lookup's. It's easier to maintain and to control such a setup.
    As you can see there's quite a big difference ;-)
    cya,
    Nail

  • Can we receive EDI as a String through WCF in to the orchestration receive location

    Hi,
    I had expose the orchestration as WCF service. Now i want to send the EDI file using WCF client to the receive location. So that my EDI file goes into the orchestration receive port and i would get the XML on the Send port.
    How i can achieve this? Please correct me if i am wrong.
    Thanks

    If you have exposed Orchestration as WCF service. And to test the process you can use test clients like
    WCFStrom,
    SOAPUI.
    With these test clients, when you add service reference to the URL of you WCF service (or provide the WSDL file) , these tools would read the contracts of the
    WCF and would provide an interface through which you can send the input parameter as expected by your exposed Orchestration.
    If your Orchestration (which is exposed as WCF service) is configured to receive EDI files, when you provide URL reference of the WCF service to the test clients
    (like WCFStrom, SOAPUI ), these test client would
    read the WSDL and would provide an interface for you to enter the data for the EDI. Look the following image of an WCFStrom which is expecting a sample schema (in your case, it would be the schema for WDI)
    So when you post a message using
    test clients, it would post the message to the WCF service. You would have got a Receive Location configured with WCF-based adapter (you get this when you
    import the binding which got created when you used the BizTalk WCF service publishing Wizard) which is bounded to orchestration (which is exposed as WCF) would receive the posted message and orchestration would process the posted message and you can see the
    output in send port as you have configured..
    If this answers your question please mark it accordingly. If this post is helpful, please vote as helpful by clicking the upward arrow mark next to my reply.

  • A page item on page 'SCHEDULE' at location 34,488 requires rasterization, but is too large to rasterize. Resize it smaller and try again

    I keep getting this error,
    A page item on page 'SCHEDULE' at location 34,488 requires rasterization, but is too large to rasterize. Resize it smaller and try again

    This basically means that there is an object that has its top left corner at x: 34 y: 488 which is too large to be rasterized, it could be due to any large object that is extreme wide or tall.
    Try to check page content and then resize it to verify the behavior. Please let me know how it goes.
    Thanks,
    Sanjit

  • Best Practice:  Using a static variables and methods vs singleton pattern

    Just curious, since when anything, within a class, is denoted as static it typically will be stored in main memory as a class property(method, variables, etc). Is it a good practice to be doing this all the time or is the singleton pattern a better choice. Please explain why. Thanks.

    I wouldnot make anything other than the constants and the instance itself static. And I cant think of a reason wjy one would.

  • What's the difference between "Receive Port" and "Receive Location" ?

    Hi Everyone,
    I am just starting to learn BizTalk, confused with some concepts, please help me figure out following doubt.
    What is the difference between  “Receive Port – Receive Location” relation and “Send Port Group – Send Port” relation?
    Thanks.
    wang

    Receive Port:
    A receive port is a logical grouping of similar receive locations through which services interact with external partners by receiving data. You can create the following types of receive ports: One-way — used for applications that drop off a message and do
    not wait synchronously for a reply; Request-response — used with applications that require a response to a message
    Receive Location:
    A receive location is the combination of a specific address at which inbound messages arrive and a specific address of the messaging pipeline that processes the messages received at the first specific address. Receive locations are created in BizTalk Explorer
    as part of the process of creating a receive port. There are two types of receive locations: One-way- Used for applications that drop off a message and do not wait for a synchronous reply; Request-response- Used with applications that require a response to
    a message. We can say that they are the physical, design-time notion of a location (such as a URL) and an adapter type. A receive location in the Host node in the BizTalk Server Administration console, defines the receive functionality. 
    Send Port: A
    send port is the location to which Microsoft® BizTalk® Server sends messages or from which BizTalk Server receives messages. It also provides the technology that BizTalk Server uses to implement the communication action.
    Send Port Group:
    A send port group is a named collection of send ports that you can use to send the same message to multiple destinations in a single configuration. This is useful when you want to send a message to multiple destinations. 
    You can find this useful
    glossary for any biztalk concepts
    When you see answers and helpful posts, please click Vote As Helpful, Propose As Answer, and/or Mark As Answer

  • Singleton pattern class and static class

    Hi,
    what is difference between the singleton pattern class and static class ?
    in singleton pattern, we declare a static member[which hold one value at a time] ,static method[ which return static member value] and a private constructor[not allow to direct instantiation]. My words are -- as a singleton pattern we implement static class .
    so can we say a singleton pattern is static class it means both are same or is any difference between these two ??

    malcolmmc wrote:
    On several occasions I've had to convert a static (never instanceated) class to a singleton type, for example because I realise I need an instance per thread, or I decide to make a program more modular, with more than one configuration.
    I've never done the opposite.
    So generally I favour the singleton if there any "state" involved, because it's more flexible in unanticipated directions. It also gives extra flexibility when building tests, especially if you program to an interface.Total agreement; if anything it is dead hard to override static method logic with mock code in unit tests (I had to do it by keeping the static methods in place but making them internally use a singleton instance of itself of which the instance variable could be overwritten with a mock implementation for unit testing purposes...).
    'Static classes' and/or methods are great for very simple util functions, but I wouldn't use them for anything other than that.

  • Receive Location - correlation

    Experts,
    i have one receive location say c:\document\Input\
    in Input folder i am having files names: ABC.xml   and   XYZ.txt
    in my orchestration i want to pickup xml file first and then all other .txt files and process it.
    how can i achive this.
    note: i am getting .txt file before .xml file.
    Thanks in adv.
    2Venture2

    This is sequential convoy/Singleton process. Here ABC.xml and XYZ.txt have predefined order. Your orchestration instance should start when the ABC.xml arrive and
    all the files of XYZ.txt (it should be *.txt is in it?, you can have multiple file with same name
    J) would be processed by the same instance of Orchestration which was triggered by the arrival/presence of ABC.xml
    file.
    Orchestration would be bound to the same Receive port but with different receive locations. One Receive Location-for ABC.xml
    and another for XYZ.txt
    Implementing Singleton pattern with BizTalk Orchestrations
    If this answers your question please mark it accordingly. If this post is helpful, please vote as helpful by clicking the upward arrow mark next to my reply.

  • The error threshold has been exceeded. The receive location is shutting down

    I am doing polling using SQL adapter......For small records polling is succesful......but for large recorde i am getting following error..
    The receive location "RcvSqlDataLoc" with URL "" is shutting down. Details:"The error threshold has been exceeded. The receive location is shutting down.".

    Hi
    You might have resolved this issue, However I would not read all those records in one go! Instead I would read them in batches and pass to BizTalk.
    I would use
    Scatter and Gather Pattern to distribute the records to multiple orchestration instance for processing and execute them using more than one BizTalk host instance?
    If you have already resolved the issue, Then it would be great if you share the solution with us!
    HTH,
    Naushad Alam (MCC,MCTS,BizTalk Server)
    If this is helpful or answers your question - please
    mark accordingly!
    alamnaushad.wordpress.com|
    My new TechNet Wiki
    "BizTalk Server: Performance Tuning & Optimization"

  • How to get application name from a receive location?

    Hi all,
    I am trying to do a custom error application, and I want to notify system admin users that "the receive port XXXX has stopped in application YYYYYY".
    Right now I do know which receive location that is being suspended. But I am not being able to get the application name, associated to this receive location (I can also get the ReceivePort, if it matters).
    I've tried 2 ways:
    > using WMI
    > Using SQL (direct query in BizTalk databases - for testing).
    I did not found any solution, even search table by table, i did not found a relation between Applications and receive location..
    Anyone? 
    Thanks in advance
    Ricardo Bessa

    The following SQL query should give you a result of every disabled receive location as well as it's receive port and application:
    SELECT
    RL.Name AS ReceiveLocationName
    , RP.nvcName AS ReceivePortName
    , APP.nvcName As Application
    FROM [BizTalkMgmtDb].[dbo].[adm_ReceiveLocation] AS RL WITH(NOLOCK)
    INNER JOIN [BizTalkMgmtDb].[dbo].[bts_receiveport] AS RP WITH(NOLOCK)
    ON RL.ReceivePortId = RP.nID
    INNER JOIN [BizTalkMgmtDb].[dbo].[bts_application] AS APP WITH(NOLOCK)
    ON RP.nApplicationID = APP.nID
    WHERE
    RL.[Disabled] = -1
    ORDER BY Application ASC
    Christian @ IntegrationAdmin.com

  • How we can limit the number of concurrent calls to a WCF service without use the Singleton pattern or without do the change in BizTalk Configuration file?

    How can we send only one message to a WCF service at a time? How we can limit the number of concurrent calls to a WCF service without use the Singleton pattern or without do the change in BizTalk Configuration file? Can we do it by Host throttling?

    Hi Pawan,
    You need to use WCF-Custom adapter and add the ServiceThrottlingBehavior service behavior to a WCF-Custom Locations.
    ServiceThrottlingBehavior.MaxConcurrentCalls - Gets or sets a value that specifies the maximum number of messages actively processing across a ServiceHost. The MaxConcurrentCalls property specifies the maximum number of messages actively
    processing across a ServiceHost object. Each channel can have one pending message that does not count against the value of MaxConcurrentCalls until WCF begins to process it.
    Follow MSDN-
    http://msdn.microsoft.com/en-us/library/ee377035%28BTS.10%29.aspx
    http://msdn.microsoft.com/en-us/library/system.servicemodel.description.servicethrottlingbehavior.maxconcurrentcalls.aspx
    I hope this helps.
    Rachit
    If this answers your question please mark it accordingly. If this post is helpful, please vote as helpful by clicking the upward arrow mark next to my reply.

  • Create receive location refer to web service

    Hi all
    I have a web service, that contains some methods such as SubmitOrder, TransferOrder, SynchOrder.
    Now I want create a receive location refer to this web service and start process biztalk application (Orchestration) when the TransferOrder method are invoked (called by SoapUI). Anyone please help me how to do it?
    Thanks in advance!

    If I understand it correctly you have following situation –
    Problem- You have a web service e.g. OrderProcessing which has following methods
    SubmitOrder
    TransferOrder
    SyncOrder
    And you have a BizTalk Orchestration say. TransferOrderOrch, Now you want to process a message by this orchestration when your webservice’s TransferOrder method is invoked. ?
    In summary you want to connect /integrate your webservice to this BizTalk process?
    Solution-
    1-  
    Either you can expose your TransferOrderOrch orchestration as a WCF service ( if you are using BizTalk 2010 onwards or as a web service) and then consume this web service to your OrderProcessing Orchestration.
    This will be loosely coupled and you can write maps to perform any type of message
    translation at receive side suing a pipeline or BizTalk Map.
    You can refer following articles to understand how to expose orchestration as web services
    http://msdn.microsoft.com/en-us/library/bb745989.aspx
    Or expose your TransferOrderOrch with same signature as your TransferOrder method has , and ask your consumers to use this. This may require significant changes at your Orderprocessing  Web Service end, it all depends upon type of messages and how
    many consumers are already consuming your service.
    This is better approach in my opinion because this will reduce some latency as you are directly submitting messages to the BizTalk isolated host service layer.
    Hope this helps. Feel free to post if you have any further queries. 
    Greetings,HTH
    Naushad Alam
    When you see answers and helpful posts, please click Vote As Helpful, Propose As Answer, and/or
    Mark As Answer
    alamnaushad.wordpress.com

Maybe you are looking for