ColdFusion Event Gateway - Data Processing Delay

Hi,
We are running ColdFusion 9 inside JBOSS.
We have event gateways that communicate with JMS Queue from JBOSS to acquire the vehicle related data for further processing (about 2 millions data per day in real time).
However, we facing performance degradation for the event gateways after it runs for 8hrs (initially data processing at 0 sec delay, subsequently built up to almost 10 mins delay) and it requires restart to resume back to normal speed.
Further checking on the servers show that the event gateways occupied all CPU processing power after running for few hrs.
May I know what is the issue and what can i do improve the performance. Your help shall be high appreciated.
Below the settings for event gateways.
Queue Size : 10000
Thread : 30
Server Spec
IBM x3650 series
RAM : 6GB
Processor : Dual Quad Core Processors 2.5GHZ

Hi, anyone can assist for my question posted ?
TQ

Similar Messages

Coldfusion Event Gateway Adapter exercise

I am doing this exercises in the FlexBuilder SDK .
Use Coldfusion Event Gateway Adapter exercise
When I get to
Select Event Gateways > Gateway Instances.
Enter Flex2CF2 as the Gateway ID.
Select Flex Messaging - Flex as the Gateway Type.
There is no Flex Messaging-Flex option
The closest two are
DataManagement Notifies Data Management Services of data
changes coldfusion.eventgateway.flex.FlexDataManagementGateway
DataServicesMessaging Handles Data Services Messaging
messages coldfusion.eventgateway.flex.FlexMessagingGateway
I try the example @
http://localhost:8700/samples/dataservice/myapp/flexemail2cf.mxml
The flex server is running fine but generates an RMI error in
the cmd console
flex.messaging.MessageException: Unable to find ColdFusion
gateway 'Flex2CF2 in RMI registry on host localhost
the gateway may not be running.
The gateway shows as running in the coldfusion
adminstrator.

You need to ask them on Adobe forum: https://forums.adobe.com/community/coldfusion
This posting is provided AS IS with no warranties or guarantees , and confers no rights.
Ahmed MALEK
My Website Link
My Linkedin Profile
My MVP Profile

ColdFusion Event Gateway Flex CF 8

I am doing this exercises in the FlexBuilder SDK .
Use Coldfusion Event Gateway Adapter exercise
When I get to
Select Event Gateways > Gateway Instances.
Enter Flex2CF2 as the Gateway ID.
Select Flex Messaging - Flex as the Gateway Type.
There is no Flex Messaging-Flex option
The closest two are
DataManagement Notifies Data Management Services of data
changes coldfusion.eventgateway.flex.FlexDataManagementGateway
DataServicesMessaging Handles Data Services Messaging
messages coldfusion.eventgateway.flex.FlexMessagingGateway
I try the example @
http://localhost:8700/samples/dataservice/myapp/flexemail2cf.mxml
The flex server is running fine but generates an RMI error in
the cmd console
flex.messaging.MessageException: Unable to find ColdFusion
gateway 'Flex2CF2 in RMI registry on host localhost
the gateway may not be running.
The gateway shows as running in the coldfusion adminstrator.
Please help

I think Flex messaging event gateway capability first was
available in CF 7.

LiveCycle 2.6.1 Data Management with The ColdFusion 8.0 DataManagement Event Gateway Issue

Hello all,
I've recently been developing a project that involves sending out events from ColdFusion to LiveCycle 2.6.1 using the Data Management event gateway to Flex 4.0 clients (LiveCycle and ColdFusion are on different Instances, but the same server). To begin with, I used ColdFusion assemblers, DAO's, and models and everything worked fine locally. After deploying this setup to a beta site, I decided that this setup would be very troublesome in terms of configuring clustered instances across multiple servers. I then decided to convert my assemblers, DAO's, and models to Java. The conversion went well and the flex clients see the exact same data as they did with the ColdFusion adapter.
Once I tried to send an update through from my ColdFusion application to a Flex client, I get an error stating that:
"Unable to find the Flex adapter for destination My_Dest in the RMI registry on localhost:1099.The Flex adapter may not be running or the destination may be incorrect."
After seeing this error, downloaded a Java-based RMI inspector to see what was going on. To get a good idea of what was happening when the ColdFusion adapter was being used, I switched my data-management-config file back to the CF adapter. I noticed that the RMI entry was as follows:
localhost:1099/cfdataserviceadapter/My_Dest
localhost:1099/cfassembler/my_cf_instance
Once I gathered this data as the base, I converted back to the Java adapter in my data-management-config file, restarted the servers, and ran the RMI inspector again. Only the "localhost:1099/cfassembler/my_cf_instance" was showing. (This one shows because I have "Enable Remote Adobe LiveCycle Data Management Access" checked in my CF instance's CF Admin -> Flex Integration). Since I don't need this checked anymore, I unchecked it and re-ran the RMI inspector. As it should, the "localhost:1099/cfdataserviceadapter/My_Dest" went away. Since no destination shows up, it means that the Flex adapter isn't registering my "my_Dest" destination with RMI. Since it isn't registered, I can't see it when I try to send a message through the CF Data Management event gateway.
Can anyone help me out here? I certainly may be missing something when it comes to RMI (I don't work with Java very often). Any advice would be greatly appreciated!
Thank you,
Dustin Blomquist

Dustin,
Without the ColdFusion based data management destination defined on the LCDS server, the destination will not show up in the RMI registry. It is only the CF adapter code that does this. The 'stock' LCDS adapter does not support invoking via RMI the way the CF version does.
I would recommend you run the LCDS MessageBrokerServlet inside the ColdFusion web application. This will give you two things:
1. You will not have the overhead of RMI between CF and LCDS as they will share the same VM (better performance!).
2. You will be able to use the CF Data Management Gateway to pass messages to Java-based destinations. The APIs the gateway uses should work fine with either CF or Java based Data Management destinations.
The CF/LCDS integration doesn't support what you are trying to do when you run two seperate instances.

Event Gateways in Multiserver Environment

Hi all
As a company we are looking to deploy a Flex application
using 'push' messaging.
The backend is a multiserver Coldfusion 8 setup behind a load
balancer. My concern is around the multiserver setup while using
event gateways. If a client consumes from a particular gateway on a
particular server, and a CF process on
another server produces for a gateway on it's server how
will the data push occur thus updating all clients consuming from
the gateways on
all servers.
I'm afraid we will have to get the CF process on one server
produce for it's own gateway and we'll have to get all clients to
consume from that gateway - thus not allowing us to spead the load
across all CF servers using thier own gateways in the multi-server
environment.
Am I missing something here? This area appears to be fairly
new, so any and all comments appreciated.
Cheers,
Ciaran

Hey guys, just wanted to register that I am having the same
exact problem! Let me know about any solution!
bhyder
Richard
Gere Fansite

Asynch Event Gateway not multi-threaded?

Hi there,
I've stumbled upon a huge problem and I would greatly
appreciate your help otherwise I am looking at a major rework of my
application :S
I thought that any event gateway (asynch) would be
multi-threaded? that is, multiple users can call the event gateway
at the same time and this would just fire up threads of the event
gateway and parallel processing of the cfc will occur?
However, that is not whats happening in my application. My
event gateway is called from a webpage and it computes (very
complicated and time consuming data) its data and creates a file
with a unique filename. However, if more then 1 user submits a job
at the same time, it only takes the first job, finishes it and then
starts working on the next job in the queue. So if there were 5
users who submitted their jobs one after the other, the event
gateway will take their jobs one by one which is very bad for me.
Is there a way to have event gateway work on multiple jobs at
the sametime??? or is this just not possible?
I will greatly appreciate your reply. thank you
using Coldfusion MX 7, WINXP

My event gateway is called from a webpage and it computes (very
complicated and time consuming data) its data and creates a file
with a unique filename. However, if more then 1 user submits a job
at the same time, it only takes the first job, finishes it and then
starts working on the next job in the queue. So if there were 5
users who submitted their jobs one after the other, the event
gateway will take their jobs one by one
I'm not so sure about your conclusion in the case of 5 jobs.
The 5 jobs may start in the order in which they were triggered, but
might end in a different order. Also, there could be some
multithreaded processing between the first job starting and the
last job finishing. Threads can be unpredictable worms.

ProjectUID selected from the reporting database from Project Server OnPublished event is taking some delay

Hi all,
I'm invoking a stored procedure from the reporting database from the project server onPublished event. I'm having some issues in getting the details of that particular project from the published event handler. This i noticed that there is some
delay is taking place in moving the data(project uid ) from the published to the reporting database .
My stored procedure select statement is as follows:
SELECT ProjectUID, ProjectName FROM MSP_EpmProject_UserView
　WHERE ProjectUID=value
This stored procedure is invoked from the onPublished event of the project server event handler where that particular projectuid is passed in that stored procedure.I have noticed that some delay is taking palce to move the data into the reporting database.
My stored procedure got invoked from the onPublished event of the project server event handler before the delay is complete. So the row selected from the stored procedure is alwys zero (0). Is there any other method to wait for repoting database
to get refreshed. Please help me .

Hi sabithad,
The issue you are facing is because of the conflict of permissions, event handlers always works through administrative user credentials whereas the project is open in edit mode by the logged-in user. And you are tyring to update custom field at the back
end with Administrative credentials, while project queue is in process by logged-in users credentials. Its not gonna work this way :)
The best option is to use Javascript on your PDP using CEWP, you can either build your complete logic in JavaScript and place that code on your PDP. OR you can use the combination of JavaScript and PSI and create a webpart to be placed on PDP.
below blog link willgive you directions to go ahead with, but for sure the solution to your query is
JavaScript :)
http://epmxperts.wordpress.com/2012/05/21/generate-unique-id-for-project-using-a-webpart/
hope this helps.
Khurram Jamshed - MBA, PMP, MCTS, MCITP (
Blog, Twitter, Linkedin )
If you found this post helpful, please “Vote as Helpful”. If it answered your question, please “Mark as Answer”.

Data processing in multiple caches in the same node

we are using a partitioned cache to load data ( multiple types of data) in multiple named caches. In one partition, we plan to have all related data and we have got this using key association.
Now I want to do processing with in that node and do some reconciliation of the data from various sources. We tried entry processor, but we want to consolidate all data from multiple named caches in the node. In a very naive form, I am thinking each named cache as a table and i am looking at ways to have a processor that will do some some processing on the related data.
I see we could use a combination of Invocable object, Invocation service and entry processors, but I am unable to implement it successfully.
Can you please point me to any reference implementation where I can do processing at the data node without transfering data back to client.
Also any reference implementation of Map reduce (at the server side) in coherence would be helpful.
Regards
Ganesan

Hi Ganesan
A common approach to perform processing in the grid is to execute background threads in response to backing map listener events. The processing is co-located with data because the listener will be called in the JVM that owns the data. The thread can then make Coherence calls to access caches just like any other Coherence client.
The Coherence Incbutor has numerous examples of this at http://coherence.oracle.com/display/INCUBATOR/Home. The Incubator Common component includes an event processing package that simplifies the handling of events. See the messaging pattern for an example: Message.java and MessageEventManager.java
I am not sure I answered your question but I hope the information helps.
Paul

Socket event gateway error in CF8?

Having upgraded to CF8 with no code changes I am seeing a
socket event gateway accept an xml message line by line and not the
entire xml packet in one shot (each line is shown in the log
separately). Passing a simple sentence with a return in it also
becomes 2 events. This obviously causes problems, as the xml cannot
be recognized and processed when it comes in line by line. The xml
is coming from a daemon writen in Perl, and I can't immediately
make any changes to it, as these requests are also sent out to
other Perl systems, so any changes would have to be tested against
them as well.
I couldn't find anything online about this being a bug in
CF8, has anyone else seen it?

Hi,
Remove this (And never put it in again)
EXCEPTION
when others then
raise_application_error(-20001,'Error sending email.Error: '||sqlerrm);
--Dbms_Put.Put_Line('Error sending email.Error: '||dbms_util.get_detailed_sqlerrm);This will tell you more about what is going on, such as giving you the location of the error.
But 20999, is a user defined exception, are you putting objects into system schema?
Regards
Peter

EVENT RAISE IN PROCESS CHAIN

Hi mates,
How to Raise events in process chain, for example i want to raise a event like ( Data loaded in ODS) after ODS is loaded.
thanks

Hi,
Look at the threads below. There are very good discussions on this topic :
Re: Process Chains Events
events and process chains
Query on using EVENTS in process chains.
Cheers,
Kedar

Sending above 254 characters through coldfusion sms gateway

I have been sending SMS using coldfusion sms gateway. If am sending below 254 characters, it works perfectly but when I need to send above 254 characters, I use the messagePayload attribute and convert the message to binary. The message sends OK but only the first 25 characters are displayed
on the phone. I have gone through the coldfusion documentation but what I don't understand is "The messagePayload field can contain up to 64K bytes;
it must start with 0x0424, followed by 2 bytes specifying the payload length, followed by the message contents."
Adobe didn't state any specific example on how this would be implemented. Kindly send an example on how this would be implemented.
If anyone knows any other method of sending long messages, please share it cos my .Net friends are laughing at me, telling me coldfusion is wack

quote:
are the unicode SMS getting thru? have you updated your cf
server, there's one
Are you referring to ColdFusion 7.0.1 Updater Paul?
Yes I did install the updater. I just found out that my
problem is not unicode problem at all. Basically every return
receipt that I received from the SMPP Gateway has err:1 in the
message. All I did is log every CFEvent that I received from the
SMPP server if error occurs. The example is:
STRUCT ->
CFCPATH -> ....
GATEWAYID -> My Gateway
ORIGINATORID -> 86???????????
CFCMETHOD -> onIncomingMessage
DATA ->
esmClass -> 4
sourceAddress -> 86???????????
priority -> 0
registeredDelivery -> 0
messageLength -> 135
protocol -> 0
destAddress -> *my number*
MESSAGE -> id:*some hex message id* sub:001 dlvrd:0
submit date:0606270957 done date:0606270957 stat:REJECTD err:1
text:esting routing,
dataCoding -> 0
GATEWAYTYPE -> SMS
and everytime I got an error, I only receive that message.
However after I called the SMPP support team just now, they said
that on their end they actually send out two messages to my server
and in my end, my logger only log one message for every SMS sent.
The thing really bugging me is, does coldfusion hiding some
of the messages they received from the server, or does it have some
kind of funny thing going on. Well, frankly I don't know much about
Coldfusion, I just been using it for about 6 months.

Events + Message Data Templates

I have defined my own versions of the events TaskCreated and TaskComplete in which I specify the XSD Event Message Template so that I can pass "complex" information.
In my event thrower, I ...
* setup an event data map to pass my process instance id
* provide an xml variable as the event message data.
In my event receiver, I have my process data map set up with 3 values.
* assign the process instance id passed in the event data map to a local variable
* assign a particular string element in my event message data (xml) to a local variable.
These both work.
What doesn't work is assigning the entire event message data (xml) to a local process variable of type xml. My local xml process variable is always null.
I don't understand how I can map a single value from within the xml, but I cannot map the entire xml.
Any thoughts?

And if you look at the provided event samples.
In the POCancel event, they specify an event message template which contains "Order" information, but they never use it in the samples.

Event Gateways missing from CF Administrator

I see other people have had this issue but I have not seen a
solution posted ...
I do not have any "Event Gateway" options in my CF
Administrator - nothing at all - no links on the left under Data
& Services or anywhere else.
Please ... any suggestions! Please!!!

Only the Enterprise edition has the Gateway feature enabled.
if you are running Standard Edition, you will not see links to
Event Gateway, because they are not available.

RAC data propagation delay?

Hi Experts,
I have a multi-threaded app that connects to a RAC DB using OCI.
Flow:
1.) Get an expired resource
2.) Assigned it to a user
Each thread executes the following sequence of queries:
1.) SELECT id, data FROM table_name WHERE date_expiry = :min_date AND rownum = 1 FOR UPDATE;
2.) UPDATE table_name SET date_expiry = trunc(sysdate) + 30, user = :user WHERE id = :id
3.) COMMIT;
-- :min_date is always <= trunc(sysdate)
I expect that each row will only be assigned to a unique user.
Apparently, this is not the case. It seems that some threads can still get a row even after the date_expiry has been updated.
Is there a data propagation delay between RAC nodes?
If there is, a thread can fetch a row even after another thread have updated it if the threads are connected in different nodes.
This is the only reason I can think of.
I tested this many times on a stand alone DB, but I can't replicate the error.
Please help!
BTW, our oracle version is:
Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 - 64bit Production With the Partitioning, Real Application Clusters, OLAP, Data Mining and Real Application Testing options
Edited by: user11912154 on Sep 17, 2009 3:01 AM

user11912154 wrote:
Each thread executes the following sequence of queries:
1.) SELECT id, data FROM table_name WHERE date_expiry = :min_date AND rownum = 1 FOR UPDATE;Bad method IMO. I have experimented with it when designing a PL/SQL replication system (Oracle SE) and this approach was not very robust and did not work properly.
A better method would be to use something like a dispatcher/thread manager that hands out the work. So instead of each thread trying to discover what work needs to be done - and running into concurrency issues - the manager process picks up a batch of work to be done and distributed that amongst threads.
Simplistic example. The thread manager fires of the SQL to find work and bulk fetches the 1st 50 rows. It closes the cursor, caches the 50 rowids and fires off 10 threads to process the 1st 10 rows. Each sleeps for a few seconds, wakes up, checks the thread number, finds that 8 threads are still busy and fires off 2 more threads to do the next 2 rowids. Repeat. When the cache has been processed (or when it is down to the last 10 rowids), the manager finds the next batch of work to do.
The key design issue is not to have threads competing to find work. As this means competing for access to the same resource, and potential serialisation issues - threads stepping on one another's toes and getting hurt.

Some kind of technical problems (like slow data processing)...

Some kind of technical problems (like slow data processing) with Leopard appears during the working in the local network.
OS Snow Leopard 10.6 is installed; including all updates, the computer I currently use is Mac Pro. So I have some kind of technical problems (like slow data processing) with Leopard during the working in the local network. (Windows network).
During the attempt to connect with network sharing (the computer is in the network) automounter process took approximately one minute.
Later on, after the automounter process is finished it’s working quite normally over remote computer, but when you try to automount again, everything starts from the very beginning.
The internet is working very well, it’s connected with domain, and everything is adjusted.
I use to ask our system administrator; maybe it is necessary to activate the some kind of protocol?
The OS Tiger is installed on the second hard disk, and it’s working well.
Maybe someone could help me, I try to find the similar problems, but I could not find them.
Thanks//

A couple of things.
First, thanks for the tips. I was starting to suspect the network might be an issue (bad nic, bad router, something).
But netstat -s was showing an average of about .15% packet loss. For a web server that seems to be a pretty good value so I'm not sure.
I did, however, ask someone else to look at the server because I'd spent the whole week on just server performance and was going out of my mind. This second pair of eyes really helped as he found that the pagefile for the server was set at 2mb!? Still trying to figure out how that happened. But we increased the pagefile to a much better value. There wasn't an immediate change in performance, but over time the server appears to have begun to respond better.
Occasionally I still get these 'lag spikes' but nowhere near what they were and the delay is much smaller as well, which is probably being caused by something else entirely (like bad programming on my part, hah).

ColdFusion Event Gateway - Data Processing Delay

Similar Messages

Maybe you are looking for