Entry Processor and worker thread question
We have a cache service configured with a thread count of 10 . When i run a Entry processor to do some processing on all entries of the cache, i see in the logs that each of the entry is processed sequentially on same worker thread even though multiple threads are configured for cache service . I was hoping Entry processor would run parallely on multiple threads for different entry's . Am i missing something here?? Appreciate all the help .
Here is thelog:: Looks like entry processor runs sequentially on same thread.
011/04/06 15:58:55:489 [*DistributedCacheWorker:6*] INFO ENTRY PROCESSOR ON ACCOUNT 1872
2011/04/06 15:58:55:490 [*DistributedCacheWorker:6]* INFO ENTRY PROCESSOR ON ACCOUNT 38570
2011/04/06 15:58:55:490 [*DistributedCacheWorker:6*] INFO ENTRY PROCESSOR ON ACCOUNT 38856
2011/04/06 15:58:55:491 [*DistributedCacheWorker:6*] INFO ENTRY PROCESSOR ON ACCOUNT 1586279
2011/04/06 15:58:55:492 [*DistributedCacheWorker:6*] INFO ENTRY PROCESSOR ON ACCOUNT 38570
Hello,
Entry Processors will use multiple threads. They will however only use one thread per partition.
My guess is what you are seeing is that the EP is working on the same partition.
hth,
-Dave
Similar Messages
-
DVD formatting and work flow questions
It’s been about ten years since I was involved in a DVD project. I am trying to catch up and thought a post on this forum might speed my way considerably. TIA to anyone who can help me out.
Project info: I have authored a Keynote presentation on California and the Civil War. (Visit www.bearriverbooks.com if you're interested in the content). I have two versions of the show – one at 4:3 (1024 x 768 pixels) and one at 16:9 (1280 x 720 pixels). I want to produce a DVD of the show to sell to the general public.
Work flow: The workflow is Keynote exported as QT.mov to AfterEffects, where timing is adjusted and animations refined. I then want to build a project in Encore and import my AE composition (via dynamic link, if possible). I have built the 16:9 menus in Photoshop and tested them successfully in Encore. The total Keynote show comes to 45 minutes, and the data load is light, since much of the presentation is composed of still images. Should be loads of room on the DVD. All Adobe apps are CS5.
Questions:
1) Is 16:9 the industry standard DVD aspect ratio these days? (I assume that a big slice of DVD-watching occurs on 16:9 HDTV sets. Any current stats on HDTV vs. computer-based watching?)
2) I notice that some DVDs I rent from Netflix are dual formatted (4:3 and 16:9). Any point to this now, or is the a relic of the analog past?
3) What should my QT export specs from Keynote be? I’d like to get the best resolution I can from my Keynote graphics and sound. (Sound is 16-bit 48000 uncompressed.)
4) What should my AfterEffects composition and Encore project transcoding specs be?
This is where I am really hurting. I have no idea what formats are best suited for the general marketplace. Frame resolution, pixel aspect ratio, and frame rates still seem pretty arbitrary to me after my initial reading of the Encore docs and various Wikipedia articles. Interleave? Progressive scan? I have no idea what to shoot for. Any help in this area would be gratefully accepted, as would referrals to lucid research documents.
Richard Hurley
Bear River BooksWhat other options instead of QT do you have available?
QT is the only animation export option. You can kick out png.s, jpg.s, and tif.s as stills but you only get one image per "build" state (which you can think of as a kind of animation keyframe). This route would require a lot of animation & transition re-building in AfterEffects, which I'd like to avoid. (Animating in Keynote is a bit like building a ship in a bottle, but Apple has created a remarkable set of capabilities, given that there is no timeline or layer palette. Truth is, I only see some tweaking necessary in AfterEffects.)
If you want 1280x720 on a disk you have to make a BluRay disk.
It looks like Encore is equipped to do this with a minimum of fuss. Does anyone have any rule of thumb about current DVD vs Blue Ray market share? This product would be sold at historical societies, museums, and in the educational market. Probably not cutting edge technology users, but not altogether out of the loop. (I.e., no VHS tapes!)
Lord, I thought I was coming back to a digital wonderland. Damn. -
Very strange, Entry Processor block the entire coherence node
I have a client node(storage= false) client1 which continues do get from cache1, the code is below:
for(int i=0;i<10000000;i++){
System.out.println("get:"+ cache1.get(i));
and have another client node(storage= false) client2 which invoke a Entry Processor. this Entry Processor is work on cache2(not cache1)
cache2.invoke(keys,new MyEntryProcessor());
the MyEntryProcessor code is as below:
public Object process(Entry entry) {
for(int i=0;i< 1000000;i++){
entry.getKey();
entry.getValue();
System.out.println(i);
return null;
when client2 begin run, client1 will be blocked, until client2's Entry Processor is finished.
who can tell me why. it's very strange, because client1 and client2 are work on two different cache. and it's only do getIf these two caches belong to same cache service, then their requests are handled by the same service thread. Coherence only use single service thread per service per node. So if the get() for cache1 and the entry processor for cache2 go to the same node, you will see that behavior if cache1 and cache2 belong to same cache service.
Either turn on thread pool (if you want to use same cache service( or use different cache service. -
Spawning new entry processors from within an existing entry processor
Is it possible / legal to spawn a new entry processor (to operate within a different cache) from within an existing entry processor.
E.g we have a parent and a child cache, We will receive an update of the parent and start an entry processor to do this. Off the back of the parent update we will also need to update some child entries in another cache and need to start a new entry processor for the child entries. Is it legal to do this?Hi Ghanshyam,
yes, in case of (a), you would be mixing different types in the same cache. There is nothing wrong with that from Coherence's point of view, as long as all code which is supposed to access such objects in their deserialized form is able to handle this situation.
This means that you need to use special extractors for creating indexes, and you need to write your filters, entry processors and aggregators appropriately to take this into account. But that's all it means.
The EntryProcessor on the child could be invoked, so long as there are more service
threads configured. This allows retaining partition affinity. I don't think this is technically
illegal.It is problematic, as invoking an entry-processor from another entry-processor in the same cache service can lead to deadlock/livelock situations. You won't find it out in a simple test as you get an exception or not.
But even if it is technically not guarded against, firing a second entry-processor consumes an additional thread from the thread-pool. Now if you get to a situation when all (or at least more than half of the thread-pool size) of your entry-processors try to fire an additional entry-processor, and there are no more threads in the thread-pool, then some or all would be waiting for a thread to be available, and of course none would be available, because there are not enough single-thread entry-processors to leave to get a thread to everyone.
However, none of them can back off as all are waiting for the fired entry-processor to complete. Poof, no processing is possible on your cache service.
Another problematic situation which can arise if entry processors are fired from entry processors is that your entry-processors may deadlock on entries (entry processors executing on some entries and trying to execute on another entry on which another entry processor executes and also tries to execute on the first entry). In this case the entry-processors would wait on each other to execute.
No code running in the cache server invoked by Coherence is supposed to access a cache service from code running in the threads of the same cache service, except for a couple of specifically named operations which only release resources but not consume additional new ones.
Best regards,
Robert -
Entry processor performance and serialization
Our application invokes an entry processor from a .Net extend client. We are seeing very poor performance the first few times the processor is invoked (typically 60ms vs 2ms). I was expecting this to be due to the overhead of the initial TCP connection but this doesn't appear to be the case. Here is a typical set of timings from the beginning of invoking the EP on the client:
5ms: Object starts being deserialized on proxy
7ms: Deserialization is finished, and serialization begins
9ms: Serialization is complete.
55ms: Deserialization begins on the storage node
58ms: Deserialization complete on storage node
58.5: Entry processor invocation complete
60ms: Serialization of result is complete
My questions are:
- What is happening between the end of serialization of the object by the proxy and deserialization starting on the storage node?
- Given that the cache is POF enabled, why is the proxy deserializing and reserializing the entry processor?
Version is 3.7.1.6Hi Rob,
Config is below. defer-key-association-check is not set. The timings were established by logging the POF serialization in each process so its definitely the proxy that is doing it.
Regards,
Dave
Client config:
<?xml version="1.0"?>
<cache-config xmlns="http://schemas.tangosol.com/cache">
<caching-scheme-mapping>
<cache-mapping>
<cache-name>Cache</cache-name>
<scheme-name>remote-scheme</scheme-name>
</cache-mapping>
</caching-scheme-mapping>
<caching-schemes>
<remote-cache-scheme>
<scheme-name>remote-scheme</scheme-name>
<service-name>ExtendProxyService</service-name>
<initiator-config>
<tcp-initiator>
<remote-addresses>
<socket-address>
<address>localhost</address>
<port>1234</port>
</socket-address>
</remote-addresses>
<connect-timeout>5s</connect-timeout>
</tcp-initiator>
<outgoing-message-handler>
<request-timeout>1s</request-timeout>
</outgoing-message-handler>
<serializer>
<class-name>Tangosol.IO.Pof.ConfigurablePofContext, Coherence</class-name>
<init-params>
<init-param>
<param-type>string</param-type>
<param-value>assembly://..../pof-config.xml</param-value>
</init-param>
</init-params>
</serializer>
</initiator-config>
</remote-cache-scheme>
</caching-schemes>
</cache-config>
Server cache config:
<?xml version="1.0"?>
<cache-config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns="http://xmlns.oracle.com/coherence/coherence-cache-config"
xsi:schemaLocation="http://xmlns.oracle.com/coherence/coherence-cache-config coherence-cache-config.xsd">
<defaults>
<serializer>pof</serializer>
</defaults>
<caching-scheme-mapping>
<cache-mapping>
<cache-name>Cache</cache-name>
<scheme-name>cache-distributed</scheme-name>
</cache-mapping>
</caching-scheme-mapping>
<caching-schemes>
<distributed-scheme>
<scheme-name>cache-distributed</scheme-name>
<service-name>PartitionedCache</service-name>
<thread-count>10</thread-count>
<partition-count>2039</partition-count>
<backing-map-scheme>
<local-scheme />
</backing-map-scheme>
<autostart>true</autostart>
</distributed-scheme>
<proxy-scheme>
<service-name>ExtendProxyService</service-name>
<thread-count>5</thread-count>
<acceptor-config>
<tcp-acceptor>
<local-address>
<address>localhost</address>
<port system-property="tangosol.coherence.proxy.port"></port>
</local-address>
</tcp-acceptor>
</acceptor-config>
<proxy-config>
<cache-service-proxy>
<class-name>....InteceptorCacheService</class-name>
<init-params>
<init-param>
<param-type>com.tangosol.net.CacheService</param-type>
<param-value>{service}</param-value>
</init-param>
</init-params>
</cache-service-proxy>
</caching-schemes>
</cache-config> -
Strange read-through operation after entry processor work
Hi.
We use the combination cache listener - entry processor to do some actions when the data comes to coherence. We use Oracle Coherence Version 3.5.3/465.
Just after the entry processor has set the new value for the entry, the new "get" operation is called for the cache and jdbc hit is done for this key.
Here is the entry processor:
public Object process(Entry entry) {
if (!entry.isPresent()) {
// No entities exist for this CoreMatchingString - creating new Matching unit
MatchingUnit newUnit = new MatchingUnit(newTrade);
entry.setValue(newUnit, true);
return null;
((MatchingUnit)entry.getValue()).processMatching(newTrade);
return null;
}Very interesting, that if I use entry.setValue(value) without second parameter - I recieve the db hit just on setValue method. According to docs, setValue() with one parameter returns the previous value and its logical, that the cache hit (and therefore the db hit) is done just on set. But I use the overloaded version void setValue(java.lang.Object oValue, boolean fSynthetic), which is said to be light and should not fetch previous version of the object. But this is done anyway! Not on setValue itself, but just after the process() method called.
Actually it's strange, that coherence tries to fetch the previous value in the case it didn't exist! The cache.invoke(matchingStr, new CCPEntryProcessor(ccp)) is invoked on not existing record and it is created just on invokation. May be it's the bug or the place for optimization.
Thanksbitec wrote:
Thanks, Robert, for such detailed answer.
Still not clear for me, why synthetic inserts are debatable. There are lots of cases, when the client simply updates/inserts the record (using setValue()) and does not need to recieve the previous value. If he needs, he will launch the method:Hi Anton,
it is debatable because the purpose of the fSynthetic flag is NOT so that you can optimize a cache store operation away. Synthetic event means that this is not real change in the data set triggered by the user, it is something Coherence has done on the backing map / on the cache for its own reasons and decisions to be able to provide high-availability to the data, and it only changes that particular Coherence node's subset of data but does not have a meaning related to the actually existing full data set. Such reasons are partition movement and cache eviction (or possibly any other reasons why Coherence would want to change the content of the backing map without telling the user that anything has changed).
If you set the synthetic flag, you are trying to masquerade a real data change as an event which Coherence decided to trigger. This is why it is debatable. Also, synthetic backing map events may not always lead to dispatching cache events (for partition rebalance they definitely not). This optimization may also be extended to synthetic cache events.
java.lang.Object setValue(java.lang.Object oValue)and recieve the previous value. If he doesn't, he calls:
void setValue(java.lang.Object oValue, boolean fSynthetic)and DOESN'T recieve the previous value as method is marked void. Thus he cannot get the previous value anyhow using this API, except of the direct manual db call. Yep, because Coherence is not interested in the old value in case of a synthetic event. The synthetic methods exist so that some entry can be changed in Coherence (usually by Coherence itself) in a way that it indicates a synthetic event so that listeners are not notified.
Some valid uses for such functionality for setValue invoked by user code could be compacting some cached value and replacing the value the backing map stores with the compacted representation, which does not mean a change in the meaning of the actual cached value, only the representation changes. Of course if the setValue(2) method does not actually honor the synthetic flag, then such functionality will still incur all the costs of a normal setValue(1) call.
But the previous value is anyway fetched by Coherence itself just after process() and client anyway doesn't get it! But any listeners on the cache may need to get it due to cache semantics reasons.
In this case I regard this as the bug, cause the client, which uses this API doesn't expect the cache hit to take place (no return value for this overloaded setValue() method), but it takes and leads to some extra problems, resulting from read-through mechanizm.
I would not regard it as a bug, it is probably the case of documenting a possible optimization too early, when it ultimately did not get implemented. I definitely would not try to abuse it to set a value without triggering a db fetch, as again, the intention of the synthetic flag is related not only to the cache loader functionality, but also to events and marking that a change indicates a real data change or a Coherence data management action.
Now I understand why coherence does not know, whether this is inserted or updated value, thanks for details.
Anton.
*Edited: I thought about this problem from the point of the oracle user, but may be this additional hit is necessary for event producers, which need to make the events, containing old/new values. In this case this seems to be the correct behaviour.... Seems that I need some other workaround to avoid db hit. The best workaround is the empty load() method for cachestore...You can try to find a workaround, but it is an ugly can of worms, because of the scenario when more than one thread tries to load the same entry, and some of them try to load it with a reason.
You may try to put some data into thread-local to indicate that you don't really want to load that particular key. The problem is that depending on configuration and race conditions your cache loader may not be called to clean up that thread-local as some other thread may just be invoked right now and in that case its return value is going to be returned to all other threads, too, so you may end up with a polluted thread.
Best regards,
Robert
Edited by: robvarga on Oct 15, 2010 4:25 PM -
I Forgot my Apple ID password and I can't reset it because the related email account has been closed and the security questions don't work! How do I solve this? Thank you.
Click here and contact Apple.
(113292) -
Similar to the question in cvs and eclipse thread
My Question is similar to the question in the cvs and eclipse thread, how can you know on the *.<file_extension>, v the date it was created? thank you very much to those who will reply
Hey wspearman,
Thanks for the question. Genius is still a great feature of iTunes 11, it may be that you simply need to turn it on. To do so, click Store > Turn On Genius
iTunes 11 for Windows: Use Genius Shuffle, Genius playlists, or Genius Mixes
http://support.apple.com/kb/PH12342
Note: Use Control-B on a Windows computer to show/hide the menubar
Thanks,
Matt M. -
Nvidia Graphics Processor (windows Vista) crashes on CS6 and works on CS5
Nvidia Graphics Processor (windows Vista) crashes on CS6 and works on CS5
Hi Steve in Palm Desert!
I see you are running Windows Vista.You know that Vista is not supported for CS6, right? It is not prevented from running, but not supported.
That known, what video card do you have ? Do you have the most updated driver? My laptop is brand new and I had an outdated video driver right upon delivery.
Maybe that could help.
Check out the following links about supported OS and Video cards:
FAQ: What features use the GPU and how do I troubleshoot GPU issues?
WHAT ARE THE SYSTEM REQUIREMENTS TO USE PHOTOSHOP CS6? (scroll down to bottom)
http://labs.adobe.com/technologies/photoshopcs6/
CS6 OPERATING SYSTEM SUPPORT INFORMATION:
http://blogs.adobe.com/photoshopdotcom/2012/03/photoshop-cs6-operating-system-support-and- beyond.html
Pattie -
MSI 915GM4 Motherboard and a P4 s775 2.66/1M/533 Processor - Dont Work
Hello Everyone,
I bought an MSI 915GM4 Motherboard and a P4 s775 2.66/1M/533 Processor yesterday and
assembled it by myself.But when I power it on, it won't display anything nor you can any hear beeping sounds.
So I tried to pull it out from the case to test only the Processor, Motherboard, PowerSupply, Memory and Monitor, but still It won't display nor you can hear beeping sounds that would indicate the cause of the problem.
Later then, I contacted a friend of mine who has a Celeron D s775 Processor and tried it on my MSI Motherboard. Its only the processor that was swapped and nothing else. Guess what? It displays and even loads to windows. So I was started thinking that my own processor might be the problem. But I did not conclude right away that the
processor is the problem and so what I did is I borrowed an Asrock Motherboard (s775)
from the same friend. Now, if this will not work then its confirm that my processor
might be the problem. But I was surprised when it work and still boot up to windows.
Confusion wraps my mind, the next time I started thinking about compatibility issues
and so I visited MSI site looking for CPU Support of my Motherboard and found out that
it should support. Well, to try to resolve this issues I tried flashing the bios for
the motherboard of its latest version updates, but sitll no luck..
Hmmmm.... what could be it? Did I miss something?
Well guys if you could help me on this I really appreciate it.. I don't really know now what move to take.
If anyone has an idea please email me at [email protected]
Thanks in advance...start here https://forum-en.msi.com/index.php?topic=25959.0
list your power supply specs and name brand.
clear the cmos and retry! -
Worker Thread and Helper Thread for Gauge updates
Hi
I know that in the Smart Ticket sample application there is an example with the implematation of "Each worker thread also has a helper thread that displays an animated gauge to indicate the progress of the worker thread".
But i can't find the sample on java.sun.com anymore..:-(. Do you have the source code of smart ticket or even an example of the helper thread implemetation. The worker thread i have already implementated.
Thanx
MichaelHere's a simple outline of what I do. This has been drastically stipped down from my actual code as I use teh message for all kinds of things incluing requests on a queue to the worker and resposes back. So in the real code I also have a success flag, a command type, an argument object and a result object with get and set methods for all of them.
1) define an interface that the GUI code can implement and pass a reference to the worker which stores it in myTarget.
interface MessageTarget
void postMessage(Message m);
void runMessage(Message m);
}2) define a message class
class Message implements Runnable
Object data;
MessageTarget target;
Message(MessageTarget target, Object data)
this.target = target;
this.data = data;
public void run(){target.runMessage(this);}
public Object getData(){return data;}
public void postMessage(){target.postMessage(this);}
}3) implement the MessageTarget interface in the GUI
// put a response from the database onto the swing queue
public void postMessage(Message r)
javax.swing.SwingUtilities.invokeLater((Runnable)r);
// process messages
public void runMessage(Message m)
Object data = m.getData();
}4) post messages back from the worker
new Message(myTarget,myData).post(); -
Hey, my first question is that do I need to be connected to iCloud on my phone for my Imessages to work on my Mac?
And my second question is if someone is looking into my Imessages on my Mac, how can I disconnect it from my phone right away?you will need to do a hard wipe or remember the password hard wipe can be found on youtube how to. but this will erase all data
-
I recently upgraded by iMac iOS to Yosemite, and I find that my best buy provided Trend Micro stopped working. Question, what security software is recommended, if any for an iMac. Thanks
Uninstall Trend Micro software. It is not needed and could prevent your computer from operating properly. If there is no uninstaller for the software, then see the following:
Uninstalling Software: The Basics
Most OS X applications are completely self-contained "packages" that can be uninstalled by simply dragging the application to the Trash. Applications may create preference files that are stored in the /Home/Library/Preferences/ folder. Although they do nothing once you delete the associated application, they do take up some disk space. If you want you can look for them in the above location and delete them, too.
Some applications may install an uninstaller program that can be used to remove the application. In some cases the uninstaller may be part of the application's installer, and is invoked by clicking on a Customize button that will appear during the install process.
Some applications may install components in the /Home/Library/Applications Support/ folder. You can also check there to see if the application has created a folder. You can also delete the folder that's in the Applications Support folder. Again, they don't do anything but take up disk space once the application is trashed.
Some applications may install a startupitem or a Log In item. Startupitems are usually installed in the /Library/StartupItems/ folder and less often in the /Home/Library/StartupItems/ folder. Log In Items are set in the Accounts preferences. Open System Preferences, click on the Accounts icon, then click on the LogIn Items tab. Locate the item in the list for the application you want to remove and click on the "-" button to delete it from the list.
Some software use startup daemons or agents that are a new feature of the OS. Look for them in /Library/LaunchAgents/ and /Library/LaunchDaemons/ or in /Home/Library/LaunchAgents/.
If an application installs any other files the best way to track them down is to do a Finder search using the application name or the developer name as the search term. Unfortunately Spotlight will not look in certain folders by default. You can modify Spotlight's behavior or use a third-party search utility, EasyFind, instead.
Some applications install a receipt in the /Library/Receipts/ folder. Usually with the same name as the program or the developer. The item generally has a ".pkg" extension. Be sure you also delete this item as some programs use it to determine if it's already installed.
There are many utilities that can uninstall applications. Here is a selection:
1. AppZapper
2. AppDelete
3. Automaton
4. Hazel
5. AppCleaner
6. CleanApp
7. iTrash
8. Amnesia
9. Uninstaller
10. Spring Cleaning
For more information visit The XLab FAQs and read the FAQ on removing software. -
Have 2 iPads and 2 i phones and a late 2006 Mac with 2G dual port processors and 3 G RAM, Need lion to utilize features of iCloud linking all devices. Currently running 10.6.8 with no problem. Is it likely that Lion will work for me or am I asking to much from the Mac?
Jimboastrojimbo wrote:
Is it likely that Lion will work for me or am I asking to much from the Mac?
As long as it meets the minimum technical specifications, although the more above the minimum you have, the better the experience. -
Most of the time when I turn on the Mac I get the white screen and question mark appear. Sometimes it will turn on and work. But often the coloured spinning wheel starts up especially after trying to use iphoto it freezes and a couple of times the screen has then gone black. The computer seemed pretty warm yesterday when this happened.
Any suggestions of what I should do to fix it would be greatly appreciated.
KateSee this on the question mark when you start up: http://support.apple.com/kb/TS1440?viewlocale=en_US
See this on the spinning beach ball: http://www.thexlab.com/faqs/sbbod.html
Maybe you are looking for
-
How to Uplaod Image in BW Server(Images to be used in creation of web appli
Hi All, How can we upload images in BW server, which can be used in creation of web application which will contain the links to the super user utilities such as super queries and help documentation. Regrads Premanshu
-
I would like to download songs and tv shows from USA but i cant as i am using Thai Itune. What should I do so that I can download the songs i like?
-
Apple AirPort WiFi connects to MacBook Air and iPad but not iPod Touch
I got my iPod touch 4th Gen in December for xmas and the wifi has worked perfectly fine until about a week ago only on my ipod, it just says 'Unable to Connect' everytime I try and connect but works on my MacBook Air and iPad It's a WPA2 Personal sec
-
Hi, I'm using a menu with CUT as Magic menu item and Type as PL-SQL and I havent code anything under PL-SQL code(Menu item is enabled,visible). However when I run this menu,This particular menu item remains disabled.I dont know y it is so.. Pls advic
-
I have a T420s with i7 and stock intel ssd and recently my startup time has been awful. Despite using various startup optimizers and playing with msconfig, I found the problem but don't know how to fix it. When there is a CD in the optical drive (any