Multi threading under single file channel
Hi,
We have File-Proxy scenario .
We are using fCC sender channel and using
recordset sequence as varaibale
recordsets per message was 250.
When large size file like 2GB was processed, PI is taking 45-50 min to process the file and from proxy end its taking 8 hrs to process this file.
we are using one channel and 250 recordsets per message by chunking file and process to target. We are following EOIO for sequence for these chunk files.
with this approach in PI target system proxy is taking huge time to process.
experts suggested using multi threading and multiple channels are the ideal solution for this issue.
am expecting with multiple channels and multi threading will cause issue for receiver proxy because of more objects and current approach is fine.
Please suggest any other alternatives we can acheive in PI with respect to this issue?
Note: I checked blog for file chunk option under Adv parameters and I cant use that here as FCC and Mapping is there .
Rgds,
aman
Hi Aman,
We had file to proxy interface and the XML payload of the proxy was around 400mb...to reduce the time taken in PI in mapping step we reduced the XML payload of the proxy by cutting the XML tagnames.
for example an xml field <firstname>Anand</firstname> was converted to <f>Anand</f>
with this we were able to achieve about 40-50% reduction in XML payload the file and a good improvement in the time taken by PI mapping to process the file.
Hope it helps,
Thanks,
Anand
Similar Messages
-
How should implement multi-thread in single-threaded Operating system using
How should implement multi-thread in single-threaded Operating system using java?
Java supports "Multi-threading".Is there is any way run the multiple threads (Implementing multi threading) using java in a Single-threaded Operating system (That is the operating system does not support for multi-threading).Previous questions from OP suggest they are using J2ME, so the question might be possible.
806437 wrote:
How should implement multi-thread in single-threaded Operating system using java?
What is the actual question/problem?
A java app doesn't do threads or not do threads. It uses classes. The VM does threads.
So if you have a platform that does not have threads and you want to support the thread class then the VM, not a java app, must provide some pseudo mechanism, such as green threads, to support that.
If your question is about java code and not the VM then you must build a task engine and insure that the tasks are of short enough duration that it is an effective use for your system. -
Hi,
I've got a tricky question: I have to read large files (over 150 MB) - I found here several topics about reading such a large files, but all examples were about single-threaded reading from file... Do you think, multi-threaded reading would be faster? I am in quite a dilemma: multiple threads in general means faster processing, but in this case, all threads are reading from the same source/file
(I use javax.mail.util.SharedFileInputStream for multi-threaded file reading - each stream in separated thread)
so what do you think, which is faster: single-thread or multiple-thread? and if so, how many threads would be fine - 2, 3 or 4?
thanks a lotkvasoo wrote:
Herko_ter_Horst wrote:
The I/O thread processing thread(s) approach will work when the processing takes more time than the I/O (i.e. when the medium can transfer information faster than a single thread can process it).it is very likely that processing will take more time then file reading - there will be 5-7 regex to match per line, sometimes object creation and filling object fith some data
so it seems to be just like I wrote some posts before: while one thread is reading data from file, another could process already read data - I am just considering malcolmmc's idea of blocking queue - one thread to perform I/O operations and one thread to process data - these threads would be connected through blocking queue....hmm that sounds good to me.. thanks malcolmmc
thank you guys for your help & suggestions and if you still have something to advise me, I will be thankfulSeparation of concerns, dude. Read the raw data in, in one thread, and split out the processing to multiple threads. I really advise you take the time to actually measure the performance of the multi-threaded processing vs the single-threaded one though. I mean, really do actually do that. Don't just assume - as so many people do - that throwing more threads at every problem will always speed it up, because it's so often not the case. -
DbEnv under mod_perl 2.0/Apache 2.2 worker MPM (multi-threaded)
I'd like to draw your attention to a thread I've opened on the mod_perl mailing list. I'm having difficulties figuring out how to initialize the Berkeley database environment on this particular platform. It may be that mod_perl is not the most suitable choice of platform for developing an application working on Berkeley DB, but still I'm curious to know whether or not this is at all possible, and if not, what are the reasons for this mismatch.
Note that I'm actually using the Perl interface to Berkeley DB XML instead of plain Berkeley DB. However, George Feinberg, the DB XML lead developer (I believe), pointed out that the issues I'm seeing pertain to the setup of the database environment, and that once this has been safely accomplished, the DB XML calls are safe, too. That's why he advised me to bring this issue to the attention of the Berkeley DB forum, which I'm now doing. Here we go:
Initializing Sleepycat::DbXml (Berkeley, Oracle) objects in startup.pl | ModPerl
http://www.gossamer-threads.com/lists/modperl/modperl/98849
It may be that a precise answer would require a lot of knowledge of the very particular mod_perl 2.0 multi-threaded platform and its intricacies, which I don't have myself and I know is not commonly found, as Perl has ceded a lot of ground to PHP and Java. But maybe there is someone here who can help.
The fundamental issue here is the setup of the environment. There has to be (1) a thread of control doing recovery on startup, and then (2) any number of threads of control which may or may not share environment handles. How could this be accomplished in this particular environment?
If you're interested in this issue, please follow the link to the mod_perl mailing list and see what's there. Note you can switch between flat and threaded view.
Thanks.
Michael LudwigIts been a long time since I messed with this, I ended up using Tomcat standalone, hopefully someone with more experience will reply to you after this post!
But as I remember, it was set up using a wildcard *.jsp to pass JSP files off to tomcat for processing. Apache cannot process jsp, which is why Tomcat is used as a plugin.
The page showing the examples is HTML, whereas the examples themselves are JSP.
Have you told Apache to pass all *.jsp's to Tomcat for PROCESSING.
You are not simply telling apache to open these files, rather Apache tells Tomcat to open and process the files.
Hope this makes sense and is of some use. -
Appending multiple xml files to single xml file using File channel?
Hi Experts,
I would like to get some pointers around xml files append.
My sender application will send around 1000 messages through SOAP per day. But the target application needs single xml file formed by consolidation of all these 1000 files.
I tried with txt files and the receiver file channel could append the txt files. However, since xml files allow only one root node, I am not able to achieve the requirement.
Is there any option we can do to get the result?
The target file location is UNIX folder. Do we need to create any shell script to run from the channel for appending the file?
Do we need to use any XSLT mapping to strip the xml declaration tags and append the files. Once the single file is ready, using another file channel with XSLT mapping, insert a dummy root node and xml declaration tags to build an xml file?
Any thoughts to get the result will be much appreciated.
Thanks
Ray..Hi,
You have two options
1)Using Receiver Sender File channel select mode append it does the job.
Option 2:
Using BPM we cann collect all messages per day and club them as a one message, in this case create Target data type root node occurance 1 to unbounded. so it cal hold multiples SOAP messages in only XML messages.
search in sdn for BPM collect pattrersn,its very simple .
Option 3:
Create scheell script in OS level and write a logic to append all files planced in FTP.
Regards,
Raj -
Writing more than 16 analog input channels into a single file
In DasyLAb 10.0, how can I WRITE more than 16 analog input channels (using WRITE DATA block) in one single file? I do not want to use 2 WRITE blocks, as I want all my data samples in one single file.
Another few questions:
In the merge block, what is the difference between Blockwise mode and Sample mode?
Also, now that I am using the Merge blocks (2 inputs to 1 output) to combine my analog inputs; how would I have to modify my WRITE block so that I can ultimately obtain both my inputs to be written in the same file in seperate columns. Could you please explain the settings I need to make on the WRITE block.
Thanks! -
Is rendering in Final Cut single thread or multi thread?
There seems to be this recurring message about how the new Macpros perform single-threaded tasks about the same as the previous gen of Macpros, however in multi-threaded tasks the new Pros shine.
How do I know if a task is single-threaded or multi-threaded? I do a lot of rendering video filters, generators, effects, etc. in Final Cut. Even something as (seemingly) simple as generating scrolling credits takes forever to render.
Is video rendering a multi-threaded task? Will I see time significant time savings in rendering with the new Pro machines?It's not right now but the new Mac Pros are faster in single-threaded modes anyway:
http://www.macrumors.com/2009/03/14/updated-mac-pro-benchmarks-and-video-of-inte rnals/
Turns out the initial benchmarks were incorrect. The integrated memory controller is a MASSIVE performance boost. But it is inevitable that Apple will eventually choose the multi-threaded route for Final Cut Studio anyway, even if they do not at the present. -
Multi-Instrument + Multi-Thread problem found
Right.....I have done a lot of research on why logic was not multi-threading correctly and have narrowed it down to one problem...using Kontakt as a multi instrument.
My Mac is a 2 x 2.26 GHz Quad-Core Intel Xeon. This means that it is capable of distributing work loads over the multiple cores of the CPU to prevent core audio overloads from happening.
If I run any plug ins in stereo or mono mode, the CPU acts as it should; I will load up a program, add processing and when the load is becoming a strain for the CPU, I use a bus send and work on that, then the workload is redistributed across more of the cores by logic, allowing me to continue working.
However, I need to use Kontakt as a multi instrument. I usually have 3 instances of Kontakt open and on each one of these, I usually have a minimum of 10 outputs being sent to the aux channels of logic. Therefore using Kontakt as a stereo instrument is not an option, as I would have to open around 30 instances!
Here is the problem; say for example, in Kontakt I send a kick drum out of Kontakt into AUX 3 + 4. The AUX channel in logic will being to show output. I can now EQ this single kick drum seperate from any other sound in logic.
If I began to add processor heavy plug ins to this AUX output, rather than share the workload across the cores as it should and does with stereo instruments, logic now only uses a single core and it will core audio overload very quickly.
As I said, this only happens when I set up Kontakt as a multi instrument. The main problem is when I begin to have many different outputs from my 3 instances of Kontakt, as by this stage, many different sounds will have processing. All 3 multi instruments and their processing are then controlled by the single processor and the other 7 are almost inactive! It therefore becomes impossible to use Kontakt and I have to switch to stereo instances. This happens for all multi-instruments not just Kontakt.
I have just purchased Kontakt 4, I am upgrading to OSX 10.6 (Snow Leopard) and am updating Logic to v9.1. (From v9.0) Will this help at all? Is there an answer to this problem or will I just have to open 30 stereo and mono instances of Kontakt to continue working!
I have included a link to a ZIP file with two photos of the problem. In the photo named MASSIVE. I have one instance of massive running as a stereo instrument. It is sent to a BUS where I applied the WAVES L316 Limiter. I did this multiple times and if you can see on the system performance (Top right) Logic is sharing the workload over the processors as it should.
In the second picture named KONTAKT MULTI - I have done the same thing, but I have opened Kontakt as a multi instrument, sent a kick drum to AUX 2 and then forwarded this to a send where I applied the same WAVES L316. You can see on the system performance meter (Top Right) that Logic now does not re-distribute the workload, it is using one core and anymore plug ins applied to the AUX or bus channels relating to Kontakt will now make it core audio over load, even though now I am using far less processing.
PHOTOS: http://www.sendspace.com/file/og8a94Ditto
Using Pro 8 Studio, I had this problem after installing Snow Leopard.
Invested £159 in Pro 9 update, same problem.
This has stopped most of my work since my external WK4 sounds are superior to the included software sounds.
As suggested, will now report since since it obviously seems to be a general problem. -
How to convert from Grayscale to single color channel
I hope that someone here might be able to help. I am writing an application which loads in color channels separately, to be merged by the application The images are multi-page TIFF files. The trouble that I am running into is that the application loads the images into grayscale, so when I merge the channels, they are merged as grayscale, not as the correct colors.
I use the following on each color channel to load them in:
public static BufferedImage[] loadTiff(File multiPageFile, int colour) {
BufferedImage[] img = null;
ArrayList<BufferedImage> tmp = new ArrayList<BufferedImage>();
try {
SeekableStream stream = new FileSeekableStream(multiPageFile);
String[] names = ImageCodec.getDecoderNames(stream);
ImageDecoder dec = ImageCodec.createImageDecoder(names[0], stream, null);
int numPages = dec.getNumPages();
for (int i=0; i<numPages; i++) {
try {
RenderedImage im = dec.decodeAsRenderedImage(i);
BufferedImage bi = PlanarImage.wrapRenderedImage(im).getAsBufferedImage();
//In case not all pages are valid images, we use ArrayList as it is dynamic
tmp.add(bi);
}catch (RuntimeException re) {
//Once the exception is caught, we can decode the layer here for whatever means.
}catch (FileNotFoundException fnfe) {
System.err.println("File not found.");
fnfe.printStackTrace();
}catch (Exception e) {
System.err.println("General Exception caught");
e.printStackTrace();
//Convert to an array to be returned.
img = new BufferedImage[tmp.size()];
for (int i=0; i<img.length; i++) {
img[i] = tmp.get(i);
return img;
}So a BufferedImage array is returned (one element in the array for each page in the multi-page file). Before I return the BufferedImage array, though, it would be good to convert it to the correct color. I know which channel is currently being loaded. Does anyone have any tips on how to shift from a grayscale intensity to a single color intensity? I have tried a number of things regarding byte shifting, but I am really none the wiser.
I have tried, for example, reading the int color value:
int color = bi.getRGB(w,h);The trouble is that this number is not in the region 0..255, so I don't know how I can use it. If I shift the number:
color = color>>16I get a number smaller than 255 (although it seems to be negative), but I don't know whether I can use this value for a single color channel, i.e. if the current color is red, saying:
Color redC = new Color(color, 0, 0);
bi.setRGB(w,h,redC); //Slow and inefficient. There must be a better way.When I try doing that, I end up with nothing drawn or visible.
Does anyone have any ideas? Thanks in advance.
Edited by: doughnuts64 on Jun 12, 2008 6:27 AMThanks for the response. It has been helpful in getting my problem sorted. I was not creating a new BufferedImage - I was just using the one that had been loaded in, and as such had not realised that the type was not TYPE_INT_RGB. As soon as I instantiated a new BufferedImage of the same dimensions and colour TYPE_INT_RGB as you suggested, it worked! So thanks for that.
Whilst that does work, it does slow the application incredibly as it has so much data to cycle through. Ideally, I would be able to load the image in directly as the colour channel in question, rather than loading in as grayscale. This way, the application would only require the amount of time needed to load in the images, not to process them on a pixel-by-pixel basis (typical images are made up of over 16,000,000 pixels for each colour channel, and we typically load two files in at once - both for separate colour channels, so over 32,000,000 pixels to process).
As I am loading in multi-page tiff files, I am having to use the JAI (Java Advanced Imaging) toolkit. I use the following code to load in my multi-page tiff into a BufferedImage array, where each element of the array represents one of the 'pages' in the multi-page tiff file:
SeekableStream stream = new FileSeekableStream(multiPageFile);
String[] names = ImageCodec.getDecoderNames(stream);
ImageDecoder dec = ImageCodec.createImageDecoder(names[0], stream, null);
int numPages = dec.getNumPages();
BufferedImage[] finalImage = new BufferedImage[numPages];
for (int i=0; i<numPages; i++) {
RenderedImage im = dec.decodeAsRenderedImage(i);
finalImage[i] = PlanarImage.wrapRenderedImage(im).getAsBufferedImage();
//...colour changing code currently here - loops through finalImage array, and on every pixel of every layer
...To be able to automatically convert the colour channel without having to loop on every pixel (and thus save an incredible amount of loading time) would be helpful.
Thanks. -
Sender File Channel Cluster Node issue in CC monitoring
Hi,
I have a issue in Sender File CC. The channel is active and polling every 1 min. The issue is in the CC monitoring it is just showing one single line i.e "Polling interval started.Length: 60.0 seconds" the other two lines Processing started and Processing Finished Successfully are not visible...
Please let me know the solution if you have faced a similar issue.
http://i50.tinypic.com/28sv4vr.jpg This is the refrence to the screen shot from CC monitoring...Deepak,
Please let me know the solution if you have faced a similar issue.
Delete the Enqueue-lock related to the stopped Communication Channel. Refer SAP Note 821267.
For the 7.1 versions, please note that the Lock Manager is under the /nwa/locks, or on the location "Availability and Performance Management -> Locks".
@ Ramg
If no files are placed in the source directory and the channel will keep try polling l!!! Place files in the source and check.
Even if there are no files in the source directory, the 3 steps of - polling interval started, processing started and processing finished - should take place by the file channel.
Regards,
Neetesh -
EDI X12 File Channel Configuration
Hello B2B Gurus,
I am trying to configure the following scenario on a single host to simulate trading partner communication using EDI X12 4010 850/810 over File Channels:
Inbound - 850 PO From RemoteTradingPartner to HostTradingPartner
I have configured a file delivery channel on the partner configuration screen under the "channels" tab. The channel is named: "RTP_File_Receive_Channel".
The channel has the following values set:
Polling Interval: 5
Folder Name: /home/oracle/b2b/remote/b2b_receive
Filename Format: %FROM_PARTY%_%DOCTYPE_NAME%_%DOCTYPE_REVISION%*.dat
Directory Name Format:
Minimum Age:
Archival Directory: /home/oracle/b2b/remote/archived_b2b_receive
I am dropping a file named RemoteTradingPartner_X12_4010_850.dat into the /home/oracle/b2b/remote/b2b_receive folder but for whatever reason, it is not picking it up and sending it to my SOA composite.
Should I be configuring a listening channel in the Administration tab instead of a delivery channel on the trading partner agreement? The naming convention and direction of channels is a topic that confuses me no matter how many times I go through this.Hello Johnny,
When Oracle B2B receives a transaction either from internal application (outbound) or from a trading partner (inbound), it first identifies the trading partner and then the transaction set. This combination is then used to identify the agreement.
What segment/element values within the .dat payload are introspected to lookup the appropriate trading partner agreement? If I specify those values in the filename format, does it override what is inside of the payload? In general, how do filename format, directory name format, and payload contents work together to resolve to the correct TP Agreement? What is the order of precedence when these values are specified in all three locations.The method for identifying the trading partner and transaction varies among transport, exchange and document protocols. If file name format is specified in the channel then Directory name format is ignored. If file name format/directory name format does not give info about transaction set (doc type and doc type revision) then payload is used to identify the document and it's version. It depends upon the type of document standard used that which selection criteria should be used. Generally for xml payloads x-path is used to identify a document and for non-xml payloads either start and end position are evaluated or the position of a particular element/string.
In short if you give all info for identifying an agreement (From tp, Doc type, doc type revision) in file name format/directory name format, then payload will not be checked for identifying any of these.
Regards,
Anuj -
Aggregate Storage And Multi-Threading/Multi-Core Systems
Please pardon if this question has been asked before, but the Forum search is not returning any relevant results.
We are in the process of purchasing hardware for an 11.1.2 Essbase environment. We are going 64-bit, on Windows 2008, with either 32 GB or 64 GB of system RAM. The debate we are having is the number of CPUs and cores per CPU. We have not built any ASO databases as of yet, but we plan to launch a major BSO to ASO conversion project once 11.1.2 is off the ground here.
Historically, with BSO, we did not see performance improvements significant enough to justify the cost of additional CPUs when we ran calcs on multi-CPU systems vs. single or dual CPU systems, even when the settings and design should have taken the most advantage of BSO's multi-threading capabilities. However, it would seem that ASO's design may be able to make better use of multi-core systems.
I know that there are a lot of factors behind any system's performance, but in general, is ASO in 11.1.2 written well enough to make it worthwhile to consider, say, a four CPU, total 16 core system vs. a 2 CPU, total four core system?Grand central dispatch - infancy, not really doing its job, and I don't think apps have to be specifically written for HT, but they do have to not do things that they use to - prevent threads from going to sleep! or be parked.
high usage is not necessarily high efficiency. often the opposite.
Windows 7 seems to be optimized for multi-core thanks to a lot of reworking. Intel wants and knows it isn't possible to hand code, that the hardware has to be smarter, too. But the OS has a job, and right now I don't think it does it properly. Or handle memory.
Gulftown's 12MB cache will help, and over all should be 20% more efficient doing its work.
With dual processors, and it doesn't look like there are two quick path bridges, data shuffling has led to memory thrashing. Use to be page thrashing with not enough memory. Then core thrashing but having the cores, but not integrated (2008 is often touted as being greatest design so far, but it was FOUR dual-cores, 2009 was the first with a processor that really was new design and (native) 4-core.
One core should be owned by the OS so it is always available for its own work and housekeeping.
The iTunes audio bug last year showed how damaging and not to implement code and how a thread could usurp processing and add a high cpu temperature while basically doing nothing, sort of a denial of service attack on the processor - those 80*C temps people had.
All those new technology features under development and not like OpenCL, GCD and even OpenGL are tested, mature but rather 1.0 foundation for the future. A year ahead of readiness. -
Multi-threaded performance server using 2 ports to two-way communication
Hi, I'm new in the forums so welcome everyone. I'm developing now an online computer game based on orginal Polish board-game, but it doesn't mean. Important is that I need to develope a high-performance multi-threaded server, which can serve... let's say few hundres of games, and a thousend of players simulateously. My server works on two ports/sockets.
First one is represented by "ServerSocketChannel clientConSsc" in my code. The main thread in my server class which invokes the method "run()" (which You can see below), registers clientConSsc with Selector and then repeatingly accepts incoming connections from clients on this socket. When connection(channel) is estabilished and clients wants to write some request to this channel, the main thread on server creates a new instance of class "RequestHandler" which extends Thread and handles this request (reads the request from channel, makes some changes on server, spreads some new messages to other clients in the same game and so on... everything on the same socket/port/channel). Then the channel/connection is closed by the server - I call such connections: "a short connections". In the same time the main thread continues the loop and is still able to accept connections from another players. That's the idea of multi-threaded server, right? :) And to this point, everything works fine, but... what if the server wants to trigger some new command, write new information to client?
Server could have tried to open a SocketChannel to client's IP address, but then the client programme would have to create and ServerSocketChannel object, bind it to some InetAddress and then... and then client becomes a server! - that breaks the idea of client-server cooperation and demands on players of my game to have routed some port on their machines. To resolve this problem I made some sort of "system" which, when some player/client is logging into my server, accepts second (this time constant, not "short") connection on the second port I mentoined on the beginning. This connection is held in variable "SocketChannel serverCon" of class "Player" - which reflects a player logged onto server. Server maintains every such connection till the client logs off. After the connection is accepted, the player whom connection it was is added to collection called "playersToRegisterToWrite". After the Selector.selectNow() is invoked (as You can see in the code below), each and every player's connection is registered in selector (with selection key - OP_WRITE and attachment pointing on owning player). In the same time client's code is trying to read some bytes from this connection and blocks on it until some bytes are written to this connection. When server wants to "tell" something to client, deliver some message/command, it creates a new instance of class which extends an abstract class called "ServerAction" (another new thread) and adds it to collection "actionsToDo". In ServerAction's method "run()" there are whole code to interact with client (e.g. send an update of players' list, an update of games' list, a chat message). Finnaly when Selector informs the server that some connection is writable (some client is waiting for commands), then server checks if there's any "actionToDo" involving this client/player. If there is, it unregisters the connection from Selector (so no other ServerAction (thread) can write on this connection) and starts the action. At the end of every "run()" method of subclass of ServerAction there's a code, which again adds the player to collection "playersToRegisterToWrite", so the player's connection can again be registered in Selector and the player can again receive informations from server - after it consumed THIS "ServerAction".
It looks to me like it should be working fine, but it's not. When I test my server and clients on the same machine (no ping/latency) everything seems to be working fine. But when it comes to play my game with somebody even on LAN or through the Internet there comes the problems. My first socket I describe (and first type of connections - short connections) works perfectly fine, so the requests triggered by client are delivered and handled properly. But the second socket is cirppling. For example: the server triggers a command refering to an update of clients logged to server' list. It is triggered by Selector so the client should be ready to read data from server, so the server sends data! And the client never receives it... Why?
It experimented a whole lot of time on SocketChannel's method "configureBlocking(boolean)", but it never helps. I think the client should always block it's listening thread on connection, contratry to server which is ment to be fast, so it should send the data through non-blocking channels. Am I right? Are there any rules refering blocking configuration of SocketChannels?
I will be grateful for every answer. To help You out I attach also drafts from run()'s methods of my classes.
Server's main method - main thread :
public void run() {
try {
selector = Selector.open();
clientConSsc.configureBlocking(false);
clientConSsc.register(selector , SelectionKey.OP_ACCEPT);
while(serverRunning) {
try {
selector.selectNow();
Iterator it;
synchronized(playersToRegisterToWrite) {
it = playersToRegisterToWrite.iterator();
while(it.hasNext()) {
Player player = (Player)it.next();
it.remove();
player.serverCon.configureBlocking(false);
player.serverCon.register(selector , SelectionKey.OP_WRITE , player);
Set keys = selector.selectedKeys() {
it = keys.iterator();
while(it.hasNext()) {
SelectionKey key = (SelectionKey)it.next();
if(key.isAcceptable()) {
it.remove();
clientCon = clientConSsc.accept();
clientCon.configureBlocking(false);
clientCon.register(selector , SelectionKey.OP_READ);
continue;
if(key.isReadable()) {
it.remove();
key.cancel();
new RequestHandler(this , (SocketChannel)key.channel()).start();
continue;
if(key.isWritable()) {
if(key.attachment() != null ) {
ServerAction action = null;
synchronized(actionsToDo) {
Iterator aIt = actionsToDo.iterator();
while(aIt.hasNext()) {
ServerAction temp = (ServerAction)aIt.next();
if(temp.getPlayer().equals((Player)key.attachment())) {
action = temp;
aIt.remove();
break;
if(action != null) {
key.channel().configureBlocking(false);
it.remove();
key.cancel();
action.start();
continue;
} catch(Exception e) {
statusArea.append(e.toString() + "\n");
e.printStackTrace();
continue;
} catch(ClosedChannelException e) {
statusArea.append(e.toString() + "\n");
} catch(IOException e) {
statusArea.append(e.toString() + "\n");
A part of server's RequestHandler's class' method run() - the method which handles requests triggered by client on first port:
public void run() {
boolean served = false;
try {
channel.configureBlocking(true);
retryCount++;
retry = false;
String command = ns.readText(channel).trim();
ns.statusArea.append("ktos przesyla komende: " + command + "\n");
if(command != null && !command.equals("ERROR")) {
String[] cmd = command.split("�");
ns.statusArea.append("komenda: " + cmd[0] + "\n");
if(cmd[0].equals("CHAT")) {
if(cmd.length != 5) {
retry();
} else {
if(cmd[2].equals("NOGAME")) { // jezeli nie ma okreslonej gry czyli rozsylamy do wszystkich
for(int i = 0 ; i < ns.loggedPlayersListItems.size() ; i++) {
Player current = (Player)ns.loggedPlayersListItems.get(i);
if(current.state == Player.CHOOSING && !current.equals(new Player(Integer.parseInt(cmd[1])))) // jezeli gracz jest w okienku wybierania gry to wysylamy mu broadcast
ns.actionsToDo.add(new SendMsg(ns , current , "CHAT�" + cmd[1] + "�" + cmd[3] + "�" + cmd[4]));
} else {
Game game = (Game)ns.gamesListItems.get(ns.gamesListItems.indexOf(new Game(Integer.parseInt(cmd[2]))));
for(int i = 0 ; i < game.players.size() ; i++) {
Player current = (Player)game.players.get(i);
if(!current.equals(new Player(Integer.parseInt(cmd[1]))))
ns.actionsToDo.add(new SendMsg(ns , current , "CHAT�" + cmd[1] + "�" + cmd[3] + "�" + cmd[4]));
served = true;
} else if(cmd[0].equals("GETGAMEINFO")) {
if(cmd.length != 3)
retry();
else {
int gameID = Integer.parseInt(cmd[2]);
ns.statusArea.append("poproszono o informacje o grze nr: " + gameID + "\n");
Game checkedGame = new Game(gameID);
if(ns.gamesListItems.contains(checkedGame)) {
Game game = (Game)ns.gamesListItems.get(ns.gamesListItems.indexOf(checkedGame));
channel.configureBlocking(true);
ObjectOutputStream oos = new ObjectOutputStream(channel.socket().getOutputStream());
oos.writeObject(game);
oos.writeObject(game.players);
oos.flush();
ns.statusArea.append("wyslano informacje o grze nr: " + gameID + "\n");
} else {
ns.statusArea.append("brak gry o nr: " + gameID + "\n");
served = true;
} catch(IOException e) {
e.printStackTrace();
if(!served) {
ns.findAndDisconnectPlayer(channel);
try {
channel.close();
} catch(IOException e) {
e.printStackTrace();
}An example of ServerAction's subclass - the class which triggers server commands to client on second port:
class UpdateLoggedPlayersList extends ServerAction {
public UpdateLoggedPlayersList(NeuroServer ns , Player player) {
super(ns , player);
public void run() {
try {
serverCon.configureBlocking(true);
ns.sendText("UPDATELOGGEDPLAYERSLIST" , serverCon);
if(ns.readText(serverCon).equals("OK")) {
ObjectOutputStream oos = new ObjectOutputStream(serverCon.socket().getOutputStream());
oos.writeObject(ns.loggedPlayersListItems);
oos.flush();
synchronized(ns.playersToRegisterToWrite) {
ns.playersToRegisterToWrite.add(player);
} catch(IOException e) {
e.printStackTrace();
}A part of client's CmdHandler's class' method run() - the method which handles commands triggered by server:
public void run() {
try {
while(works) {
String command = nc.readText(nc.serverCon);
System.out.println("Server przesyla komende: " + command + "\n");
String[] cmd = command.split("�");
if(cmd[0] != null && !cmd[0].equals("ERROR")) {
if(cmd[0].equals("CHAT")) {
if(nc.chooseGameDialog != null && nc.chooseGameDialog.isVisible()) { // jezeli na wybieraniu gry
newChatEntry(cmd[2] , null , cmd[3] , nc.chooseGameDialog.chatPane);
} else if(nc.readyDialog != null && nc.readyDialog.isVisible()) { // jesli na przygotowywaniu
Player sender = (Player)nc.actGame.players.get(nc.actGame.players.indexOf(new Player(Integer.parseInt(cmd[1]))));
newChatEntry(cmd[2] , sender.fraction , cmd[3] , nc.readyDialog.chatPane);
} else if(nc.ng != null) { // jesli w grze
Player sender = (Player)nc.actGame.players.get(nc.actGame.players.indexOf(new Player(Integer.parseInt(cmd[1]))));
newChatEntry(cmd[2] , sender.fraction , cmd[3] , nc.ng.inGameChatPane);
} else if(cmd[0].equals("UPDATELOGGEDPLAYERSLIST")) {
nc.sendText("OK" , nc.serverCon , false);
nc.serverCon.configureBlocking(true);
ObjectInputStream ois = new ObjectInputStream(nc.serverCon.socket().getInputStream());
DefaultListModel players = (DefaultListModel)ois.readObject();
nc.chooseGameDialog.updateLoggedPlayersList(players);
} catch(IndexOutOfBoundsException e) {
System.out.println(e);
} catch(IOException e) {
System.out.println(e);
} catch(ClassNotFoundException e) {
System.out.println(e);
}And two methods I used in codes above: sendText(String text , SocketChannel sc) and readText(SocketChannel sc) - they are my "utility" methods, which I use to send and receive Strings through specified SocketChannels.
boolean sendText(String text , SocketChannel sc) {
ByteBuffer bbuf = ByteBuffer.allocate(BSIZE);
boolean sendRetry;
boolean sent = false;
do {
sendRetry = false;
try {
StringBuffer cmd = new StringBuffer();
cmd.setLength(0);
if(text.length()+4 < 100)
cmd.append("0");
if(text.length()+4 < 10)
cmd.append("0");
cmd.append(Integer.toString(text.length()+4) + "�");
cmd.append(text);
cmd.append("\n");
bbuf = charset.encode(CharBuffer.wrap(cmd));
sc.write(bbuf);
bbuf.clear();
int n = sc.read(bbuf);
if(n == 1) {
bbuf.flip();
Byte re = bbuf.get();
if(re == 1) {
sendRetry = true;
} else {
sent = true;
} catch(Exception e) {
findAndDisconnectPlayer(sc);
try {
sc.close();
} catch(IOException f) {
f.printStackTrace();
return false;
} while(!sent && sendRetry);
return true;
String readText(SocketChannel sc) {
ByteBuffer bbuf = ByteBuffer.allocate(BSIZE);
int readRetryCount = -1;
boolean readRetry;
do {
readRetry = false;
readRetryCount++;
StringBuffer cmd = new StringBuffer();
cmd.setLength(0);
bbuf.clear();
try {
readLoop:
while(true) {
int n = sc.read(bbuf);
if(n > 0) {
bbuf.flip();
CharBuffer cbuf = charset.decode(bbuf);
while(cbuf.hasRemaining()) {
char c = cbuf.get();
if(c == '\r' || c == '\n') break readLoop;
cmd.append(c);
} else break;
statusArea.append(Thread.currentThread().getId() + " readText() odczytuje: " + cmd.toString() + "\n");
if(cmd.length() < 3 || Integer.parseInt(cmd.substring(0 , 3)) != cmd.length()) {
sc.write(ByteBuffer.wrap(new byte[] {1}));
readRetry = true;
} else {
sc.write(ByteBuffer.wrap(new byte[] {0})); // length OK
return cmd.toString().substring(4 , cmd.toString().length());
} catch(Exception e) {
findAndDisconnectPlayer(sc);
try {
sc.close();
} catch(IOException f) {
f.printStackTrace();
return "ERROR";
} while(readRetry && readRetryCount < 3);
findAndDisconnectPlayer(sc);
try {
sc.close();
} catch(IOException e) {
e.printStackTrace();
return "ERROR";
}Edited by: Kakalec on Jul 23, 2008 11:04 AMYou seem to be using a horrendous mixture of PrintWriters, BufferedReaders, ObjectOutputStreams, and no doubt ObjectInputStreams. You can't do that.
Some rules about this:
(a) Whenever you use a stream or reader or writer on a socket, you must allocate it once for the life of the socket, not every time you want to do an I/O.
There are many reasons for this including losing buffered data in the old stream and auto-closure of the socket on garbage-collection of the old stream.
(b) You can't mix stream types on the same socket. (i) If you're writing objects, use an ObjectOutputStream, you must read with an ObjectInputStream, and you can't read or write anything that isn't supported by the API of ObjectOutputStream. (ii) If you're writing lines, use a BufferedWriter and a BufferedReader at the other end. If you're writing characters, use a Writer and a Reader. (iii) If you're writing primitive data or UTF strings, use a DataOutputStream and a DataInputStream. (iv) If you're writing bytes, use DataOutputStream and a DataInputStream or an OutputStream and an InputStream. You can't mix (i), (ii), (iii), and (iv).
(c) Don't use PrintWriter or PrintStream over the network. They swallow exceptions that you need to know about. They are for writing to the console, or log files, where you don't really care about exceptions.
(d) Call flush() before reading.
(e) Always close the 'outermost' output stream or writer of a socket, never the input stream or the socket itself. Closing any of these closes the socket and therefore makes the other stream unusable, but only closing the output stream/writer will flush it. -
Trouble rationalizing use of multi-threading in run of the mill servlets
Hey everybody,
While spending time writing an internal wiki article on servlets for work, I asked myself a very basic question: What does multi-threading buy average servlets where the business logic requires procedural handling of the request?
Don't get me wrong: I appreciate the fact that servlet containers spawning a new thread being less expensive than spawning an entirely new process is helpful and efficient. Coming from a background in PHP, it is great how servlets maintain persistence. However, as more of my coworkers are required to gain proficiency in Java and designing servlets, it is a question that many will ask and aside from having real-time processing of data files and other arduous tasks, I cannot think of any instances of where multi-threading benefits the application.
What are some of the ways that you are using multi-threading with web applications?
How would you explain why and where you would want to use multi-threading to someone?
Thank you in advance for your insight,
Andyhow can we pass arguments to the run ()method?Create classes which implement Runnable that take your runtime parameters as constructor arguments and store them.
eg: if your single thread method is static void foo (int quantity, String name) {
for (int i=0; i<quantity; i++) {
System.out.println(name);
// caller code
foo(7, "wombats");Then you can make a runnable implementation thus:public class Foo implements Runnable {
final int quantity_;
final String name_;
public Foo (int quantity, String name) {
quantity_ = quantity;
name_ = name;
public void run () {
for (int i=0; i<quantity_; i++) {
System.out.println(name_);
// caller code
new Thread(new Foo(7, "wombats")).start();
You could overload this method to take parameters in
your class that implements the Runnable interface,
and then call the base run() method.I don't get what you mean by this; Runnable is an interface so there is no base class run() method, and a run() overloaded with extra parameters method wouldn't get called by the thread.
Pete -
CS6 After Effects PNG export very slow - Not Multi-Threaded?
I am rendering an after effects project consisting of over 320,000 source frames of 4000x4000 resolution, outputting over 250,000 frames as PNG files.
Rendering is instant on our machines, all time is spent "compressing and writing" frames. It is taking forever.
We have several machines that are all windows 7, with 16 physical cores, 64GB ram, connected to a 4Gbit SAN.
I did some experimenting, tried using all 16 cores, or 8 cores, or disabled multi-frame rendering even. When this project is rendering my cpu usage is almost non existent,. maybe 1%-5% at times.
with 16 frames at a time being rendered it is averaging between 1 and 7 seconds per frame depending on the content of the frame.
Does Adobe have any plans to multi-thread the PNG exporter? or is there a faster 3rd party plugin out there i could use? It is very slow and doesn't tax the CPU's at all. Even on a single core it doesnt even use 100% cpu while its saving.
I even tried loading one of the frames into photoshop and simpy hit save as a PNG and it took almost 15 seconds to compress and save. I did it again without compression and it saved the 40MB raw file instantly.
That is terrible performance for machines like mine that are pretty much the fastest things out there.
Thanks!I appreciate your attempt for a workaround solution, however as you well know you can't render to a video file from multiple machines. we must output directly to individual frames, I setup 8 machines to work on this project, if we did not use compression then we would be writing over 300MB/sec of data which would not be good for our fibre network which unfortunatly is only 4Gbit. And it would be about 10TB of data total.
As for your workaround Im not sure we could gain speed with your method either way. spending the extra time to encode to a lossless format and then process again in AME sounds like it would take even more time, unless AME could create PNG files at some sort of ridiculous speed maxxing out every core at 100% or something to compress the PNG's. Out of curiosity I will throw some frames at AME and see how long it takes to compress and how much cpu it will use.
thanks
Maybe you are looking for
-
Hi I am planning to install window 7 on my imac ( version - ox mountaion loin - 10.8.2, purchase this year in may 2012) but before installing the same pls let me know what i would required means - is there required any special software or hardware
-
I have a message on my iphone that I would like to print, how can I do that? Or, how can I send it to my gmail address in order to print it?
-
LaserJet Pro 200 color MFP M276nw cant install
while it gets ready for installation setup to appear, dont send or send error will appear but installation setup loading doesnt stop, after i pass the first page of the setup (accepting T&C) it says fatal error occrured preventing product use. :/
-
BADI to change payment terms in MIRO
Hi, Can anyone suggest a suitable BADI to change the payment terms at header level in MIRO ? BADI MRM_PAYMENT_TERMS is supposed to do so as per its documentation but it doesnt since the payment term is not an exporting parameter. Please suggest.
-
Multi-value parameters and strings with leading zeros
I have invoice number as a multi-value parameter. Invoice is a string,10 (VBRP.VBELN) . Invoice number is my group. If I enter invoices 100 and 200 as parms, then I only get data for invoice #100 (the lowest value entered). But if I enter 100, 000000