Synchronizing multiple threads = sequential processing ???

Hi !
I am trying to solicit some comments and views.
I tested a program which updates an account with two amounts each of 1000, via two different threads.
I can see thread-1 kicks up , waits for some resource (e.g. from the db server ) while thread-2 kicks in and grab the cpu.
Without sychronisation, they overwrote each other so that at the end I have only one amount credited to the account (plus the original balance).
Now I simply add the directive "synchronized" and both amounts are added to the balance correctly !
However, I can also see (via much use of PrintWriter) that thread-1 completes the synchronised block of java-code first (along with a palpable delay) before thread-2 kicks in.
In the sychronized case, the cpu was idle while thread-1 waits for an I/O and idle again while thread-2 waits for an I/O.
Isn't this simple sequential processing ?
Is this an optimal model for concurrency ?
I do not see any significant advantages !

I know the book: Zukovski Java 2 from Sybex.
Two threads are fighting for single resource. In such case synchronization does not defeat the purpose of concurrent programming. It just introduces the order: not too much of a good thing. Sure, you are better off with only one thread in your account, but think that they can service several diffrent accounts at once not just one.

Similar Messages

How to handle multiple threads to read one line from a file each time?

Dear Sir or Madam,
I'm new to multiple threads java programming. What I want to do is as following:
1. I'm writing a program to read one line of text from a very large file, and then do some process on this one line of text, then read the next line of text from this file, then process on this line of text, ...... When reach the end of the file, close the file.
This is a single thread scenario.
2. To fullly untilized computer resource, I want to create multiple threads to process this large file.
One thing is very important is that each line of text in the large file has to be read sequentially. This means that the first line of text in the large file is read first, then the second line, then the third line ......
My question is that if I create multiple threads to process the same file, how can I make sure that different threads will read the line of text from the same file sequentially?
I don't have enough experience on java multiple threads programming, so it will be very appreaciated if you can help me on this as soon as you are available.
Thanks and regards,
Steven Wu

A better solutoin would be to have a single thread that reads each line an puts it into a queue, and then multiple threads reading from the queue to process these lines.
This will only be effective if the amount of processing to be done on each line is very large compared to how long it takes to read the line, or if you have more than one CPU on your machine.

How to protect the creation of a db across multiple threads/processes?

Given a multi-process, multi-threaded application, and one database file to be created, how can I guarantee that only one of the threads in one of the processes successfully creates the database, when ALL of the threads are going to either attempt to create it, or open it (if it already exists) upon startup?
My current logic for all threads is:
set open flags to DB_THREAD
start transaction
attempt to open the db
if ENOENT
abort transaction
change open flags to DB_CREATE | DB_EXCL | DB_THREAD
retry
else if EEXIST
abort transaction
change open flags to DB_THREAD
retry
else if !ok
# some other error
end
commit transaction
I'm testing on Linux right now, with plans to move to Windows, AIX, and Solaris. What I'm experiencing on Linux is several of the threads (out of 10 threads I'm testing) will succeed in creating the database. The others will receive either either succeed in opening the first time through, or will receive the EEXIST when they do the open w/ create flags - ultimately, they open the same created db (I'm presuming the last one that's created by one of the other threads). Effectively, the open with DB_CREATE | DB_EXCL is not ensuring that only one DB is created. I was under the impression that opening in a transaction would guarantee this, but it does not, or maybe I'm doing something incorrectly?
Should DB_CREATE | DB_EXCL and opening in a transaction guarantee that only one thread can create the database? Do I need to use another synchronization method?
Note: I am running off of a local disk, not over NFS or anything like that.
I tried taking out my transaction and using DB_AUTO_COMMIT instead, still no go - multiple threads still report they successfully created the DB. Using BDB 4.5.
Thanks,
Kevin Burge

Brian,
Thanks for the reply. I think I'm doing what you said, unless I'm misunderstanding. I do have all threads try to do the DB_CREATE | DB_EXCL. Are you saying I shouldn't use the DB_EXCL flag?
The problem I was seeing with 10 threads calling open w/ DB_CREATE | DB_EXCL on the same db:
* Between 1 and 9 threads would return success from db->open with the creation flags.... but the last one to create "wins".
* All the other threads would get EEXIST, as expected.
The threads that "lost", do get a successful return code from "open" with the create flags, but all data written to them is lost. They act normally except for the fact that the have a deleted file-handle that they are writing to. There's no indicator that records written to them are going into the void.
My test:
I had 10 threads each trying to create or open a recno db, then append 10 records, for a total of 100 records expected. Ultimately, I would end up with between 20 to 100 records in the db, depending on how many of the threads said they successfully created the db. So, if 5 threads said they created the db successfully, then 40 records would be missing, because 4 of those threads were writing to deleted file handles. If 2 threads said they created the db, then 10 records would be missing....If 8 threads, then 70 records missing, etc.
In other words, multiple threads creating the db appears to work correctly, because there are no errors. It was the missing records that caught my attention, and prompted my question on this forum.
For what it's worth, I've worked around the problem by opening a similarly named file via the open() system call, with O_CREAT|O_EXCL, which is guaranteed to be atomic. The first thread that can create this temp file is the only thread that can actually create the db - all others sleep upon open with ENOENT until it's created.
Thanks,
Kevin

Processing large file in multiple threads

I have a large plain text file (~1GB) that contains lots (~500,000) of chunks of text, separated by the record separator: "\n//\n". In my application, I have a class that reads the file and turns the records into objects, which is fairly time-consuming (the bottleneck is definitely in the processing, not the IO). The app is running on a muti-core machine, so I want to split the processing up over several threads.
My plan is to write a "Dispatcher" class, which reads the file from disk and maintains a queue of records, and a "Processor" class, which requests records from the Dispatcher object, turns the records into objects, and add()s them to an array. Is this the correct way to do it? Can anyone point me in the direction of a tutorial or example of this type of multi-threaded file processing?

public class TaskExecutionr {
    private static final int NTHREADS = 10; // Probably want to do System.getNumberOfProcessors+1 here
    private static final Executor exec
            = Executors.newFixedThreadPool(NTHREADS);
    private static Collection result = new ConcurrentLinkedQueue();
    public static void main(String[] args) throws IOException {
        FileReader reader =
        while (true) {
            final String text = reader.read(500,000); // read 500.000 lines to a string.
            Runnable task = new Runnable() { // create a new runnable that deals with the text
                public void run() {
                    handle(text);
            exec.execute(task); // Place the task in the Executor
    private static void handle(Socket connection) {
        // make your objects here
        result.add(<your new object>);
}http://www.javaconcurrencyinpractice.com/listings/TaskExecutionWebServer.java
Edited by: Fireblaze-II on Jun 19, 2008 5:48 AM

First JavaFX application with long sequential processes

Hi all,
I'm working on my first JavaFX application. I have UI elements (view) in place, I have a controller that works properly (with the exception I'll mention below), and I have an model that runs correctly (processes the data properly), but the model code was not originally written to run under a UI.
What I want to happen is the following. When the "Run" button is clicked, the input of the user is passed to the model via the controller and the data is processed. It's a do-once kind of thing where the input is transformed into the output and that's the end of it. The UI can be reset and new input provided, and the whole thing happens again with the click of the Run button. Here's the problem: The processing has several steps that need to happen sequentially, but some can take a long time. I want to provide feedback via a message (what's being done), and overall progress via a progress bar (both elements in the current UI). The problem is I'm still very confused about how to accomplish the user feedback.
The UI was built in SceneBuilder, and I've assigned the onMouseClicked event to call the runButtonClicked() method in the controller.
The runButtonClicked() method calls private methods that execute the processing steps (I'm going to do something more intelligent with the exceptions - please ignore that for the time being)...
    public void runButtonClicked() {
        runButton.setDisable(true);
        sdfFileBrowseButton.setDisable(true);
        outputFileBrowseButton.setDisable(true);
        try {
            prepareSDFFile();
            prepareModels();
            collectDescriptors();
            makeSamples();
            evaluateSamples();
            writeResultsToTextFile(results, outputFile);
        } catch (IOException exception) {
            System.out.println("Caught IOException: " + exception.getMessage());
        } catch (JAXBException exception) {
            System.out.println("Caught JAXBException: " + exception.getMessage());
Each of the calls in the try-catch are tasks that need to occur sequentially in the order shown. A few can take a long time (e.g. collectDescriptors(), evaluateSamples(), and writeResultsToTextFile(). The actual work isn't done in the controller. The controller creates instances of the classes that do the actual work (this seems to be a departure from most of the tutorials I've seen - it seems the model and the controller are mixed in the tutorials).
I want to alert the user about what step is executing (by way of a message in the UI) and what stage the whole process is in (by way of a progress bar). It would be nice if the progress bar could reflect the progress of each individual step (returning to 0 at the beginning of the next step), but having the bar just reflect the over all progress (e.g., how many of the six steps have been finished) would be sufficient.
I also want to be able to handle a Cancel button event and just stop the whole process and reset the application (losing whatever work has been done is OK).
So - I really can't tell what the best approach should be. None of the examples in several tutorials I've gone over have really touched on this type of sequential process. I'd appreciate suggestions.
Thanks,
Dave.

Suggested Approach
What you describe is an absolutely classic case for using the javafx.concurrent utilities (Task and Service).
Your application is a perfect fit for using these services, which will provide you:
1. a concurrent execution thread to run your log running processes
2. a threadsafe feedback mechanism for progress that you can bind to a ProgressBar for interactive progress feedback.
3. a threadsafe feedback mechanism for messages that you can bind to a Label for interactive status feedback.
4. sample code that demonstrates the ability to cancel and restart processes.
5, a success callback method setter that can be set to provide a result and take action on success.
6. a failure callback method setter that can be set to provide an exception and take action on failure.
Task and Service Samples
There are many examples of Task usage if you search for JavaFX Task.
There is also great documentation in the Task and Service javadoc.
Here is an example of using a JavaFX service. The example is a little old and uses doesn't use some new API features such as setOnSucceeded or setOnFailed, it also uses the now-deprecated Builder patterns, however, the bulk of it should still be relevant to what you want.
Here is another more complicated sample to Render 300 charts off screen and save them to files in JavaFX, though I doubt your task needs to be that complicated, and likely a single controlling service is all you need, so don't pay too much attention to the more complicated example unless you must have that kind of functionality.
Here is another simpler example for Re: Creating multiple parallel tasks by a single service, that actually demonstrates sequential and parallel demonstration of step tasks within a service.
The progress can be reset between steps, so that the message indicates the step being performed and the progress indicates the progress of the step rather than overall progress. You could expose separate mechanisms for feeding back overall progress in addition to step progress.
For further help
If you are still having issues, I'm sure somebody on the forum could easily mock up a sample application stub that demonstrates the techniques based on the info you provided in your question, just ask if you would like that...

Breaking a Program Execution into Multiple Threads

Hi,
We want to run a BAPI with Differenet parameters synchronoulsy in the same program as the BAPI is taking a very lon time for execution.
We are Planning to break up the execution and the call o the BAPI into multiple threads that can run synchronously. How can this concept implemented in SAP ABAP.
Thanks in advance.
Arunava

I am using parallel processing in my current client. Basis define a dialog processes group that we can use. This way our program will be limited to the number of processing give in the group. For example, our production system has 5 servers and each servers has 15 dialog processes. Basis reserve in the group 5 dialog processes for each group. So when we run our program, we are limited to 25 processes. We use function module SPT_INITIALIZE to find out the number of dialog processes and free processes for our group.
The logic of the program as follow:
Do loop
Call function module with starting new task and destination in group performing end of task subroutine
Enddo
Wait until all task completed (check help on wait-you need to keep track of the number of record completed)
In end of task subroutine, use command receive results from function to get the return parameters.
Hope this helps.
Cheers

Read ini file with multiple threads

I have a state machine architecture, but I have multiple threads. For instance, one is dealing with listening for mulitple tcp connections and storing their refnums to an array in a functional global. Another thread is accessing these refnums and collecting data it's receiving from each tcp connection. This data is queued up and then dequeued in a third thread which does some modification and sends it out via serial. My question is, when you have a situation like this and have to read an ini file, where should you read it? It seems like the most logical place would be outside your loops so you can get all the tcp and serial info (port, baud rate, etc) then wire it it to your create listener or initialize serial connection despite them being in different threads. But then again, normal state machine architecture would want an "initialize" case. If you did this though which loop would you put the init case in? And you would then have to worry about synchronizing loops becuase you wouldn't want one to try and create a listener while another thread was still reading ini data which would include the port to listen on. Maybe I'm overthinking this haha. Suggestions? Edit: one more question. Does it seem overkill that I have a tcp loop listening for data and queuing it up and a separate loop sending out the processed data via serial? Or should I just have one state that gets tcp data stores it in a shift register, then another state that sends it out via serial, and returns the state machine to the tcp read state?
Message Edited by for(imstuck) on 03-03-2010 01:13 PM
Message Edited by for(imstuck) on 03-03-2010 01:17 PM
CLA, LabVIEW Versions 2010-2013

Most of the applications I work on at the moment are used for testing barcode and label printers. The test applications I design are focused on testing the printer's firmware, not the hardware. Within our applications we have three primary objects (Unfortunately they are not native LabVIEW objects yet. They were developed before native LVOOP.) The primary objects we use in our applications are a log object, a connection object (communication interface to the printer) and a printer object. In any single instance of a test we only have a single printer, a single connection to the printer and one or more discrete logs. Each instance of these objects represent a single, real physical entity. A singleton object is a virtual representation of the physical world. Let's take the log object since that is the most simple of the objects described above. Naturally for a given log file you have the log file name and path. We also provide other attributes such as the maximum size of a single file (we allow log files to span multiple files), whether it is a comma delimited file or if it contains raw data, if timestamps should be included with a log entry and so forth. Most of these attributes are static for a log file with the exception of the name and such things as whether the logging is actually enabled or disabled. If we split a wire and had multiple instances of the log file (the way native LVOOP actually works) the attribute for whether logging is currently enabled or disabled will only pertain to the specific instance, or specific wire for the that object. Since this truly represents a single item, one log file, we need that attribute to be shared for all references to the instance of the log object. Since we allow this we can set an attribute on the log object in any task and it will be reflected in any other task that is using it. Think of the way a action engine or functional global works. However, in this case we provide discrete methods for the various actions.
I hope that made some sense. If not let me know since I just whipped up this response.
Mark Yedinak
"Does anyone know where the love of God goes when the waves turn the minutes to hours?"
Wreck of the Edmund Fitzgerald - Gordon Lightfoot

Synchronize work from multiple threads using wait() and notifyAll() help

Hello folks,
(Sorry for my bad english)
My current project handles multiple access requests to a shared collection from multiple threads originating from different classess and methods. Therefor I need to keep track of the order in which the Threads access that collection. I wrote a sort of Buffer class that has a static instance entry which initiate a new Instance of my Buffer class, attributes the instance a cue number and return the instance to the caller Thread.(Just like when you go to a supermarket and draw a number to wait to get served at the cheese counter).The callerThread then uses this instance to execute a method within the buffer class. Inside the buffer class method, I want to set a while loop on wait() just like this:
while(currentCue != myCueState)
wait();
when all other prior method calls within my Buffer class are done, the loop should wake up using a notifyAll() call and check the condition "currentCue != myCueState" agen to see if its turn has come.
I am new to the wait() and notifyAll() stuff and are therefor not sure what I am dooing wrong here: The only way this buffer class finishes all it's cues is when the caller Threads are beeing executed in the same order than they have checked in to the Buffer class. Otherwise I get some sort of dead-lock in the middle. Here is my code for the Buffer class:
public class Buffer{
    private static int currentCue = 0;
    private static int lastCued = 0;
    private int myCueState;
    private Buffer myInstance = null;
    synchronized void doTaskOne(){
        try{
            while(currentCue != myCueState)
                wait();
            //Do your task now
            System.out.println("doTaskOne got Executed: "+currentCue);
            currentCue++;
            notifyAll();
        catch(Exception a){}
    synchronized void doTaskTwo(){
         try{
            while(currentCue != myCueState)
                wait();
            //Do your task now
            System.out.println("doTaskTwo got Executed: "+currentCue);
            currentCue++;
            notifyAll();
        catch(Exception a){}
    synchronized void doTaskThree(){
        try{
            while(currentCue != myCueState)
                wait();
            //Do your task now
            System.out.println("doTaskThree got Executed: "+currentCue);
            currentCue++;
            notifyAll();
        catch(Exception a){}
    synchronized Object getSomething(){
        try{
            while(currentCue != myCueState)
                wait();
            //Do your task now
            System.out.println("getSomething got Executed");
            currentCue++;
            notifyAll();
        catch(Exception a){}
        return "something";
    //Access the buffer class through a single static synchronized instance and draw a turn number
    public synchronized Buffer instance(){
        myInstance = new Buffer();
        myInstance.setMyCueState();
        return myInstance;
    private void setMyCueState(){
         myCueState = lastCued;
         lastCued++;
}and here for the Test class I have coded to test this:
public class TestBuffer{
     private Buffer accessOne;
     private Buffer accessTwo;
     private Buffer accessThree;
     private Buffer accessFour;
     public TestBuffer(){
                //Instantiate different instances from Bufferclass and draw a number
          accessThree = new Buffer().instance();
          accessOne = new Buffer().instance();
          accessTwo = new Buffer().instance();
          accessFour = new Buffer().instance();
          Thread one = new Thread(){
               public void run(){
                    accessOne.doTaskOne();
          Thread two = new Thread(){
               public void run(){
                    accessTwo.doTaskTwo();
          Thread three = new Thread(){
               public void run(){
                    accessThree.doTaskThree();
          Thread four = new Thread(){
               public void run(){
                    accessFour.getSomething();
          try{
               one.start();
               two.start();
               three.start();
               four.start();
          catch(Exception f){}
     public static void main(String args[]){
          TestBuffer myTest = new TestBuffer();
}What am I doing wrong here??
Maby this is not how I should use the notifyAll() method, but how then?
Please give me a solution!
Thanks

Ok, so if I get you guys right, the following should do it:
public class Buffer{
    private static Object sharedLock = new Object();
    public void doTaskOne(){
          synchronized(sharedLock) {
            System.out.println("doTaskOne got Executed: ");
    public void doTaskTwo(){
         synchronized(sharedLock) {
            System.out.println("doTaskTwo got Executed: ");
    public void doTaskThree(){
         synchronized(sharedLock) {
              System.out.println("doTaskThree got Executed: ");
    public Object getSomething(){
        synchronized(sharedLock) {
            System.out.println("getSomething got Executed");
            return "something";
}Lets say that each method accesses the same ressources (in this case a table model) to retreave values, delete rows and set some existing values vith new values and all this 20-30 times a minute, all processing will stay synchronised and collision is not possible?
And lets say I would update the Table model directly from the buffer Class using MyTableModel.instance().setValueAt() or watever methods I implemented on my Table model, could I safely do that using "SwingUtilities.invokeLater();" from my BufferClass just like this:
public void doTaskThree(){
        synchronized(sharedLock) {
               Runnable runme = new Runnable(){
                     public void run(){
                            MyTableModel.instance().setValueAt("abc", 5,5);
               SwingUtilities.invokeLater(runme);
}Thanks in advance for your help guys!

Multiple Thread of a xml publisher report - messed up fonts in the PDF out

We are running 60000 invoices at a given time. In order to get the job done faster, the xml publisher report is set up to run mulitple threads. Each thread running unique set of invoices.
I am seeing this issue, randomly.
The pdf output of the invoice is having the fonts messed up. font 10 shows up as 14 and the invoice looks very bad.
When i rerun the same set of invoices it looks absolutely normal.
Also if I retrieve the xml output of the bad pdf and view it through the xmlpub desktop, it looks perfectly fine.
Two times we saw this issue
PRODUCTION
- when i run 8 threads, i saw this issue in one of the thread and affected one invoice. (Production- 1 concurrent processing node+ 5 OPP processes and 5 threads + MEMORY is J:oracle.apps.fnd.cp.gsf.GSMServiceController:-mx3072m )
TEST INSTANCE
- when i ran with 16 threads, one of the thread running xml pub report affected all the invoices (UAT environment-1 concurrent processing node+ 7 OPP processes and 1 threads + MEMORY is J:oracle.apps.fnd.cp.gsf.GSMServiceController:-mx3072m )
Looks like it is happening when running multiple threads.
The formatting command or information is getting lost somewhere and the report is all messed up.
Please advise.
Thanks
SJ

Really appreciate your response. Can you pls give little more details.
1] Are there any standard API's which I can use to submit XML publisher report from the JSP pages
2]Is there any sample code snippet for any of the options that I can refer to..or pls let me know the API's, I will check on them
Appreciate any help

Multiple threads access the same method.

Hello,
I have been trying for a long time to find out multiple threads access the shared data.
I have written a sample code, there I my intention is that method has to be accessed
onlny one thread at a time., mean one thread finished the job, then next thread can
access the shared source. But for this code I am not getting the desired out put what I want. But if I am using synchronized block I am getting the output. Please correct where I got mistake. Please see my code.
public class TestThread implements Runnable {
     Shared r;
     public TestThread() {
          r = new Shared();
     public static void main(String args[]) {
          Thread t1 = new Thread(new TestThread());
          Thread t2 = new Thread(new TestThread());
          t1.setName("A");
          t2.setName("B");
          t1.start();
          t2.start();
      * (non-Javadoc)
      * @see java.lang.Runnable#run()
     @Override
     public void run() {
          // TODO Auto-generated method stub
          r.count();
class Shared {
     public synchronized void count() {
          String name = Thread.currentThread().getName();
          System.out.println(name + ":accessed...");
          try {
               for (int i = 0; i < 5; i++) {
                    System.out.println(name + ": " + i);
          } catch (Exception e) {
               // TODO: handle exception
}Thanks
Bhanu lakshmi.

It depends on what you synchronize. Non-static methods synchronize on the object, so if you're using several objects, you'll be able to call each from their own thread.
Make your method synchronized or use only a single object and see the difference.

Can multiple threads write to the database?

I am a little confused from the statement in the documentation: "Berkeley DB Data Store does not support locking, and hence does not guarantee correct behavior if more than one thread of control is updating the database at a time."
1. Can multiple threads write to the "Simple Data Store"?
2. Considering the sample code below which writes to the DB using 5 threads - is there a possibility of data loss?
3. If the code will cause data loss, will adding DB_INIT_LOCK and/or DB_INIT_TXN in DBENV->open make any difference?
#include "stdafx.h"
#include <stdio.h>
#include <windows.h>
#include <db.h>
static DB *db = NULL;
static DB_ENV *dbEnv = NULL;
DWORD WINAPI th_write(LPVOID lpParam)
DBT key, data;
char key_buff[32], data_buff[32];
DWORD i;
printf("thread(%s) - start\n", lpParam);
for (i = 0; i < 200; ++i)
memset(&key, 0, sizeof(key));
memset(&data, 0, sizeof(data));
sprintf(key_buff, "K:%s", lpParam);
sprintf(data_buff, "D:%s:%8d", lpParam, i);
key.data = key_buff;
key.size = strlen(key_buff);
data.data = data_buff;
data.size = strlen(data_buff);
db->put(db, NULL, &key, &data, 0);
Sleep(5);
printf("thread(%s) - End\n", lpParam);
return 0;
int main()
db_env_create(&dbEnv, 0);
dbEnv->open(dbEnv, NULL, DB_CREATE | DB_INIT_MPOOL | DB_THREAD, 0);
db_create(&db, dbEnv, 0);
db->open(db, NULL, "test.db", NULL, DB_BTREE, DB_CREATE, 0);
CreateThread(NULL, 0, th_write, "A", 0, 0);
CreateThread(NULL, 0, th_write, "B", 0, 0);
CreateThread(NULL, 0, th_write, "B", 0, 0);
CreateThread(NULL, 0, th_write, "C", 0, 0);
th_write("C");
Sleep(2000);
}

Here some clarification about BDB Lock and Multi threads behavior
Question 1. Can multiple threads write to the "Simple Data Store"?
Answer 1.
Please Refer to http://docs.oracle.com/cd/E17076_02/html/programmer_reference/intro_products.html
A Data Store (DS) set up
(so not using an environment or using one, but without any of the DB_INIT_LOCK, DB_INIT_TXN, DB_INIT_LOG environment regions related flags specified
each corresponding to the appropriate subsystem, locking, transaction, logging)
will not guard against data corruption due to accessing the same database page and overwriting the same records, corrupting the internal structure of the database etc.
(note that in the case of the Btree, Hash and Recno access methods we lock at the database page level, only for the Queue access method we lock at record level)
So,
if You want to have multiple threads in the application writing concurrently or in parallel to the same database You need to use locking (and properly handle any potential deadlocks),
otherwise You risk corrupting the data itself or the database (its internal structure).
Of course , If You serialize at the application level the access to the database, so that no more one threads writes to the database at a time, there will be no need for locking.
But obviously this is likely not the behavior You want.
Hence, You need to use either a CDS (Concurrent Data Store) or TDS (Transactional Data Store) set up.
See the table comparing the various set ups, here: http://docs.oracle.com/cd/E17076_02/html/programmer_reference/intro_products.html
Berkeley DB Data Store
The Berkeley DB Data Store product is an embeddable, high-performance data store. This product supports multiple concurrent threads of control, including multiple processes and multiple threads of control within a process. However, Berkeley DB Data Store does not support locking, and hence does not guarantee correct behavior if more than one thread of control is updating the database at a time. The Berkeley DB Data Store is intended for use in read-only applications or applications which can guarantee no more than one thread of control updates the database at a time.
Berkeley DB Concurrent Data Store
The Berkeley DB Concurrent Data Store product adds multiple-reader, single writer capabilities to the Berkeley DB Data Store product. This product provides built-in concurrency and locking feature. Berkeley DB Concurrent Data Store is intended for applications that need support for concurrent updates to a database that is largely used for reading.
Berkeley DB Transactional Data Store
The Berkeley DB Transactional Data Store product adds support for transactions and database recovery. Berkeley DB Transactional Data Store is intended for applications that require industrial-strength database services, including excellent performance under high-concurrency workloads of read and write operations, the ability to commit or roll back multiple changes to the database at a single instant, and the guarantee that in the event of a catastrophic system or hardware failure, all committed database changes are preserved.
So, clearly DS is not a solution for this case, where multiple threads need to write simultaneously to the database.
CDS (Concurrent Data Store) provides locking features, but only for multiple-reader/single-writer scenarios. You use CDS when you specify the DB_INIT_CDB flag when opening the BDB environment: http://docs.oracle.com/cd/E17076_02/html/api_reference/C/envopen.html#envopen_DB_INIT_CDB
TDS (Transactional Data Store) provides locking features, adds complete ACID support for transactions and offers recoverability guarantees. You use TDS when you specify the DB_INIT_TXN and DB_INIT_LOG flags when opening the environment. To have locking support, you would need to also specify the DB_INIT_LOCK flag.
Now, since the requirement is to have multiple writers (multi-threaded writes to the database),
then TDS would be the way to go (CDS is useful only in single-writer scenarios, when there are no needs for recoverability).
To Summarize
The best way to have an understanding of what set up is needed, it is to answer the following questions:
- What is the data access scenario? Is it multiple writer threads? Will the writers access the database simultaneously?
- Are recoverability/data durability, atomicity of operations and data isolation important for the application? http://docs.oracle.com/cd/E17076_02/html/programmer_reference/transapp_why.html
If the answers are yes, then TDS should be used, and the environment should be opened like this:
dbEnv->open(dbEnv, ENV_HOME, DB_CREATE | DB_INIT_MPOOL | DB_INIT_LOCK | DB_INIT_TXN | DB_INIT_LOG | DB_RECOVER | DB_THREAD, 0);
(where ENV_HOME is the filesystem directory where the BDB environment will be created)
Question 2. Considering the sample code below which writes to the DB using 5 threads - is there a possibility of data loss?
Answer 2.
Definitely yes, You can see data loss and/or data corruption.
You can check the behavior of your testcase in the following way
1. Run your testcase
2.After the program exits
run db_verify to verify the database (db_verify -o test.db).
You will likely see db_verify complaining, unless the thread scheduler on Windows weirdly starts each thread one after the other,
IOW no two or ore threads write to the database at the same time -- kind of serializing the writes
Question 3. If the code will cause data loss, will adding DB_INIT_LOCK and/or DB_INIT_TXN in DBENV->open make any difference?
Answer 3.
In Your case the TDS should be used, and the environment should be opened like this:
dbEnv->open(dbEnv, ENV_HOME, DB_CREATE | DB_INIT_MPOOL | DB_INIT_LOCK | DB_INIT_TXN | DB_INIT_LOG | DB_RECOVER | DB_THREAD, 0);
(where ENV_HOME is the filesystem directory where the BDB environment will be created)
doing this You have proper deadlock handling in place and proper transaction usage
so
You are protected against potential data corruption/data loss.
see http://docs.oracle.com/cd/E17076_02/html/gsg_txn/C/BerkeleyDB-Core-C-Txn.pdf
Multi-threaded and Multi-process Applications
DB is designed to support multi-threaded and multi-process applications, but their usage
means you must pay careful attention to issues of concurrency. Transactions help your
application's concurrency by providing various levels of isolation for your threads of control. In
addition, DB provides mechanisms that allow you to detect and respond to deadlocks.
Isolation means that database modifications made by one transaction will not normally be
seen by readers from another transaction until the first commits its changes. Different threads
use different transaction handles, so this mechanism is normally used to provide isolation
between database operations performed by different threads.
Note that DB supports different isolation levels. For example, you can configure your
application to see uncommitted reads, which means that one transaction can see data that
has been modified but not yet committed by another transaction. Doing this might mean
your transaction reads data "dirtied" by another transaction, but which subsequently might
change before that other transaction commits its changes. On the other hand, lowering your
isolation requirements means that your application can experience improved throughput due
to reduced lock contention.
For more information on concurrency, on managing isolation levels, and on deadlock
detection, see Concurrency (page 32).

Berkeley DB Java Edition - multiple threads accessing database

Hi,
I would like to access (read & write) a Berkeley DB from multiple threads (all same process/jvm).
Should I use a single com.sleepycat.je.Database object for all threads, or should I create an instance for each?
Or is there any other way of doing it?
Thanks

The Database object may be used by multiple threads, so you can use a single instance or multiple ones, as best suits you. From http://docs.oracle.com/cd/E17277_02/html/java/com/sleepycat/je/Database.html,
Database handles are free-threaded and may be used concurrently by multiple threads. 'If you read through the javadoc, you'll find other notes on using classes from multiple threads.
Regards,
Linda

How to proces the record in Table with multiple threads using Pl/Sql & Java

I have a table containing millions of records in it; and numbers of records also keep on increasing because of a high speed process populating this table.
I want to process this table using multiple threads of java. But the condition is that each records should process only once by any of the thread. And after processing I need to delete that record from the table.
Here is what I am thinking. I will put the code to process the records in PL/SQL procedure and call it by multiple threads of Java to make the processing concurrent.
Java Thread.1 }
Java Thread.2 }
.....................} -------------> PL/SQL Procedure to process and delete Records ------> <<<Table >>>
Java Thread.n }
But the problem is how can I restrict a record not to pick by another thread while processing(So it should not processed multiple times) ?
I am very much familiar with PL/SQL code. Only issue I am facing is How to fetch/process/delete the record only once.
I can change the structure of table to add any new column if needed.
Thanks in advance.
Edited by: abhisheak123 on Aug 2, 2009 11:29 PM

Check if you can use the bucket logic in your PLSQL code..
By bucket I mean if you can make multiple buckets of your data to be processed so that each bucket contains the different rows and then call the PLSQL process in parallel.
Lets say there is a column create_date and processed_flag in your table.
Your PLSQL code should take 2 parameters start_date and end_date.
Now if you want to process data say between 01-Jan to 06-Jan, a wrapper program should first create 6 buckets each of one day and then call PLSQL proc in parallel for these 6 different buckets.
Regards
Arun

How to read from and write into the same file from multiple threads?

I need to read from and write into a same file multiple threads.
How can we do that without any data contamination.
Can u please provide coding for this type of task.
Thanks in advance.

Assuming you are using RandomAccessFile, you can use the locking functionality in the Java NIO library to lock sections of a file that you are reading/writing from each thread (or process).
If you can't use NIO, and all your threads are in the same application, you can create your own in-process locking mechanism that each thread uses prior to accessing the file. That would take some development, and the OS already has the capability, so using NIO is the best way to go if you can use JDK 1.4 or higher.
- K
I need to read from and write into a same file
multiple threads.
How can we do that without any data contamination.
Can u please provide coding for this type of task.
Thanks in advance.

Sequential Processing in XI

Hi Experts,
I have to execute a scenario which involves sequentially processing of files coming from legacy and sending data to R/3 via proxy.We have files coming in for different insp lots.Each insp lot has 1 or more results files.
My requirement is :
all the files pertaining to same lot must be sequentialy processed.If one of the file fails all other must be blocked.But Xi should not block files of other onsp lot.
Options that I have tried:
1) Setting EOIO in channel and giving queue names.But this works fine for a particular batch lot,but even blocks files of other batchlot.
2) I tried configuring rules on R/3 moni to break the Qos and make it Eo and let R/3 code handle the sequence.
3) Tried BPM with option of "Mulitple queues(Content specific)" and using corelation on Insp lot.Did all the settings in SWF_INB_CONF.But the I cannot see separate queues as per the correlation.
Can anyone help in solving this.Is this possible in XI?
Thanks in advance.
Pragati

Ur third option seems to be the most logical option. There could be some config problem in BPM. U will find multiple queues only if ur correlation instantiate multiple integration processes in parallel. Also make sure that u have only One correlation in ur scenario.
Regards,
Prateek

Synchronizing multiple threads = sequential processing ???

Similar Messages

Maybe you are looking for