Alternatives to waiting for Future.get()?

Question

Hi,
I'm new to the Java5 and 6 concurrency classes. But so far they seem very cool. My goal is to bulk load data concurrently by using multiple threads. A FixedThreadPool seems like it will do the job just fine.
My question has to do with retrieving the value from Future objects. It seems that since Future.get() blocks, it would kind of slow down my entire process if say for example, the first thread takes 10 minutes to complete (whereas all the other threads would take just a few seconds).
I was using Runnable instead of Callable, but the reason I changed to use Callable was so I could return a boolean if an error occurred during processing. Then I could abort the entire batch process if for example, the first 20 tasks were failing due to a server being down, etc.
Any ideas on how I could improve the design to accomplish my goal?
Here is a little proof-of-concept I put together for demonstrating:
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
public class CallableExample {
    public static void main(String[] args) throws Exception {
        CallableExample callableExample = new CallableExample();
        callableExample.runExample();
    private void runExample() throws Exception {
        System.out.println("Starting threads");
        ExecutorService executorService = Executors.newFixedThreadPool(2);
        List<Future<String>> futures = new ArrayList<Future<String>>();
        for (int i = 0; i < 2; i++) {
            final int finalI = i;
            Future<String> future = executorService.submit(new Callable<String>() {
                public String call() {
                    if (finalI == 0) {
                        try {
                            Thread.sleep(30000);
                            System.out.println("Running thread: " + finalI);
                        } catch (InterruptedException e) {                           
                    if (finalI == 1) {
                        try {
                            Thread.sleep(5000);
                            System.out.println("Running thread: " + finalI);
                        } catch (InterruptedException e) {                           
                    return Integer.toString(finalI);
            futures.add(future);
        executorService.shutdown();
        System.out.println("Threads started, main ended");
        for (Future<String> future : futures) {
            System.out.println("Future value: " + future.get());
}Here is the output I get:
Starting threads
Threads started, main ended
Running thread: 1
Running thread: 0
Future value: 0
Future value: 1Thanks!
Edited by: user13426024 on Dec 6, 2010 12:45 PM

Answer 1

user13426024 wrote:
My question has to do with retrieving the value from Future objects. It seems that since Future.get() blocks, it would kind of slow down my entire process if say for example, the first thread takes 10 minutes to complete (whereas all the other threads would take just a few seconds).
I think in reality this is a non-scenario: the Future.get() is called where the result is actually used, right? While a thread is blocked in the get(), the other threads in your thread pool will continue to proceed and any other thread requesting the result of those futures will not block in their get() calls.