Chapter 76 of 77

Executor Framework & Virtual Threads in Java

So far, we have created threads by hand with new Thread(...) and start(). That's fine for two or three threads. But imagine a web server that handles 5,000 requests per second. Creating a brand-new thread for every request, and throwing it away afterwards, has real problems:

  • Creating a normal thread is expensive. Each one gets its own memory for its stack and is managed by the operating system.

  • There is no limit. Too many threads can slow the machine down or crash it with an OutOfMemoryError.

  • A Runnable cannot return a result or throw a checked exception.

Java solves this with the Executor framework in the java.util.concurrent package, and, since Java 21, with lightweight virtual threads.

The Executor framework separates what to run (a task) from how it runs (which thread, when, and how many threads). You submit tasks, and an executor runs them for you, usually on a reusable pool of threads.

The Idea of a Thread Pool

Think of a restaurant kitchen. The restaurant doesn't hire a new cook for every order and fire them after the dish is served. It has a fixed team of cooks, and new orders wait in a queue until a cook is free.

   submit tasks            task queue             worker threads
  ─────────────▶   [T6] [T5] [T4]   ─────▶   pool-1-thread-1 → T1
                                             pool-1-thread-2 → T2
                                             pool-1-thread-3 → T3

When a worker finishes its task, it picks the next one from the queue. The threads are created once and reused.

Your First ExecutorService

ExecutorService is the main interface, and the Executors class has factory methods to create one:

import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;

class Main {
    public static void main(String[] args) {

        ExecutorService executor = Executors.newFixedThreadPool(3);

        for (int i = 1; i <= 6; i++) {
            int taskId = i;
            executor.submit(() -> {
                String worker = Thread.currentThread().getName();
                System.out.println("Task " + taskId + " run by " + worker);
            });
        }

        executor.shutdown();   // no new tasks; finish the submitted ones
    }
}

One possible output:

Task 2 run by pool-1-thread-2
Task 3 run by pool-1-thread-3
Task 1 run by pool-1-thread-1
Task 4 run by pool-1-thread-3
Task 5 run by pool-1-thread-2
Task 6 run by pool-1-thread-1

The order changes from run to run, but notice that six tasks were handled by only three threads, which were reused. (We copy i into taskId because a lambda can only use local variables that are effectively final.)

Always call shutdown() when you are done. The pool's threads are not daemon threads, so if you forget, your program may keep running and never exit.

Getting Results with Callable and Future

A Runnable returns nothing. When a task should produce a value, use Callable<V>. Its single method, call(), returns a value and is allowed to throw checked exceptions.

When you submit a Callable, you immediately get back a Future<V>: a "receipt" for a result that will be ready later.

import java.util.concurrent.Callable;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;

class Main {
    public static void main(String[] args) throws InterruptedException, ExecutionException {

        ExecutorService executor = Executors.newFixedThreadPool(2);

        Callable<Long> sumTask = () -> {
            long sum = 0;
            for (int i = 1; i <= 1_000_000; i++) {
                sum += i;
            }
            return sum;
        };

        Future<Long> future = executor.submit(sumTask);

        System.out.println("Task submitted, main thread is free to do other work...");

        Long result = future.get();   // waits until the result is ready
        System.out.println("Sum = " + result);
        System.out.println("Done? " + future.isDone());

        executor.shutdown();
    }
}

Output:

Task submitted, main thread is free to do other work...
Sum = 500000500000
Done? true

Useful Future methods:

Method

What it does

get()

Waits (blocks) until the result is ready, then returns it

get(timeout, unit)

Waits at most the given time, otherwise throws TimeoutException

isDone()

Returns true if the task has finished

cancel(true)

Tries to cancel the task (interrupting it if it's running)

Running Many Tasks with invokeAll()

invokeAll() submits a whole list of tasks and waits until all of them are complete. Here we "fetch" three prices from slow services in parallel:

import java.util.List;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;

class Main {

    static Callable<String> fetchPrice(String item, int price, long delayMs) {
        return () -> {
            Thread.sleep(delayMs);        // pretend to call a slow service
            return item + " = Rs. " + price;
        };
    }

    public static void main(String[] args) throws Exception {

        ExecutorService executor = Executors.newFixedThreadPool(3);

        List<Callable<String>> tasks = List.of(
                fetchPrice("Laptop", 55000, 300),
                fetchPrice("Phone", 20000, 100),
                fetchPrice("Headphones", 2500, 200)
        );

        List<Future<String>> futures = executor.invokeAll(tasks);

        for (Future<String> f : futures) {
            System.out.println(f.get());
        }

        executor.shutdown();
    }
}

Output:

Laptop = Rs. 55000
Phone = Rs. 20000
Headphones = Rs. 2500

Two things to notice. First, the futures come back in the same order as the tasks, even though "Phone" actually finished first. Second, because the three tasks ran at the same time, the whole thing took about 300 ms (the slowest task), not 600 ms (the sum).

There is also invokeAny(), which returns the result of whichever task finishes successfully first and cancels the rest.

Shutting Down Properly

Method

What it does

shutdown()

Stops accepting new tasks; already submitted tasks still run

shutdownNow()

Tries to stop running tasks (by interrupting them) and returns the tasks that never started

awaitTermination(time, unit)

Waits until all tasks finish after shutdown, or the timeout passes

close() (Java 19+)

Calls shutdown() and waits for all tasks to finish

Since Java 19, ExecutorService implements AutoCloseable, so on Java 19 or newer you can use try-with-resources and never forget to shut down:

try (ExecutorService executor = Executors.newFixedThreadPool(2)) {
    executor.submit(() -> System.out.println("Task A"));
    executor.submit(() -> System.out.println("Task B"));
}   // close() waits for both tasks, then shuts the pool down

System.out.println("All tasks finished");

Output (Task A and Task B may swap places):

Task A
Task B
All tasks finished

Types of Executors

Factory method

Behaviour

Good for

newFixedThreadPool(n)

Exactly n threads; extra tasks wait in a queue

CPU-heavy work (n ≈ number of CPU cores)

newCachedThreadPool()

Creates threads as needed and reuses idle ones

Many short-lived tasks

newSingleThreadExecutor()

One thread; tasks run one after another, in order

Tasks that must not overlap

newScheduledThreadPool(n)

Runs tasks after a delay or repeatedly

Timers, periodic jobs

newVirtualThreadPerTaskExecutor()

A new virtual thread for every task (Java 21+)

Large numbers of waiting (I/O) tasks

A useful tip: Runtime.getRuntime().availableProcessors() tells you how many CPU cores are available, which is a good starting size for a pool doing pure computation.

For production servers, be aware that newFixedThreadPool() uses an unbounded queue and newCachedThreadPool() can create an unbounded number of threads. Under heavy load either can exhaust memory, so large systems often create a ThreadPoolExecutor directly with explicit limits.

A Glimpse of CompletableFuture

future.get() blocks the calling thread while it waits. CompletableFuture lets you describe what should happen when a result arrives, as a chain of steps:

import java.util.concurrent.CompletableFuture;

class Main {
    public static void main(String[] args) {

        CompletableFuture<Void> pipeline = CompletableFuture
                .supplyAsync(() -> "john")              // runs on a background thread
                .thenApply(name -> name.toUpperCase())  // transform the result
                .thenApply(name -> "Hello, " + name)
                .thenAccept(msg -> System.out.println(msg));

        pipeline.join();   // wait for the whole chain to finish
    }
}

Output:

Hello, JOHN

Notice the functional interfaces at work: supplyAsync() takes a Supplier, thenApply() takes a Function, and thenAccept() takes a Consumer. CompletableFuture is a big topic of its own; for now, just recognise it when you see it.

Virtual Threads (Java 21+)

The threads we have used so far are platform threads. Each one is a thin wrapper around an operating system thread. They are heavy (each typically reserves around 1 MB for its stack), so a machine can only run a few thousand of them comfortably.

That's a problem for servers, because most of a request's time is spent waiting: for the database, for another web service, for a file. While a platform thread waits, it sits idle but still occupies an OS thread.

Virtual threads, a standard feature since Java 21 (and available in the current LTS release, Java 25), fix this. They are managed by the JVM, not the OS, and are very cheap: you can create millions of them. When a virtual thread blocks on I/O or sleep(), the JVM simply parks it and uses the underlying OS thread (called a carrier thread) to run another virtual thread.

Platform threads:  1 Java thread  = 1 OS thread   (heavy, thousands)
Virtual threads:   many virtual threads share a few carrier OS threads (light, millions)

  virtual threads:  V1  V2  V3  V4  V5  V6 ... V10000
                      \   |   /      \   |   /
  carrier threads:     Carrier-1     Carrier-2      (about one per CPU core)

Creating virtual threads directly:

class Main {
    public static void main(String[] args) throws InterruptedException {

        Thread vt = Thread.ofVirtual().name("my-virtual").start(() -> {
            Thread t = Thread.currentThread();
            System.out.println(t.getName() + " virtual? " + t.isVirtual());
        });
        vt.join();

        Thread vt2 = Thread.startVirtualThread(() ->
                System.out.println("Daemon? " + Thread.currentThread().isDaemon()));
        vt2.join();

        System.out.println("main virtual? " + Thread.currentThread().isVirtual());
    }
}

Output:

my-virtual virtual? true
Daemon? true
main virtual? false

Virtual threads are always daemon threads, so the JVM won't wait for them when main() ends. That's why we call join().

10,000 Tasks: Virtual vs Platform Threads

The easiest way to use virtual threads is through the executor you already know. Let's run 10,000 tasks that each "wait for the network" for 1 second:

import java.time.Duration;
import java.time.Instant;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.atomic.AtomicInteger;

class Main {
    public static void main(String[] args) {

        AtomicInteger completed = new AtomicInteger();
        Instant start = Instant.now();

        try (ExecutorService executor = Executors.newVirtualThreadPerTaskExecutor()) {
            for (int i = 0; i < 10_000; i++) {
                executor.submit(() -> {
                    Thread.sleep(1000);          // simulate a 1-second network call
                    completed.incrementAndGet();
                    return null;
                });
            }
        }   // waits for all 10,000 tasks

        long seconds = Duration.between(start, Instant.now()).toSeconds();
        System.out.println("Completed: " + completed.get());
        System.out.println("Time: about " + seconds + " second(s)");
    }
}

Output on our machine:

Completed: 10000
Time: about 1 second(s)

All 10,000 tasks waited at the same time. When we replaced the executor with Executors.newFixedThreadPool(100), the same program printed Time: about 100 second(s), because only 100 tasks could wait at once (10,000 ÷ 100 × 1 second). (The lambda ends with return null; so that it becomes a Callable, which is allowed to throw the checked InterruptedException from sleep().)

When to Use Virtual Threads

Platform thread pool

Virtual threads

Cost per thread

High (OS thread, large stack)

Very low (managed by the JVM)

How many

Thousands at most

Millions

Best for

CPU-heavy work (calculations, image processing)

I/O-heavy work (web requests, database calls, file and network access)

Pooling

Yes, reuse threads

No, create one per task

Virtual threads do not make code run faster. A CPU-heavy calculation takes just as long. They help when you have many tasks that spend most of their time waiting, by letting far more of them wait at the same time.

Things to keep in mind:

  • Don't pool virtual threads. They are cheap, so create a new one for each task. To limit access to a scarce resource (like 10 database connections), use a Semaphore instead of a small pool.

  • Everything from the synchronization chapter still applies. Shared data still needs locks or atomic variables.

  • In Java 21 to 23, a virtual thread that blocked inside a synchronized block was "pinned" to its carrier thread, which could hurt scalability, and ReentrantLock was recommended instead. Since Java 24 (JEP 491), synchronized no longer pins virtual threads in this way.

  • Be careful with ThreadLocal caches. With millions of threads, a per-thread copy of a large object can use a lot of memory.

Java is also developing structured concurrency, an API for treating a group of related tasks as one unit. It is still a preview feature (not yet final) as of Java 27, so you can skip it for now.

Common Mistakes

1. Forgetting shutdown()
   → the program never exits (use try-with-resources on Java 19+)

2. Calling future.get() right after each submit()
   → each task waits for the previous one, so nothing runs in parallel

3. Ignoring exceptions from submit()
   → the exception is stored inside the Future and is silently lost
     unless you call get()

4. Using a huge fixed pool for I/O, or a tiny one
   → use virtual threads for I/O-heavy work instead

5. Using virtual threads for CPU-heavy work and expecting a speed-up
   → the number of CPU cores is still the limit

Mistake 3 surprises many developers. If a task submitted with submit() throws an exception, nothing is printed. The exception only appears when you call get(), wrapped in an ExecutionException:

Future<Integer> f = executor.submit(() -> 10 / 0);
try {
    f.get();
} catch (ExecutionException e) {
    System.out.println("Task failed: " + e.getCause());
}

Output:

Task failed: java.lang.ArithmeticException: / by zero

The Main Idea

Instead of creating threads by hand, submit tasks to an ExecutorService. Use a fixed pool of platform threads for CPU-heavy work, and virtual threads (Java 21+) for large numbers of tasks that spend most of their time waiting.

Key points to remember:

  • Runnable returns nothing; Callable returns a value and can throw checked exceptions.

  • submit() returns a Future; get() waits for the result.

  • Always shut down an executor, or use try-with-resources (Java 19+).

  • Virtual threads are cheap, JVM-managed threads. Create one per task and never pool them.

Common interview questions: What is a thread pool and why use one? What is the difference between Runnable and Callable? What does Future.get() do? What is the difference between shutdown() and shutdownNow()? How are virtual threads different from platform threads, and when should you use them?