Why three threads? (in progress)

An unfinished rabbit hole. This is the plan and the open question, written down before I have answers, for a Rabbit Holes 2026 entry (Sapling track). I'll update it as results come in. If you already know the answer, I'd like to hear it.

The question

A lock-based stack (LockStack: optimistic read of top, non-fair ReentrantLock, validate, publish) runs at 29 MOPS on one thread, 8.5 on two, and about 20 on three or more. On macOS I've shown why the dip exists: with two threads nearly every unlock has to wake a parked waiter, and from three up, running threads barge past parked ones. A fair lock collapses to 0.2 MOPS, a spin lock has no dip, and JFR counts 6.6 times more parks per operation at two threads than at four. The details are in Five ways my benchmarks lied to me and in lockstep/experiments/lockstack-dip.

What I can't explain is why the switch is sharp, all of it between two and three threads. That's this investigation.

Hypotheses

1. A barger needs a second running thread. With two threads, when the holder unlocks, the only other thread is parked, so the unlock has to wake it, and nobody else can take the lock in between except the releaser itself. With three, there's usually another running thread arriving at the lock, so it changes hands between running threads while the queue head stays parked.
2. Wake-up latency dwarfs the critical section. A push or pop holds the lock for tens of nanoseconds; waking a parked thread takes microseconds. If that ratio is what matters, making the critical section longer should move the switch point.
3. AQS's spin-before-park. In JDK 17 a woken waiter that loses spins a little longer each time before parking again (postSpins doubles). The queue length could change how often that spin catches the lock.
4. It's macOS. The scheduler or thread QoS on macOS might be part of it. The first step is checking whether Linux shows the dip at all.

Plan

On a native Linux machine with at least four cores. Not a VM: perf and core pinning need real hardware. Run from lockstep/experiments/lockstack-dip, with caffeinate removed from run.sh.

1. Reproduce. Turbo off, same sweep as macOS, then 1 to 8 threads in steps of one.

   javac -d build $(ls ../../src/*.java | grep -v '/Bench.java$') *.java
   echo 1 | sudo tee /sys/devices/system/cpu/intel_pstate/no_turbo
   java -Xms2g -Xmx2g -cp build Bench LockStack 1000 0C-50I-50D 1,2,3,4,5,6,7,8 5 2000

   If there's no dip on Linux, hypothesis 4 wins and the post becomes about macOS scheduling.
2. Count barging directly. A fifth variant that does if (!lock.tryLock()) { contended++; lock.lock(); } and records how often the lock is free on arrival, at each thread count. Hypothesis 1 predicts a jump between two and three.
3. Pin threads. Two threads on one core vs two cores; three threads on two cores.

   taskset -c 0   java ... Bench LockStack 1000 0C-50I-50D 2 5 2000
   taskset -c 0,1 java ... Bench LockStack 1000 0C-50I-50D 2 5 2000
   taskset -c 0,1 java ... Bench LockStack 1000 0C-50I-50D 3 5 2000

4. Stretch the critical section. Add a fixed amount of busy work inside the lock (for example, a loop of Blackhole-style work) at a few lengths. If the switch point moves to more threads as the critical section grows, that supports hypothesis 2.
5. Measure wake-up latency and cache traffic.

   sudo perf sched record -- java ... Bench LockStack 1000 0C-50I-50D 2 1 2000
   sudo perf sched latency | head -20
   sudo perf c2c record -- java ... Bench LockStack 1000 0C-50I-50D 2 1 2000
   sudo perf c2c report --stdio | head -60

   Compare two and three threads. The question is whether the woken thread is usually still waiting to run when the releaser comes back for the lock.
6. Profile where threads wait. async-profiler in lock mode (-e lock), producing a flame graph for two and three threads.
7. JFR on Linux. Repeat the park counts with park.jfc and CountParks, so macOS and Linux are directly comparable.

Controls

• Turbo off, AC power, nothing else running, five runs per point, report the spread.
• Randomise the order of thread counts so slow drift can't make a fake trend.
• Note the CPU model, core count, kernel, JDK and governor in every results file.
• Keep every raw file. Check that elapsed times are close to 2 s before trusting a run.

The write-up, when it's done

1. The dip, and what I already showed on macOS (short).
2. Does Linux agree?
3. Each hypothesis, the experiment that tests it, and the result, including the ones that failed.
4. Charts: throughput against threads, barge rate against threads, switch point against critical-section length.
5. What I still don't know.
6. Code and raw data in the repo, and references (the JDK sources, AQS, Shipilëv).