Fix macros which call Bun.file operations causing build commands to hang

[brainkim]

What does this PR do?

• Documentation or TypeScript types (it's okay to leave the rest blank in this case)
• Code changes

Users reported in #7611 and #14104 that calling Bun.file().text() in a macro would cause builds to hang. This bug was introduced in v1.0.16, likely when some changes related to how Bun handles multithreading occurred.

I went through a minimal reproduction with a debugger and determined that work would get stuck in ThreadPool threads because the ThreadPool was initialized twice. The double instantiation of ThreadPool, which is assumed to be a global singleton, caused issues in macros because file I/O would get scheduled on a thread, the work would be pushed to the thread local queue/buffer, and then the thread would wait for the promise to resolve, causing a dead lock. There is code which allows for “work stealing” but threads initialized from different thread pools do not steal work each other.

The least invasive fix I found was to consistently call WorkPool.get() in the bundle_v2’s initialization code.

Fixes #7611 and fixes #14104.
Happy holidays!

How did you verify your code works?

• I included a test for the new code, or existing tests cover it
• I ran my tests locally and they pass (bun-debug test test-file-name.test)

[brainkim]

I guess things are never this easy...

[Jarred-Sumner]

My guess is that by not using the bundler’s threadpool, the threads never call their init function

it sounds like we should have a global variable for “threadpool ID” and only use the global one in that case maybe?

[brainkim]

I tried building on ubuntu and the failing test passes locally. I also tested it by hand.

Is it possible that the CI is running the tests against a stale build? I’m not sure how this failure could be platform dependent. Perhaps it is due to the number of threads available?

Running the build process in a container takes a painfully long time, wishing for cross-compilation here.

[brainkim]

Sorry about the failures. The additional Windows failure may indicate that this is a race condition of some sort. I searched the codebase for additional direct uses of ThreadPool.init() and found one in installer.zig, which might be getting called during the test running process. Beyond that, I’m stumped unless I figure out how to reproduce the test failure locally.

Again, sorry about the test failures!

[brainkim]

@Jarred-Sumner
Took another look at this and I think I figured out the problem. The platform differences are likely due to maximum number of threads for the work pool being set to the number of logical cores available, which is probably 2 for the linux environments. The deadlock in the test suite happens when the max_threads is set to anything less than 3.

This would have been fine, except there’s a bug in ThreadPool.warm function:

bun/src/thread_pool.zig

Lines 416 to 435 in 7830e15

	pub fn warm(self: *ThreadPool, count: u14) void {
	var sync = @as(Sync, @bitCast(self.sync.load(.monotonic)));
	if (sync.spawned >= count)
	return;
	const to_spawn = @min(count - sync.spawned, @as(u14, @truncate(self.max_threads)));
	while (sync.spawned < to_spawn) {
	var new_sync = sync;
	new_sync.spawned += 1;
	sync = @as(Sync, @bitCast(self.sync.cmpxchgWeak(
	@as(u32, @bitCast(sync)),
	@as(u32, @bitCast(new_sync)),
	.release,
	.monotonic,
	) orelse break));
	const spawn_config = std.Thread.SpawnConfig{ .stack_size = default_thread_stack_size };
	const thread = std.Thread.spawn(spawn_config, Thread.run, .{self}) catch return self.unregister(null);
	thread.detach();
	}
	}

In short, cmpxchngWeak will return null on success, but this code breaks on null, meaning the spawned property is incremented without actually spawning any new threads, causing an off-by-one error when spawning new threads normally. This, combined with the task dependency created by macros, where the parsing thread is blocked until the macro finishes, causes a deadlock at two max threads when the macro call schedules file I/O. You can confirm this caused the bug by commenting out the warm() call here (

bun/src/bundler/bundle_v2.zig

Line 169 in 7830e15

this.pool.warm(8);

) at two max threads. I assumed that there are performance reasons for warming the thread pool so I fixed the function instead.