testmthread: Contains an unsupported closure type (whose implementation is missing)

[runeksvendsen]

Using GHC 8.6.5 on macOS, the testmthread test suite occasionally fails with the following output:

Multithreaded test program for serialisation
normal evaluation (take 3), should be
[10946,17711,28657]
[10946,17711,28657]
014256f379testmthread: Contains an unsupported closure type (whose implementation is missing)
8testmthread: Contains an unsupported closure type (whose implementation is missing)
otestmthread: Contains an unsupported closure type (whose implementation is missing)
testmthread: Contains an unsupported closure type (whose implementation is missing)
testmthread: Contains an unsupported closure type (whose implementation is missing)
testmthread: Contains an unsupported closure type (whose implementation is missing)
testmthread: Contains an unsupported closure type (whose implementation is missing)
testmthread: rContains an unsupported closure type (whose implementation is missing)
k::e  dOO,KK 		w11a07i97t41i61n

g
testmthread: thread blocked indefinitely in an MVar operation

It doesn't always happen, but if I keep running the given test suite repeatedly as follows, it happens after a few runs.

for n in $(seq 1 1000); do if ! cabal run test:testmthread; then break; fi; done

The test failure can be observed in the following CI run: https://github.com/runeksvendsen/packman/actions/runs/12525280654/job/34936688671

TODO

• Revert CI: ignore failures due to issue #18 #25 once this issue is fixed

[runeksvendsen]

This error has been observed also on ubuntu-22.04 using GHC 7.10.3 and Clang 18 (CI run: https://github.com/runeksvendsen/packman/actions/runs/12557619285/job/35010538362).

Logs:

Multithreaded test program for serialisation
normal evaluation (take 3), should be 
[10946,17711,28657]
[10946,17711,28657]
fo1263750498rktestmthread: Contains an unsupported closure type (whose implementation is missing)
testmthread: Contains an unsupported closure type (whose implementation is missing)testmthread: Contains an unsupported closure type (whose implementation is missing)

testmthread: Contains an unsupported closure type (whose implementation is missing)
testmthread: Contains an unsupported closure type (whose implementation is missing)
testmthread: Contains an unsupported closure type (whose implementation is missing)
testmthread: Contains an unsupported closure type (whose implementation is missing)
testmthread: Contains an unsupported closure type (whose implementation is missing)
e::d  ,OO KKw		a12i78t76i15n17g


testmthread: thread blocked indefinitely in an MVar operation

Test suite testmthread: FAIL

[runeksvendsen]

Note: the CI run for master (18cf106) had four failures, all of which are due to this issue.

I know this because I wrote a script for grepping the logs of failed jobs within a GitHub CI run (see https://stackoverflow.com/a/79326305/700597), and all of the logs contain the string testmthread: Contains an unsupported closure type (whose implementation is missing).

[runeksvendsen]

In CI I'm seeing the following error as well for the testmthread test suite. See e.g. this CI run: https://github.com/jberthold/packman/actions/runs/12726238305/job/35474390752. Not sure if it's the same error as above.

testmthread: Unpacking unexpected closure type (26)
testmthread: Garbled data for deserialisation

Full logs:

Test suite testmthread: RUNNING...
Multithreaded test program for serialisation
normal evaluation (take 3), should be 
[10[9](https://github.com/jberthold/packman/actions/runs/12726238305/job/35474390752#step:12:10)46,17711,28657]
[10946,17711,28657]
fo0189testmthread: Unpacking unexpected closure type (26)

254736rtestmthread: Garbled data for deserialisation
ked, waiting
: OK	17711
: OK::::	    1OOOO0KKKK9				4524161869
4636256427819

8

: O:K 	O3K1	7785[10](https://github.com/jberthold/packman/actions/runs/12726238305/job/35474390752#step:12:11)12
5
: OK	121393
testmthread: thread blocked indefinitely in an MVar operation
Test suite testmthread: FAIL
Test suite logged to: /Users/runner/work/packman/packman/ci-logs/testmthread-test-fail.log

[runeksvendsen]

@jberthold in #19 (comment) you say (regarding this issue):

[...] first thing would be to run with debug logging

How do I do this? I've tried running with the debug flag by using the command cabal test -fdebug test:testmthread but the output appears the same as without that flag.

[jberthold]

There are macros PACK_DEBUG (some detail) and PACKET_DEBUG (more detail) in the code, see

packman/cbits/Pack.c

Lines 75 to 90 in 58b5a6d

	// debugging macros for library and in-RTS version
	#if defined(LIBRARY_CODE)
	// for the library version, borrow flags "scheduler" and "sparks"
	# define PACKDEBUG(s) IF_DEBUG(scheduler, s)
	# define PACKETDEBUG(s) IF_DEBUG(sparks, s)
	#else
	// for the in-RTS version, use the usual macros
	// XXX maybe drop support for the non-parallel in-RTS version
	#if defined(PARALLEL_RTS)
	# define PACKDEBUG(s) IF_PAR_DEBUG(pack, s)
	# define PACKETDEBUG(s) IF_PAR_DEBUG(packet, s)
	# else
	# define PACKDEBUG(s) IF_DEBUG(scheduler, s)
	# define PACKETDEBUG(s) /* nothing */
	# endif
	#endif

.
When using the library, the debug logging is activated together with other debug options (scheduler and sparks), originally from https://github.com/ghc/ghc/blob/f21fbb795b39a3ac0cf56940be67daf4ce4369fa/includes/rts/Flags.h#L90-L110
In order to use the flags, you need to link the program with debugging enabled (-debug), and then pass +RTS -Ds -Dr -RTS as command line flags for the runtime system (to activate both scheduler and sparks) when you run the program, see https://downloads.haskell.org/ghc/9.12.1/docs/users_guide/runtime_control.html#rts-options-for-hackers-debuggers-and-over-interested-souls
(How exactly to do this with a distro GHC I don't quite remember, TBH, I was always using the debugging with GHCs built locally with debugging enabled).

[jberthold]

Just re-read your last sentence...

I've tried running with the debug flag by using the command cabal test -fdebug test:testmthread but the output appears the same as without that flag.

What is missing is the activation of the scheduler and packet debug messages when running the program. You could set the variable GHCRTS to pass the -Ds and -Dr, but I would rather suggest you run the test manually by invoking the compiled program (somewhere in dist-newstyle, not sure where) because the log output will be noisy.

[runeksvendsen]

@jberthold thank you. The command cabal run -fdebug test:testmthread -- +RTS -Ds -Dr did the trick. Console output: https://gist.github.com/runeksvendsen/2bb0c57285a9ead73d3352d3edae612a.

[jberthold]

@jberthold thank you. The command cabal run -fdebug test:testmthread -- +RTS -Ds -Dr did the trick. Console output: https://gist.github.com/runeksvendsen/2bb0c57285a9ead73d3352d3edae612a.

Sorry for the delayed response, busy getting back to the job...
Strangely, your output does not show any logs from the packing code, but it was certainly executed based on the test outcome (at least there should be log entries reporting that the packing code was called).
Maybe that logging broke silently in the newer version, I am not sure . Will try to test it using your nix setup in the weekend.

[jberthold]

As it turns out, ghc-8.6.5 provided by nix via ghc865Binary (as in the .github/workflow file) indeed does not emit the log messages from the packing code. The prior and subsequent versions ghc844 and ghc884 log the messages as expected:

$ nix-shell --argstr ghcVersion ghc844 --run "cabal -fdebug test"
$ dist-newstyle/build/x86_64-linux/ghc-8.4.4/packman-0.5.1/t/testmthread/build/testmthread/testmthread +RTS -Ds -Dr 2>&1 | grep -i pack
$ nix-shell --argstr ghcVersion ghc884 --run "cabal -fdebug test"
$ dist-newstyle/build/x86_64-linux/ghc-8.8.4/packman-0.5.1/t/testmthread/build/testmthread/testmthread +RTS -Ds -Dr 2>&1 | grep -i pack

IDK why 865 does not (maybe something that the Binary suffix hints at - ??).

Also, I observed the multi-threading error (...unsupported closure) twice while trying different versions under Linux, this error is not Mac-specific. These two observations were during the cabal build/test. Unfortunately, none of my re-runs with debug logging enabled hit the error any more. Obviously the error happens when there is additional load on the CPU (such as concurrently running cabal test for another test).