Evaluate Testing framework(s)

[JohnLCaron]

1. We are using the kotlin.test standard library from JetBrains (https://kotlinlang.org/api/latest/kotlin.test/)

library("kotlin-test-common", "org.jetbrains.kotlin", "test-common").versionRef("kotlin-version")
library("kotlin-test-annotations-common", "org.jetbrains.kotlin", "test-annotations-common").versionRef("kotlin-version")

kotlin-test-common – assertions for use in common code;
kotlin-test-annotations-common – test annotations for use in common code;

2. We use it on top of junit5 in order to get parellel execution, but junit is only for the JVM:

library("kotlin-test-junit5", "org.jetbrains.kotlin", "kotlin-test-junit5").versionRef("kotlin-version")
library("kotlin-test-junit", "org.jetbrains.kotlin", "kotlin-test-junit").versionRef("kotlin-version")

kotlin-test-junit – maps the test annotations from kotlin-test-annotations-common to JUnit test annotations;
kotlin-test-junit5 – maps the test annotations from kotlin-test-annotations-common to JUnit 5 test annotations;

parellel executions are enabled in egklib build:

        testRuns["test"].executionTask
            .configure {
                useJUnitPlatform()
                minHeapSize = "512m"
                maxHeapSize = "2048m"
                jvmArgs = listOf("-Xss128m")

                // Make tests run in parallel
                // More info: https://www.jvt.me/posts/2021/03/11/gradle-speed-parallel/
                systemProperties["junit.jupiter.execution.parallel.enabled"] = "true"
                systemProperties["junit.jupiter.execution.parallel.mode.default"] = "concurrent"
                systemProperties["junit.jupiter.execution.parallel.mode.classes.default"] = "concurrent"
            }

So JVM tests run in parellel, and native tests do not.

3. Also using mockk, also only for the JVM

library("mockk", "io.mockk", "mockk").version("1.13.2")

4. We are also using the kotest.io property-based testing (https://kotest.io/docs/proptest/property-based-testing.html), (not from JetBrains, apparenly just a community supported project).

library("kotest-property", "io.kotest", "kotest-property").version("5.5.4")

This is in commonTest, so is multiplatform.
kotest.io also has a framework and an assertions package.
Apparently uses Junit5 under the hood. Not sure how native works, but cleary more complex.

Also see

• https://levelup.gitconnected.com/on-the-mysteries-of-kotlin-test-444cf094e69f
• https://www.reddit.com/r/Kotlin/comments/kccm6b/kotest_vs_spek_how_do_they_compare/
• https://webcodr.io/2020/08/kotest-and-junit-with-intellij-or-dont-frak-up-your-toolchain-upgrades/

[JohnLCaron]

Ive got a class with 4 tests that is failing because the tests needs to be not run in parellel. But I dont see a way to do that.

I could use Junit4 annotations, but then I would need to drop that test down into the jvm branch.

We could evaluate using kotest instead of kotlin.test, with the hope of getting parellel tests in native.

[JohnLCaron]

Currently testing in native: 1) asserts dont show line number, and 2) cant run in the debugger. So testing/debugging is seriously painful.

Both are probably due to how we are building, rather than testing framework, but ??