Adds FreeSpec-style DSL, data-driven and property testing, per-test fixture generation, and improved test-name/display handling with compacting and collection/byte-array stringification for test suites.
This project provides addons for TestBalloon, the next generation KMP-first, coroutine-first testing framework.
TestBalloon Addons started as migration helpers for people coming from Kotest. That still matters, but the newer Matrix Testing module is the greenfield version of the idea: What would data-driven testing, property testing, fixtures, and compact reports look like if they were designed directly for TestBalloon's KMP-first, coroutine-first execution model?
The answer is a single DSL that keeps Kotest's excellent assertion and generator libraries, but gives TestBalloon full control over registration, execution, concurrency, compaction, reporting and replaying.
val combinedFeaturesSuite by matrixSuite(matrixConfig { execution = ExecutionMode.Concurrent(12) }) {
fixture { Random.nextBytes(16) } - {
"data, properties, fixtures, and compact reports" - { freshBytes ->
data("multiplier", listOf(, , )) - { multiplier ->
compact() { report = CompactReport.FailuresOnly } - {
property(, Arb.int(), iterations = ) test { offset ->
result = freshBytes.size * multiplier + offset
result shouldBeGreaterThan
}
}
}
}
}
}
(Replay in action)
[!TIP]
Looking for a smooth migration path from Kotest?
Check out the Coming from Kotest section!
| Maven Coordinates | at.asitplus.testballoon:matrix:$version |
|---|
The matrix module provides an original, advanced, next-generation testing DSL.
It may not be for the faint of heart, but it combines data-driven testing,
property testing, FreeSpec-style names, fixture generation, concurrency controls, compact reports, and targeted replay of failed tests – things you will need
for truly powerful, comprehensive test suites.
Matrix tests are built from composable layers. A data layer, a property layer, a generated fixture, and a plain
FreeSpec-style suite can be stacked into an n-dimensional test matrix, where every leaf test runs once for each path
through those layers. When that would create too many framework test elements, compact can flatten the subtree into one real
test while still executing and reporting the full virtual matrix. This way, you still get full insights including clickable
stacktraces taking you to the failing assertion(s)!
The top-level matrixSuite(matrixConfig { ... }) { ... } is a regular TestBalloon suite so IDE gutter actions can discover and run it.
Inside the suite, every nested layer is configured first, then either opened with - { ... } for more nesting or finished
with test { ... } to create row test elements.
val quickstart by matrixSuite(matrixConfig { execution = ExecutionMode.Sequential }) {
data("input", listOf("foo", "bar")) - { input ->
property("offset", Arb.int(0..100), iterations = 25) test { offset ->
val result = "$input-$offset"
result.startsWith(input) shouldBe true
}
}
}
A layer's leading name is just a label for a grouping node that wraps the rows it generates. Naming a layer is shorthand for wrapping a nameless one in a labeled suite by hand — these two produce the same test-tree shape:
data("numbers", listOf(1, 2, 3)) - { /* … */ } // named layer
"numbers" - { // equivalent structure
data(listOf(1, 2, 3)) - { /* … */ }
}
Both nest the three rows under a numbers node. Drop the name entirely (data(listOf(1, 2, 3))) and the rows attach
directly to the surrounding scope with no grouping node at all. The name, when present, additionally becomes the
layer's label in failure/replay output ((data) numbers: …); a nameless layer just shows its kind ((data) …). The
same applies to property. The sections below use named layers throughout, but every / form has a
nameless overload.
data layers accept Iterable values, lazy Sequence values, and a Map (each entry comes through as a
destructurable Pair<K, V>, named "<index>: (key: value)" by default). Use test { ... } when each row is the test,
and when each row is a dimension that contains more layers or explicit tests.
Every collection also reads fluently as a receiver via .asData(…), with the exact same options as data:
listOf(1, 2, 3).asData() test { it shouldBeGreaterThan 0 }
mapOf("dev" to 8080, "prod" to 443).asData(nameFn = { (env, _) -> env }) - { (env, port) -> /* … */ }
nameFn resolves by lambda arity: a single-parameter namer ({ v -> … }, { it }, or destructured { (k, v) -> … })
names by value alone, while the two-parameter { index, value -> … } form keeps the index.
dataDrivenMatrix matrixSuite(matrixConfig { execution = ExecutionMode.Sequential }) {
(, listOf(, , ), nameFn = { index, value -> }) test { number ->
number shouldBeGreaterThan
}
(, generateSequence() { it + }, limit = ) {
execution = ExecutionMode.Concurrent(parallelism = )
} test { number ->
number shouldBeGreaterThan
}
(, listOf(, )) - { word ->
{
word.length shouldBeGreaterThan
}
}
}
Layer config is written as the trailing lambda before test or -. For data, the most important layer option is
execution, which can be ExecutionMode.Sequential or ExecutionMode.Concurrent(parallelism = ...).
Concurrency bounds are per layer: nested concurrent layers can multiply the number of active tests or virtual checks.
For large compacted matrices, prefer to use one compact-wide worker budget.
property layers use Kotest generators directly. You get Kotest's Arb ecosystem, edge cases, shrinking-friendly data
models, and concise generator composition, while TestBalloon owns the test tree.
val propertyMatrix by matrixSuite(matrixConfig { defaultPropertyIterations = 100 }) {
property("bytes", Arb.byteArray(Arb.int(1, 64), Arb.byte())) {
seed = 0xC0FFEE
nameFn = { index, bytes -> "$index: ${bytes.size} bytes" }
} test { bytes ->
bytes.toHexString(HexFormat.UpperCase).hexToByteArray() shouldBe bytes
}
}
property supports iterations, seed, edgeConfig, nameFn, and per-layer execution. If iterations is omitted,
the suite default is used.
Deep data/property nesting can generate very large test trees. compact collapses the virtual subtree into one real
TestBalloon test and reports the virtual rows itself.
val compactMatrix by matrixSuite(matrixConfig { execution = ExecutionMode.Concurrent() }) {
compact("all generated checks") {
concurrency = CompactConcurrency.Shared(16)
report = CompactReport.FailuresOnly
reportRows = 256
progressIndicator = Indicator.Heartbeat(every = 1.seconds)
} - {
data("word", listOf("foo", , )) - { word ->
property(, Arb.int(), iterations = ) test { number ->
(word.length + number) shouldBeGreaterThan
}
}
}
}
CompactReport.FailuresOnly renders only failing virtual rows, AllCases also renders successes, and SummaryOnly
keeps the report short. reportRows bounds rendered row details. addSuppressedErrors controls whether retained
failures are attached as suppressed exceptions. controls where compact virtual children run.
keeps per-layer concurrency behaviour, while uses one
compact-wide worker budget so nested virtual layers cannot multiply coroutine counts.
Progress heartbeats are printed separately from the final failure report, for example:
all generated checks: compact progress: 512 of 900 queued completed (1200 source cases), 3 failed
[!NOTE] Compact virtual children are not real TestBalloon test elements, so virtual
test/testSuitedeclarations and terminaldata(...) test { ... }/property(...) test { ... }rows insidecompactcannot honor per-childTestConfig. Put on real matrix tests/suites outside compact, or configure the compact block itself.
Every matrix failure — a real test-tree leaf or a compacted virtual row — carries an Error replay info
block whose detail lines are valid property / data arguments. Copy them back onto the failing layers and the
matrix re-runs exactly those cases. Each frame is tagged with its layer kind — (property) or (data) — so the
marker still pinpoints the right layer even for nameless layers (see below), where no layer name is printed.
Paste the text after each layer name into that layer's call:
val replay by matrixSuite {
property("first", Arb.int(), replay = Cases(seed = 4779463605442148766L, iter = 4L)) - { first ->
property("second", Arb.double(), replay = Cases(seed = -, iter = )) - { second ->
(, listOf(, , , , ), replay = Indexes()) - { third ->
property(, Arb.byte(), replay = Cases(seed = , iter = )) test { fourth ->
}
}
}
}
}
Each pinned layer collapses to just the recorded case, so the whole matrix narrows to the failing leaf.
The path on the first line mirrors the compact report path in full: it includes the enclosing grouping suites
("name" - { ... }) as plain segments, with the (property) / (data) layers carrying the markers. Only the
replayable layers get a - ... argument line below — grouping suites appear in the path but have nothing to paste.
A compacted leaf therefore records the full chain (outer real layers included), and the data index is recorded even
when a custom nameFn omits it from the displayed name.
fixture { ... } produces a fresh value, and every test or suite you declare inside its - { ... } block gains an
extra lambda parameter that hands you that value. You name that parameter yourself — it is the freshly generated
fixture, scoped to the element it is attached to:
fixture { Random.nextBytes(16) } - {
// ^ open the fixture scope with `- { }`
"a single test" { bytes -> // <-- `bytes` is the fixture, freshly generated for THIS test
bytes.size shouldBe 16
}
"a nested suite" - { bytes -> // <-- `bytes` is one fixture, shared by everything in this suite
data("word", listOf("foo", "bar")) test { word ->
bytes.isNotEmpty() shouldBe true
}
}
}
The parameter appears on every nested form — "name" { value -> }, "name" - { value -> },
test(...) { value -> }, and testSuite(...) { value -> } — so the fixture threads in wherever you declare a child.
Freshness follows the element the lambda is attached to:
"name" { value -> } / test { value -> }) gets its own fresh value;"name" - { value -> }) gets one fresh value that all of its children share.So fixtures isolate per test by default, but give you a single shared instance when you group children under a suite.
Directly inside the fixture block you can only declare tests and suites — not data/property/compact layers. To add
those, open a suite first: a "name" - { value -> ... } body is a normal matrix scope, so every layer you declare in it
closes over that suite's value. Nesting composes the other direction too: a inside a row regenerates
once per row, and a inside another fixture's suite threads both values down to the leaves.
fixtureMatrix matrixSuite(matrixConfig { execution = ExecutionMode.Concurrent() }) {
fixture { Random.nextBytes() } - {
{ bytes ->
bytes.size shouldBe
}
- { bytes ->
(, listOf(, )) test { word ->
bytes.isNotEmpty() shouldBe
word.length shouldBeGreaterThan
}
}
}
compact() { report = CompactReport.AllCases } - {
fixture { Random.nextBytes() } - {
{ bytes ->
bytes.size shouldBe
}
}
}
}
Project-wide matrix defaults are best configured based on the configuration of a TestSession:
object ProjectTestSessionConfig : TestSession(testConfig = TestConfig.apply {
MatrixTestDefaults {
execution = ExecutionMode.Sequential
defaultPropertyIterations = 250
defaultCompactConcurrency = CompactConcurrency.Shared(16)
defaultCompactReport = CompactReport.FailuresOnly
defaultCompactReportRows = 128
defaultProgressIndicator = Indicator.Heartbeat(every = 2.seconds)
}
})
Individual suites can override those defaults with matrixSuite parameters. Layers can override their own execution or
generation settings. Real tests and suites also accept TestConfig, which is chained onto the current matrix config.
, , and their FreeSpec forms accept a full matrix config too — pass a value to set concurrency (and any other matrix setting) for that subtree. Unset fields inherit the enclosing scope, and is one of the fields (so covers etc.). The -only forms shown above are just a shorthand wrapping . The fixture-scope / (and their FreeSpec forms, inside ) accept the same way.
val perScopeConfig by matrixSuite(matrixConfig { execution = ExecutionMode.Sequential }) {
// run just this group's children concurrently (also auto-disables the TestScope — see the concurrency note above)
testSuite("heavy group", matrixConfig { execution = ExecutionMode.Concurrent(8) }) {
data(1..1000) test { /* … */ }
}
"freespec group"(matrixConfig { execution = ExecutionMode.Concurrent() }) - {
{ }
}
test(, matrixConfig { execution = ExecutionMode.Concurrent() }) { }
}
val configuredMatrix by matrixSuite(matrixConfig {
execution = ExecutionMode.Concurrent(parallelism = 4)
defaultPropertyIterations = 50
}) {
test("explicit test", testConfig = TestConfig) {
// green code
}
testSuite("explicit suite", testConfig = TestConfig) {
"bare FreeSpec-style test"(testConfig = TestConfig) {
// green code
}
}
"!temporarily disabled" {
error("will not run")
}
}
Layers (data / property) don't take a themselves — their config block only carries layer settings
(, , , replay, …). To apply a such as , , or a
to a layer's cases, wrap the layer in a suite that does. Pair it with a layer so no extra
node appears:
testSuite("with shared setup", testConfig = TestConfig.aroundAll { action ->
// one-time setup around every case of this layer
action()
}) {
data(listOf("foo", "bar", "baz")) test { word ->
word.length shouldBeGreaterThan 0
}
}
[!NOTE]
compactvirtual children are not real TestBalloon test elements, so this wrapping (and per-childTestConfigin general) cannot be honored insidecompact. Wrap real (non-compact) layers, or configure thecompactblock itself.
Prefix any matrix name with ! to disable it. This works for test, testSuite, bare FreeSpec-style strings, data,
property, and compact. Generated row names from nameFn don't get disabled when they start with a bang.
Coming from KotestIf you want APIs that mirror Kotest more closely, the original addon modules are still available:
datatest replicates Kotest's data-driven testing features for TestBalloonproperty brings Kotest's property testing to TestBalloonfixturegen introduces per-test fixture generation for TestBalloon without boilerplate, and beyond TestBalloon's current fixture generation capabilitiesfreespec emulates Kotest's FreeSpec test style for TestBalloon[!TIP]
freespecandfixturegenare modulated into thefixturegen-freespecmodule. This means: if you add theat.asitplus.modulatorGradle plugin to any project that uses both, you can automagically combine FreeSpec syntax and per-test fixture generation.
[!CAUTION]
TestBalloon jumps through quite some hoops to avoid the shortcomings of the underlying Gradle-based test infrastructure and file system limitations eating your cat. However, deep nesting and exceptionally long test names (both of which are easily produced when using data-driven testing or property testing) can still cause errors or even crashes.This is especially true for Android device/emulator-based test execution, which is a wondrous mess!
Because TestBalloon can only shorten test names (not suite names), truncation becomes useful.
All modules allow setting global defaults with regard to test name truncation. These properties are called:
defaultTestNameLengthThe former generally defaults to 64 characters (15 on Android). Display names are not truncated by default.
Both properties can be set in two ways:
TestBalloonAddons.defaultTestNameLength = 15)FreeSpec.defaultTestNameLength = 10)Per-style configuration takes precedence over global configuration. Hence, per-style configuration property setters are nullable, even though their getters will never return null, as they fall back to the global configuration properties automatically.
It is also possible to set test name length for individual tests by passing the maxLength arameter.
Truncated names are ellipsised in the middle, not just cut off at the end.
→ Check out the full API docs for each test style for all configuration options!
TestBalloon Addons use sane default stringification for test names of collection and array types inside data-driven tests and property tests:
[-1, 4, -643, 34310])[9, 76, 145, 9365])ByteArray and UByteArray use hex uppercase notation (i.e. CA:FE:BA:BE)Data-driven testing and property testing can easily produce millions of individual cases being tested.
To avoid making the test runner's heap explode in such cases, the datatest and property modules allow for compacting
test series.
Just pass the compact = true parameter when creating data-driven tests or property tests (see examples in the module
descriptions for data-driven testing and property testing).
The names of compacted test series consist of an uppercase sigma (Σ) followed by the test series' datatype (e.g.,
ΣULong, ΣByteArray, …).
To still get intelligible output about which precise data point(s) caused failing tests, the error message of the resulting failed assertion contains a compact summary and then lists the relevant child rows:
If you only care about failing cases, set suppressCompactSuccesses = true. This can be configured globally through
TestBalloonAddons.suppressCompactSuccesses, per module through DataTest.suppressCompactSuccesses or
PropertyTest.suppressCompactSuccesses, and per terminal withData / checkAll call. Successful cases are still counted
in the summary, but individual rows are omitted.
The stack trace of the thrown exception is the stack trace of the first error (which is equal to the stack traces of all failed assertions). As such, you can directly navigate to the error with the same convenience as ever!
By default, compacted reports keep only the first failure as the cause. To attach all failed child exceptions as suppressed
exceptions, set addSuppressedErrorsToCompactedFailures = true globally, per module, or per compacted run where the API
offers the override.
Suspending terminal compacted withData and checkAll leaves run child bodies sequentially by default. Set
compactConcurrent = true globally through TestBalloonAddons.compactConcurrent, per module through
DataTest.compactConcurrent or PropertyTest.compactConcurrent, or per terminal call if concurrent
execution is desired for that series. Intermediate suite builders keep their existing suite registration behaviour.
Long-running compacted terminal leaves also print periodic progress heartbeats while the compacted body is still running, for example:
ΣByteArray: compact progress: 124/1000000 completed, 3 failed
This output is intentionally separate from the compacted failure report.
To globally enable compacting test series for data-driven testing and property testing, set
DataTest.compactByDefault = true and PropertyTest.compactByDefault = true, respectively.
Compacting works on test and suite level!
In addition, it is possible to specify a prefix parameter when defining data-driven tests or property tests. The prefix
is prepended to generated test names (in front of the sigma), which helps navigate large test graphs.
→ Check out the full API docs for each test style for all configuration options!
| Maven Coordinates | at.asitplus.testballoon:datatest:$version |
|---|
[!NOTE]
Deep nesting will produce a large number of tests, making the heap explode. Either manually compact tests as in the example below (works for bothwithDatatest series andwithDatasuite series), or set the globalDataTest.compactByDefault = trueto automatically compact all data-driven tests.
TestBalloon makes it ridiculously easy to roll your own data-driven testing wrapper with just a couple of lines of code. So we did, by replicating Kotest's data-driven testing API:
val aDataDrivenSuite by testSuite {
// -> NOTE the minus ↙↙↙, it creates a suite
withData(1, 2, 3, 4) - { number ->
// Will create only a single test, but the error will contain all failed inputs
withData("one", "two", "three", "four", compact = true) { word ->
//your test logic being run 16 times
}
}
}
It is possible to specify a prefix parameter when defining data-driven tests and suites. The prefix is prepended to
generated test names, which helps navigate large test reports.
Running individual tests from the gutter is not possible, as the test suite structure and the names of suites and tests are computed at runtime. Hence, you must run the entire suite (but you can manually filter using wildcards).
| Maven Coordinates | at.asitplus.testballoon:property:$version |
|---|
[!NOTE]
Deep nesting will produce a large number of tests, making the heap explode. Either manually compact tests as in the first example below (works for bothcheckAlltest series andcheckAllsuite series), or set the globalPropertyTest.compactByDefault = trueto automatically compact all data-driven tests.
Although it comes with some warts, kotest-property is still extremely helpful for generating a large corpus of test
data—especially as it covers many edge cases out of the box. Again, since TestBalloon has been specifically crafted to be
flexible and extensible, we did just that:
val propertySuite by testSuite {
// DON'T generate a suite for each item. Instead: aggregate >->-->------↘↘↘↘↘↘↘↘↘↘↘↘
checkAll(iterations = 100, Arb.byteArray(Arb.int(100, 200), Arb.byte()), compact = true) - { byteArray ->
checkAll(iterations = 10, Arb.uLong()) { number ->
//test with byte arrays and number for fun and profit
}
}
checkAll(iterations = 100, Arb.byteArray(Arb.int(, ), Arb.byte())) - { byteArray ->
checkAll(iterations = , Arb.uLong()) { number ->
}
}
}
It is possible to specify a prefix parameter when defining property tests and suites. The prefix is prepended to
generated test names, which helps navigate large test reports.
Running individual tests from the gutter is not possible, as the test suite structure and the names of suites and tests are computed at runtime. Hence, you must run the entire suite (but you can manually filter using wildcards).
| Maven Coordinates | at.asitplus.testballoon:fixturegen:$version |
|---|
TestBalloon enforces a strict separation between blue code and green code. This is a good thing—especially for deeply
nested test suites—and it supports deep concurrency. Hence, ye olde JUnit4-style @Before and @After hacks mutating
global state are deliberately not supported.
Sometimes, though, you really want fresh data for every test or suite—in effect, you want to generate a fresh test fixture for every test/suite.
[!NOTE]
Fixture generation as provided by the addons does not use TestBalloon's native fixtures, as those only work in green code. The flavour of fixture generation provided by TestBalloon Addons works for suites (blue code) and tests (green code), as shown below.
[!WARNING]
A fixture-generating scope is intended to be consumed by the scope directly below it (i.e. the outermost test suite, or directly by a test). Programmatically, you can mix this up and it will compile, but it will not run!The following is an antipattern:
val outermostSuite by testSuite { withFixtureGenerator(random::nextFloat) - { testSuite("outer") { /*fixture implicitly available as `it`*/ test() { float -> } } } }
| Maven Coordinates | at.asitplus.testballoon:freespec:$version |
|---|
At A-SIT Plus, we've been using Kotest's FreeSpec for its expressiveness, as it allows modeling tests and test dependencies close to natural language.
TestBalloon is flexible enough to emulate FreeSpec with very little code, if you have context parameters enabled for your codebase:
// build.gradle.kts
kotlin {
compilerOptions {
freeCompilerArgs.add("-Xcontext-parameters")
}
}
Running individual tests from the gutter is not (yet) possible, due to the intricacies of how code analysis works.
Hence, you must run the entire suite (but you can manually filter using wildcards).
(You can, of course, just migrate off FreeSpec and use TestBalloon's native functions to create suites and tests.)
External contributions are greatly appreciated! Just be sure to observe the contribution guidelines (see CONTRIBUTING.md).
The Apache License does not apply to the logos, (including the A-SIT logo) and the project/module name(s), as these are the sole property of A-SIT/A-SIT Plus GmbH and may not be used in derivative works without explicit permission!
| TestBalloon Addons | TestBalloon |
|---|
0.9.0+ | 1.0.0 (Kotlin 2.3.0+) |
0.7.0 - 0.8.0 | 0.8.2+ (Kotlin 2.3.0+) |
0.7.0-RC | 0.8.0-RC (Kotlin 2.3.0) |
0.1.1–0.6.1 | 0.7.1 (Kotlin 2.2.21) |
0.1.0 | 0.7.0 (Kotlin 2.2.21) |
dataproperty- { ... }CompactConcurrency.Shared(n)Concurrency and TestScope. Real concurrency cannot run inside TestBalloon's virtual-time TestScope
(TestBalloon forbids the combination, as it can deadlock through thread starvation). So wherever matrix
parallelizes — any ExecutionMode.Concurrent layer/suite, and every compact block (which executes on real
dispatchers) — matrix automatically disables the TestScope for that scope, even if the surrounding
TestSession enabled it (the default). You therefore do not need testScope(isEnabled = false) to combine
matrix concurrency with the default session — in fact, leave TestScope enabled (the default) and let matrix turn
it off only where it parallelizes. Sequential matrix execution leaves the inherited TestScope untouched, so a mix
works: sequential suites get virtual time (and its timeout), concurrent suites/compact run for real. The disable also
wins over a testConfig you pass yourself, so a testScope(true) on a concurrent scope is overridden.
Timeouts: for a virtual-time (sequential) test, use testScope(isEnabled = true, timeout = …) (or the 60s
default). The TestScope timeout is per test element, not per suite — a large matrix is many tests, each with
its own budget, so the case count never drains one shared timeout; raise it only if a single case is genuinely
slow. Compact and concurrent tests run with TestScope disabled, so its timeout doesn't apply to them at all —
for a wall-clock cap there, use a real-time timeout via aroundEachTest { action -> withTimeout(…) { action() } }
(per leaf) or aroundAll { action -> withTimeout(…) { action() } } (per block). That's a plain testConfig wrapper,
not invocation/testScope, so it doesn't conflict with execution.
coroutineContextCompactConcurrency.LayeredCompactConcurrency.Shared(n)TestConfigat.asitplus.AssertionError: 360888 should be < 256000
Error replay info: (property) first: 4: 2018089192 ↘ (property) second: 2: 2796 ↘ (data) third: 1: 2 ↘ (property) fourth: 3: 1
- (property) first: replay = Cases(seed = 4779463605442148766L, iter = 4L)
- (property) second: replay = Cases(seed = -1353176301820643450L, iter = 2L)
- (data) third: replay = Indexes(1L)
- (property) fourth: replay = Cases(seed = 5014696554795393980L, iter = 3L)
replay = Cases(seed = …, iter = …) for one case (what the report prints).
Pass several Cases(Input(...), Input(...)) to reproduce several recorded cases — each with its own seed — at once,
or Cases(seed = …, iter = 1L..5L) for several iterations of one seed.replay = Indexes(n) (printed), Indexes(a, b, c) for several, or Indexes(0L..9L) for a
range. Data is deterministic, so only the index is needed.replay is independent of its seed: seed pins a deterministic full run, while replay selects
specific recorded cases (which carry their own seeds).fixturedatafixturetesttestSuite"name"(…)matrixConfig { … }testConfigmatrixConfig { testConfig = … }aroundAlltestConfigmatrixConfig { testConfig = … }testtestSuitefixture { … } - { … }matrixConfig { … }⚠️ Set concurrency with execution, not a testConfig. A testConfig is for aroundAll / aroundEach /
context / timeout wrappers. Never put TestConfig.invocation(...) in a matrix testConfig — matrix derives
invocation from execution, can't strip a conflicting one out of an (opaque) TestConfig, and the clash can break
test discovery ("did not discover any tests"). Likewise, a virtual-time testScope(...) only applies to sequential
execution (matrix disables it when concurrent), so enable TestScope on the TestSession, not per matrix scope.
Wrong: matrixSuite(matrixConfig { testConfig = TestConfig.invocation(Sequential) }). Right: matrixSuite(matrixConfig { execution = ExecutionMode.Sequential }).
TestConfigexecutionnameMaxLengthseedTestConfigaroundAllaroundEachcoroutineContextdata(...) test { ... } / property(...) test { ... } creates row test elements; data(...) - { ... } / property(...) - { ... } creates row suite or dimension test elements.test { ... } or - { ... } leaves a configured layer unopened, so no child tests are registered. The layer itself does nothing until its trailing lambda receives content; either tests or more rows.data and property can each appear multiple times in the same suite.nameFn. Use - { ... } plus an explicit "name" { ... } leaf when the invariant itself needs a separate name.matrixSuite, and a data / property / compact / test / testSuite placed at the top
level or under purely static nesting (only test / testSuite / named suites in between), get a working run-gutter:
clicking a data("name", …) / property("name", …) / compact("name") line runs the whole layer (all its cases).
A test / testSuite nested inside a data / property / compact layer does not get a usable gutter,
because that element's path includes a per-case segment (nameFn(index, value), e.g. 0: x) that is generated at
runtime and is invisible to the IDE's static analysis. To run such an inner element, click the enclosing layer's
gutter (runs all cases) or set a filter by hand, e.g. TESTBALLOON_INCLUDE_PATTERNS='*name' (patterns are anchored
full-match with * → .* across the ↘ path separator, so a leading * matches an arbitrarily deep element).
Nameless data(…) / property(…) layers carry no gutter (they have no name to select on).compact when the test tree itself becomes too large.If you don't want to use modulator, you can add the
at.asitplus.testballoon:fixturegen-freespec:$version
dependency manually to your project.
java.lang.AssertionError: ΣString
Summary: 1 OK, 7 failed
Error: 1: 4: expected:<three> but was:<4>
Error: 2: one: expected:<three> but was:<one>
Error: 3: null: Expected "three" but actual was null
Error: 4: null: Expected "three" but actual was null
Error: 5: null: Expected "three" but actual was null
Error: 6: two: expected:<three> but was:<two>
OK: 7: three
Error: 8: four: expected:<three> but was:<four>
----------------------------------------
OKimport at.asitplus.testballoon.withFixtureGenerator //<- Look ma, only a single import!
import de.infix.testBalloon.framework.core.testSuite
import kotlin.random.Random
import kotlinx.coroutines.delay //just to get some suspending demo generator
val aGeneratingSuite by testSuite {
//seed before the generator function, not inside!
val byteRNG = Random(42);
//We want to test with fresh randomness, so we generate a fresh fixture for each test
withFixtureGenerator { byteRNG.nextBytes(32) } - {
repeat(5) {
test("Generated test with fresh randomness") { freshFixture ->
//your test logic here
}
}
testSuite("Generated Suite with fresh randomness") { freshFixture ->
test("using the outer fixture") {
//your logic based on freshFixture here
}
}
repeat(5) {
//✨ it ✨ just ✨ werks ✨
test("Test with implicit fixture name `it`") {
//do something with `it`, it contains fresh randomness!
}
}
}
//seed the RNG for reproducible tests
val random = Random(42)
//reference function to be called for each test inside withFixtureGenerator
withFixtureGenerator(random::nextFloat) - {
repeat(10) {
test("Generated test with random float") {
//test something floaty!
}
}
test("And some other test that des not conform to the shema from the loop") {
//test something different, with a fresh float
}
}
//always-the-same fixtures also work, of course
withFixtureGenerator {
object {
var a: Int = 1
val b: Int = 2
}
} - {
test("one") {
it.a++ //and we can even modify them in one test
println("a=${it.a}, b=${it.b}") //a=2, b=2
}
test("two") {
//without affecting the other!
println("a=${it.a}, b=${it.b}") //a=1, b=2
}
}
//Let's test some nasty bug that shows itself only sometimes functionality
val ageRNG = Random(seed = 26)
withFixtureGenerator {
class ABuggyImplementation(val age: Int) {
fun restrictedAction(): Boolean =
if (age < 18) false
else if (age > 18) true
else Random.nextBoolean() //introduce jitter to simulate a faulty implementation
}
//create new object for each test
ABuggyImplementation(ageRNG.nextInt(0, 99))
} - {
repeat(1000) {
test("Generated test accessing restricted resources") {
//test `restrictedAction` across a wide age range
//a thousand times to unveil the bug
}
}
}
}
import at.asitplus.testballoon.invoke
import at.asitplus.testballoon.minus
import de.infix.testBalloon.framework.core.TestConfig
import de.infix.testBalloon.framework.core.TestInvocation
import de.infix.testBalloon.framework.core.invocation
import de.infix.testBalloon.framework.core.singleThreaded
import de.infix.testBalloon.framework.core.testSuite
val aFreeSpecSuite by testSuite {
//testConfigs are supported for suites
"The outermost blue code"(testConfig = TestConfig.singleThreaded()) - {
"contains some more blue code" - {
", some green code inside the lambda" {
// your test logic here
}
//testConfigs are supported for Tests
", and some more green code inside the second lambda"(testConfig = TestConfig.invocation(TestInvocation.SEQUENTIAL)) {
// more test logic here
}
}
"And finally some more blue code" - {
"!With some final disabled green code in this lambda" {
//additional, disabled test logic here
}
}
}
}
| Maven Coordinates (if not using modulator) | at.asitplus.testballoon:fixturegen-freespec:$version |
|---|
[!WARNING]
As without FreeSpec syntax, a fixture-generating scope is intended to be consumed by the scope directly below it (i.e. the outermost test suite, or directly by a test). To disambiguate and be explicit about this, explicit parameter specification is required, starting with TestBalloon Addons 0.6.0.
import at.asitplus.testballoon.withFixtureGenerator // <- Look ma, only regular generatingFixture import!
import at.asitplus.testballoon.invoke // <- Look ma, only regular freespec import!
import at.asitplus.testballoon.minus // <- Look ma, only regular freespec import!
import de.infix.testBalloon.framework.core.testSuite
import kotlin.random.Random
val aGeneratingFreeSpecSuite by testSuite {
//any lambda with any return type is a fixture generator. Type is reified.
withFixtureGenerator { Random.nextBytes(32) } - {
"A Suite with fresh randomness" - { freshFixture ->
"Consuming outer fixture" {
//your freshFixture-based test logic here
}
withFixtureGenerator { Random.nextBytes(32) } - {
"With fresh inner fixture" { inner ->
//your test logic here with always fresh inner
//and fixed freshFixture from outer scope
}
}
}
repeat(100) {
"Generated test with fresh randomness" { freshFixture ->
//some more test logic; each call gets fresh randomness
}
}
//parameter must be explicitly specified to disambiguate
"Test with fixture name `it`" { it ->
//no need for an explicit parameter name here, just use `it`
}
"And we can even nest!" - {
withFixtureGenerator { Random.nextBytes(16) } - {
repeat(10) {
"pure, high-octane magic going on" { it ->
//Woohoo! more randomness each run
}
}
}
}
}
}
Surfaced from shared tags and platforms — no rankings paid for.