Overview
When you run a test set, multiple test cases, or multiple test sets, you pick an execution mode in the Execute modal : Parallel (the default) or Sequential. A single test case always runs on its own. This guide explains what each mode does and when to choose it.
Parallel (default)
Tests run concurrently, several at a time, up to your Max parallelism limit. This is today's behavior and the fastest option.
Use Parallel when your tests are independent — they don't rely on each other's data or on a specific order.
Sequential (alphabetical)
Tests run one at a time, in alphabetical order of their names, so tests that depend on each other run in a predictable order.
Use Sequential when your tests depend on each other (for example Create → Edit → Delete), or you want the run to stop as soon as a critical test fails.
What runs, and in what order:
A test set, or multiple test cases → the test cases run in order, by test case name.
Multiple test sets → the test sets run in order, by set name — and inside each set, its cases also run one at a time, alphabetically. Sequential applies all the way down.
Failure behavior: Continue vs Stop
In Sequential mode you also choose how the run reacts when a test fails:
Continue (default) : every test runs, even if an earlier one fails; each reports its own result. Pick this when you want a full picture of every test's result.
Stop : the first test that fails (after its retries) cancels everything after it; the remaining tests appear as Canceled. Pick this for dependent flows, where later tests are meaningless once an earlier one has failed.
Running with a data file
When you run Sequential with a data file, each row becomes its own independent chain — rows don't share data or state. Within a chain, the cases run one at a time in alphabetical order; across chains, the rows still run in parallel, up to your plan's concurrency limit. Stop is scoped to a single row: a failure in one row cancels only that row's remaining cases, while the other rows keep running.
Control the order with names
Sorting is natural and case-insensitive, so 02_Create comes before 10_Delete. To pin the order, prefix names with numbers: 01_Login, 02_Create_Report, 03_Edit_Report.
How retries interact with Sequential and Stop
With Max retries set, a failed test is retried up to that many times. In Sequential + Stop, the run only stops after a test has failed and used up its retries — a test that passes on a retry won't stop the run.
Related reads
→ Execution Behavior: Max Retries & Max Parallelism — the Max parallelism and Max retries settings
→ CI/CD — trigger parallel or sequential runs from a pipeline