How to Compare Test Runs Side by Side

Runs Diff lets you compare two executions of the same test case side by side across screenshots, page HTML, logs, and run parameters. Both runs appear in a split panel, with changes highlighted action by action.

It's most useful when a test that used to pass suddenly starts failing and you need to figure out what changed.

Investigating a regression: compare a passing run against a failing one to pinpoint what broke.

Spotting visual differences: the screenshot diff overlay detects pixel-level UI changes between two runs.

Comparing configurations: check whether a different browser, environment, or persona caused a different outcome.

Debugging log differences: compare console logs side by side. Strip timestamps for a cleaner view.

Reviewing HTML changes: see exactly which parts of the page source changed between the two executions.

Runs Diff loads the recorded data for each action in both runs and presents them in four view modes:

Each panel header shows the run status, date and time, duration, and the current action's description, timing, and status (Passed, Failed, Skipped, or Not Executed).

The page is split into two resizable panels — Run A on the left, Run B on the right. Drag the divider to resize them.

At the top you'll find a back arrow, the page title, and sync navigation arrows in the center to move both panels forward or back at the same time. Use the view mode tabs (Screenshot, HTML, Logs, Execution Params) to switch how you compare.

Each panel also has its own independent navigation arrows at the bottom, which is helpful when the two runs have a different number of actions.

The Screenshot tab is the default view. Each panel shows the screenshot captured at the current action.

Click the "Show Diff" button below a screenshot to toggle a pixel-difference overlay. This overlay highlights every pixel that changed between the two runs, making it easy to spot visual regressions — even subtle ones like a shifted button or a missing icon.

Click the button again to go back to the original screenshot.

The HTML tab shows a code-level diff of the page's HTML source at the current action.

This is useful when a visual change is caused by a DOM modification — for example, a missing element, a changed class name, or an updated attribute.

The Logs tab shows a side-by-side diff of the browser console logs captured during each run.

Just like the HTML tab, additions appear in green and removals in red. This helps you spot new errors, missing log entries, or changed API responses between the two runs.

A "Strip timestamps" checkbox is available at the top. When enabled, it removes the [YYYY-MM-DD HH:MM:SS.mmm] prefix from each log line so you can focus on the actual content without time-related noise causing false differences.

The Execution Params tab shows a diff of the configuration used for each run. This includes:

This is useful when two runs of the same test case produce different results and you want to check whether the difference comes from a configuration change rather than a code change.