Diffy Steps

Overview

The Diffy step definitions add opt-in visual-regression testing to Webship JS 2.0.x. They capture full-page Playwright screenshots at one or more breakpoints and talk directly to the Diffy REST API: uploading screenshot batches as named custom snapshots, triggering Diffy-side screenshot runs per environment, creating comparisons (Diffs) between snapshots or environments, and polling a comparison until it completes. The result is a pixel-level diff report in your Diffy project, driven entirely from plain Gherkin.

Key characteristic: "Kept in a separate folder (tests/step-definitions) so the default npm test run does not load them." To enable them, uncomment the Diffy line in the generated cucumber.js require list:

require: [
  'node_modules/webship-js/tests/step-definitions/**/*.js',
  'node_modules/@webship-js/diffy-steps/tests/step-definitions-diffy/**/*.js', // Diffy (optional)
  'tests/step-definitions/**/*.js',
],

The typical flow has three phases: capture (resize, take screenshots), upload (send the batch as a named snapshot - this clears the in-memory buffer), and compare (create a Diff from the last two uploaded snapshots, then wait for it to complete). Screenshots are held in memory per scenario, and optionally written to disk when a screenshots directory is configured.

Quick reference

StepWhat it does
When I resize window to "1200"Resize the viewport to a breakpoint width (plus Diffy window padding).
Then I take screenshotCapture a full-page screenshot at the current viewport.
Then I take screenshots for all breakpointsResize to each configured breakpoint and capture one screenshot each.
Then send screenshots to diffy with name "baseline"Upload the captured batch as a named Diffy custom snapshot.
Then create diffy comparisonCreate a Diff from the last two uploaded snapshots.
Then create diffy comparison with name "release-2.0"Same, with a custom comparison name.
Then create diffy screenshot from "production" environmentAsk Diffy to capture a snapshot remotely from a project environment.
Then compare diffy "prod" with "stage"Kick off a Diffy comparison between two project environments.
Then upload folder "./shots" to diffy as "baseline"Upload every PNG/WebP in a local folder as a named snapshot.
Then wait for diffy comparison to completePoll the last created comparison until it reaches a terminal state.

Available Step Definitions

Steps in detail

When I resize window to "1200"

Resizes the browser viewport to the given breakpoint width plus 8 px of Diffy window padding, with the configured window height (default 2000 px). Screenshots taken afterwards report the padded-back breakpoint width to Diffy so widths line up in comparisons.

When I resize window to "640"
Then I take screenshot

Then I take screenshot / Then I take screenshots for all breakpoints

Captures a full-page screenshot at the current viewport, recording the current URL path and breakpoint alongside it. The plural form loops over every configured breakpoint (from DIFFY_BREAKPOINTS or worldParameters.diffy.breakpoints, e.g. "640,1200"), resizing before each capture. Shots stay in memory - and are also written to DIFFY_SCREENSHOTS_DIR if set - until you upload the batch. Both accept I, we, or no pronoun.

Given I am on "/diffy/baseline.html"
Then I take screenshots for all breakpoints

Then send screenshots to diffy with name "baseline"

Uploads every captured screenshot as one named custom snapshot via create-custom-snapshot, then clears the in-memory buffer (and any PNGs in the on-disk screenshots directory) so the next batch starts clean. Fails with a clear message if nothing has been captured yet. The returned snapshot id is remembered for the comparison steps.

Then I take screenshots for all breakpoints
Then send screenshots to diffy with name "homepage-release-2.0"

Then create diffy comparison / ... with name "release-2.0"

Creates a Diffy Diff from the first two snapshot ids uploaded in the scenario - so upload a baseline set, then a changed set, then create the comparison. Requires at least two uploaded snapshots. The named variant sets the comparison name shown in the Diffy UI.

Then send screenshots to diffy with name "baseline"
Then send screenshots to diffy with name "changed"
Then create diffy comparison with name "PR-smoke"

Then create diffy screenshot from "production" environment

Asks Diffy itself to capture a snapshot from an environment configured on your Diffy project - no local browser involved. Accepts short (prod, stage, dev) or long (production, staging, development) forms, plus custom. The resulting snapshot id joins the same queue used by the comparison steps.

Then create diffy screenshot from "production" environment
Then create diffy screenshot from "staging" environment
Then create diffy comparison with name "env-pair"

Then compare diffy "prod" with "stage"

Kicks off a fully server-side comparison between two project environments in one call. Allowed values: prod, stage, dev, baseline, custom (long forms are normalized). When either side is custom, the URL comes from DIFFY_ENV1_URL / DIFFY_ENV2_URL (or the matching worldParameters.diffy keys).

Then compare diffy "prod" with "stage"
Then wait for diffy comparison to complete

Then upload folder "./screenshots/baseline" to diffy as "baseline"

Uploads every PNG/WebP under a local folder (recursively) as one named custom snapshot - useful for importing screenshots produced by another tool or a previous run. The breakpoint is read from each PNG's actual pixel width, and the URL slug is derived from the file name, so home-page.png becomes /home-page.

Then upload folder "./screenshots/baseline" to diffy as "folder-baseline"
Then upload folder "./screenshots/feature" to diffy as "folder-feature"
Then create diffy comparison with name "folder-pair"

Then wait for diffy comparison to complete

Polls the most recently created comparison every 10 seconds until it reaches a terminal state, up to DIFFY_MAX_WAIT seconds (default 1200). The step's own Cucumber timeout is raised to one hour, so long Diffy runs won't trip the default step timeout. Fails if no comparison has been created in the scenario.

Then create diffy comparison
Then wait for diffy comparison to complete

Complete example

From tests/features/diffy/test--diffy-02-all-steps.feature:

Feature: Diffy - visual regression

  @diffy @smoke
  Scenario: Resize, take screenshot, breakpoints loop, send
    Given I am on "/diffy/baseline.html"
    When I resize window to "640"
    Then I take screenshot
    Given I am on "/diffy/changed.html"
    Then I take screenshots for all breakpoints
    Then send screenshots to diffy with name "smoke-set-1"

    Given I am on "/diffy/baseline.html"
    Then I take screenshots for all breakpoints
    Then send screenshots to diffy with name "smoke-set-2"

    Then create diffy comparison
    Then wait for diffy comparison to complete

Configuration

Each setting is resolved per scenario with this priority: DIFFY_* environment variable (highest, ideal for CI), then worldParameters.diffy.* in cucumber.js, then the built-in default. Get an API key at app.diffy.website/#/keys.

Env varworldParameters keyDefaultPurpose
DIFFY_API_KEYdiffy.apiKey-Diffy API key (required).
DIFFY_PROJECT_IDdiffy.projectId-Diffy project id (required).
DIFFY_BREAKPOINTSdiffy.breakpoints1200Comma list of breakpoint widths, e.g. 640,1200.
DIFFY_WINDOW_HEIGHTdiffy.windowHeight2000Viewport height used when resizing.
DIFFY_SCREENSHOTS_DIRdiffy.screenshotsDir(off)Also write captured screenshots to this folder.
DIFFY_API_BASE_URLdiffy.baseUrlhttps://app.diffy.website/api/API endpoint (point at a mock for offline runs).
DIFFY_MAX_WAITdiffy.maxWait1200Comparison polling timeout in seconds.
DIFFY_ENV1_URLdiffy.env1Url-URL for side 1 when comparing with custom.
DIFFY_ENV2_URLdiffy.env2Url-URL for side 2 when comparing with custom.

Tag Diffy scenarios (e.g. @diffy) and run them separately from the functional suite: npx cucumber-js --tags @diffy.

Back to all step groups