Skip to content

Local release and Pages smoke tests

Run from the repository root with act and Docker installed:

python3 tools/test_release_workflows.py run

The helper uses the installed act; it does not download another copy. The runner image is pinned by digest in the helper. Docker must be running and able to pull that image on the first run. Logs, event payloads, local artifacts and report.json are written under ignored build/artifacts/act-smoke-<UTC timestamp>/. Downloaded actions share build/artifacts/act-action-cache/ between runs.

The source snapshot is a disposable local clone of committed HEAD, with only the release/Pages/CI workflows, workflow guard tests and smoke helpers overlaid from the working tree. Other uncommitted installer/firmware work is deliberately excluded. The report records the snapshot's commit. Nothing changes the source repository's remotes, tags or history.

What runs

The Pages case builds the combined installer and MkDocs artifact, verifies local documentation links and confirms the install guide is present alongside retained firmware assets. Publication, Pages configuration and deployment remain disabled.

The real release and Pages workflows run inside act, with these local substitutions:

  • The reusable full CI call targets a generated fixture workflow that produces non-flashable ZIPs for every committed build preset. It runs the release, approval-helper, installer and workflow guard tests instead of compiling firmware or mobile apps. Production ci.yml stays present for workflow regression tests, including container lock checks.
  • The real pinned artifact upload/download actions move those ZIPs through act's local artifact server, followed by real release package-set validation.
  • A read-only gh shim supplies fixture Release listings and downloads for Pages. Every other gh operation fails. There is no real GitHub token, persisted host credential configuration or mounted Docker socket in the job containers.
  • Pages checks out the local snapshot without an explicit checkout ref, which avoids act requiring a token. The production workflow retains its exact SHA checkout.
  • Workflow steps guarded by !env.ACT skip release publication, Pages configuration and Pages deployment. The helper also rejects logs showing those steps executing.

Five scenarios must pass their expected outcome: stable tag, RC tag, rejection of an invalid tag, Pages after a successful release, and skipped Pages after a failed release. Pages verifies retained tag ordering and generated manifest/app/checksum paths. The fixtures cover four majors and four minors per major, with older patches, superseded RCs and numeric rc9/rc10 ordering on every line. The default window must retain three minors in each of three majors, with a stable patch and an eligible RC on each line. The assembled site must include the same-origin /espurr-hw/ hardware link.

S3 fixtures model the recovery bootstrap/normal OTA pair for each supported image family and flash capacity. Site verification checks every configured installer profile, reset/preserve/app-install manifests and their segment paths, the bootstrap ZIP, the matching normal app and both checksum links. S3 app payloads contain only a checksummed image descriptor and an explicit smoke marker, with entry point zero and no executable code. That exercises the normal-app identity validation used by the slot-A installer. The release/package tests separately exercise packaging rules with test images.

Fixture bytes are not firmware. Do not flash the local smoke artifacts.

act compatibility

act 0.2.94 rejects the unpadded base64url signatures generated by upload-artifact@v6 (upstream issue). The local fixture upload preloads tools/act_artifact_compat.cjs to restore signature padding only for requests to the local artifact service. Production action pins and the installed act binary remain unchanged. Remove the shim when the upstream fix lands and the pinned actions pass without it.

The fixture gh PATH override is scoped to the site-generation shell step because act can lose Node's PATH inside a subsequent composite action after a GITHUB_PATH update. This workaround also exists only in the disposable workflow.

These tests establish local workflow execution, artifact transfer, package validation and site assembly. They do not exercise GitHub's event delivery, hosted evaluator, repository rules, live PR approval state, release API writes, Pages OIDC/deployment, full target builds or physical flashing. Publication behavior and approval checks also have mocked API regression tests; validating those services end to end requires a separately authorised GitHub run.