A first run you can inspect.

Start / Check your API contract

Use an API you control and are authorized to test. Its target must be publicly accessible over HTTPS. No model key or generated test code is needed for this flow.

1. Sign in and choose your organization. An organization Owner can create and run checks. Open Contract checks in the hosted portal.
2. Add the API. Enter a check name and target URL, then paste one self-contained OpenAPI 3.0 JSON document. Choose Preview contract probes. RunSpec lists supported GET operations and explains exclusions; authenticated endpoints, writes, external references and unsupported schemas are not included.
3. Select requests and provide known inputs. Select up to ten GET requests and supply any required path or query values. You can require a documented HTTP status or add an explicit response-value criterion for a documented status. These are your expectations, not inferred business rules. Choose Validate inputs and freeze plan; values are omitted from the review.
4. Verify the target domain. In Domain control, choose Issue fresh DNS challenge. Add the exact TXT record name and value shown to your domain's DNS, then choose Check DNS after propagation. Choose Refresh domain proof and exact review. Verification establishes domain control; you still need permission to test the API. A fresh challenge, revocation or expiry requires a new review.
5. Review and run. Check the selected requests, documented responses, optional criteria, exclusions and limits. The permission checkbox starts unchecked. Select it only when you authorize those exact requests, then choose Approve exact checks and run. Open the recorded-run link to return to the same evidence later.

Each run allows at most ten requests, five seconds per request and thirty seconds overall, with a bounded response size. Unselected operations remain unchecked. Unsupported or precision-losing numeric inputs are rejected instead of silently changing your test.

Read the result

• Documented response observed: the observed success status is documented and its supported body schema matched, when one was provided. A missing schema leaves body conformance unverified.
• Contract mismatch: the observed status or supported response schema did not match the document. A schema mismatch may identify the documented field; response values stay hidden.
• Owner expectation mismatch: an optional status or value you chose did not match. This does not automatically mean the API violated its documentation.
• Success path not exercised: a documented error response was observed. That can be valid without verifying the operation's success path. If you explicitly selected that error status and its documented schema (when supplied) and applicable value criteria match, the selected check can pass; the success path remains unexercised.
• Inconclusive, cancelled or no result: the run did not establish a complete verified result. Missing evidence is not a pass.

Reports omit response bodies, expanded request URLs and expected literal values. These finite checks do not prove all business behavior or complete API coverage. For access questions, contact Runspec Support without sending credentials or customer payloads.

01 / Local quickstart

Work from the RunSpec repository root with Bun 1.4 and Docker running. Configure the local database using the supplied .env.example; preserve any existing secrets. If you need repository access, contact Runspec Support.

bun run dev:setup
bun run check

Setup installs pinned dependencies, starts local PostgreSQL, generates its client and applies forward-only migrations. It does not require a model key or reset the database. The deterministic checks do not call a model.

Start the fixture stack without a model key

On a qualified Linux development host with rootless Podman, use the core profile:

RUNSPEC_AUTH_MODE=local bun run dev --profile core

On macOS, or for explicitly trusted fixture-only development, select the unsafe local-process provider:

RUNSPEC_AUTH_MODE=local RUNSPEC_EXECUTION_PROVIDER=local-process bun run dev --profile core

Local-process execution is not isolation. Use this opt-out only for trusted local fixture testing. It must never run customer workloads. The core profile excludes the generation worker and does not read a model-key file.

The service profile does not change authentication. Firebase is the default and fails closed until configured. RUNSPEC_AUTH_MODE=local explicitly selects trusted fixture development only; never use it on an exposed server, and production rejects it. For account-based development, configure the dedicated Firebase project and exact trusted portal origins instead of enabling local mode.

Submit one owned fixture run

Keep that stack running. In a second terminal:

curl --fail-with-body --request POST \
  --header 'Idempotency-Key: local-fixture-first-run' \
  http://127.0.0.1:3000/v1/fixture/runs

Open the local Runs view and select the returned run ID. The same submission key recovers the same run; use a new key only when you intend to start another run. Fixture submissions use the fixed, owned pack and do not call a model.

Stop the stack with Ctrl+C. PostgreSQL remains running; bun run docker:down stops it separately while retaining its volume. Do not run mvp:demo alongside this stack: both own the fixture port.

02 / Supported documents

For the separate generation/discovery workflow, start from one self-contained OpenAPI 3.0 or 3.1 document in JSON or YAML. A direct specification URL works; bounded discovery can also inspect an HTML documentation page for specification links and common OpenAPI locations.

• Discovery URLs must be anonymous HTTP(S) URLs. A short query that selects the document (such as ?format=json) is kept; one that could carry a credential is refused. A #fragment is ignored.
• When no specification is found, bounded same-origin HTML/Markdown and llms.txt collection can reconstruct OpenAPI with a documentation-extractor agent. This requires the generation worker and incurs model usage charges. Direct OpenAPI discovery does not need a model key.
• Prose reconstruction retains immutable pages, exact-source citations and explicit uncertainty for review. Citations establish traceability; they do not prove the inferred API is correct. Missing pages or exceeded limits block extraction rather than silently produce complete coverage.
• JavaScript rendering, authenticated documentation, cross-origin crawling and combining external OpenAPI references are not supported. The bounded collector has page, size and deadline limits; it is not an unrestricted web crawler.
• Discovery presents the source, API version, documented targets, authentication declarations and operation count for review. It does not silently select a target.
• Select a target declared in the specification. A missing or incorrect deployment URL requires a corrected specification or explicit manual suite setup.

bun run discover start http://127.0.0.1:4010/openapi.json --wait
bun run discover show DISCOVERY_ID

Replace DISCOVERY_ID with the returned ID. In the local portal, choose Discover API to review and confirm the same information. Confirmation creates the suite; it grants no execution permission.

03 / Generate, then review execution

Generation in this local evaluation requires your own model-provider key and incurs provider usage charges. Configure OPENAI_API_KEY or the launcher's OPENAI_API_KEY_FILE privately; never place it in a browser, documentation URL or test pack. Stop the core stack before restarting the full profile.

RUNSPEC_AUTH_MODE=local bun run dev --profile full

The development execution provider remains a separate choice. The unsafe local-process option above remains fixture-only; enabling generation does not qualify it for customer workloads. Generations queued in the core profile cannot advance until a generation worker starts.

Retain the explicit local-process provider setting from the macOS fixture command if you are using it. For authenticated development, retain Firebase mode and configured account credentials instead. The full profile selects services; it does not waive authentication, authority review or isolation requirements.

1. Confirm the documentation, target and environment. Start generation as a separate action.
2. Inspect the generated pack, diagnostics, eligible operation coverage and explicit exclusions. Promotion means the trusted offline checks accepted the pack; it does not mean a real API passed.
3. Configure required inputs and logical actor credentials for an executable fixture suite. Saved values are encrypted; read APIs show configured names rather than secret values.
4. Choose Review and run tests. Review the exact pack, target, possible writes, request ceiling, timeout and cleanup plan before authorizing execution.
5. Approval can be reused only for the same generation, target and policy until revoked. Changes require new review. Cancelling review submits no run.

Customer targets for generated-code execution remain blocked in this release. Do not enter customer credentials merely to evaluate generation. Documentation changes also block an old pack: checking changes, regenerating tests and approving execution are separate actions.

04 / Read the evidence

Use the overall suite result first, then inspect individual batches, tests and cleanup. A passing batch is not the result of the complete suite.

Report retrieval failures are shown explicitly and can be retried. When a terminal record lacks per-step history, the progress view says completion is unverified rather than animating a finished job or inventing passes.

05 / Troubleshooting

• Queued generation: the core profile intentionally omits the generation worker. Start the full profile with a privately configured key when you want paid generation.
• Run waiting in the queue: RunSpec runs one test run at a time. A queued run shows why it is waiting under its status: another run is in progress, the previous batch has not finished, or another run needs attention or is being recovered. Resolve that run in Recovery and queued runs continue. If no reason is shown, check the workers below.
• Queued execution: verify the execution worker and private gateway are running. The worker does not bypass the gateway when it is unavailable.
• Port conflict: stop another local stack or standalone fixture demo. Startup checks ports before launching services.
• Database or migration error: run bun run dev:setup against your local database. Do not reset existing product records to clear an error.
• Lost submission response: retry the same action. The portal retains a tab-local submission identity through review reopening or reload, preventing a second job for the same uncertain submission.
• Cleanup needs attention: inspect the recovery evidence. Cancellation stops further work while preserving cleanup; it does not undo all writes or guarantee resource removal.