Skip to content
RunSpec
Sign in

Documentation-led API testing

Check your API against what your docs say.

RunSpec reads your public documentation, writes repeatable tests, rehearses them independently, and records evidence from approved fixture runs. No source access to the API under test. No internal test suite.

Check public GET responses

Existing organization Owners can check unauthenticated public HTTPS GET responses against a supported OpenAPI 3.0 contract and optional criteria. Domain verification and review come before requests; only selected checks are covered.

New registrations are paused. Existing accounts can sign in; start a local evaluation or review execution requirements.

Example workflowapi.acme.comRunning
  1. Reading your documentationOpenAPI 3.1
  2. Planning what to test24 operations
  3. Writing the tests18 tests
  4. Rehearsing against a simulator0 calls to your API
  5. Running against your API46 requests
  6. Recording cleanup attempts6 cleanup attempts

Illustration only: these counts are not a live result. Real runs report failed cleanup and unresolved effects separately.

Why

Your tests and your API share the same blind spots.

Internal suites are written by the people who built the endpoint, against the same fixtures, the same client library, and the same assumptions. They pass. Meanwhile the customer holding only your published documentation gets a 500, a field that is not in the schema, or a workflow that stops halfway.

RunSpec only ever knows what a customer knows: your documentation, your public endpoints, the credentials you hand it, and the permissions you grant.

No source code
RunSpec never reads your repository, your CI, or your existing tests.
No shared fixtures
Every test is derived from your published contract, not from internal seed data.
No internal client
Requests use the same public surface your customers use. Runs against your own API aren't enabled yet; today they reach only the built-in example API.

Onboarding

Start from a link, not a project.

Point RunSpec at an OpenAPI document or a documentation URL. It reads, cites what it read, and then waits for you to confirm the contract and the target.

Example contractsnapshot 9f3c1e7a
  • GET /orders/{id}Returns 404 when no order matches the ID.docs.acme.com/api/orders#retrieve-an-order
  • POST /ordersRequests without an Idempotency-Key header are rejected.docs.acme.com/api/orders#create-an-order
Needs reviewConfirmation does not authorize execution.
  1. OpenAPI 3.0 and 3.1, JSON or YAML. If your docs are prose, RunSpec rebuilds a spec from them
  2. A spec rebuilt from prose cites the page and passage behind each claim
  3. Documentation pinned to a version and hash, so a run is reproducible
  4. Confirmation starts neither test generation nor target execution

Over time

Documentation moves. The tests follow, when you say so.

RunSpec re-reads your documentation and shows you what changed. Regeneration is scoped to the workflows actually affected, and only after you confirm it. Everything else keeps its history.

Example checkGET /orders
  1. Check history
  2. Documentation diff
  3. Assessed reasons and risks
  4. Scoped regeneration
limit:
maximum: 100
maximum: 50
page: integer
cursor: string
Behavior change2 of 9 workflows affected. Regenerate on confirm.

Start with a fixture. Inspect what actually passed.

Run a documented API you own locally, review the plan, and inspect the assertions and cleanup evidence. Runs against customer APIs aren't enabled yet, and new registrations are paused.

Open the local quickstart