# Test mode

Mock providers with fixed answers: build the pass and the fail path without spending a credit.

Keys that start with `sk_test_` are test keys. A test key sends the same requests as a live key, with the same headers and the same JSON bodies, and gets responses of the same shape. The difference is what answers: a mock provider with fixed answers instead of a real one. No credits move, no real provider is called, and the real pass rule still decides whether the result passed.

The mock providers are deterministic. The input decides the outcome, so your agent can build the pass path, the partial path and the fail path, and run each of them again and again with the same result.

> Before launch, keys go to design partners. Everyone else can [join the waitlist](/waitlist). The `@arettic/sdk` and `arettic` (PyPI) packages and the `@arettic/mcp` bridge are published at launch; the samples on this page are written against their source.

## Get a test key

Every agent has one mode, test or live, chosen when the agent is made. The mode can't be changed later; make a second agent for the other mode. The key is shown once.

- In the dashboard: sign in at [/app](/app), open the [Agents page](/app/agents), add an agent and pick Test. The key appears on the page right after you add it.
- From the org API: `POST /v1/orgs/{orgId}/agents` with a `name` and `mode: "test"` (`mode` defaults to `test`). The response carries the key in `key`, once, with a `key_note`. Org keys (`ok_…`) act as the owner who made them; see [authentication](/docs/authentication) and the [org API](/docs/org-api).

**curl**

```bash
curl https://api.arettic.com/v1/orgs/$ORG_ID/agents \
  -H "Authorization: Bearer $ARETTIC_ORG_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "name": "Test agent", "mode": "test" }'
```

Put the test key in the `ARETTIC_API_KEY` environment variable. Both SDKs and the MCP bridge read it, and every request sends it as `Authorization: Bearer sk_test_…`. The examples below assume that variable holds a test key.

## What a test key changes

- `POST /v1/recommend` returns only mock tools, one per task type, and says so with `test_mode: true`.
- `POST /v1/execute` runs the mock provider for the tool's task type, applies the real pass rule with the same `check_version` as live, and answers with `test_mode: true`, a `charged` of 0 and `would_have_charged`: the price a live call would have paid.
- The input is validated before the call, the same way as live, except that domains are not looked up in DNS. Made-up domains like `example.com` or `acme.catchall.test` are fine.
- A fail withholds the result, in test mode too. You get the `reason` and nothing else, which is exactly what a live fail looks like.
- A live key can't reach a mock tool: it gets `404 unknown_tool`. A test key can name any tool id from the catalog, mock or real. The mock for that tool's task type answers, and `would_have_charged` uses that tool's price.

## The mock tools

There is one mock tool per task type. Its id is `mock-` plus the task type with underscores turned into hyphens. The input fields are the published ones for the task type; a request missing one is rejected as `invalid_input` before anything runs.

The mock tools and their input fields

| Tool id | Task type | Input |
|---|---|---|
| mock-find-email | find_email | `first_name`: string; `last_name`: string; `domain`: company domain, e.g. acme.com |
| mock-verify-email | verify_email | `email`: string |
| mock-enrich-company | enrich_company | `domain`: company domain; `name`: company name (if no domain); `country`: optional, with name |
| mock-enrich-person | enrich_person | `first_name`: string; `last_name`: string; `company_domain`: company domain |
| mock-web-search | web_search | `query`: string, up to 500 characters; `n`: 1–25, default 5 |
| mock-extract-url | extract_url | `url`: http(s) URL |

List them without a key: `GET https://api.arettic.com/v1/tools?mode=test`. Each has a `price_per_success`, which is what `would_have_charged` reports on a pass, and `test_mode: true`. `GET https://api.arettic.com/v1/tools/mock-find-email` shows one in detail. Without `mode=test`, mock tools never appear.

**curl**

```bash
curl "https://api.arettic.com/v1/tools?mode=test"
```

**Response (example, one of the tools shown)**

```json
{
  "tools": [
    {
      "tool_id": "mock-find-email",
      "name": "Mock Email Finder",
      "provider": "Arettic Mock Provider",
      "task_type": "find_email",
      "regions": ["GLOBAL", "US"],
      "purchasable": true,
      "price_per_success": { "credits": "38", "usd": "0.038" },
      "price_valid_from": null,
      "pass_rule": "find_email@v1",
      "scores": [{ "region": "GLOBAL", "score": 56.53, "week": "2026-09-28", "sample_size": 10 }],
      "test_mode": true
    }
  ],
  "license": {
    "id": "CC-BY-4.0",
    "url": "https://creativecommons.org/licenses/by/4.0/",
    "attribution": "Arettic (arettic.com)"
  },
  "generated_at": "2026-09-29T18:00:00.180Z"
}
```

### What each mock returns on a pass

Use these shapes in your assertions. Placeholders in angle brackets come from your input.

The fixture each mock tool returns when the input has no fail trigger

| Tool id | Result |
|---|---|
| mock-find-email | `{ "email": "<first>.<last>@<domain>", "verification_status": "valid" }`. Names are lowercased and stripped to letters. |
| mock-verify-email | `{ "email": "<email>", "status": "valid" }`, lowercased. |
| mock-enrich-company | `{ "name": "<name>", "domain": "<domain>", "employee_range": "51-200", "industry": "Software", "country": "<country>" }`. `name` is your `name`, or the first label of the domain with a capital letter. `country` is your `country`, or `US`. |
| mock-enrich-person | `{ "name": "<first> <last>", "company_domain": "<company_domain>", "title": "Head of Marketing", "email": "<first>@<company_domain>", "linkedin_url": "https://www.linkedin.com/in/<first>-mock" }` |
| mock-web-search | `{ "results": [ { "url": "https://example.test/<query>/<k>", "title": "Result <k> for <query>", "snippet": "Mock snippet <k>." } ] }` with `n` results (default 5). The URL uses the first 20 characters of the query, URL-encoded. |
| mock-extract-url | `{ "url": "<url>", "status_code": 200, "title": "Mock page", "content": "<a fixed paragraph of over 200 characters>" }` |

## A full request and response

Ask which tool to use, then run it. With a test key the first option for `find_email` is `mock-find-email`, and the response says `test_mode: true`. Everything else in that response is as described on the [recommend](/docs/recommend) page.

**curl: recommend**

```bash
curl https://api.arettic.com/v1/recommend \
  -H "Authorization: Bearer $ARETTIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "task_type": "find_email" }'
```

**curl: execute**

```bash
curl https://api.arettic.com/v1/execute \
  -H "Authorization: Bearer $ARETTIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "tool_id": "mock-find-email",
    "input": { "first_name": "Emily", "last_name": "Carter", "domain": "example.com" }
  }'
```

**Response (example)**

```json
{
  "test_mode": true,
  "tool_id": "mock-find-email",
  "task_type": "find_email",
  "check_version": "v1",
  "charged": { "credits": "0", "usd": "0.000" },
  "status": "passed",
  "result": { "email": "emily.carter@example.com", "verification_status": "valid" },
  "would_have_charged": { "credits": "38", "usd": "0.038" }
}
```

The same call with the SDKs. Both clients read `ARETTIC_API_KEY` (and `ARETTIC_API_URL`, if set) from the environment when you pass nothing.

**TypeScript**

```ts
import { Arettic } from "@arettic/sdk";

const arettic = new Arettic(); // reads ARETTIC_API_KEY, here a test key

const r = await arettic.execute({
  tool_id: "mock-find-email",
  input: { first_name: "Emily", last_name: "Carter", domain: "example.com" },
});

if ("test_mode" in r && r.status === "passed") {
  console.log(r.result); // { email: "emily.carter@example.com", verification_status: "valid" }
  console.log(r.would_have_charged.credits); // "38"
}
```

**Python**

```python
from arettic import Arettic

arettic = Arettic()  # reads ARETTIC_API_KEY, here a test key

r = arettic.execute(
    tool_id="mock-find-email",
    input={"first_name": "Emily", "last_name": "Carter", "domain": "example.com"},
)
print(r["status"])  # passed
if r["status"] == "passed":
    print(r["result"])  # {'email': 'emily.carter@example.com', 'verification_status': 'valid'}
    print(r["would_have_charged"]["credits"])  # 38
```

Over MCP, put the test key in your client's config (`ARETTIC_API_KEY=sk_test_…`; the client configs are on the [MCP](/docs/mcp) page) and call the `execute` tool with the same arguments. The tool result is the same JSON as above, as text content. `get_receipt` and `open_dispute` need a live key.

**MCP tool call: execute**

```json
{
  "name": "execute",
  "arguments": {
    "tool_id": "mock-find-email",
    "input": { "first_name": "Emily", "last_name": "Carter", "domain": "example.com" }
  }
}
```

## The response, field by field

Fields of a test-mode execute response for one input

| Field | Present | Meaning |
|---|---|---|
| test_mode | always | `true`. A live response never has this field. |
| status | always | `passed`, `partial` or `failed`. |
| tool_id | always | The tool's id (its slug), even if you sent its UUID. |
| task_type | always | The task type the mock ran and the rule that judged it. |
| check_version | always | The version of the pass rule that was applied, the same as live (`v1` today). |
| charged | always | Always `{ "credits": "0", "usd": "0.000" }`. |
| result | passed, partial | The mock's data. Withheld on a fail. |
| reason | failed, partial | A reason code from the pass rule, such as `catch_all`, `field_missing:industry` or `results:2/5`, or `provider_error` when the mock simulated an outage. The codes are listed on the [pass rules](/docs/pass-rules) page. |
| would_have_charged | always | The tool's `price_per_success` on a pass. The price times the fraction, rounded up to a whole credit, on a partial. 0 on a fail or a provider error. |
| receipt_id, execution_id, refunded | never | Test calls write no receipt and hold no credits, so a live response's settlement fields are absent. |

## Make a mock fail on purpose

Put these values in the input and the mock returns data that fails its check, or simulates an outage. The pass rule then reports the same reason it would report live. Anything not listed here passes.

Inputs that make each mock tool fail, and what comes back

| Tool id | Input | What comes back |
|---|---|---|
| any mock tool | any field contains `provider-error` | `failed`, reason `provider_error`, `would_have_charged` 0. The provider is simulated as down. |
| any mock tool | any field contains `nomatch` | `failed`, reason `no_result`: the provider returned nothing. Two rules name the gap differently: `mock-verify-email` reports `status:unknown` and `mock-extract-url` reports `http:none`. |
| mock-find-email | `domain` ends in `.catchall.test` | `failed`, reason `catch_all`. |
| mock-verify-email | `email` starts with `unknown` | `failed`, reason `status:unknown`: the verifier could not decide. |
| mock-verify-email | `email` starts with `bad` | `passed`, with `status: "invalid"` in the result. A definite invalid is a true answer and would be charged. |
| mock-enrich-company | `domain` is `missing-fields.test` | `failed`, reason `field_missing:industry`. |
| mock-enrich-person | `company_domain` contains `nocontact` | `failed`, reason `field_missing:contact`: a title but no email, phone or LinkedIn URL. |
| mock-web-search | `query` contains `few results` | `partial`, reason `results:2/5`: 2 results instead of `n` (default 5). `would_have_charged` is the price times 2/5, rounded up. With `n` of 1 or 2 it passes. |
| mock-extract-url | `url` contains `blocked` | `failed`, reason `blocked_page`: a captcha page came back. |

A simulated provider error is a normal `200` response with `status: "failed"` and `reason: "provider_error"`, not an error envelope. Live, a provider that still fails after one retry is never charged either.

**curl: a fail**

```bash
curl https://api.arettic.com/v1/execute \
  -H "Authorization: Bearer $ARETTIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "tool_id": "mock-find-email",
    "input": { "first_name": "Emily", "last_name": "Carter", "domain": "example.catchall.test" }
  }'
```

**Response (example)**

```json
{
  "test_mode": true,
  "tool_id": "mock-find-email",
  "task_type": "find_email",
  "check_version": "v1",
  "charged": { "credits": "0", "usd": "0.000" },
  "status": "failed",
  "reason": "catch_all",
  "would_have_charged": { "credits": "0", "usd": "0.000" }
}
```

**curl: a partial**

```bash
curl https://api.arettic.com/v1/execute \
  -H "Authorization: Bearer $ARETTIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "tool_id": "mock-web-search", "input": { "query": "few results please" } }'
```

**Response (example)**

```json
{
  "test_mode": true,
  "tool_id": "mock-web-search",
  "task_type": "web_search",
  "check_version": "v1",
  "charged": { "credits": "0", "usd": "0.000" },
  "status": "partial",
  "reason": "results:2/5",
  "result": {
    "results": [
      {
        "url": "https://example.test/few%20results%20please/1",
        "title": "Result 1 for few results please",
        "snippet": "Mock snippet 1."
      },
      {
        "url": "https://example.test/few%20results%20please/2",
        "title": "Result 2 for few results please",
        "snippet": "Mock snippet 2."
      }
    ]
  },
  "would_have_charged": { "credits": "4", "usd": "0.004" }
}
```

## Batches with a test key

Send `inputs` instead of `input`: an array of up to 1,000 objects. With a test key every batch runs at once and returns per-item results, even over 25 items. No job is created, so the response is never `queued`, and `GET /v1/jobs/{id}` has nothing to show for a test key. Live, a batch over 25 items becomes a job; see [jobs](/docs/jobs).

Each input is validated first. If any input is invalid the whole request is rejected with `invalid_input`, the same as live.

**curl**

```bash
curl https://api.arettic.com/v1/execute \
  -H "Authorization: Bearer $ARETTIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "tool_id": "mock-verify-email",
    "inputs": [{ "email": "emily@example.com" }, { "email": "unknown@example.com" }]
  }'
```

**Response (example)**

```json
{
  "test_mode": true,
  "status": "completed",
  "tool_id": "mock-verify-email",
  "charged": { "credits": "0", "usd": "0.000" },
  "would_have_charged": { "credits": "6", "usd": "0.006" },
  "summary": { "items": 2, "passed": 1, "partial": 0, "failed": 1 },
  "items": [
    {
      "index": 0,
      "status": "passed",
      "result": { "email": "emily@example.com", "status": "valid" },
      "would_have_charged": { "credits": "6", "usd": "0.006" }
    },
    {
      "index": 1,
      "status": "failed",
      "reason": "status:unknown",
      "would_have_charged": { "credits": "0", "usd": "0.000" }
    }
  ]
}
```

`would_have_charged` at the top is the sum over the items. `summary` counts the items by status. Items carry `index`, `status`, `result` on a pass or partial, `reason` on a fail or partial, and their own `would_have_charged`. They carry no `charged` field, because nothing was charged.

**TypeScript**

```ts
const batch = await arettic.execute({
  tool_id: "mock-verify-email",
  inputs: [{ email: "emily@example.com" }, { email: "unknown@example.com" }],
});

if ("test_mode" in batch && batch.status === "completed") {
  console.log(batch.summary); // { items: 2, passed: 1, partial: 0, failed: 1 }
  for (const item of batch.items) console.log(item.index, item.status, item.reason);
}
```

**Python**

```python
batch = arettic.execute(
    tool_id="mock-verify-email",
    inputs=[{"email": "emily@example.com"}, {"email": "unknown@example.com"}],
)
print(batch["summary"])  # {'items': 2, 'passed': 1, 'partial': 0, 'failed': 1}
for item in batch["items"]:
    print(item["index"], item["status"], item.get("reason"))
```

## What test mode does not do

- No receipts. There is no `receipt_id`, `execution_id` or `refunded` in the response, and `GET /v1/receipts` lists live purchases only.
- No disputes. `POST /v1/disputes` with a test key answers `400 invalid_input`: test-mode purchases are free, so there is nothing to dispute.
- No jobs. Batches run inline, and `GET /v1/jobs/{id}` answers `404 not_found` for a test key.
- No holds, budgets or approvals. Nothing is reserved, so the agent's monthly budget and approval threshold are not checked, and you never see `approval_required`, `insufficient_credits`, `over_budget`, `price_above_max` or `trial_limit`. `max_price`, `approval_id`, `fallback` and `idempotency_key` are accepted and ignored.
- No replay. There is nothing to replay: a repeated request runs the mock again and gets the same answer, because the mock is deterministic.
- No DNS check. Live, a domain that definitely does not exist is rejected before the call. Test mode checks only the shape of the input.
- No real data. Every result is a fixture. Never treat a mock result as a fact about a real company or person.
- Still on: the input check, the agent key's rate limit and IP allowlist, and the plan's daily lookup quota for `recommend`. `GET /v1/balance` works and shows the org's credits; a test key never changes them. See [rate limits](/docs/rate-limits).

## Errors you can get with a test key

Error codes a test key can receive from execute

| Code | HTTP | When |
|---|---|---|
| invalid_input | 400 | `tool_id` missing, `input` not an object, `inputs` empty or over 1,000, or a field fails the task type's check. The message names the field, for example `input.email: must be an email address. Nothing was charged.` See [invalid_input](/docs/errors#invalid_input). |
| unknown_tool | 404 | No tool with that id. A live key naming a mock tool gets this too. See [unknown_tool](/docs/errors#unknown_tool). |
| unauthenticated | 401 | No `Authorization: Bearer` header, or a revoked key. See [unauthenticated](/docs/errors#unauthenticated). |
| ip_not_allowed | 403 | The agent has an IP allowlist and the call came from elsewhere. See [ip_not_allowed](/docs/errors#ip_not_allowed). |
| rate_limited | 429 | Too many calls from this agent. Wait for `Retry-After`. See [rate_limited](/docs/errors#rate_limited). |

Every error has the same envelope: `{ "error": { "code", "message", "doc_url", "retryable" } }`. `doc_url` points at the code's entry on the [error codes](/docs/errors) page.

## Mock tools on the scores page

Mock tools have scores. They are published on [/scores](/scores) under the heading "Test mode (mock providers)" and at `GET https://api.arettic.com/v1/scores?mode=test`. They come from the same weekly scoring pipeline as live scores, from benchmark runs against mock test sets that ship with the platform. A few expected answers in those sets differ from what the mock returns, on purpose, so accuracy sits realistically below 100%.

These scores prove the scoring pipeline end to end. They say nothing about any real provider. Mock tools never appear in the live catalog, the live scores, `GET /v1/pricing` or a live key's recommendations. How scores are computed is on the [scores](/docs/scores) page.

## Switch to a live key

Change the key from `sk_test_…` to `sk_live_…` and keep the code. Then:

- `recommend` returns real tools. Pick the first option with `purchasable: true`.
- `execute` returns `receipt_id`, `execution_id`, `charged` and `refunded`, and no `test_mode` or `would_have_charged`.
- Credits are held before the call and settled after the check: charged on a pass, pro rata on a partial, released on a fail.
- Budgets, approvals, idempotency keys, `max_price` and `fallback` all take effect.
- A batch over 25 items runs as a job.
- Charged items can be disputed within 7 days.

Next: [Quickstart](/docs/quickstart), [Execute](/docs/execute), [Receipts](/docs/receipts), [Budgets and approvals](/docs/budgets-and-approvals), [Jobs](/docs/jobs), [MCP](/docs/mcp), [SDKs](/docs/sdks).

Updated 2026-09-29. This page as HTML: https://arettic.com/docs/test-mode · Markdown: https://arettic.com/docs/test-mode.md · JSON: https://arettic.com/docs/test-mode.json