Test mode

Mock providers with fixed answers: build the pass and the fail path without spending a credit.

Keys that start with sk_test_ are test keys. A test key sends the same requests as a live key, with the same headers and the same JSON bodies, and gets responses of the same shape. The difference is what answers: a mock provider with fixed answers instead of a real one. No credits move, no real provider is called, and the real pass rule still decides whether the result passed.

The mock providers are deterministic. The input decides the outcome, so your agent can build the pass path, the partial path and the fail path, and run each of them again and again with the same result.

Before launch, keys go to design partners. Everyone else can join the waitlist. The @arettic/sdk and arettic (PyPI) packages and the @arettic/mcp bridge are published at launch; the samples on this page are written against their source.

Get a test key

Every agent has one mode, test or live, chosen when the agent is made. The mode can't be changed later; make a second agent for the other mode. The key is shown once.

• In the dashboard: sign in at /app, open the Agents page, add an agent and pick Test. The key appears on the page right after you add it.
• From the org API: POST /v1/orgs/{orgId}/agents with a name and mode: "test" (mode defaults to test). The response carries the key in key, once, with a key_note. Org keys (ok_…) act as the owner who made them; see authentication and the org API.

curl https://api.arettic.com/v1/orgs/$ORG_ID/agents \
  -H "Authorization: Bearer $ARETTIC_ORG_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "name": "Test agent", "mode": "test" }'

Put the test key in the ARETTIC_API_KEY environment variable. Both SDKs and the MCP bridge read it, and every request sends it as Authorization: Bearer sk_test_…. The examples below assume that variable holds a test key.

What a test key changes

• POST /v1/recommend returns only mock tools, one per task type, and says so with test_mode: true.
• POST /v1/execute runs the mock provider for the tool's task type, applies the real pass rule with the same check_version as live, and answers with test_mode: true, a charged of 0 and would_have_charged: the price a live call would have paid.
• The input is validated before the call, the same way as live, except that domains are not looked up in DNS. Made-up domains like example.com or acme.catchall.test are fine.
• A fail withholds the result, in test mode too. You get the reason and nothing else, which is exactly what a live fail looks like.
• A live key can't reach a mock tool: it gets 404 unknown_tool. A test key can name any tool id from the catalog, mock or real. The mock for that tool's task type answers, and would_have_charged uses that tool's price.

The mock tools

There is one mock tool per task type. Its id is mock- plus the task type with underscores turned into hyphens. The input fields are the published ones for the task type; a request missing one is rejected as invalid_input before anything runs.

List them without a key: GET https://api.arettic.com/v1/tools?mode=test. Each has a price_per_success, which is what would_have_charged reports on a pass, and test_mode: true. GET https://api.arettic.com/v1/tools/mock-find-email shows one in detail. Without mode=test, mock tools never appear.

curl "https://api.arettic.com/v1/tools?mode=test"

{
  "tools": [
    {
      "tool_id": "mock-find-email",
      "name": "Mock Email Finder",
      "provider": "Arettic Mock Provider",
      "task_type": "find_email",
      "regions": ["GLOBAL", "US"],
      "purchasable": true,
      "price_per_success": { "credits": "38", "usd": "0.038" },
      "price_valid_from": null,
      "pass_rule": "find_email@v1",
      "scores": [{ "region": "GLOBAL", "score": 56.53, "week": "2026-09-28", "sample_size": 10 }],
      "test_mode": true
    }
  ],
  "license": {
    "id": "CC-BY-4.0",
    "url": "https://creativecommons.org/licenses/by/4.0/",
    "attribution": "Arettic (arettic.com)"
  },
  "generated_at": "2026-09-29T18:00:00.180Z"
}

What each mock returns on a pass

Use these shapes in your assertions. Placeholders in angle brackets come from your input.

A full request and response

Ask which tool to use, then run it. With a test key the first option for find_email is mock-find-email, and the response says test_mode: true. Everything else in that response is as described on the recommend page.

curl https://api.arettic.com/v1/recommend \
  -H "Authorization: Bearer $ARETTIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "task_type": "find_email" }'

curl https://api.arettic.com/v1/execute \
  -H "Authorization: Bearer $ARETTIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "tool_id": "mock-find-email",
    "input": { "first_name": "Emily", "last_name": "Carter", "domain": "example.com" }
  }'

{
  "test_mode": true,
  "tool_id": "mock-find-email",
  "task_type": "find_email",
  "check_version": "v1",
  "charged": { "credits": "0", "usd": "0.000" },
  "status": "passed",
  "result": { "email": "[email protected]", "verification_status": "valid" },
  "would_have_charged": { "credits": "38", "usd": "0.038" }
}

The same call with the SDKs. Both clients read ARETTIC_API_KEY (and ARETTIC_API_URL, if set) from the environment when you pass nothing.

import { Arettic } from "@arettic/sdk";

const arettic = new Arettic(); // reads ARETTIC_API_KEY, here a test key

const r = await arettic.execute({
  tool_id: "mock-find-email",
  input: { first_name: "Emily", last_name: "Carter", domain: "example.com" },
});

if ("test_mode" in r && r.status === "passed") {
  console.log(r.result); // { email: "[email protected]", verification_status: "valid" }
  console.log(r.would_have_charged.credits); // "38"
}

from arettic import Arettic

arettic = Arettic()  # reads ARETTIC_API_KEY, here a test key

r = arettic.execute(
    tool_id="mock-find-email",
    input={"first_name": "Emily", "last_name": "Carter", "domain": "example.com"},
)
print(r["status"])  # passed
if r["status"] == "passed":
    print(r["result"])  # {'email': '[email protected]', 'verification_status': 'valid'}
    print(r["would_have_charged"]["credits"])  # 38

Over MCP, put the test key in your client's config (ARETTIC_API_KEY=sk_test_…; the client configs are on the MCP page) and call the execute tool with the same arguments. The tool result is the same JSON as above, as text content. get_receipt and open_dispute need a live key.

{
  "name": "execute",
  "arguments": {
    "tool_id": "mock-find-email",
    "input": { "first_name": "Emily", "last_name": "Carter", "domain": "example.com" }
  }
}

The response, field by field

Make a mock fail on purpose

Put these values in the input and the mock returns data that fails its check, or simulates an outage. The pass rule then reports the same reason it would report live. Anything not listed here passes.

A simulated provider error is a normal 200 response with status: "failed" and reason: "provider_error", not an error envelope. Live, a provider that still fails after one retry is never charged either.

curl https://api.arettic.com/v1/execute \
  -H "Authorization: Bearer $ARETTIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "tool_id": "mock-find-email",
    "input": { "first_name": "Emily", "last_name": "Carter", "domain": "example.catchall.test" }
  }'

{
  "test_mode": true,
  "tool_id": "mock-find-email",
  "task_type": "find_email",
  "check_version": "v1",
  "charged": { "credits": "0", "usd": "0.000" },
  "status": "failed",
  "reason": "catch_all",
  "would_have_charged": { "credits": "0", "usd": "0.000" }
}

curl https://api.arettic.com/v1/execute \
  -H "Authorization: Bearer $ARETTIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "tool_id": "mock-web-search", "input": { "query": "few results please" } }'

{
  "test_mode": true,
  "tool_id": "mock-web-search",
  "task_type": "web_search",
  "check_version": "v1",
  "charged": { "credits": "0", "usd": "0.000" },
  "status": "partial",
  "reason": "results:2/5",
  "result": {
    "results": [
      {
        "url": "https://example.test/few%20results%20please/1",
        "title": "Result 1 for few results please",
        "snippet": "Mock snippet 1."
      },
      {
        "url": "https://example.test/few%20results%20please/2",
        "title": "Result 2 for few results please",
        "snippet": "Mock snippet 2."
      }
    ]
  },
  "would_have_charged": { "credits": "4", "usd": "0.004" }
}

Batches with a test key

Send inputs instead of input: an array of up to 1,000 objects. With a test key every batch runs at once and returns per-item results, even over 25 items. No job is created, so the response is never queued, and GET /v1/jobs/{id} has nothing to show for a test key. Live, a batch over 25 items becomes a job; see jobs.

Each input is validated first. If any input is invalid the whole request is rejected with invalid_input, the same as live.

curl https://api.arettic.com/v1/execute \
  -H "Authorization: Bearer $ARETTIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "tool_id": "mock-verify-email",
    "inputs": [{ "email": "[email protected]" }, { "email": "[email protected]" }]
  }'

{
  "test_mode": true,
  "status": "completed",
  "tool_id": "mock-verify-email",
  "charged": { "credits": "0", "usd": "0.000" },
  "would_have_charged": { "credits": "6", "usd": "0.006" },
  "summary": { "items": 2, "passed": 1, "partial": 0, "failed": 1 },
  "items": [
    {
      "index": 0,
      "status": "passed",
      "result": { "email": "[email protected]", "status": "valid" },
      "would_have_charged": { "credits": "6", "usd": "0.006" }
    },
    {
      "index": 1,
      "status": "failed",
      "reason": "status:unknown",
      "would_have_charged": { "credits": "0", "usd": "0.000" }
    }
  ]
}

would_have_charged at the top is the sum over the items. summary counts the items by status. Items carry index, status, result on a pass or partial, reason on a fail or partial, and their own would_have_charged. They carry no charged field, because nothing was charged.

const batch = await arettic.execute({
  tool_id: "mock-verify-email",
  inputs: [{ email: "[email protected]" }, { email: "[email protected]" }],
});

if ("test_mode" in batch && batch.status === "completed") {
  console.log(batch.summary); // { items: 2, passed: 1, partial: 0, failed: 1 }
  for (const item of batch.items) console.log(item.index, item.status, item.reason);
}

batch = arettic.execute(
    tool_id="mock-verify-email",
    inputs=[{"email": "[email protected]"}, {"email": "[email protected]"}],
)
print(batch["summary"])  # {'items': 2, 'passed': 1, 'partial': 0, 'failed': 1}
for item in batch["items"]:
    print(item["index"], item["status"], item.get("reason"))

What test mode does not do

• No receipts. There is no receipt_id, execution_id or refunded in the response, and GET /v1/receipts lists live purchases only.
• No disputes. POST /v1/disputes with a test key answers 400 invalid_input: test-mode purchases are free, so there is nothing to dispute.
• No jobs. Batches run inline, and GET /v1/jobs/{id} answers 404 not_found for a test key.
• No holds, budgets or approvals. Nothing is reserved, so the agent's monthly budget and approval threshold are not checked, and you never see approval_required, insufficient_credits, over_budget, price_above_max or trial_limit. max_price, approval_id, fallback and idempotency_key are accepted and ignored.
• No replay. There is nothing to replay: a repeated request runs the mock again and gets the same answer, because the mock is deterministic.
• No DNS check. Live, a domain that definitely does not exist is rejected before the call. Test mode checks only the shape of the input.
• No real data. Every result is a fixture. Never treat a mock result as a fact about a real company or person.
• Still on: the input check, the agent key's rate limit and IP allowlist, and the plan's daily lookup quota for recommend. GET /v1/balance works and shows the org's credits; a test key never changes them. See rate limits.

Errors you can get with a test key

Every error has the same envelope: { "error": { "code", "message", "doc_url", "retryable" } }. doc_url points at the code's entry on the error codes page.

Mock tools on the scores page

Mock tools have scores. They are published on /scores under the heading "Test mode (mock providers)" and at GET https://api.arettic.com/v1/scores?mode=test. They come from the same weekly scoring pipeline as live scores, from benchmark runs against mock test sets that ship with the platform. A few expected answers in those sets differ from what the mock returns, on purpose, so accuracy sits realistically below 100%.

These scores prove the scoring pipeline end to end. They say nothing about any real provider. Mock tools never appear in the live catalog, the live scores, GET /v1/pricing or a live key's recommendations. How scores are computed is on the scores page.

Switch to a live key

Change the key from sk_test_… to sk_live_… and keep the code. Then:

• recommend returns real tools. Pick the first option with purchasable: true.
• execute returns receipt_id, execution_id, charged and refunded, and no test_mode or would_have_charged.
• Credits are held before the call and settled after the check: charged on a pass, pro rata on a partial, released on a fail.
• Budgets, approvals, idempotency keys, max_price and fallback all take effect.
• A batch over 25 items runs as a job.
• Charged items can be disputed within 7 days.

Next: Quickstart, Execute, Receipts, Budgets and approvals, Jobs, MCP, SDKs.

Updated 2026-09-29 · This page as Markdown · JSON