Test your agent against a frozen GitHub.

01 / Why a frozen replica

Three things an agent test needs.

• Determinism. gh-2026-03-g12 is a frozen snapshot, content addressed at c27f762cc1d2. Every byte it serves today is a byte it served yesterday, so an assertion on a reply count or a commit SHA is a test rather than a flake.
• One coherent story. These rows are one simulated company, olympus-labs, rendered into each vendor's dialect. The same incident is a thread on the Slack host, FATE-51 on the Jira host and pull request 53 on the GitHub host, which is what lets an agent that crosses tools be judged on whether it crossed them correctly.
• Nothing to babysit. No vendor sandbox to provision, no seed script, no token to rotate, nothing to clean up between runs. Every host is read-only, so the worst an over-eager tool call can do is produce the response in section 06.

02 / The host

One request, no key, no account.

The base URL is https://gh-2026-03-g12.snap.sandboxapis.dev. It serves GitHub's own paths, JSON and headers, so the client you already use needs one line changed and nothing else. Send this before you write any code:

curl -s https://gh-2026-03-g12.snap.sandboxapis.dev/repos/olympus-labs/parthenon/pulls/53 \
  | python3 -c "import json,sys; p = json.load(sys.stdin); print(p['number'], p['title'], '| merged:', p['merged'], '|', p['head']['sha'])"

53 Fix the backfill job OOMing | merged: True | 94dd1fdf3b96e96c242a869801e448edeca9fd85

Anonymous callers get 60 req/hour, shared across every sandbox host rather than one bucket per host, which is enough to work through this page. A free key raises that to 600 req/hour and rides GitHub's own auth slot, so an eval loop does not run out halfway. Keys and rate limits.

03 / The agent task

Did the fix for the incident ship, and did anyone other than its author sign it off?

Save this as github_agent.py and run it. It is plain requests, in the three beats an agent runs in: fetch the rows a tool would return, decide something from them, then check the decision. The middle beat is a pure function on purpose, so you can replace it with a model call and leave the assertions exactly as they are.

"""Agent task: did the fix for the incident ship, and did anyone sign it off?

Every request goes to a frozen SandboxAPIs snapshot host, so this file is a
test: the answer is the same today, in CI tonight, and in a year.
"""
import requests

GITHUB = "https://gh-2026-03-g12.snap.sandboxapis.dev"  # the only line that changes
REPO = "olympus-labs/parthenon"

http = requests.Session()
http.headers["Authorization"] = "token any-token"  # any token is accepted


def get(path):
    res = http.get(GITHUB + path, timeout=30)
    res.raise_for_status()
    return res.json()


# 1. FETCH. The same three calls your agent's GitHub tools already make.
pull = get("/repos/%s/pulls/53" % REPO)
reviews = get("/repos/%s/pulls/53/reviews" % REPO)
branch = get("/repos/%s/branches/%s" % (REPO, pull["head"]["ref"]))


# 2. REASON. The agent step: a pure function of the fetched data, so the same
# input always produces the same verdict. Swap in a model call here and the
# assertions below become an eval:
#   answer = json.loads(client.messages.create(model="claude-sonnet-4-5",
#            max_tokens=256, messages=[{"role": "user", "content": prompt}]) ...)
def verdict(pr, pr_reviews):
    author = pr["user"]["login"]
    approvals = [r for r in pr_reviews if r["state"] == "APPROVED" and r["user"]["login"] != author]
    if not pr["merged"]:
        return {"shipped": False, "reason": "closed without merging"}
    if not approvals:
        return {"shipped": True, "reason": "merged with no independent approval"}
    return {"shipped": True, "approved_by": approvals[0]["user"]["login"]}


answer = verdict(pull, reviews)
print("PR #%d %s -> %s" % (pull["number"], pull["title"], answer))

# 3. ASSERT. Known facts of the olympus-labs data set, frozen in this snapshot.
assert pull["user"]["login"] == "tiresias", pull["user"]["login"]
assert answer == {"shipped": True, "approved_by": "athena"}, answer
assert pull["head"]["ref"] == "hotfix/backfill-job-ooming", pull["head"]["ref"]
assert pull["head"]["sha"] == "94dd1fdf3b96e96c242a869801e448edeca9fd85", pull["head"]["sha"]
# The branch is a real, fetchable ref at that exact commit, not a string on the
# pull request. Every reference in this data set resolves like this.
assert branch["commit"]["sha"] == pull["head"]["sha"], branch["commit"]["sha"]
print("ok")

PR #53 Fix the backfill job OOMing -> {'shipped': True, 'approved_by': 'athena'}
ok

The commented line inside the reasoning step is where a real model call goes. Everything above it is tool output and everything below it is judgement, which is the split that makes an eval readable: when the assertion fails you know whether the tools returned the wrong rows or the model read the right ones wrongly.

04 / In CI

Five lines that drop into a suite.

The same claim, as a test the runner you already have will collect. No fixtures to check in, no recorded cassettes to refresh, no credential in the job's environment:

import requests

PIN = "https://gh-2026-03-g12.snap.sandboxapis.dev/repos/olympus-labs/parthenon"

def test_the_incident_fix_shipped():
    pr = requests.get(PIN + "/pulls/53", timeout=30).json()
    assert pr["merged"] and pr["head"]["sha"] == "94dd1fdf3b96e96c242a869801e448edeca9fd85"

Because the host is pinned rather than live, this test has no expiry date. The live host at https://gh.sandboxapis.dev re-rolls its data daily, which is what you want for a demo and the one thing you must not point a test at. Versioning and pinning covers the registry and how to choose a pin.

05 / MCP instead of HTTP

Same universe, through tools.

If your agent prefers tools to raw HTTP, @sandboxapis/mcp serves the same rows through orient, the read tools and check_budget. One environment variable points it at the pin instead of the live host, which makes the tool results as reproducible as the request above. Drop this into Claude Desktop's claude_desktop_config.json or Cursor's mcp.json:

{
  "mcpServers": {
    "sandboxapis": {
      "command": "npx",
      "args": ["-y", "@sandboxapis/mcp"],
      "env": {
        "SANDBOXAPIS_BASE_URL_GITHUB": "https://gh-2026-03-g12.snap.sandboxapis.dev"
      }
    }
  }
}

The variable is SANDBOXAPIS_BASE_URL_GITHUB, and every provider has one, so an agent can be pinned on the surfaces a test cares about and left live everywhere else. MCP setup has the full tool list and the one-line install for Claude Code.

06 / When the agent writes

The refusal is legible, not a crash.

An agent under test will eventually try to change something. Here is what that costs, captured from this host rather than described:

curl -s -i -X POST https://gh-2026-03-g12.snap.sandboxapis.dev/repos/olympus-labs/parthenon/issues \
  -H "Content-Type: application/json" \
  -d '{"title":"opened by an agent under test"}'

{"message":"This universe is read-only. See what's covered at https://sandboxapis.dev/roadmap. (You tried to POST /repos/olympus-labs/parthenon/issues.) Tell us what you needed to write: https://sandboxapis.dev/feedback","documentation_url":"https://sandboxapis.dev/roadmap","status":"403"}

It is GitHub's own error envelope, so your client raises the exception it would normally raise, and the message names the exact request that was refused. The response also carries a header a harness can branch on, which is how a read-only refusal is told apart from a coverage 404 or a rate limit without parsing prose:

x-sandboxapis-read-only: true

Nothing is mutated, and the next run reads exactly the same rows.

Next

Where to go from here.

The other two tutorials read the same incident from the other side: Frozen Jira and Frozen Slack.

All agent tutorials · SandboxAPIs for agents · Verified quickstarts · GitHub quickstart · Versioning and pinning · Coverage manifest