A test bench for AI agent tools

MockAgentTest the tool calls your agent hasn't made yet.

Build and import mock tools, connect agent clients over MCP, and test edge cases before production. Run saved regression suites in GitHub Actions and review supported HAR captures as local scenarios—without sending requests to captured hosts.

Start building Log in to your workspace

JSON Schema validation No live service calls Inspect every run

workspace / tools / issue_refund

INTERACTIVE DEMO

Execution log

Recent tool calls

SAMPLE

issue_refund

{ "user_id": "vip" }

SUCCESS

42 ms

issue_refund

{ "reason": "missing id" }

SCHEMA ERROR

58 ms

Every request leaves a trace you can inspect and replay.

Interactive demo

Try a tool call

BROWSER ONLY

Output

SUCCESS

Matched scenario: VIP user

{
  "success": true,
  "message": "VIP refund processed."
}

Runs locally in your browser with the sample schema and responses. It does not call a live endpoint.

In your workspace, each test also appears in execution logs. Add bearer-key protection when a mock is ready to share.

Explore the workflow

Why MockAgent

Build the integration before the real integration is ready.

Agent tool calls are hard to test when the service is unfinished, expensive, rate-limited, or capable of real side effects. MockAgent gives each tool a predictable HTTPS endpoint, so you can iterate on schemas and agent behavior in a controlled workspace.

Make the contract explicit

Define the arguments your tool accepts with JSON Schema. Invalid calls come back with specific errors your agent can correct.

SCHEMA VALIDATION

Exercise more than the happy path

Match inputs to named scenarios, sequence changing responses, and schedule deterministic transient failures to exercise retries.

CONDITIONAL RESPONSES

Replay and compare

Replay a saved successful call against the current mock. See whether a prompt, schema, or response change altered the result.

REGRESSION CHECKS

Protect the endpoint

Issue revocable bearer keys, require them per tool, and enforce monthly usage limits at the gateway.

API KEY CONTROLS

Draft tools with Muse Spark

Turn a plain-language requirement into an editable tool contract, mock response, and edge-case scenarios. Validate before saving.

META AI TOOL STUDIO

Connect over MCP

Expose a workspace toolset through a hosted MCP URL with API-key or token-based access.

HOSTED MCP

Test untrusted tool output

Add a controlled prompt-injection fixture and check whether a saved regression answer repeats its known marker.

PROMPT-INJECTION CHECKS

Spot contract drift

Compare a pasted API specification with saved mock names, descriptions, and schemas, then review reliability trends.

DRIFT & RELIABILITY

The full toolkit

Everything to mock, exercise, and improve agent tool behavior.

Explore MockAgent's testing, Meta AI, observability, integration, and workspace features at a glance. AI-generated content stays reviewable; coverage and reliability reflect the saved traces that have actually been run.

Create, validate, and version mock tools

Define the tool contract, predictable response, access policy, and revisions in one workspace.

Tool authoring. Create tools with a name, description, JSON Schema input contract, JSON response, scenarios, and optional per-tool API-key protection.
HTTP mock endpoints. Give each tool a stable HTTPS endpoint; GET describes the tool and POST executes a mock call.
Actionable validation. Validate required fields, types, enums, formats, and extra properties; return field-specific errors and record schema violations.
Conditional scenarios. Match arguments with JSON Schema and return scenario-specific responses; preview a matcher against sample input in the editor.
Response sequences. Serve ordered responses for matching calls within a run ID, repeating the final response after the sequence ends.
Scheduled transient failures. Inject deterministic 429, 500, 502, or 503 faults at selected steps, with retry guidance and Retry-After behavior.
Assertions and semantic criteria. Check expected ordered tool calls and final-answer phrases; optionally judge answers against successful tool evidence.
Revision history. Compare tool configuration revisions, inspect changed fields, and restore a prior configuration.
Pasted-spec drift check. Compare saved mock names, descriptions, and input schemas with pasted OpenAPI, function-tool, or MCP definitions. This is a point-in-time comparison; MockAgent does not poll upstream APIs.

Agent Playground and Meta AI

Exercise tools with a model and use AI assistance for editable drafts, not automatic production changes.

Agent Playground. Choose a configured model and active mock tools, run a prompt, and inspect its final answer and tool calls.
Protected-tool support. Provide temporary keys for selected protected tools; credentials are sent to the matching gateway call, not the model prompt or saved run.
Meta AI Tool Studio. Draft an editable tool name, schema, response, and scenarios from a natural-language description.
Workspace assistant. Chat for tool-design and testing ideas; it does not automatically access workspace data.
Model selection. Use the configured Meta-compatible provider and available model catalog; model credentials stay server-side.
Reasoning continuity. Native Meta Responses API Playground runs can replay encrypted reasoning between tool turns with provider-side storage disabled.

Regression tests, coverage, and run workflows

Save checks, run cases, investigate failures, and target unobserved behavior with reviewable drafts.

Saved suites and prompt cases. Create suites with up to 20 named cases, selected tools, and expected behavior.
Ordered tool-call checks. Assert the exact tool names and order for a run, including unexpected or missing calls.
Final-answer and response checks. Check phrases and JSON response fields using equality, contains, existence, numeric comparisons, or JSON Schema shape assertions.
Prompt-injection fixture. Add a known untrusted-output marker and assert it is absent from the final answer. This deterministic check is not a general prompt-injection detector.
Semantic answer checks. Optionally evaluate an answer against successful tool responses and timestamps using the configured model judge.
Suite execution controls. Run one case or all cases with confirmation, see per-check outcomes and traces, and stop after the active case.
Run history, replay, and branches. Replay prompt, model, checks, and current or saved tool configuration; compare with the source run or branch into an edited test.
Coverage-guided drafts. Target arguments, enum values, scenarios, and response steps not observed in the recent saved traces.
Auto-Improve rounds. Measure baseline coverage, review generated cases, and confirm bounded rounds to measure coverage changes.
Failure explanations and follow-ups. Generate evidence-linked explanations and follow-up cases from failed checks and bounded traces.
Contract-risk review. Ask for advisory schema/scenario risks with contract pointers and draft prompts; findings do not modify tools.
Suite portability. Export versioned JSON bundles and import them after validating suites and mapping tool names to active workspace tools.

Execution logs, traces, and analytics

Inspect mock calls and saved test evidence, then understand trends in the data available to your workspace.

Execution logs. Inspect tool arguments and responses, status, latency, validation errors, matched scenario/step, run ID, and transport.
MCP-aware logs. Identify MCP requests by transport, JSON-RPC method, and called tool; logs use workspace retention settings.
Final-answer submissions. Review configured run-level phrase, tool-sequence, and semantic assertion outcomes.
Bounded run traces. Review saved prompt and trace, ordered tool calls, model/provider metadata, and reasoning-replay status without exposing encrypted reasoning.
Mock-call replay comparison. Replay a saved successful request and compare its response with the current mock configuration.
Regression analytics and coverage. Explore pass rates, recent trends, recurring failures, model breakdowns, and coverage observed in available saved traces.
Suite reliability score. See the pass percentage across up to 20 recent saved case runs and the change against the preceding equal-size window in loaded history.
Usage analytics. View daily calls, average latency, schema violations, and injected failures over the available reporting period.
Advanced insights. Inspect service health integrations and workspace-level operational indicators.

SDK, API, and CI integrations

Call mocks from agent code and run saved tests from local scripts or GitHub Actions.

JavaScript and TypeScript SDK. Use the published trymockagent npm package for tool calls with run IDs and optional bearer-key authentication.
HTTP API. Call the HTTPS gateway from any language; use run IDs to group calls and make sequence behavior repeatable.
Python SDK source. Use the source package in the repository; it is not currently published on PyPI.
CI regression runner. Run selected or all saved suites locally or in a manual GitHub Actions workflow; failures return a nonzero exit code.
Pull request feedback. Optionally update one PR comment with hosted suite results and changed files; the workflow checks out the trusted base revision.

MCP, imports, and captured traffic

Start from existing tool definitions or turn a reviewed capture into reusable mocks.

Hosted MCP toolsets. Connect compatible MCP clients to workspace tools over a stateless HTTP endpoint with API-key or open-token authentication.
Import existing definitions. Preview and create tools from OpenAPI JSON/YAML, OpenAI or Anthropic function definitions, and MCP tool lists.
Offline HAR capture import. Convert exported JSON request/response exchanges into inferred contracts, exact-match scenarios, and response sequences.
Capture limitations. HAR import does not contact upstream hosts or replay status codes and headers. Review captures for sensitive data before importing.

Access, usage, plans, and retention

Scope data to workspaces, protect tool calls, and manage usage within plan entitlements.

Workspace isolation and roles. Organize tools, suites, runs, logs, keys, settings, and members with owner, admin, and member roles.
Workspace API keys. Create named bearer keys with monthly limits, reveal a key once, and revoke it when no longer needed.
Tool and toolset access. Protect individual HTTP tools and configure MCP toolsets for workspace-key or open-token access.
Rate limits and quotas. Apply gateway limits and shared monthly workspace call and AI-request quotas.
Plan and billing controls. Review Free (3 tools), Solo (25), and Team (250) entitlements, usage meters, and paid subscriptions where billing is configured.
Workspace-level limits. Plan entitlements include monthly call and AI quotas, seats, API-key counts, and maximum log retention; AI requests also have a shared per-minute limit.
Log retention. Choose a retention period permitted by the plan; scheduled cleanup removes expired execution logs.
Self-hosting and OpenTelemetry. Run a separate Next.js deployment and optionally export low-cardinality operational OTLP spans and metrics without prompts, arguments, or mock payloads.

Hosted MCP and tool imports

Connect MCP clients or create mocks from existing contracts and supported traffic captures.

Hosted MCP toolsets. Expose selected workspace tools at a stateless HTTP MCP endpoint for compatible agent clients.
Protocol compatibility. Support discovery, tool listing/calling, legacy initialization and ping; no session IDs or deprecated HTTP+SSE.
MCP authentication and execution. Use workspace bearer keys or a one-time open URL token; calls share HTTP validation, scenario, quota, and logging behavior.
Import from OpenAPI. Parse JSON or YAML operations and resolve local references without fetching external references.
Import function tools. Preview OpenAI and Anthropic function definitions, plus MCP tool-list results.
Import HAR 1.2 captures. Convert supported JSON request/response exchanges to inferred schemas, exact-match scenarios, and same-run response sequences.
Import review and safety limits. Preview before saving; external hosts are never contacted, import sizes are bounded, and common credentials are scrubbed. HAR status and headers are not replayed.

Safe, reproducible demo

Explore a realistic agent-tool regression without triggering a real business action.

Delayed-order refund walkthrough. Reproduce an agent bug involving a pending order, then inspect an evidence-based review and corrected rerun.
Local deterministic sample. The seeded browser demo uses sample tools and a deterministic sample agent; it does not create workspace records or issue a refund.
Demo AI review. The optional explanation is generated by the configured model and consumes normal workspace AI usage.

Prompt-injection checks use a known test marker and are not a general security detector. Drift checks compare pasted specifications only; they do not poll live APIs. Reliability is based on loaded suite history. Optional OpenTelemetry export is operational-only; paid checkout depends on each deployment's billing configuration.

A simple loop

From tool definition to repeatable test.

Keep the mock endpoint alongside your agent while the real service evolves. Swap the mock for the production tool when you are ready.

Create your first workspace

Describe the tool

Add a name, argument schema, default response, and any conditional scenarios.

Point your agent at it

Use the generated HTTPS endpoint as the tool handler in your agent or test harness.

Inspect and iterate

Review calls, validation failures, latency, and replay comparisons in one workspace.

Integrate with the SDK

Call your mocks from your code.

Install the published trymockagent npm package and call your mock endpoints directly from JavaScript or TypeScript. Pass run IDs, add bearer keys for protected tools, and inspect structured errors when a call needs fixing.

View trymockagent on npm

Python client source is available in the project but has not been published to PyPI yet. Any language can call the HTTP endpoint directly.

JavaScript / TypeScripttrymockagent

# Install
npm install trymockagent

# Use the tool in your app
import { MockAgent } from "trymockagent";

const mock = new MockAgent({
  toolId: "YOUR_TOOL_ID",
});

const result = await mock.call(
  { user_id: "vip", reason: "duplicate" },
  { runId: "test-run-1" },
);

Frequently asked questions

Build a repeatable tool-testing workflow.

Connect your agent, bring an existing contract or capture, then validate behavior with saved tests.

Can I connect an agent to MockAgent over MCP?

Yes. Create a workspace toolset and use its hosted, stateless MCP HTTP endpoint. Authentication can use a workspace API key or an open URL token. MCP calls share the mock execution engine, quotas, and execution logs.

What can I import into MockAgent?

You can preview and create tools from OpenAPI JSON or YAML, OpenAI or Anthropic function tool definitions, MCP tool lists, and exported HAR 1.2 captures. Imports create local mocks; they do not call pasted MCP servers or captured API hosts.

Does HAR import record live traffic or replay the original service?

HAR import processes a capture file you provide. It creates mock contracts and scenarios from supported JSON exchanges; it is not a live proxy, and it does not replay captured response status codes or headers. Inspect data for secrets and personal information before import.

Can regression suites run in GitHub Actions?

Yes. The included Node runner runs saved suites, and the optional pull-request workflow updates a bot comment with hosted-suite results and changed files. Configure a dedicated workspace API key and base URL as GitHub Actions secrets.

Start with one tool

Make the next agent run observable.

Create a workspace, define a mock contract, and see exactly what your agent sends back.