LLM Test Serverby Managed Code
GitHubGet started

.NET-native · Docker for any stack

Your tests.
Your responses.

A deterministic LLM API server for your tests — in .NET, Python, Node, Go or any language that speaks HTTP.

Script responses, streaming, tool calls and failures. Call it with your regular provider SDK, then see exactly what your application sent and where every token went.

Get started View on GitHub
.NET dotnet add package ManagedCode.LlmTestServer.Aspire
Docker docker run -p 8080:8080 ghcr.io/managedcode/llm-test-server:1.0.0
Tokens tab for a five-request conversation: token flow from system prompt, tool schemas, user, assistant and tool messages into cache read, cache write, uncached input, response and reasoning.
A real capture of the dashboard served by the package.

13 provider API families, each under its own local namespace

  • OpenAI
  • Azure OpenAI
  • Microsoft Foundry
  • Anthropic
  • Gemini
  • Amazon Bedrock
  • Groq
  • Mistral
  • Ollama
  • Cohere
  • OpenRouter
  • DeepSeek
  • Perplexity

How it works

From fixture to a
request you can inspect.

Keep your real provider integration in the test. Choose the behavior on the other side.

  1. Set the scene.

    Script a model, a reply, a tool call or a failure.

    .UseChatScenario("greeting", s => s
  .ForModel("gpt-6.1-sol")
        .WhenUserContains("hello")
  .Responds("Hello from your test!"))

  2. Make the call.

    Point your provider SDK or IChatClient at the namespaced route.

    POST /openai/v1/chat/completions
→ 200 OK
→ "Hello from your test!"

  3. Check what happened.

    Assert on the reply and the recorded requests.

    var state = await control
  .GetAssertionsAsync();
// state.Matched == 1

The built-in request dashboard

Less reconstruction.
More context.

Provider, model, status, latency and token usage stay with each request. Select a call to read the response, the conversation behind it and the wire payloads. Hover any chart for exact values; JSON payloads and tool-call arguments are highlighted.

Request overview with the conversation strip, per-message token bars, a tool-call card with highlighted arguments and a tooltip.
One entry per provider request, with the selected response in the reading pane.

Traffic insights

All traffic at a glance.

Providers, models, outcomes, latency, cache and context mix as charts. Click a slice to filter the request inbox.

Traffic insights view with donut, ring, capsule-bar and dot-matrix charts across providers, models, outcomes and latency.

Runtime diagnostics

Look inside your test.

Inspect models, fixtures, provider endpoints, assertions and runtime events from the same debugging surface.

Runtime state dialog showing configuration and diagnostic information.

Request lifecycle

Every call, end to end.

Your test calls through its usual client. The server answers from your fixtures and records the exchange for the dashboard and your assertions.

Token analytics

See where every
token goes.

The Tokens tab splits each request's input by source: system prompt, tool schemas, user, assistant, tool calls and tool results. The parts add up to the reported input. Then it shows what the prompt cache read, what it wrote and what stayed fresh.

Your tests read the same numbers from GET /admin-api/requests: each request's usage carries the per-role split as InputComposition.

Read the analytics guide

Input token flow

Where 3,140 input tokens came from, and how the prompt cache handled them. Example request.

Example request

Context mix

What the prompt is made of.

  • System prompt1,86059.2%
  • Tool schemas42013.4%
  • User2146.8%
  • Assistant34210.9%
  • Tool calls882.8%
  • Tool results2166.9%

Prompt cache

How the input was served.

81.5%from cache
  • Cache read2,56081.5%
  • Cache write2568.2%
  • Uncached32410.3%

Context growth

Input per request in one labeled conversation.

  1. 1,204#1
  2. 2,150#2
  3. 2,736#3
  4. 3,140#4
  • From cache
  • Fresh input

Output55 tokens

  • Visible text32
  • Reasoning23

Example values. Reasoning is part of output and cache reads/writes are part of input, so they are never added twice. Requests without reported usage show as missing, not as zero.

Traffic insights

The same journal, sliced by provider, model, outcome, latency, operation and delivery. Open it with /?view=insights; clicking a slice filters the request inbox.

Example values

Providers

Requests per provider surface.

  • OpenAI22
  • Anthropic11
  • Gemini7
  • Azure OpenAI5
  • Other3

Models

One dot per request.

  • gpt-6.1-sol14
  • claude-sonnet-5-511
  • gpt-6-luna9
  • gemini-3.8-flash7
  • text-embedding-3-small4
  • Other3

Outcomes

Share of requests that returned 2xx.

85%success
  • 4xx 4
  • 5xx 1
  • transport 1
  • in flight 1

Latency

Requests per latency band.

  1. 30<10 ms
  2. 1210–100 ms
  3. 50.1–1 s
  4. 1≥1 s

Operations

Routed provider operations.

  • chat.completions.create44%
  • messages.create23%
  • models.generateContent15%
  • responses.create10%
  • embeddings.create8.3%

Delivery

Streamed chunks or one body.

  • Single response31
  • Streaming17

Beyond a single reply

Test the conversation.
And the unexpected.

Every behavior is a fixture you choose. Nothing is hidden behind retries or randomness.

Tool loops, step by step.

Conversation graphs match each call and result, then choose the next reply.

Conversation DSL

Streaming, chunk by chunk.

Script exact chunks: one response id, then the terminal marker.

Streaming chunks

Failures on demand.

Rate limits and content filters on every provider family.

Fault simulation

More than text.

Explicit embeddings, images, audio and video jobs where supported.

Modalities

Parallel tests, separate state.

Each session owns its configuration, queue and journal.

Test sessions

Verify, then reproduce.

Check order, counts and tool arguments; fix ids with a seed.

Verification

Provider-aware, by design

Your provider.
A local endpoint.

Every provider family has its own namespace and wire format. Operation coverage varies by provider and grows incrementally.

Read the operation matrix
  • OpenAI/openai/v1Official SDK checked
  • Azure OpenAI/azure-openai/openai/deployments/…Official SDK checked
  • Microsoft Foundry/microsoft-foundry/modelsOfficial SDK checked
  • Anthropic/anthropic/v1/messages
  • Gemini/gemini/v1beta/models/…
  • Amazon Bedrock/bedrock/model/…/converseOfficial SDK checked
  • Groq/groq/openai/v1
  • Mistral/mistral/v1
  • Ollama/ollama/api/chat
  • Cohere/cohere/v2/chat
  • OpenRouter/openrouter/api/v1
  • DeepSeek/deepseek/v1
  • Perplexity/perplexity/v1/sonar
  • Native decision APIs/systemone · /cloudflareTypeSafe Jev · Workers AI Clef

Each hosted operation is listed in a provider contract with documentation links and behavior tests. A provider profile does not imply every API of that provider is implemented: check the operation and client version your application uses.

Get started

A local server.
Any stack.

On .NET, add it to your Aspire AppHost. Everywhere else, run the Docker image and point your provider SDK at it.

.NET · Aspire

Add it to your AppHost

Aspire starts the packaged .NET server and allocates its endpoint. No container runtime needed.

Terminal 
dotnet add package ManagedCode.LlmTestServer.Aspire
dotnet add package ManagedCode.LlmTestServer.Client
AppHost.cs 
using ManagedCode.LlmTestServer.Aspire;

var builder = DistributedApplication.CreateBuilder(args);
builder.AddLlmTestServer("llm").WithApiKey("test-key");
builder.Build().Run();
ChatTests.cs 
var control = new LlmTestServerClient(
    app.CreateHttpClient("llm", "http"), "test-key");
await control.ConfigureAsync(config => config
    .RequireBearerToken("test-key").UseModel("gpt-6.1-sol", LlmTestServerModelKind.Chat)
    .UseChatScenario("greeting", scenario => scenario
        .ForModel("gpt-6.1-sol")
        .WhenUserContains("hello").Responds("Hello from your test!")));

using var chat = control.CreateChatClient("gpt-6.1-sol");
var reply = await chat.GetResponseAsync(
    [new ChatMessage(ChatRole.User, "hello")]);
// reply.Text == "Hello from your test!"
Open the complete integration test
Any language · Docker

Run the container

Python, Node, Go, Java or anything else: the image serves the same provider routes, dashboard and /admin-api. Load fixtures with one request.

Terminal 
docker run --rm -p 8080:8080 \
  -e LlmTestServer__RequiredBearerToken=local-test-key \
  ghcr.io/managedcode/llm-test-server:1.0.0
Configure 
curl http://localhost:8080/admin-api/configure \
  -H 'Authorization: Bearer local-test-key' \
  -H 'Content-Type: application/json' \
  --data '{"requiredBearerToken":"local-test-key",
    "models":[{"id":"gpt-6.1-sol","kind":"chat"}],
    "chatScenarios":[{"id":"greeting","modelId":"gpt-6.1-sol",
      "responses":[{"content":"Hello from LLM Test Server"}]}]}'
test_chat.py 
from openai import OpenAI

client = OpenAI(base_url="http://localhost:8080/openai/v1",
                api_key="local-test-key")
reply = client.chat.completions.create(
    model="gpt-6.1-sol",
    messages=[{"role": "user", "content": "Hi"}])
assert reply.choices[0].message.content == "Hello from LLM Test Server"
Docker quick start

NuGet packages

A few useful details

Before your first test.

Does it call a live model?

No. The server returns the fixtures configured for your test. It models API behavior so your application can make real HTTP or SDK calls against a local server.

Is it only for .NET?

No. The Docker image serves the same provider routes, dashboard and management API to any language. Point your Python, Node, Go or Java provider SDK at the container and configure fixtures over HTTP.

Do I need Docker?

Not on .NET. The package-owned Aspire integration starts the packaged .NET server and exposes its allocated resource endpoint. For other stacks, the container image is the way in.

Can I keep my provider SDK?

Use the supported routes with the SDKs listed in the operation coverage documentation. Compatibility is verified per operation; check your client version and API requirements.

How are tokens counted?

Usage is deterministic. Text is counted with a tiktoken-compatible tokenizer, so the same request always reports the same input, output, reasoning and cache values. It does not reproduce a provider's billing.

Can tests share a server?

Yes. Create separate test sessions for independent configuration, scenario queues, history and diagnostics. Dispose each session when its test ends.

Is every provider API implemented?

No. Coverage varies by provider. The operation matrix describes the claimed routes, streaming modes and SDK evidence. Provider profiles grow incrementally.

Does this evaluate model quality?

No. It verifies your application's behavior against scripted API fixtures. It does not measure the quality of a live model's answers.

Make the next call predictable

Know what you send.
Choose what comes back.

Start the server, script a response and inspect your first request.

Open on GitHub Read the quick start

Dashboard preview