Is the relay swapping models? Test before you accuse

Signals of model substitution and a repeatable way to collect evidence.

Reviewed 2026-11-12 · Chinese detail: 完整中文页

Signals worth taking seriously

A sudden drop in answer quality, a context window that truncates earlier than advertised, and usage fields that do not match the requested model are all signals. None of them proves substitution on its own.

Collect evidence, not impressions

1. Use a fixed set of prompts and run them against the relay and a known-good endpoint.

2. Record the model name returned in the response, the usage fields, and the finish reason.

3. Test the advertised context window with a long input and check whether the middle is actually retained.

4. Repeat each prompt several times. A single sample cannot distinguish a bad model from a bad day.

What the evidence can show

Consistent differences in tokenizer ratios, refusals, or self-reported identity suggest that the backend is not the advertised model. They do not identify which model is behind it. Write the report in those terms.

Talk to the relay

Send request IDs, timestamps, the requested model, and the returned model. Ask for the routing rule. If the answer is vague, test a second relay before moving production traffic.

This page is general guidance, not a statement about any specific provider. For measured data, see the relay directory and method.