benchmarks

Measured on our production API.

Newest run: Nov 9, 08:00. Each row shows when it ran.

1OpenRouter bench

How Logits ranks against OpenRouter's providers.

We ran OpenRouter's harness ourselves, on our API. OpenRouter did not run or review these results. Our proxy replaces OpenRouter's gateway and sends the requests to our API.

How our runs differ from OpenRouter's

Every benchmark

not through openrouter.ai: parity proxy (Chat Completions to api.logits.com, reasoning replayed as plain reasoning_content)
concurrency 32 (harness default 8; scheduling only)

GPQA Diamond

epochs 3 (harness default 10; OpenRouter's published runs use 3)

τ²-bench Verified airline

epochs 3 (harness default 1; OpenRouter's published runs use 3)

Standings against OpenRouter providers

Logits
OpenRouter provider
Peer median
Derank line

GPQA DiamondGraduate-level science questions

Model	Logits	Peer median	Derank line	vs median	Rank	Status	OpenRouter providers
DeepSeek V4.1 Flash198 questions × 3 epochsrun Nov 8, 08:04	90.8%32 days, 12 runsthis run 90.9%	90.6%	88.7%+2.2 pp margin	+0.2 pp	19 / 52	At or above median
Kimi K3.1 Pro198 questions × 3 epochsrun Nov 8, 08:04	93.3%32 days, 6 runsthis run 93.8%	92.9%	91.5%+1.7 pp marginprovisional	+0.3 pp	10 / 26	At or above median

Provider scores for GPQA Diamond

DeepSeek V4.1 Flash, 51 providers
#	Provider	Score
1	Tenstorrent	92.8%
2	Near AI	92.3%
3	DeepInfra	92.3%
4	DigitalOcean	92.1%
5	SiliconFlow	91.9%
6	PrimaLabs	91.8%
7	DekaLLM	91.6%
8	CoreWeave	91.5%
9	Baidu	91.3%
10	Alibaba	91.3%
11	InferenceNet · provider	91.2%
12	Venice	91.2%
13	GMICloud	91.1%
14	Fireworks · ac9eda21	91.1%
15	Sail Research · 8f31c510	91.1%
16	InferenceNet · a74ca7c9	90.9%
17	StepFun	90.9%
18	StreamLake	90.9%
19	Logits	90.8%
20	Fireworks · c857cf2a	90.8%
21	Cohere	90.7%
22	Nebius	90.7%
23	River	90.7%
24	Wafer · 66d4851d	90.7%
25	Wafer · provider	90.7%
26	Modal	90.7%
27	Relace · 485d24f3	90.6%
28	Morph	90.6%
29	Decart	90.6%
30	BaseTen · 007b95bb	90.6%
31	Makora	90.5%
32	InferenceNet · 4231aea9	90.5%
33	AtlasCloud	90.5%
34	Parasail	90.5%
35	Mistral	90.4%
36	Relace · 301a2173	90.4%
37	PrimeIntellect	90.3%
38	Particle AI	90.3%
39	Together	90.3%
40	Sail Research · a79e9269	90.2%
41	Sail Research · provider	90.1%
42	Fireworks · 9f90ca83	90.1%
43	Cloudflare	90.1%
44	BaseTen · b7371050	89.9%
45	Ionstream	89.8%
46	Crusoe	89.7%
47	Wafer · 913e3d9a	89.7%
48	Phala	89.4%
49	OpenInference	89.2%
50	ModelRun	88.2%
51	Reka	88.2%
52	Novita	87.6%

Kimi K3.1 Pro, 25 providers
#	Provider	Score
1	InferenceNet · bed5b2b6	94.1%
2	Sail Research · 68e8eb89	94.0%
3	Nebius	93.9%
4	Sail Research · provider	93.9%
5	Sail Research · c0a8a518	93.6%
6	InferenceNet · 4889efe8	93.6%
7	Logits	93.3%
8	Decart	93.3%
9	DigitalOcean	93.3%
10	Moonshot AI · da2df19f	93.3%
11	Modal	93.2%
12	Parasail	93.1%
13	AkashML	92.9%
14	Morph	92.9%
15	Phala · d6759daf	92.9%
16	Fireworks	92.8%
17	InferenceNet · provider	92.8%
18	PrimaLabs	92.7%
19	Moonshot AI · a3fafaac	92.6%
20	Phala · 44be9331	92.4%
21	Together	92.4%
22	Wafer · ef01e5c5	92.3%
23	InferenceNet · ff212546	92.1%
24	Relace	91.9%
25	DeepInfra	91.6%
26	Wafer · 9dc2f681	91.6%

Logits runs for GPQA Diamond

DeepSeek V4.1 Flash, 12 runs behind the 32-day score
#	Run	Score	Questions
1	Oct 31, 17:05	90.9%	198
2	Nov 1, 22:17	90.2%	198
3	Nov 2, 02:00	92.1%	198
4	Nov 2, 06:56	90.4%	198
5	Nov 3, 01:05	91.9%	198
6	Nov 3, 02:00	90.4%	198
7	Nov 4, 02:03	91.8%	198
8	Nov 5, 02:02	91.1%	198
9	Nov 5, 08:04	89.4%	198
10	Nov 6, 08:02	89.7%	198
11	Nov 7, 08:00	91.3%	198
12	Nov 8, 08:04	90.9%	198

Kimi K3.1 Pro, 6 runs behind the 32-day score
#	Run	Score	Questions
1	Nov 5, 00:15	92.8%	198
2	Nov 5, 02:02	93.1%	198
3	Nov 5, 08:04	92.8%	198
4	Nov 6, 08:02	93.6%	198
5	Nov 7, 08:00	93.6%	198
6	Nov 8, 08:04	93.8%	198

τ²-bench Verified airlineAirline customer-service agent tasks

Model	Logits	Peer median	Derank line	vs median	Rank	Status	OpenRouter providers
DeepSeek V4.1 Flash50 tasks × 3 epochsrun Nov 9, 08:00	73.0%32 days, 15 runsthis run 72.0%	72.5%runs since Oct 2132 days: 73.3%	69.7%+3.4 pp margin	+0.5 pp	21 / 49	At or above median
Kimi K3.1 Pro50 tasks × 3 epochsrun Nov 9, 08:00	75.5%32 days, 7 runsthis run 77.3%	75.3%	71.6%+4.0 pp marginprovisional	+0.2 pp	11 / 25	At or above median

Provider scores for τ²-bench Verified airline

DeepSeek V4.1 Flash, 48 providers
#	Provider	Score
1	River	77.7%
2	AtlasCloud	77.3%
3	Wafer · provider	77.1%
4	ModelRun	76.7%
5	BaseTen	76.3%
6	CoreWeave	76.3%
7	Fireworks · 9f90ca83	75.6%
8	Fireworks · c857cf2a	75.5%
9	GMICloud	75.5%
10	Decart	75.3%
11	Alibaba	75.1%
12	Parasail	75.0%
13	Fireworks · ac9eda21	74.3%
14	Mistral	74.1%
15	Baidu	74.1%
16	Near AI	74.0%
17	DekaLLM	73.3%
18	Sail Research · a79e9269	73.3%
19	Sail Research · provider	73.3%
20	OpenInference	73.1%
21	Logits	73.0%
22	Together	73.0%
23	Wafer · 66d4851d	73.0%
24	Modal	72.7%
25	StepFun	72.6%
26	Phala	72.4%
27	Cloudflare	72.3%
28	Morph	72.3%
29	SiliconFlow	72.3%
30	Nebius	72.2%
31	Particle AI	72.2%
32	Ionstream	72.1%
33	Reka	72.1%
34	DeepInfra	72.0%
35	Wafer · 913e3d9a	72.0%
36	Novita	71.7%
37	PrimeIntellect	71.7%
38	Relace · 485d24f3	71.7%
39	StreamLake	71.7%
40	Venice	71.7%
41	Sail Research · 8f31c510	71.6%
42	Crusoe	71.3%
43	InferenceNet · a74ca7c9	71.3%
44	Cohere	71.0%
45	InferenceNet · provider	71.0%
46	DigitalOcean	70.7%
47	InferenceNet · 4231aea9	70.7%
48	Relace · 301a2173	70.7%
49	Makora	70.4%

Kimi K3.1 Pro, 24 providers
#	Provider	Score
1	InferenceNet · bed5b2b6	78.9%
2	DigitalOcean	78.7%
3	Fireworks	78.7%
4	Moonshot AI · da2df19f	78.7%
5	Phala · d6759daf	78.7%
6	Nebius	77.3%
7	Relace	77.3%
8	Wafer · 9dc2f681	76.7%
9	Decart	76.0%
10	Morph	76.0%
11	Logits	75.5%
12	InferenceNet · 4889efe8	75.3%
13	InferenceNet · provider	75.3%
14	Moonshot AI · a3fafaac	75.3%
15	Phala · 44be9331	75.0%
16	AkashML	74.7%
17	Alibaba	74.7%
18	DeepInfra	74.7%
19	InferenceNet · ff212546	74.7%
20	Wafer · ef01e5c5	74.7%
21	PrimaLabs	74.5%
22	Parasail	73.3%
23	Together	73.3%
24	Modal	73.0%
25	Sail Research	73.0%

Logits runs for τ²-bench Verified airline

DeepSeek V4.1 Flash, 15 runs behind the 32-day score
#	Run	Score	Tasks
1	Nov 3, 02:31	73.3%	50
2	Nov 4, 02:35	70.7%	50
3	Nov 5, 02:34	72.7%	50
4	Nov 5, 08:53	71.3%	50
5	Nov 6, 09:10	72.0%	50
6	Nov 6, 17:23	73.3%	50
7	Nov 6, 17:24	74.0%	50
8	Nov 6, 17:35	73.3%	50
9	Nov 6, 17:44	73.3%	50
10	Nov 6, 17:45	72.7%	50
11	Nov 6, 17:55	73.3%	50
12	Nov 6, 17:56	76.7%	50
13	Nov 7, 08:48	72.7%	50
14	Nov 8, 08:33	74.0%	50
15	Nov 9, 08:01	72.0%	50

Kimi K3.1 Pro, 7 runs behind the 32-day score
#	Run	Score	Tasks
1	Nov 5, 00:51	76.7%	50
2	Nov 5, 03:30	75.3%	50
3	Nov 5, 08:39	77.3%	50
4	Nov 6, 09:44	74.0%	50
5	Nov 7, 09:47	72.7%	50
6	Nov 8, 09:03	75.3%	50
7	Nov 9, 08:01	77.3%	50

2How to read these results

You can read every number on this page, OpenRouter's provider scores included, from one public API call: api.logits.com/status/v1/benchmarks.
The harness is OpenRouter's repository at commit c6a05aa. Provider scores come from OpenRouter's published benchmark runs of Nov 12. Source: OpenRouter (openrouter.ai), licensed under CC BY 4.0. Medians, ranks and derank lines are our calculations from those runs.
Each score in the standings covers the runs in the last 32 days, for Logits and for every OpenRouter endpoint. The number of runs behind the Logits score is shown under it, each of those runs is listed below its table, and some scores rest on few runs. When a benchmark's conditions changed for every provider at once inside that window, the peer scores and ours count only the runs from that day, which is shown under the peer median beside the median over the whole window. The derank line keeps OpenRouter's own window.
We count each OpenRouter endpoint as one provider, so a provider with several endpoints appears more than once, with part of its endpoint ID. Rank is one plus the number of endpoints strictly above Logits, and the total includes Logits.
We calculate the derank line with the rule from Auto Exacto, OpenRouter's provider routing: the median minus two standard deviations of endpoint scores over the first 21 days of a model version. The line is provisional while those days are still running. It is approximate when they began before the 90 days of runs that OpenRouter's export keeps.
We calculate gaps from unrounded scores, so a gap can differ by 0.1 pp from the rounded scores beside it.
This page shows a fixed list of benchmarks and their latest results. We run other tests that are not listed here. Times are UTC.