Measured, not claimed.
How MONOVION measures the AI model market
MONOVION publishes short, data-driven videos about AI models: which ones got cheaper or more expensive, which ones answer fastest, which ones climbed the leaderboards. Every number in a video comes from the time series described on this page — most of them from our own measurements. This page documents how we collect them and lets you download the raw data.
What we measure
| Probe | Schedule | Source | Values |
|---|---|---|---|
| Price probe | daily 06:00 UTC | Public model price lists via the OpenRouter models API; cross-checked against models.dev (provider list prices). | USD per 1M input / output tokens |
| Latency probe | every 3 hours | Our own measurement harness: one fixed prompt sent to each model from a fixed server in Berlin, Germany. | Time to first token (ms), output speed (tokens/s), availability, HTTP status |
| Leaderboard probe | daily 06:30 UTC | LMArena public leaderboard dataset (Hugging Face) and OpenRouter usage rankings. | Rank, score, change vs. previous update |
| Capability probe | weekly, Monday | Model catalogs and published deprecation schedules. | Context window, max output, shutdown dates |
| Change watcher | every 30 minutes | Provider status pages, model catalog changes, official news feeds and deprecation pages. | Incidents, new/removed models |
The latency probe in detail
Prices and rankings are public — anyone can copy them. Our latency series is not: it only exists because a specific prompt was sent
at a specific time from a specific place. This is how it works (probe version v1, frozen — any change gets a new version):
- Location: a fixed server in Berlin, Germany (region label
eu-de-berlin). - Prompt: the same ~520-token prompt for every model (full text below), temperature 0, seed 7 where supported.
- Output length: max. 256 tokens; 1024 for models that always reason before answering (lowest reasoning effort the model allows).
- Time to first token (TTFT): milliseconds from sending the request until the first streamed token arrives.
- Output speed: generated tokens divided by the time between the first and the last token.
- Availability: a failed request (server error, timeout, broken stream) is recorded as a data point with value 0. Failures caused by our own account (authentication, credit, rate limits) are not recorded as model outages.
- Routes: models are measured through OpenRouter; Google models additionally direct via the Gemini API; as a fallback, open-weight models via the Hugging Face inference router. Results of different routes are stored as separate series and never mixed.
- Models: 16 widely used models in a fixed priority order, limited by a daily measurement budget.
Full probe prompt (v1)
Read the following passage carefully. Lighthouses have guided sailors for more than two thousand years. The Pharos of Alexandria, completed around 280 BC, stood on a small island at the entrance to the harbour and is thought to have been well over one hundred metres tall. Its fire was kept burning at night, and polished bronze mirrors may have been used to project the light far out to sea. For centuries afterwards, most lighthouses were simple towers with open fires or candles at the top, and their range was limited by weather, smoke and the brightness of the flame. The eighteenth and nineteenth centuries brought rapid change. Oil lamps with circular wicks produced a steadier and brighter flame, and parabolic reflectors concentrated that light into a beam. In 1822 the French physicist Augustin Fresnel introduced a lens built from concentric rings of glass prisms. The design captured light that earlier systems wasted and bent it into a narrow, powerful beam that could be seen more than twenty nautical miles away. Fresnel lenses were graded into orders by size, and the largest first-order lenses were used on major coastal lights. Each lighthouse was given a distinctive pattern of flashes, called its characteristic, so that navigators could identify it at night by timing the intervals between flashes. By day, towers were painted with bands, stripes or checks for the same reason. Keepers lived at the stations, often with their families, trimming wicks, cleaning lenses, winding the clockwork that rotated the optics and recording the weather in logbooks. In fog, bells, horns and later sirens gave audible warnings. During the twentieth century electricity replaced oil, rotating optics were driven by motors, and automatic systems made resident keepers unnecessary. Radio beacons and later satellite navigation reduced the practical importance of lighthouses, yet many remain in service as a backup and as landmarks, and a large number have been preserved as museums. Task: Write a detailed, continuous essay of at least 400 words that explains how lighthouse technology developed and why each change mattered to sailors. Write in plain prose without lists, headings or bullet points. Do not ask questions and do not stop early.
How the videos are made
- Topics are selected by a deterministic scoring of measured changes (size of the change, how well-known the model is, freshness).
- Scripts are drafted with the help of language models, but they are not allowed to write any number or model name themselves: they may only reference placeholders that are filled from the database. A validator rejects every script that contains a bare number.
- Charts are rendered from our own data; the voice-over is synthetic (text-to-speech).
- Each video description lists the exact data points used, their sources and measurement times.
Corrections
The measurement database is append-only: values are never edited or deleted. If a measurement turns out to be wrong, a correction is added as a new data point with a new probe version. If a video contains an error, we correct it in the pinned comment and the description.
Raw data (last 90 days, updated daily)
| File | Content | Rows |
|---|---|---|
| latency.csv | Own latency probe: TTFT, output speed, availability, HTTP status per model and route | 372 |
| prices.csv | Input/output prices per 1M tokens (OpenRouter models API) | 1348 |
| leaderboard.csv | LMArena ranks/scores and OpenRouter usage ranks | 2679 |
| provider_uptime.csv | Provider uptime as published by OpenRouter | 639 |
Data is provided as-is under CC BY 4.0 — please credit "MONOVION". Third-party figures (prices, leaderboards) remain subject to the terms of their original sources.