Arena is launching the Alignment Index, a public leaderboard that scores how frontier AI models behave when real people use them to get work done. The index evaluates 27 models across more than 90,000 real-world agent sessions , grading them on three concrete failure modes: Unauthorized Action, where the model acts beyond what the user asked, False Attribution, where the model attributes a statement or fact to the user that is contradicted by their own evidence, and Deceptive Completion, where the model tells a user a task is done when it is not. These signals are inspired by the same definitions that labs like OpenAI and Anthropic have published in their system cards , which lets the index function as an outside check on claims the labs make about their own models.
The launch is paired with a $200M Series B at a $3.1B valuation, co-led by Lightspeed Venture Partners and Khosla Ventures, with participation from Salesforce Ventures, 01 Advisors, Dell Technologies Capital, Acrew Capital, Endeavor Catalyst, and existing investors including a16z, Felicis, AMP PBC, QuantumLight, and The House Fund . The company is the same group behind Chatbot Arena and the Agent Arena leaderboard, started as a research project out of UC Berkeley's Sky Computing Lab by Wei-Lin Chiang, Anastasios N. Angelopoulos, and Ion Stoica . The Alignment Index sits alongside their existing capability rankings, extending the methodology from "which model is best" to "which model can be trusted to do what it was asked."
The timing matters for anyone shipping agents into production. Arena is releasing methodology and initial results for 20+ frontier models and plans to expand the benchmark signal by signal, giving builders, procurement teams, and policy watchers a third-party reference point as agents move from answering questions to taking actions on users' behalf in code, analysis, and document work.
Comments (8)
Sign in to join the discussion.
N
Nadia Vukovic3d ago
A neutral third party funded by a $200M round is doing a lot of lifting in that sentence. Who audits the auditor?
T
Tareq H.3d ago
The Alignment Index landing page cold open is clean but that voiceover is doing Werner Herzog cosplay for 11 seconds before you say what the product is. Hook belongs in the first 3.
C
Chiamaka O.3d ago
Rewriting the tagline for you at no charge: 'The scoreboard for AI behavior.' Yours sounds like a compliance brochure.
P
priyanka3d ago
What's the inter-rater reliability on the alignment scores and are the rubrics public? Hoping this isn't another leaderboard where the methodology is a PDF nobody reads.
D
Devon A.3d ago
The reply ratio on this tweet is suspiciously healthy for a Series B announcement. Either the thread is landing or the quote tweets are about to be rough.
Y
yuki3d ago
Rate limits on the Index API? Webhook when a model's score drops below a threshold would be genuinely useful for anyone shipping to prod.
M
Matteo R.3d ago
Evals as a service has brutal margins unless you productize the rubric. Curious if the labs pay to be measured or if enterprise buyers subsidize the whole thing.
L
Lena Voss3d ago
Would love to chat about which frontier labs are participating in the Index vs. opting out. Happy to go on background, DMs open.
Work With Us
Book an intro call to explore our suite of services. We might have a waitlist.
Arena
Measuring how safe and aligned AI actually is
Measuring how safe and aligned AI actually is
Tags
TLVC Rating
Community Rating
No ratings yet
to rate this launch.
Arena is launching the Alignment Index, a public leaderboard that scores how frontier AI models behave when real people use them to get work done. The index evaluates 27 models across more than 90,000 real-world agent sessions , grading them on three concrete failure modes: Unauthorized Action, where the model acts beyond what the user asked, False Attribution, where the model attributes a statement or fact to the user that is contradicted by their own evidence, and Deceptive Completion, where the model tells a user a task is done when it is not. These signals are inspired by the same definitions that labs like OpenAI and Anthropic have published in their system cards , which lets the index function as an outside check on claims the labs make about their own models. The launch is paired with a $200M Series B at a $3.1B valuation, co-led by Lightspeed Venture Partners and Khosla Ventures, with participation from Salesforce Ventures, 01 Advisors, Dell Technologies Capital, Acrew Capital, Endeavor Catalyst, and existing investors including a16z, Felicis, AMP PBC, QuantumLight, and The House Fund . The company is the same group behind Chatbot Arena and the Agent Arena leaderboard, started as a research project out of UC Berkeley's Sky Computing Lab by Wei-Lin Chiang, Anastasios N. Angelopoulos, and Ion Stoica . The Alignment Index sits alongside their existing capability rankings, extending the methodology from "which model is best" to "which model can be trusted to do what it was asked." The timing matters for anyone shipping agents into production. Arena is releasing methodology and initial results for 20+ frontier models and plans to expand the benchmark signal by signal, giving builders, procurement teams, and policy watchers a third-party reference point as agents move from answering questions to taking actions on users' behalf in code, analysis, and document work.
Comments (8)
A neutral third party funded by a $200M round is doing a lot of lifting in that sentence. Who audits the auditor?
The Alignment Index landing page cold open is clean but that voiceover is doing Werner Herzog cosplay for 11 seconds before you say what the product is. Hook belongs in the first 3.
Rewriting the tagline for you at no charge: 'The scoreboard for AI behavior.' Yours sounds like a compliance brochure.
What's the inter-rater reliability on the alignment scores and are the rubrics public? Hoping this isn't another leaderboard where the methodology is a PDF nobody reads.
The reply ratio on this tweet is suspiciously healthy for a Series B announcement. Either the thread is landing or the quote tweets are about to be rough.
Rate limits on the Index API? Webhook when a model's score drops below a threshold would be genuinely useful for anyone shipping to prod.
Evals as a service has brutal margins unless you productize the rubric. Curious if the labs pay to be measured or if enterprise buyers subsidize the whole thing.
Would love to chat about which frontier labs are participating in the Index vs. opting out. Happy to go on background, DMs open.