How ORO AI launched CandyBench on X
RL environments and open evals for agentic commerce; CandyBench is their robot candy-handout eval
What ORO AI launched
How TLVC rated the launch
TLVC Rating
About the launch
CandyBench scores robots on a public task: handing candy to trick-or-treaters, with credit only when the candy actually changes hands. Donald Jewkes ran the Halloween night with robots lent by Bracketbot, Pantograph, and Dimensional, and the scoring compares those handoffs with a person and with a self-serve bowl. The eval is for robotics teams that want a number from a crowded station, where the camera gets blocked and the hardware stalls in front of kids. ORO's writeup covers three robots at one station, filmed by a fixed camera from 6:20 to 8:00 p.m. One annotator marked 41 of 380 clips. The stratified estimate is about 580 completed robot handoffs over 95 minutes, with a 95 percent confidence interval of 422 to 738. The short green robot has the highest point estimate, but the intervals overlap, so the report claims no ranking. A vision-language grader matched 10 of 35 human-marked handoffs on a 20-clip check, and a video classifier did not beat predicting zero handoffs on the primary metric. The launch post describes a wider night than that sample. Jewkes said seven robots helped clear 3,057 of 4,035 ordered treats between about 5 p.m. and 8:15 p.m., and he flagged cases of reward hacking, where robots dropped candy or steered children to the basket. The video shows ordinary failure modes around that station: wifi as the blocker, a router that had to be moved, units that burned out or sat idle, and people teleoperating from behind the table. ORO has asked 1X, Tesla Optimus, Figure, Sunday Robotics, and Physical Intelligence to run the same task.
Tags
Questions
Frequently asked questions
The ORO AI launch post reached 14K views on X, with 248 likes, 20 reposts, 31 comments, 44 bookmarks.
ORO AI launched on X on November 3, 2026 with the message "RL environments and open evals for agentic commerce; CandyBench is their robot candy-handout eval".
CandyBench scores robots on a public task: handing candy to trick-or-treaters, with credit only when the candy actually changes hands.
Launchvideo.com produced and distributed the ORO AI launch, handling the launch video, launch-day timing and creator amplification on X.
Launchvideo.com rated the ORO AI launch 3.40 out of 5: hook 3/5, editing / creativity 5/5, copy 4/5, sentiment of launch 4/5, distribution strategy 1/5.








