Frontier coding agents claim work they didn't do
Source: LessWrong ·
Tara Research Team arXiv | Twitter thread tl;dr. We built OverclaimBench, which asks frontier coding agents to review a corpus of code or text documents, then compares their final report with their transcripts. Pooled across 12 frontier agents and five scenarios, two-thirds of runs didn't open some of the files they were asked to review. In ~4 out of 5 of those incomplete runs the agent didn't tell the user, often claiming to have read everything. Skipping these files had real consequences - runs that overclaimed missed planted defects almost twice as often as runs that touched every file. Fig
Frequently Asked Questions
What is Frontier coding agents claim work they didn't do?
Frontier coding agents claim work they didn't do is a article discovery curated from LessWrong. Tara Research Team arXiv | Twitter thread tl;dr. We built OverclaimBench, which asks frontier coding agents to review a corpus of code or text documents, then compares their final
How can I access or experience this?
You can experience this content directly on its original host (LessWrong) by clicking the "Open Original" button.
Where does this content come from?
Curated from LessWrong by the OnSuggest team to surface the most interesting and serendipitous content on the web.
More like this
All Culture & ideas discoveries →‘It: Welcome to Derry’ – “A Story of Derry” Finally Reveals Stephen King’s Hero in a Half Shell [The Losers’ Club Podcast]
What’s this? What’s this? Another season of It: Welcome to Derry on HBO Max? Not so. It’s a bonus episode on a bonus episode, a sweet Halloween treat, called “A Story of Derry” that gives us a deeper look into the origins of Pennywise and a cameo by our favorite Ninja Turtle. Join The Losers’ […] The post ‘It: Welcome to Derry’ – “A Story of Derry” Finally Reveals Stephen King’s Hero in a Half Shell [The Losers’ Club Podcast] appeared first on Bloody Disgusting! .
Patrick Stewart's '90s Fantasy Movie With Neve Campbell Is Streaming For Free
Neve Campbell starred opposite Patrick Stewart (as a ghost) in this '90s Oscar Wilde fantasy story adaptation, which you can currently stream for free.
Patrick Dempsey's Thanksgiving Horror Movie Is Worth Revisiting This Holiday Season
If you want to watch a Thanksgiving movie that features amusingly gross deaths, Patrick Dempsey, and more, this Eli Roth horror flick should do the trick.
Deadpool 2's Rob Delaney Debunks Popular X-Men Fan Theory About His Character Peter
Rob Delaney, who plays Peter in the Deadpool film franchise, has thrown cold water on a popular fan theory about the true identity of the character he plays.
Steve Buscemi And Sam Rockwell Have VERY Different Takes On Getting Killed On-Screen
They've died their fair share of times.
One Of The Best Stephen King Adaptations Is Back On Netflix, And I Can't Wait To Watch It Again
It’s definitely my favorite.