software by Stafford Williamsstafford williams
home blog devlog notes links talks apps about

2025-01-26 10:03am

devin, code-assist

The “First AI Software Engineer” Is Bungling the Vast Majority of Tasks It’s Asked to Do - It took longer than a human, and failed at the vast majority of tasks.

Out of 20 tasks we attempted, we saw 14 failures, three inconclusive results, and just three successes,” the researchers found — a meager success rate of just 15 percent.

  • If this was helpful, please share:

  • Reddit icon
  • Y Combinator icon
  • Twitter icon
  • LinkedIn icon
  • software by Stafford Williams
about
  • LinkedIn icon
  • Icon Letter Mail image/svg+xml Openclipart icon_letter_mail 2010-01-29T13:59:32 https://openclipart.org/detail/29117/icon_letter_mail-by-jean_victor_balin jean_victor_balin icon letter mail mailing unchecked