Comment by UltraSane
10 days ago
AGI should be able to do every job a human can do using a computer at least as well as the average human.
10 days ago
AGI should be able to do every job a human can do using a computer at least as well as the average human.
That's already been true for a while, you're overestimating the average human. They just have different failure modes.
It isn't even close to true. The biggest problem is that humans performance improves over time.
https://www.linkedin.com/pulse/announcing-aa-briefcase-bench...
AA-Briefcase is a new benchmark for testing models on realistic knowledge work tasks in complex projects built by industry experts. Models are evaluated on multi-week knowledge work projects, each with many linked tasks and thousands of input source files. AA-Briefcase combines rubric and pairwise grading to evaluate verifiable task success, analytical quality, and presentation quality, giving a holistic view of overall agentic capability in knowledge work.
Tasks with many messy input files, conflicting information, and complex deliverables remain difficult for all models. Under a strict all-or-nothing grading scheme per task, Claude Fable 5 leads overall, but achieves a perfect task score on only 3% of tasks. On 31 of 91 tasks, no model scores above 50%.
And what is it worse at than an average human today that can be done on a computer?
https://www.linkedin.com/pulse/announcing-aa-briefcase-bench...
AA-Briefcase is a new benchmark for testing models on realistic knowledge work tasks in complex projects built by industry experts. Models are evaluated on multi-week knowledge work projects, each with many linked tasks and thousands of input source files. AA-Briefcase combines rubric and pairwise grading to evaluate verifiable task success, analytical quality, and presentation quality, giving a holistic view of overall agentic capability in knowledge work.
Tasks with many messy input files, conflicting information, and complex deliverables remain difficult for all models. Under a strict all-or-nothing grading scheme per task, Claude Fable 5 leads overall, but achieves a perfect task score on only 3% of tasks. On 31 of 91 tasks, no model scores above 50%.
almost everything? AGI has to be able to completely replace a human in any information worker role indefinitely.
I think you're speeding past the word "average" in the sentence. I'd argue that current frontier models already exceed the abilities of average humans across the majority of tasks you can do on a computer, although you might be able to argue that they tend to be a bit slower?
That latter part is debatable though - have you seen a non-technical person try to figure out something new on a computer?
8 replies →