Comment by payneio

FWIW, finished an eval of claude code against various tasks that amplifier works well on:

The agent demonstrated strong architectural and organizational capabilities but suffered from critical implementation gaps across all three analyzed tasks. The primary pattern observed is a "scaffold without substance" failure mode, where the agent produces well-structured, well-documented code frameworks that either don't work at all or produce placeholder outputs instead of real functionality. Of the three tasks analyzed, two failed due to placeholder/mock implementations (Cross-Repo Improvement Tool, Email Drafting Tool), and one failed due to insufficient verification of factual claims (GDPVAL Extraction). The common thread is a lack of validation and testing before delivery, combined with a tendency to prioritize architecture over functional implementation.