Comment by tptacek

I shouldn't keep leaving this comment because it's not that useful but: AI-era work samples are a fun problem (you definitely can't just use your pre-agent default work samples!), and we've come up with a couple useful solutions.

If you're in an environment that is open to agents to begin with, one simple thing to do is just tell people to use agents, and ask for the prompts. Prompts are a high-signal artifact, and you can construct rubrics to evaluate them objectively.

I'll get around to writing a piece about this within the next couple months.

Ultimately work samples aren't a technology technique; they're a longstanding concept in management science. Long before the first coding take home was ever given, firms were using work samples to qualify salespeople, support professionals, factory workers, whatever. AI agents are a part of what the work of software development involves, and there's obviously no fundamental reason why you can't work sample them like anything else.