Comment by rockemsockem

It usually takes significant time for someone to get up to their base level of effectiveness in a new organization, so I don't really think this is better, in fact it's much worse because "doing the job" includes a lot of ancillary things like familiarizing oneself with the tools, metrics, codebase, etc.

Much better to just have a live coding test that tries to measure your communication, effectiveness at working on a problem, and your raw coding skills.