Comment by kenschu

Founder of new tech assessment company / mentioned in article here.

We're biased, but we think the old form of take-home assessments (+ classic Leetcode tests, etc) are completely broken. Beyond reasons you all mention - they're completely unreliable today in the age of ChatGPT, etc. Way too easy to cheat.

We're seeing candidates copy takehome instructions into an LLM, paste the solution and submit in <5 minutes. It's hard to write a problem that's (1) low enough scope to solve in a short time, but (2) hard enough that LLMs can't step towards sovling it.

At Ropes - we're using LLMs to evaluate how candidates code, not just the final solution. So HM's can step in and look at a timeline of actions taken to reach the solution, instead of just the final answer. How do candidates debug? What edge cases do they consider? Etc. We think these answers hold real signal and can be answered for the first time async.

We're trying to make this better for candidates too. E.g. (1) shorter assessments, (2) you can often use your own IDE, (3) you're not purely evaluated on test cases, etc. But we're not yet perfect. If this sounds interesting / you have strong thoughts I'd love to talk to you - email is in my bio.