Comment by KalskiTheDan

I'd bet that still to this day, most interview questions test recall (of XYZ) and recall is what LLMs are best at. "Explain the difference between X and Y" or "what would you do in scenario Z" are search queries now. A lot more streamlined than individual google searches people would patch together from their search journey...

If I'd ever be doing a hiring spree, I'd give them a real problem. Put them in a position to think out loud. Not "what's the answer" but "walk me through how you'd approach this." Probe the reasoning. When they hit a fork, ask why they'd go left instead of right. Ask about tradeoffs they see. The best/real candidates will naturally say "I'd need to know more about X before deciding" because they're actually thinking through it.

IMO acknowledging the lack of information (without coming from a place of uninformed/non-experienced) almost impossible to fake with an overlay. LLMs always have an answer. Good engineers know when they don't.