Comment by codeflo
1 day ago
I don't think those explanations are mutually exclusive.
Yes, there's a large cohort of "senior" software engineers who can't actually code. They bullshit their way into jobs until they're fired and then apply for the next one. These are the people you want to filter out with live coding.
But also, someone can fail a live coding interview for reasons other than belonging to that group.
I think there's a lot of developers who can ace a live-coding interview but who lack the understanding of engineering systems at scale so they'll make your whole codebase worse over time by introducing tech debt, anti-patterns, and inconsistencies. These are the people you really want to avoid, but very few interview processes are set up to filter them out.
There's an assumption that the company's existing senior architects and developers will stop a new person from making the code worse, but also devs at every company thinks their codebase is terrible so it obviously isn't working.
I've seen lots of devs who think their codebase is the only correct way to do things. Lots of overconfident people out there. Inconsistencies are fine as long as there's file level consistency. All that really matters is if you can relatively quickly understand what you are working with. What you really want to avoid is having functions doing 20 different things from 5 different contexts.
Exactly. One negative productivity technical tornado can cause more damage than 10 people who lied about their coding ability.
What engineering interviewing process catches things like this?
Most coding interviews are accompanied by work experience discussions which CAN identify stuff like this, although obviously people can BS.
1 reply →
I agree. Live coding always has a much smaller scope than real software, and after a few interviews it is easy to learn to read the room, even for the worst developers.
I think we can leave companies who don't care about quality out of the discussion, but for those who do, the time to detect those developers is in a probational period, which is not something that most companies really use on their favor.
The problem is this requires a good management that is able to actively paying attention to the work of the developer. Which is often not in place, even in companies who want to prioritize quality :/
Have you interviewed people recently? The "worst developers" absolutely cannot solve basic problems.
1 reply →
You could filter then out much more effectively by letting them sit in a room by themself to write the code, that way you aren't missing out on good candidates who can't function when under abnormal stress(that has nothing in common with the actual job).
I've had take home problems for job interviews that were given a few days before and during the actual interview I only had to explain my code. But I wouldn't be sure this still works as a useful candidate filter today, given how much coding agents have advanced. In fact, if you were a sr dev and had a bunch of guys to bounce this problem back and forth, it wouldn't even have filtered out the bad ones back in the old days. There is very little that is more telling than seeing a person work out a problem live, even if that sucks for smart people who can't handle stress.
I have found over the years that I learn more by asking easier questions and interacting with candidates as they think through problems out loud. Little things that test the critical ability to craft a Boolean expression to accurately characterize a situation can be explored in a brief interview where you have some assurance that they're working on their own, and not just getting an answer online or from a smart roommate. (Sample: given two intervals [a,b] and [c,d], write an expression that determines whether they intersect.). Candidates that have lots of trouble programming "in the small" are going to have trouble programming in the large as well.
1 reply →
If people can explain their decisions, I'd say it's fair game. It would be nice to know up front if someone used AI of course.
The other implication here is that if a candidate can use AI for a take home and ace the interview, then maybe the company doesn't have as tough of problems as it thought and it could fill this seat quickly. Not a bad problem to have.
Leetcode for CRUD app positions is overkill.
I don’t use those LLM tools, but if someone can pass the test with LLM tools, then they can pass the test unless there’s something special about the environment that precludes the LLM tools they use.
Even before the coding assistants I was not happy with homework. It was not discriminatory, everyone looked the same, https://andrewpwheeler.com/2022/03/24/i-have-no-clue-how-to-....
> But I wouldn't be sure this still works as a useful candidate filter today, given how much coding agents have advanced.
Prior to ChatGPT coming out, I gave a take home test to sort roman numerals.
What before was a "here's a take home that you can do in an hour and I can check the 'did you write the code reasonably?'" is now a 30 seconds in ChatGPT with comments embedded in it that would make an "explain how this function works" to be less useful. https://chatgpt.com/share/688cd543-e9f0-8011-bb79-bd7ac73b3f...
When next there's an interview for a programmer, strongly suggest that it be in person with a whiteboard instead to mitigate the risks of North Korean IT workers and developers who are reliant on an LLM for each task.
My solution for this was a propose a problem obscure enough that no LLM tool really knows how to deal with it. This involved some old Fortran code and obscure Fortran file format.
How would someone explain code that was vibe-coded?
5 replies →
We live in a remote world where a hiring process like this is less of an option in most cases
Leaving aside that many companies have pulled back from remote to at least some degree, I'd always push for an in-person day for a variety of reasons. In general, the cost is nothing for a late-stage/end-stage confirmation. And, honestly, a candidate that just doesn't want to do that is a red flag.
4 replies →
I don't know about that. Long ago I interviewed with someone that wanted some trivial C++ thing written on their laptop. I hadn't seen a Windows dev machine before and had no Internet access. I think I'd worked out the compiler was called visual studio and how to compile hello world by the time limit. Not sure that told either of us much.
I share your point of view, but live coding these days are just beyond that testing programming skills. You must know by heart the most common algorithms out there and design solutions that might involve two or three of them to solve a problem in 30 minutes.
Sometimes you spend the whole time trying to figure out how to solve the puzzle that don't even have time to show that you can - actually - code.
> You must know by heart the most common algorithms out there and design solutions that might involve two or three of them to solve a problem in 30 minutes.
You're not going to pass every interview. Some of them are truly outlandish, and require all of the above.
What you need is the technical knowledge to translate requirements into a loose pattern like "This looks like a search problem", then have the charisma (or more accurately, practice) to walk the interviewer through how each search algorithm you know could apply there. Then of course be able to actually write some code.
I've passed interviews where I had never heard of the datastructure they wanted me to solve it with; I just walked them through the tradeoffs of every data structure that I knew applied to it instead.
I’ve been doing this for 20 years. I’ve never worked with a single one of those people. I don’t think I’ve ever even interviewed one where I couldn’t have screened them out based on their resume and a 15 minute conversation.
I’ve worked with plenty of people who passed a whiteboard interview and then spent years actively reducing aggregate productivity.
Why do you need arbitrary (and very short) deadlines, and for someone to stand up at a whiteboard while simultaneously trying to solve a problem and "walk you through their thought process" to filter out people who can't write code on the job?
The short deadlines are because neither the company nor the candidate wants to spend a month on an extended interview. Solving a problem and walking through the thought process are because that's what "coding" is.
> neither the company nor the candidate wants to spend a month on an extended interview.
So says the companies that insist on multi-round, multi-week interview loops.
I don't know about you, but I've never had to live code a PR and explain to my reviewer what I was thinking while writing the code. By "deadlines" I'm referring to the length of the interview. Take home problems theoretically solve both these issues, but they need to be properly scoped and specified to be valid assessments.
8 replies →
In most of the western world, firing employees is a high risk, high cost task. Ideally companies would hire quickly and fire poor matches just as quickly once they've been evaluated in the real world environment of the company. For this to work, on the employee side there needs to be knowledge that this is the company's process, financial depth to deal with the job not being stable, and a savviness to not relocate into a job that's risky. On the employer side, there needs to be a legal and social environment that doesn't punish removing non-productive employees.
The legal environment is what it is and unlikely to change. The social environment is fickle and trend driven. Workers can't always fully evaluate their odds of success or the entirety of risk of leaving a job that's valuable for the employee and employer for one that might end up as a poor match, even if both sides have been transparent and honest. It's a difficult matchmaking problem with lots of external factors imposed and short term thinking on all sides.
Ideally young workers would have an early career period that involves a small number of short lived jobs, followed up by a good match that lasts decades, providing value to both the employee and employer. Much like finding a spouse used to be a period of dating followed by making a choice and sticking with it so a life could be built together, employment ideally should result in both sides making the other better. Today however everyone seems focused on maximizing an assortment of short term gains in search for the best local timescale deal at the expense of the long term. It's interesting how the broken job market and broken family formation process in the western world mirror each other so much.
I suspect the similarity is just a coincidence.
There was a lot of social pressure in the past for permanent marriages. That doesn't mean they were happy marriages. With the social changes in the west in the 1960s, divorce became more socially acceptable. Legal changes meant women had the ability to join the workforce and support themselves. People in unhappy marriages had options to seek happiness elsewhere. Those options didn't exist before.
For job retention, the problem is that changing jobs is often the only way to advance. I lost my best worker because the suits wouldn't give him a raise. He now makes more than I do at a different company. He liked his job with us, but he tripled his pay by leaving. My coworkers all tell the same story. I'm one of the lucky ones that managed to move up in the company, and that's only because I had the boss over a barrel.
There are two ways to interview:
1. Make sure you pick every good candidate, but some bad candidates will slip through as well.
2. Make sure you reject every bad candidate, but some good candidates will fail as well.
Candidates want process #1, but companies have no reason to push for it. The cost of accidentally hiring a bad employee is simply too high, way more than rejecting a good employee. The current system in place prioritizes #2. Yes they are rejecting great candidates, and they are aware of it.
The article is suggesting that #2 will end up rejecting LOTS of good candidates (and potentially ALL female candidates)
Also I don't think being artificially picky is a better filter than just going with some gut feeling after weeding out candidates with fake credentials.
Being picky gives the illusion of choosing when you in practice are bound down by the process.
> Yes, there's a large cohort of "senior" software engineers who can't actually code. They bullshit their way into jobs until they're fired and then apply for the next one. These are the people you want to filter out with live coding.
Genuinely, are there any amount of these at any significant scale in a place like Silicon Valley? I'm not sure I've ever met someone who couldn't code at any of the places I've worked.
Senior engineers are heavily evaluated for their ability to pump out code. If you're not coding, what the hell are you doing at a startup that needs features built to generate any revenue?