Comment by m11a
2 days ago
I dislike pair programming interviews - as they currently exist - because they usually feel like a time-crunched exam. You don't realistically have the freedom to actually think as you would in actual pair programming. i.e. if you wag your tail chasing a bad end for 15 mins, this is a fail in an interview, but it's pretty realistic of real life work and entirely a non-problem. It's probably even good to test for at interview: how does a person work when they aren't working with an oracle that already knows the answer (ie: the interviewer)?
Pair programming with the person for a couple hours, maybe even on an actual feature, would probably work, assuming the candidate is compensated for their time. I can imagine it'd especially work for teams working on open source projects (Sentry, Zed, etc). Might not be as workable for companies whose work is entirely closed source.
Indeed, the other problem is what you mention: it doesn't scale to high throughput.
> i.e. if you wag your tail chasing a bad end for 15 mins, this is a fail in an interview
In all pair programming interviews I have run (which I will admit have been only a few) I would fail myself as an interviewer if I was not able to guide the interviewee away from a dead end within 15 minutes.
If the candidate wasn't able to understand the hints I was giving them, or just kept driving forward, then they would fail.
Exactly! Who calls "researching how to build X, but then letting my pair-programming partner fall down a rabbit hole so I can feel superior" "pair programming".
Way too many career and remuneration focused techbros. Often driven by bullshit management and stack ranking style promotion/firing pathways.
> I would fail myself as an interviewer
You are not most interviewers, alas.
That's definitely up to the interviewer, in which a lot of discretion and trust has been placed. I think a lot of it also comes down to the culture of the company—whether they're cutthroat or supportive. As you get better people into the company, hopefully this improves over time. I know that when we did it, it was never about nailing it on the first try, it was literally about proving you knew how to program and were not an asshole. So, not the equivalent of reversing a binary tree on a whiteboard. The kinds of problems we worked on in the interviews weren't leetcode type problems, they were real tickets from our current project. Sometimes it was just doing stuff like making a new component or closing a bug, but those were the things we really did, so it felt like a better test.
Hiring doesn't scale, period; deal with it.
Exactly. People go like “the ideal way to interview is <whatever they themselves are the best at>”. Pair programming interviews suck and don’t scale, just like every other alternate way of hiring.
IMO interviewing is the biggest bottleneck and if interviewing was decoupled from hiring then it wouldn't be a problem. But this requires a Guild-like organization to manage interviewing/vetting and for companies to use said Guild for hiring. The companies could then do a single team culture meeting (if they wanted) before hiring.
I wish I could remember who, but there is a company out there who conducts tech interviews to create a pool of candidates for their customers. Pretty sure there was a post here about it, but it's lost in my ocean of unread favorites.
5 replies →
For companies that successfully scale their team size, it literally did scale, right? I think you mean that hiring is very difficult to scale.
Define successfully, I'm pretty sure they could have been a lot more successful by giving hiring the attention it deserves.
Exactly. Its almost like optimising for finding your best possible match for marriage. You don't go over a billion prospects, you choose from the ones locally available to you, as they come.
You're right, premature optimization is exactly what it is.
it isn't about scale. It is about core principle of the tech hiring - all the companies hire only the best. Not only it is impossible to scale, it is plain impossible. Even if all the companies hired only "above the average" it would still be a pretty tall order :)
There is no one-size-fits-all for software developers. Different companies are looking to hire people at different experience/pay levels, with different skill sets, etc. Just as with dating, most companies are also going to have some understanding of their own "attractiveness", and not try to date out of their league.
FAANG companies offering industry leading compensation packages and prestige are in a position to be able to hold out for the best (even if their interviewing practices may fail to achieve that), but most companies are just looking for someone that checks the boxes and seems like a good fit.
11 replies →
in the end it did not even achieve that. Those who spent 6 months at last job practicing leet code were not necessarily the best of the best
When I was doing interviews at my old job we did an hour of pair programming and we very intentionally designed the exercise such that
1) if you're qualified to do what we're hiring you for this stuff should be your bread and butter. We were doing restful web apps so the prompt might be "A restful API that takes in a string and returns that same string but backward." If you chase your tail on that for 15 minutes, you're not right for this job.
2) We weren't looking for a specific implementation. You want to make it a GET call with a query param that returns just the string? Neat. A POST with request and response DTOs? Also neat. We're not gonna ding you for doing it one way rather than the other but we're definitely gonna talk to you about why you made the choice that you did and maybe try to tease out another choice you could have made so that we can ask you to compare them. Again, the only wrong answer here is one you can't defend.
3) I'm not here to test your ability to memorize a particular language's syntax, your IDE's autocomplete or your ability to google. If you can articulate that you're trying to create a POST endpoint I'll give you that the correct annotation with Spring is @PostMapping or little things like that.
> i.e. if you wag your tail chasing a bad end for 15 mins, this is a fail in an interview
That’s an assumption. Perhaps following a dead end for a while, realizing it, pivoting, etc is a valuable, positive, signal?
I agree. But what I mean is: that's not how it's perceived in the current interview structure, which lasts maybe 45 minutes or so. Ultimately, going down a dead end means you'd now have 30 minutes to find the right solution and code it up. So the oracle (the interviewer) would probably help you realise sooner that it's a bad idea, so you don't waste your time. That's assuming they know the problem and solution well; if they don't, you'll just lose them and burn through your time.
In a 2 hour pair programming session on an 'unsolved' problem (like an open issue / minor bug / minor feature in a public repo), yes, it will likely not matter if you tried a bad idea, and would both be more realistic and a positive signal.
this is an interviewer problem, unless the candidate is totally silent. A candidate that can't ask questions isn't going to succeed, anyway.
If the candidate will communicate with me, I will offer them a LOT of guidance. It is still very, very easy to tell who knows what they are doing and who does not. You give them a basic but domain-specific task, you give them whatever extra context they need to do it, and you watch them hammer out the code.
It should be a task that is sufficiently familiar to the person applying to the role that they do not need to do -much- looking at docs, and as the interviewer you should be prepared to help them quickly find the docs you already know they will need -- you designed the task after all -- so that they don't waste time with that.
What's important is that they ask for docs when they need them, and that they can understand them, quickly, and use them. It will be obvious if they are using AI because of how long things will take. It will be obvious if they don't know when to reach for documentation, and it will be obvious if they cannot understand the documentation.
Then, they should write a test for their solution. This weeds out 95% of candidates. Talk to the other 5% and you'll find someone who can both actually write code and also discuss design.
3 replies →
I do 1hr pair programming interviews for my company and you have to strike a balance between letting candidates think through the problem even when you think it won't work (to see their thought process and maybe be surprised at their approach working/see how quickly they can self-correct) and keeping them on track so that the interview still provides a good signal for candidates who are less familiar with that specific task/stack.
I'm also not actually testing for pair programming ability directly, moreso ability to complete practical tasks / work in a specific area, collaborate, and communicate. If you choose a problem that is general/expandable enough that good candidates for the position are unlikely to go down bad rabbit holes (eg for a senior fullstack role, create a minimal frontend and api server that talk to each other) it works just fine. Actually with these kinds of problems it's kind of good if your candidates end up "studying" them like with leetcode, because it means they are just learning how to do the things that they'll do on the job.
> maybe even on an actual feature
I don't think this would work unless the feature were entirely self-contained. If your workaround is to give the candidate an OSS project they need to study beforehand, I think that would bias candidates' performance in ways that aren't aligned with who you want to hire (eg how desperate are they for the role and how much time outside of work are they willing to put into your interview).
Another problem is it is difficult to compare candidates whose interviews involved working on completely different problems.
> if you wag your tail chasing a bad end for 15 mins, this is a fail in an interview
Eh, if it's a reasonable bad end and you communicate through it, I wouldn't see it as a fail. Particularly if it's something I would have tried, too. (If it's something I wouldn't have thought of and it's a good idea, you're hired.)
I did a couple of rounds of this with my manager as the interviewer. Personally I really liked the process, and the feedback I got from the candidates was positive (but then again it always would be).
What worked well for me was that I made it very clear to my manager, a man who I trust, that I would not be able to provide him with a boolean pass/fail result. I couldn't provide him any objective measure of their ability or performance. What I could do was hang out with the canditates for an hour while we discussed some concepts I thought were important in my position. From that conversation I would be able to provide him a ranking along with a personal evaluation on whether I would personally like to work with the candidate.
I prepared some example problems that I worked for myself a bit. Then I went into the interviews with those problems and let them direct those same explorations of the problem. Some of them took me on detours I hadn't taken on my own. Some of them needed a little nudge at times. I never took specific notes, but just allowed my brain to get a natural impression of the person. I was there to get to know them, not administer an exam.
I feel like the whole experience worked super well. It felt casual, but also very telling. It was almost like a focused date. Afterwards I discussed my impression of the candidates with my manager to ensure the things I was weighing was somewhat commutable to what he desired.
All in all it was a very human process. It must have taken an enormous amount of trust from my manager to allow me the discretion to make a subjective judgment. I was extremely surprised at how clearly I was able to delineate the people, but also how that delineation shifted depending on which axis we evaluated. A simple pass/fail technical interview would have missed that image of a full person.
This is exactly how I interviewed people and what my replies were like. But I was one of many interviewers, so I would only take 15 minutes, which always annoyed my manager, but I never gave a bad "fine by me", so I still did interviews...
[dead]