Comment by jakubmazanec
2 days ago
The problem isn't AI, the problem is companies don't know how to properly select between candidates, and they don't apply even the basics of Psychometrics. Do they do item analysis of their custom coding tests? Do they analyse the new hires' performances and relate them to their interview scores? I seriously doubt it.
Also, the best (albeit the most expensive) selection process is simply letting the new person to do the actual work for a few weeks.
> Also, the best (albeit the most expensive) selection process is simply letting the new person to do the actual work for a few weeks.
What kind of desperate candidate would agree to that? Also, what do you expect to see from the person in a few weeks? Usual onboarding (company + project) will take like 2-3 months before a person is efficient.
Candidate would be compensated, obviously. That's why it's expensive.
You don't need him to become efficient. Also I don't think it is always necessary to have such long onboarding. I'll never understand why a new hire (at least in senior position) can't start contributing after a week.
> Candidate would be compensated, obviously. That's why it's expensive
Ok... take me through it. I apply to your company and after a short call you offer me to spend 4 weeks working at your place instead of an interview.
I go back to my employer, give them resignation letter, work the rest of my notice period (2 months - 3 months), working on all handovers, saying goodbyes.
Unless the idea is to compensate me for the risk (I guess at least 6 months salary, probably more), then I do not see how you'd get anyone who is just a poor candidate to sign up for this.
> You don't need him to become efficient
So what will you see? Efficiency, being independent and being a good team player are the main things that are difficult to test during a regular interview.
And so that self-selects for people who already are unemployed then, right? Most developers I know (including myself) look for a new job while still having a job, as to not create a financial hole in-between. I'd be curious if that doesn't then end up with lower quality candidates who ended up unemployed to begin with?
4 replies →
I'd argue the bigger expense is on the team having to onboard what could potentially be a revolving door of temporary hires. Getting a new engineer to the point where they understand how things work and the specific weirdness of the company and its patterns is a pretty big effort at anywhere I've worked.
> can't start contributing after a week.
Because you have zero context of what the org is working on.
If you work with Boring Technology, your onboarding process has no reason to be longer than a week, unless you're trying to make the non-tech parts of the role too interesting.
> unless you're trying to make the non-tech parts of the role too interesting.
Unless your role is trivial to replace with an LLM, you need to understand the business. Maybe not for really junior role, but everything above - you need to solve issues. Tech is just a tool.
3 replies →
How do you control for confounders and small data?
For data size, if you're a medium-ish company, you may only hire a few engineers a year (1000 person company, 5% SWE staff, 20% turnover annually = 10 new engineers hired per year), so the numbers will be small and a correlation will be potentially weak/noisy.
For confounders, a bad manager or atypical context may cause a great engineer to 'perform' poorly and leave early. Human factors are big.
Sure, psychological research is hard because of this, but that's not what I'm proposing - I'm talking about just having some data on predictive validity of the hiring process. If there's some coding test: is it reliable and valid? Aren't some items redundant because they're too easy or too hard? Which items have the best discrimination parameter? How the total scores correlate with e.g. length of the test takers tenures?
Sure, the confidence intervals will be wide, but it doesn't matter, even noisy data are better than no data.
Maybe some companies already do this, but I didn't see it (though my sample is small).
Might as well use https://en.wikipedia.org/wiki/E-meter