Comment by mikek
11 years ago
At a certain point, your resume should speak for itself. The fact that experienced engineers with impressive resumes are put through these types of interviews is insulting and frustrating to the interviewees.
Succeeding at these whiteboard questions requires weeks of preparation. You need to practice, practice, practice. After enough practice, you are pretty likely to pass. So ultimately, it is more of a test of "how much do you want to work here." If you care enough, you can pass. That said, these types of interviews do favor fresh grads over experienced programmers (you have algorithms and data structures fresh in your mind), which means that they are flawed in IMHO.
These types of interviews work really well for the "I got 1600 on my SATs and went to {insert high profile school here} crowd" There are books out there just to prep you for Google interviews I see these as very similar to SAT test prep books. I'm not so sure Google is really interested in hiring the best engineers but rather a specific type of engineer.
Think of the problem from Google's perspective though. At some point, you have tens of thousands of candidates and you need a system to quantify how good they are. Further, it's reasonable to have false negatives (people you don't hire that should have been hired) but really bad to have false positives (people that you hire that you should not have). Together, these boil down into the de facto whiteboarding interview process.
This has got to be the biggest hiring fallacy I've ever heard. "It's better to reject a good candidate than hire a bad candidate."
That's completely false, and anyone who says that is completely ignorant of Bayesian logic.
Here are some simple numbers.
Suppose that a "good" candidate is a 1-in-100 find. Suppose that a "bad" candidate has a 1% chance of tricking you into hiring them anyway.
Every time you pass on a "good" candidate, that is greater opportunity for a "bad" candidate to trick you into hiring them!
Counterintuitively, if you pass on too many of the "good" candidates, in your overzealousness to reject bad candidates, you're actually INCREASING THE ODDS OF A BAD HIRE.
This is management porn. "It's better to reject a good candidate than hire a bad candidate." It makes the manager feel good. "Wow! That candidate seemed smart, but I rejected him anyway! I'm such a great leader! I make the tough decisions!"
tl;dr summary
Because "good" candidates are rare, every time you pass on a good candidate that increases your odds of making a bad hire! This is simple Bayesian reasoning!
53 replies →
If only whiteboarding interview processes actually weeded out false positives.
In practice, they select for people with good memorization skills. If you can remember the details on a ten dozen different algorithms and data structures, you can pass one of these without having a single lick of creativity or skill.
I say this as an employee who has worked alongside many unskilled drones who made it past the algorithmic interview process.
43 replies →
No, it doesn't.
Say I pass the invert a tree question. Does that prove I am good? That I can design a product, listen to customer requirements, come up with ideas that elude others, read a research paper and turn it into something commercial, solder a circuit board, make a schedule, mentor junior engineers, write documentation? No. All inverting a tree tells them is I studied trees recently, and/or I'm at least superficially clever.
Google lets go of plenty of people. They aren't making perfect hires.
3 replies →
> you need a system to quantify how good they are
But do you? How catastrophical would it be if Google accidentally hired not the top 0.001% but instead somebody from top 0.01%? I mean, same people that build everything else around us and the world still didn't collapse yet? I get it, you have to have standards, and you don't want to waste time and money on somebody that can't pass fizzbuzz test. But once you've moved past that - do you need to obsess so hard on ranking and quantifying it? There are probably some positions in Google that require people to invert binary trees (speaking figuratively) day in and day out, but my decades of experience in the industry shows that most positions aren't like that. I've not worked for google but I'm pretty sure it's also this way there. If you hire smart and competent people, it's OK even if they don't reverse a binary tree here or there.
Yes, exactly, since it's Google, they can afford to have strong filters which will definitely filter out a bad hire, may be at the expense of rejecting a good candidate. Their policy might expect good candidate to get absorbed in Google one way or other, but they definitely want to filter a bad hire. However, in the recent years, I have seen this to be not working as expected. Still, I don't think it matters to Google as an big organization.
12 replies →
This cracks me up a bit. I don't have anything against whiteboarding, even though I find it meaningless. With this you more often than not have false positives. And if Google had so many resumes they have to filter through, then why do they have all these recruiters literally spamming you to join? they have recruiting issues like many others tech companies, but the whiteboard is really a bad filter. I've seen very much capable,competent and experienced engineers being denied just because they didn't spend too much time learning cute algorithms or coding in the whiteboard, in front of interviewers sometimes more focused in showing off how smart they are.
"Google's perspective" shouldn't be a monolithic thing. Sure if you're sorting through candidates that were dropped on you then this process makes sense. But if you've discovered someone that by reputation and demonstrated code base has already excelled, and you're actively chasing them, then the interview process for that individual has to change. It's more about seeing if they'll thrive in your environment, and convincing them that you're an inevitable part of their career.
It's true that evaluating candidates is a hard problem, but you'd think that at the very least they would try to evaluate them based on skills that will actually be used in the job. Judging programmers based on their whiteboard skills is like judging a ballet dancer based on how eloquently he can describe his training routine.
Problem is if your system is actually capable measuring up to the claimed degree of accuracy, or just stabbing in the dark after a certain point.
IQ tests supposedly become inaccurate after 150/160, so you wouldn't really want to put too much emphasis on 180 vs 170 at those levels.
Well, my suggestion would be to use Page Rank to sift through the resumes. If you are respected by respected computer programmers then your "score" goes up. They could measure that by mentions in a blog, twitter "conversations", Github commits on the same project, etc.
"Hmmm... it looks like this guy really gets around. He's on the same project at GoodProgrammer A and GoodProgrammer B. He also tweets about functional programming and was mentioned in GoodProgrammer C's latest blog."
People have previously mentioned hallway discussions at Google routinely devolve into bragging about SAT scores and GPAs.
Nothing says Google must accept sub-par technical people, but Google fails to realize you don't have to be the top 0.0001% in absolute algorithms intelligence to do great work.
Google's basic theme seems to be "you must be low latency ultra smart in our narrow interest areas" to pass an interview (or well-connected in other ways to lower scrutiny), but Google's main problem these days isn't lack of smart people — it's lack of projects with meaning and lack of how to organize smart people into anything resembling a coherent world-moving force.
In The Plex is great read for insight into their entire process. Basically, Google is run by people who have never been told "no" to anything in their lives, so they continue to think they are the best at everything until reality forces them to reconsider their delusional assumptions ("montessori naivety").
In The Plex is great read for insight into their entire process. Basically, Google is run by people who have never been told "no" to anything in their lives, so they continue to think they are the best at everything until reality forces them to reconsider their delusional assumptions ("montessori naivety").
Which is unfortunate, because objectively Google doesn't actually have a very impressive track record of creating successful products and services given its size and resources. It has a goose that lays golden eggs (the on-line advertising business), two strong on-line services (search and mail, both over a decade old) and two well-established platforms (Android and Chrome, both nearly seven years old). Other than those, most things Google tries seem to land somewhere between unremarkable and complete failure, with Google+ surely the most obvious example of the latter. They also seem to be developing an unenviable reputation for killing things off as a result, which is going to make it more difficult for them to succeed with future endeavours. Whatever workforce their hiring process is producing for them, it doesn't seem to be one that is very good at creating successful new projects.
4 replies →
> People have previously mentioned hallway discussions at Google routinely devolve into bragging about SAT scores and GPAs.
I've been at Google for 3.5 years (in the NY office) and have never heard this. I don't think I've ever even seen anyone wear a college t-shirt/sweater.
5 replies →
My friend worked at Google a few years ago (in a non-engineering role). After his in-person interviews, the recruiter called him back to ask him how many hours he worked during college (to pay his way), so as to excuse is less than 4.0 GPA to the hiring review board. He had completed his undergrad degree was 20 years in a subject not directly related to the Google position. He since had an relevant MS degree and real-world work experience, but that undergrad GPA was apparently still really important.
> Nothing says Google must accept sub-par technical people, but Google fails to realize you don't have to be the top 0.0001% in absolute algorithms intelligence to do great work.
But if you're Google, why wouldn't you aim for the top 0.0001%? It's not as though you have a shortage of candidates.
2 replies →
What is "Montessori naivety"?
2 replies →
> Google is really interested in hiring the best engineers but rather a specific type of engineer.
I think you made an excellent point. It seems that outside a tiny minority of world-class experts, Google is more interested in hiring interchangeable workers.
That's how big corporations in general work.
Hence Go.
2 replies →
Exactly. I see it as a tacit admission that you are effectively a cog in a wheel if those are the kinds of questions they are asking.
I know some people get frustrated by those kinds of questions, but I'd just see it as a reverse weeding out process. If you still want to work at (insert company here), just build something valuable and have them acquire you. At that point they'll acqui-hire you and you'll leap frog all the people who got in because they had Knuth books memorized.
Really? Being able to solve basic CS101 problems requires some sort of privileged upbringing? I grew up poor - my family lived off food stamps for a while. This stuff is easy. I find it disgusting the level of entitlement swept throughout this thread. Oh Google HAD to give you that job, it was yours and they stole it by testing basic skills.
You know, there is something to be said for someone who has a good product sense, but not good developer skills. They sound very valuable, so why are they applying to a developer position? Just because you are unqualified for one job, doesn't mean you deserved that job, or that you're worthless.
Go do what you're good at.
https://www.youtube.com/watch?v=qr638pCfPxs
I suspect the issue is that programmers right out of college are strong on testable skills (like how to do X to a linked list) and weak on experience-based skills that are harder to test (like how to keep projects under budget or how to manage technical debt). Whereas veterans naturally tend to be the opposite. And hiring processes naturally tend towards what's easier to test.
As such, the best approach to these interviews might be to say, "Look, I could take a stab at that algo question, but I haven't touched that stuff since school, and if I needed it I'd google it. It might be more productive if we talked about what I've been doing day-to-day for the last ten years to ship successful products, because that's the main value I can bring to your company."
Not just books. There are even courses: http://Interviewkickstart.com.
I aced some data analyst interviews fresh out of grad school. A lot of the questions were excel and probability questions, which I used to teach. Now, I have more experience and knowledge, but I would be much slower in those interviews.
>not so sure Google is really interested in hiring the best engineers but rather a specific type of engineer.
Any mature company should focus on finding those specific candidates - the ones who fit the company and specific team, on several dimensions beyond standard skillsrequired for the job. That's someone who will be called the best candidate in that specific company. And it's absolutely natural that s/he won't be considered "best" insome other company.
I interviewed @ google several years ago as part of some due-diligence they were doing to decide if they wanted to acquire the startup I was working at, [redacted].
It was several hours, with many interviewers one at a time, of academic exercises. Some I could see were relevant to some of Google's projects, but that wasn't the emphasis- it was very much a "how much do you remember from CS courses?" (with a few exceptions- one had me explain why a snippet of C would crash- something more what I would expect a senior engineer to know that was more practical-oriented).
I _think_ I did pretty well (again, it was part of a bigger due-diligence-- another story altogether, so I never heard), but in the end I was struck by the fact that (a) not a single interviewer had looked at or asked about any of my publicly available work, and (b) not a single one even asked something along the lines of "what are your strengths / what do you think your contribution would be" (or weaknesses, for that matter, but that seems less important).
Of course, one of the other guys I worked with lucked out and got interviewed by one of the paypal-mafia, and ended up with a reportedly very stimulating hour long discussion about actual work, strengths, and actual situations that one would hypothetically need to handle at google.
Your previous work is what gets you in the door, but interviews at Google are explicitly structured so that you need to demonstrate competence right then and there.
And ye olde "what's your greatest strength/weakness" question sounds like a terrible way to judge an engineer.
Makes sense for google. And it's not that I didn't appreciate doing all that (and I did do well), it just struck me how they knew nothing else about me afterward except that I could answer academic questions, and how different that was from the startup world- where, at the very least, you want to know what they're passionate about and that they are productive.
Asking an engineer what their strengths are and then to demonstrate or explain in concrete terms is how you find out their unique value proposition. I'm not talking about "I'm a hard worker" type garbage, I mean "my strongest experience is in systems programming on Linux." "How so?" "For example, I wrote a specialized TCP kernel module that works like this..."
Again, not saying that should be the whole interview- but there are some non-academic skills and experience (like managing and leading a critical system used by thousands of people) that are worth more than a few weaknesses in problems that were already solved years ago and that are easily looked up. It's how you should judge people looking to get into a graduate program, but falls short if you're trying to build teams of awesome fury that need more broad experience and even (gasp) unique experience that you don't have a ready-made question for ;-)
1 reply →
> not a single interviewer had looked at or asked about any of my publicly available work
It could be dangerous. Let's say you explain some interesting techniques you used on your GPL'd software and a Google product starts doing something suspiciously like what you showed them. You may even sue and end up making their source public after some litigation.
You can't make anyone making source code public, only stopping them from distributing copies of code which you have authored. Copyright and patents works very differently, and its quite hard (impossible) to copy source code wholesale from a interview by accident.
If they did makes copies of an applicants copyrighted work and distribute it, then they could get sued. Some companies has been know to explicitly ask applicants to bring source code from previous work places (also called industry espionage), and those been clearly intended crimes and not something that just happened to company during the process of an interview.
Replying to myself- too late to edit. Some other comments in here lead me to believe there would be experience-related questions ideally these days. Again, my experience was a few years ago :-)
"How much do you want to work here?" is probably a better predictor of your performance on the job than your experience is. I don't see the problem with that: IMHO, the fresh grad who really really wants to be there should be hired over the experienced veteran who's just putting in time. I'd like to think that an experienced veteran who really really wants to be there would also get the job, though, over both of the other candidates.
> At a certain point, your resume should speak for itself. The fact that experienced engineers with impressive resumes are put through these types of interviews is insulting and frustrating to the interviewees.
Only to bad interviewees.
You'd be shocked to hear the number of impressive résumés I have seen who turned out to be absolute no-hires after 20mn on a white board. By "absolute no hire", I mean the kind where some of the interviewers tell me in the debriefing "I will quit if we hire this guy".
Not all excellent engineers have impressive résumés but they all share a common characteristic in my experience: they all really enjoyed working on the problems I asked them to solve on the white board. I even had some of them be disappointed when we ran out of time because we were having interesting conversations about the problem.
> You'd be shocked to hear the number of impressive résumés I have seen who turned out to be absolute no-hires after 20mn on a white board.
Hmm, at least in my experience it's really quick and easy to tell if someone is bullshitting about their resume or not by simply talking about the different projects they worked on. I know a few developers who didn't go to a 4 year school so they don't know all of the correct terms or algorithms (one even interviewed for Google and failed) but I would hire them in a heart beat because they are really awesome.
I still feel some sort of pair-programming exercise or something similar is better than simply trying to solve problems on a white board. Unless you're in a SCIF you're going to have access to the internet to refresh your memory, etc so I've never found utility in asking academic questions. In fact I've had a few people who could pass the academic questions but couldn't build an application to save their life so I don't even bother anymore.
Google doesn't have any SCIFs...do they?
1 reply →
>By "absolute no hire", I mean the kind where some of the interviewers tell me in the debriefing "I will quit if we hire this guy".
That's just a self-reinforcing selection bias - you hire people that are good at these kinds of problems/enjoy them - you filter out the rest by default and it becomes ingrained in the culture - it doesn't say anything about the suitability of the candidates you filtered out. If you said "we hired a guy who had a great resume but he couldn't do algorithmic problems, turned out to be a bad fit for the company" - that would still be a single data point but at least it would be relevant.
I would agree with what you said if you replaced resume with reputation. Anyone can fill out a nice resume.
One of the challenges with Google's interview process [0] is they are very worried about hiring mistakes, and also know that the process is very noisy. As such, they interview a tremendous amount of people for hiring. In addition, they've found it more efficient to just keep a high bar (missing many good hires to keep out 1 bad hire) than to conduct more than 5 interviews.
The one thing I've wonder is - if they want the top 100 computer scientists in the world to work for them, will this process produce that? Or is there another process for them?
I think the most effective way to get a job there is to build a small company and get acquihired.
[0] I don't work at Google, but I've researched the process in anticipation of an interview.
Well I know one mistake they may have made in hiring. Guy I used to work with had a totally unclear design process and couldn't complete his work. He would design flawed algorithms that made no sense to anyway and then he asked for lead a project which he dug himself into a hole and had me pulled in to help him and then abandoned me and left the company. Spent 3 months cleaning up his mess. I wonder how he is working out for them...
Ah, they are very unlikely to get the top 100, but very likely to get e.g. 100 of the top 200? Now I think I understand the trade off they are going for (e.g. Getting 100 of the top 200 is more efficient and they are "good enough")
Eh I don't know that they're getting 100 out of the top 200 either. At some point Google's reputation for an insane hiring process causes the best computer scientists to not even bother applying because it's a waste of their time. Those people have many, many options that would pay them handsomely for their talents.
Look at it this way. If you're a cream of the crop programmer and you're looking for a job, Google has the most drawn out and arduous hiring process. Even if you apply there and start the interview process it often takes a lot longer than the industry average to get to an offer. In the meantime a multitude of other companies have probably already determined that you are an excellent programmer and tried their best to snap you up.
Google seems to be happy with the tradeoff they're making but I'm pretty sure their process is not optimized for getting the best people in the field.
Most of those 200 would not even apply for Google knowing their process.
Acquihired people still are interviewed.
It's also stressful. To be fair the tree problem he got was easy, but brain freeze can kill you. I flubbed a trivial problem at a FB interview that still haunts me today.
Often it's completely free-form too. So you have to pick the language, the data model, the data structures, the traversal mechanism (recursive vs. iterative), state to track (allow duplicates? prevent infinite loops?) and how to test the example all at once before even getting _started_ with their problem description.
Many of the interviews could actually work if they broke the process down into 8 questions instead of one (how would you represent a tree? a binary tree? how would you find node X? how would you move node X to the other side of the tree?). But, the ability to accommodate understanding takes empathy and human connection, not being a jerk programmer on interview duty who is browsing HN while listing to the interviewee mumble on your speakerphone.
Never underestimate the lack of ability in the interviewer and not the interviewee. Except, the company always believes itself to be more competent than outsiders, so you're ultimately left with being condescended to unnecessarily.
Ha, I totally messed up a question for a front end interview today that could have easily been solved with timeouts. I didn't realize timeouts had resets as I haven't used them extensively and bam I'm trying to essentially design a reset for a generic counter. Boy I felt ridiculous. As soon as he said use timeout reset it's immediately obvious what I had to do. I'm not a bad programmer but between brain freeze and whiteboarding, and interviewing day after day for a few weeks, it just didn't occur. Killed me and I hope I don't miss out on an offer from a company I was actually really interested in because of some bullshit brainfart on something I'm not particularly well versed in.
Whatever! They asked me back but it sure is a frustrating feeling even if I tend not to appear flustered until I walk out, introspect, and realize what a doofus I was.
>>To be fair the tree problem he got was easy
They don't just stop at binary trees. Think of it more like memorizing hundreds of magic tricks that could be done using a deck of cards. The tree is more analogous to those deck of cards. The data structure remains constant but the tricks you can do keep changing.
I know of situations where candidates have been grilled whole full hours on tree traversal methods, and then asked to code up perfectly working solutions for them.
Do these people seriously think people who can invent 50 years of CS research by sheer analysis over an hour on the whiteboard, will be ready to work for them to write shell scripts? If not you are only testing memory skills.
In my academic days I used to refer to say "physics-standing-up" is a different subject from "physics-sitting-down" and needs to be studied separately. Some years after I left academia I had the good fortune to get in contact with the person who introduced me to acting at a very young age and thank her for her contribution to my academic career. About 30% of my success was due to my comfort working in front of an audience--even a panel of examiners is "an audience"--and I had a huge advantage over my peers because of it.
One thing that I'd recommend to anyone seeking any kind of job is taking some improv courses--they are pretty common in major centers and even some smaller towns, and can really improve your ability to deal with this kind of nonsensical interview situation, which tests a bunch of skills that have nothing to do with your ability to actually do the job.
Bingo, unfortunately, this does happen. Happened to me at Google interview with one of the rather trivial question.
Happened to me at Twilio.
The answer was transactions (database). You never forget that sort of mistake.
2 replies →
Yup. I could see myself getting brain freeze on this question in an interview setting.
I'll disagree that these questions require weeks of preparation. I think it's more just unfair because they favor a certain type of person: those whose interests align more strongly with the theory and algorithms and less with building things.
Of course, the irony is that for vast majority of the people they hire, most of the work they do will be building and not much algorithms work. So then what will happen is either you have people who are great at building things and force them to suffer through the interviews or you have people who are great at algorithms but not that interested in programming.
Of course, you might also have people who are great at both, but I wonder how many of them there are..
I don't necessarily agree though, I think if you can reason about a problem, you are infinitely more valuable than someone who memorizes problems and just applies them blindly.
Correct me if I'm wrong, but a binary search tree is a very simple data structure. It has rules, and just by it's definition, and the definition of "inverted", you should be able to come up with an iterative solution, even if it's just walking the tree and re-creating it.
It may not be the best, but it's a solution, it's a starting point.
But if you're interviewing for a job solving problems, shouldn't you be able to solve problems? Even if they aren't the best solution? Genuinely asking here, as this approach seems to have a 85-90% success rate for me.
It depends how your interviewer is ranking/scoring you. One interviewer might be impressed with your ability to talk through and solve the problem from first principles, where another might ding you for not instantly knowing the answer. The former type of interviewer would be happy to give you hints and prompts, the second is a stone wall. I try to be the first -- it helps assess how a candidate responds to coaching too! -- but there's a spectrum between "nudging" and "telling them the answer straight out".
> where another might ding you for not instantly knowing the answer
While there certainly are incompetent interviewers, I think ones that extreme are rare.
2 replies →
Is a binary search tree a problem that the Dev would have needed to solve? In over 5 years as a dev, I have never once needed to solve a binary search tree. Why not ask the candidate how to perform a kidney transplant.. That's solving a problem too. White boarding algorithms is not relevant to the majority of Dev jobs. It's related, but not particularly valuable as a screener.
I'm not sure how asking someone who's going to be working in computer programming something about computer science is equal to asking someone who's going to be working in computer programming something about medicine. Those two things aren't equivalent. Could you perhaps clarify?
Engineering is the use of knowledge/science to solve problems, right? So why is asking someone to use knowledge to solve a problem not applicable to the hiring process?
To be clear, this isn't the same question as "why are manhole covers round", or "how would you move a mountain".
1 reply →
Typing "how to reverse a binary search tree" into google is not the same thing as solving a problem.
If these problems were so reliant on individual problem solving ability, there wouldn't be dozens of books with reams of answers to these questions on the market. These questions test memorization and nothing else.
I mean, that's a little strong. My undergrad degree is in math, so I would be thrilled if I was sitting in an interview and was being asked questions like that instead of stuff about how computers actually work.
I wonder how much of it is legacy.
Google said early on they want to hire the best and brightest and that was the most important. We got the mantra that passing over a good person is better than hiring a bad one. And they were right. But they're a huge company now. They, and other gigantic tech companies are always complaining they can't find 'good people'.
But how do you make this philosophy scale? Do you say, "This person is smart but lacking some skills so we'll invest in them"?
I have no idea. :(
Smart people will learn what they need to to do a job if they're interested in it.
Google has decided that the only way to tell if someone is smart is by asking academic questions.
Even though anyone with half a brain knows that there are other ways to gauge how smart a person is... such as experience.
Does anyone really think Max can't look up how to invert a binary tree if he needs to do that? He obviously knows how to read and learn. So why is that a reason to reject someone?
Google is filled with smart people.. but their interview process is pretty stupid.
> Google is filled with smart people.. but their interview process is pretty stupid.
Google is filled with people who can solve Data Structure / Algorithm questions in 50 minutes on a whiteboard. Their interview process selects for people whose past experiences led them to being able to solve Data Structure / Algorithm questions in 50 minutes on a whiteboard.
The extent to which solving Data Structure / Algorithm questions in 50 minutes has anything to do with building maintainable production software, developing talent, a good working environment, or even this mysterious concept known as "being smart" is up for debate.
1 reply →
> Google is filled with smart people.. but their interview process is pretty stupid.
My guess is that they're unable or unwilling to improve the process but they don't really have to since they're deluged with candidates.
1 reply →
> The gigantic tech companies are always complaining they can't find 'good people'.
The "can't find good people" argument can usually be solved either 1) raising the pay, or 2) increasing the labor pool (i.e. H1B, etc). Raising the pay is what is done in finance because it's harder to make the argument that the required "skills" are lacking in the labor pool. To enter investment banking, and then private equity, it often only requires a B.A. in French literature and willingness to work 90 hours.
The "supply problem of engineers" is the great lie of our industry of our time.
I guarantee if dev positions paid as much as medicine or ibanking you'd have more devs. Not necessarily really good devs, but certainly people who pass the Ivy + 1600 SAT sniff test and can memorize the SAT book of Google interview questions.
They'll do it when they start to care more about how much their employees cost. Screening too narrowly drives up the price.
I wonder about that. What percentage of their employees are recent college graduates? From all of these anecdotes, it seems like their process is optimized to select cheap recent CS grads.
I agree. I feel the overpowering urge, when asked this sort of basic interview question, to shout back "WHY?". What does inverting a binary tree have to do with software engineering, at all?
Google (and other interviewers) might as well be asking me to name the last 1000 digits of PI. The Kafkaesquen ability to recite how to do the task correctly reflects nothing about the candidate, except the strength of their memory.
It boggles the mind that the field of computer science does not have any standard way to detect the competency of a person. It is extremely discongruous for a company with as many smart people as Google to have such an absurd hiring policy.
This spoof comment is hilarious. The last 1000 digits of π!
At my job, our interview guidelines explicitly say that we don't expect non-graduates to be able to whip out fancy data structures. We want to see that the person can program, how he thinks about problems and system design, and how they deal with being stuck. I've seen people with impressive CVs and PhDs that were unable to solve very simple programming tasks.
Google interviewers are strictly told to ask these data structures and algorithms questions, it doesn't matter to them if they are relevant for future work. I have even read reviews by PhD candidates where even they were asked these questions in the interviews. When I interviewed with Google last year, I remember that the interviewers had a blueprint of the type of questions they were allowed to ask. They hardly concentrate on your resume during the interviews.
At what point during one's career should one simply be hired at face value and be exempt from interviewing? After 10 years? 20? 30?
"I see your resume says you've been writing software for 30 years, and wrote this really well known tool. Wellllll....That's good enough for me. You're hired!"
Background: I worked at Google for 5 years and did a bunch of interviewing for them.
I don't support skipping interviews even for good candidates, but if you were going to it would work the other way around -- you'd value recent accomplishments over more distant ones.
People don't understand that cutting out folks who had their name attached to some big successes but can't solve a toy problem in an interview is usually a really good thing. It's a strength of the system more often than a weakness. You wouldn't believe the number of senior candidates I interviewed who had really impressive resumes but were clearly checked out from doing real technical work -- folks who obviously no longer lived close to the code and either were very rusty or just no longer had the temperament to spend time thinking through a slightly tricky piece of code.
One of the reasons not to skip an interview for a candidate you're confident is good is that a good interview can be a great sales pitch to come work for your company.
I've interviewed and worked with 30 year programmers and for some of them it was obvious could not program and their resumes were concocted. I would never rely on a resume.
When they buy your funky startup
> At a certain point, your resume should speak for itself.
IIRC, a Portuguese proverb states that paper accepts whatever is written on it. One should always take a self-assessment with a grain of salt.
He self-assessed writing Homebrew?
"Self-assessed" as a resume speaking for the candidate.
Without running a `git blame`, the interviewer cannot say how much of the code is from the author and how much is contributed by someone else and even then asking questions is an important part of any interview.
9 replies →
Another way of looking at it is that they do these kinds of interviews particularly because they do want to hire fresh grads (and ideally, the best 5% of those) over experienced programmers. And if that's the case, then it's not really flawed, it's doing what it's supposed to do.
Actually, Max himself said that it "Felt like they would have preferred a fresh comp sci grad."[1]
Investment banks recruit bright young graduates, putting them through a tough selection process (c.f. all those hard-working interns we keep reading about), and select the ones who perform/conform the best. The rationale for bringing in young (as opposed to experienced) people is that they'll work their asses off, can be moulded to fit the culture (i.e. independent thinkers aren't necessarily sought), and (perhaps most importantly) are too junior to present a threat to the incumbents.
If Google's going down the same path, it may be a sign that their corporate culture is solidifying, which would be a bad sign, IMHO.
1: https://twitter.com/mxcl/status/608698037100244992
I think this is probably overall true. I'm a bit cynical, maybe, but the set of tech used by developers here at Google does not necessarily intersect with what is used in the rest of the industry. So much is proprietary, and best practices are heavily codified and well understood. So in reality I suspect that new grads are as valuable or more valuable to Google as experienced engineers since it's unlikely the specific technologies an experienced engineer has under their belt will be specifically applicable here.
For the top elite of experienced engineers that may not be the case, as they will come with some overall good analytical skills that Google wants. But for most in our industry, what we have learned over the years is how to be good in various sets of tech around our platforms of choice... and that is mostly irrelevant to Google, as they have their own platforms.
It always surprises me when people like Pacino or De Niro talk about going for auditions - I'd imagine they had a large enough body of work that directors would know what they can do.
But then it occurs to me that they won't have seen them in -this- role acting -this- way for -this- film/play/whatnot and that's what the director needs. And given the amount of money and risk involved, it's probably wise to check first...
That's true. However In Google defense, they tell you upfront what kind of interviews you're going to get and their process it's pretty well known anyway (with all its flaws). I went through the same pain myself a few years ago and I'm not disagreeing that the chap should have just got the job there, but their process is well known and perhaps he just had the wrong expectations.
How do you practice answering whiteboard questions?
By re-learning computer science concepts you haven't had to use in real life without Googling in a decade.
(I'm at Microsoft, FWIW, every engineering team here does hiring their own way)
My team does whiteboard questions, but we try to keep them practical.
Typically they are the types of problems that we'd expect new engineers to have to look into on their first day. Oftentimes the questions are less "come up and an answer" and more "let's explore this problem domain and see what we can uncover."
As an example, the interviewer will write up on the board a data structure that is too big and has redundant fields in it, and start a discussion about how to optimize that structure. The optimizations go more and more in-depth as the candidate gains experience thinking within the problem space.
Another problem, the interviewer will give an example of a synchronous operation and work with the candidate to break it up into a set of asynchronous calls.
My team tries to avoid "puzzle questions with 1 correct answer".
I do ask simple data structure questions just to test basic programming skills. On occasion someone does make it through who cannot even write a FOR loop.
My standard go to question is "Please do an in order print of this already sorted binary tree."
It is 5 lines of code, and ~80% of candidates who haven't seen a tree in a decade can still figure it out.
IMH the goal isn't to try and trick people, it is to try and see if people can hold a conversation about a technical topic and make valuable contributions to a discussion.
8 replies →
Even if you have used them like a few months ago. Very few people work on variation of the same problem over and over again, except in research.
Generally, you have a problem, you research, you experiment and then you wrap all that in a nice easy to use utility class or api and forget the details because there is something else you need to work on and that problem was just one piece of the puzzle anyway.
The worst hypocrisy in all that, is if you ask a specific question to a veteran developer on a average size codebase, he will very often need to look in the code and you will not think any less of him. But if it is an interview and he does not remember some trivia in a piece of code he didn't write - my god, he is a loser.
I found actually using a whiteboard, or pen & paper, is the best.
No computers. Pen only.
For me, rehearsing an answer works out. Then, I try to see how to deconstruct it and apply it to a similar problem I don't know.
I can deliver an answer sounding confident and competent because I practiced an answer. It's a bit of a crap shoot.
What irks me the most is that people are doing the interviews. They can be fooled and persuaded with tones and intonations without them realizing it.
How is it the best way if you are not going to be writing code on a whiteboard on a daily basis on the job you are going to be interviewing for?
3 replies →
Look at glassdoor.com or careercup.com for questions from a certain company. Practice using paper and pencil.
Have a friend do a mock interview with you.
They all just seem to be varying levels of fanciness over relatively common data structures and algorithms. I was only applying for internships but the whiteboard stuff wasn't too bad when I did (and other questions I practiced online), I imagine almost completely because we'd done 2 related modules at university.
You spot patterns but there's so many different ways they can ask things and things that you just remember from studying it, that I imagine I'd do worse now unless it was close to what I've done since. On the spot I couldn't invert a BST right now.
I suspect you could probably pass a whiteboard interview a few weeks after taking a mooc on data structures/algorithms with enough depth for example.
You might want to check the latest post on this topic on Eric Lippert's blog: http://ericlippert.com/2015/06/01/writing-code-on-whiteboard...
Get a couple books on algorithms, memorize like its CS 101 basically.
I interviewed (unsuccessfully) at google. If you want to be prepped, yes, I would recommend going over the algorithm books, and make sure that you can do all the basics in your sleep.
That said, memorization, in my limited experience, won't be that helpful, because the questions will likely come with a twist that requires that you understand the algorithms and are good at adapting and applying them - and you need to be able to do it in 45 minutes on a whiteboard. It is very difficult.
I did find "cracking the coding interview" to be helpful prep, and really, the hard problems are fair game. Again, memorizing won't really help you, instead, view these as the kinds of problems you need to be able to solve in 45 min at the whiteboard.
I am kind of blown away that people can do this. I went home and did one of the questions I remembered from my interview with a compiler, and it took me several hours (though I didn't look up a solution, I did use the compiler constantly to check logic, and so forth, something that isn't available of course if you write on paper or a whiteboard). I suppose might be able to get this down to 45 min at a whiteboard, but it would take me a lot of time to get that sharp, months or more of study and practice.
None of this is mean to comment on whether this is or isn't a good interview process, it's just advice on how to prepare.
3 replies →
This is something I could agree over, Thoughts on technical interviewing. http://ericlippert.com/2015/06/08/interviewing-candidates/
Thanks for the link. My personal favorite http://rajeshanbiah.blogspot.com/2009/01/how-to-interview-ca...
inverting a binary tree is a pretty basic question though, if you understand recursion. i can understand why google would ask simple algo questions
Could you illustrate? I can find plenty of examples of mirroring/reversing a tree (below the root node) but none that really invert it. I haven't even found a definition of what inverting a tree means (aside from some academic papers I haven't been able to read yet), despite quite a bit of googling.
i'm not exactly sure either, too bad he didnt define "invert".
i based my comment on the only thing I could find: https://wwwx.cs.unc.edu/~duozhao/entry/2014/12/binary-tree-u...
this is only slightly harder than the trivial (if u know recursion) "mirroring" a tree problem
The fact is, that reliably separating bad from good engineers is incredibly difficult at best.
For big companies that want to scale hiring process, there's really not much you can do.
You put difficult questions and you hope that bad engineers will not be able to solve them. Then you absolutely have to commit to not hiring the guys who can't solve the assignment, otherwise it's all for naught.
'Bring in something you've built and we'll go through it' is getting pretty common.
I spent Xmas break reviewing the most recent TopCoder problems, C++ and STL, and read Sedgewick's algorithms book cover to cover. I drank two quad espressos right before the interview. The interview was a straightforward process and I didn't miss a single question. The offer came that evening.
I should have turned it down. I've written on this thread already what a career-churning waste of time my brief stint at Google turned out to be because I was blind allocated.
Fortunately, the skills that got me noticed by Google were in demand elsewhere then (and, ironically, in demand at Google now) so I just exited the building when it became clear HR had labelled me a troublemaker for trying to find a position that made use of my talents rather than opt for the career reboot they seemed to expect of me. Great perks, but the 2nd worst gig I've had my entire career.
What was the worst?
A consulting company that hired me to develop PS2 tools and then yanked the rug out from under me on day one and told me I had to write a fast MPEG-2 codec from the ground up in the next 60 days despite zero zip nada null knowledge of MPEG-2.
I quit after a month or so. Later I learned their bait and switch tactics were S.O.P. and the company mostly existed to fund the CEO's various mistresses.