← Back to context

Comment by kccqzy

11 hours ago

Great insight. But this is sadly not applicable to interviews.

> It's easy to do in O(n^2) time, or if you are clever, you can do it in O(n). Or you could be not clever at all and just write it as a constraint problem

This nails it. The point of these problems is to test your cleverness. That's it. Presenting a not-clever solution of using constraint solvers shows that you have experience and your breadth of knowledge is great. It doesn't show any cleverness.

>The point of these problems is to test your cleverness.

In my experience, interviewers love going to the Leetcode "Top Interview 150" list and using problems in the "Array String" category. I'm not a fan of these problems for the kind of jobs I've interviewed for (backend Python mostly), as they are almost always a "give me a O(n) runtime O(1) memory algorithm over this array" type challenge that really doesn't resemble my day to day work at all. I do not regularly do in-place array algorithms in Python because those problems are almost always handled by other languages (C, Rust, etc.) where performance is critical.

I wish interviewers would go to the "Hashmap" section for interviews in Python, JavaScript, etc., type of languages. They are much less about cleverness and more about whether you can demonstrate using the appropriate tools in your language to solve problems that actually do resemble ones I encounter regularly.

There's also the problem of difficulty tuning on some of these. Problem 169 (Majority Element) being rated "Easy" for getting a O(n) runtime O(1) memory solution is hilarious to me. The algorithm first described in 1981 that does it (Boyer–Moore majority vote algorithm) has a Wikipedia page. It's not a difficult to implement or understand algorithm, but its correctness is not obvious until you think about it a bit, at which point you're at sufficient "cleverness" to get a Wikipedia page about an algorithm named after you. Seems excessive for an "Easy" problem.

  • Interviews should not be about cleverness. They should test that you can code. I almost never write an algorithm because all the important algorithms are in my standard library already. Sure back in school I did implement a red-black tree - I don't remember if it worked, but I implemented it: I can do that again if you need me to, but it will take me several days to get all the details right (most of it looking up how it works again). I use red-black trees all the time, but they are in the language.

    You need to make sure a candidate can program so asking programing question make sense. However the candidate should not be judged on if they finish or get an optimal or even correct answer. You need to know if they write good code that you can understand, and are on a path that if given a reasonable amount of time on a realistic story would finish it and get it correct. If someone has seen the problem before they may get the correct answer, but if they have not seen it they won't know and shouldn't expected to get the right answer in an hour.

  • Majority Element is rated easy because it can be trivially solved with a hashmap in O(N) space and that's enough to pass the question on Leetcode. The O(1) space answer is probably more like a medium.

    • Yeah it just depends on whether your interviewer considers that "solved". To test this out, I wrote a one liner in Python (after imports) that solves it with a hashmap (under the hood for Counter, which uses a heap queue to find the most common one):

      return Counter(nums).most_common(1)[0][0]

      And that's 50th percentile for runtime and memory usage. Doing it with another one liner that's 87% percentile for time because it uses builtin Python sorting but is 20th percentile for memory:

      return sorted(nums)[len(nums) // 2]

      But the interviewer might be looking for the best approach, which beats "100%" of other solutions in runtime per Leetcode's analysis:

        m, c = -1, 0
        for x in nums:
            if not c:
                m = x
                c = 1
            elif m == x:
                c += 1
            else:
                c -= 1
        return m
      

      If I were interviewing, I'd be happy with any of these except maybe the sorted() one, as it's only faster because of the native code doing the sort, which doesn't change that it's O(n log n) time and O(n) space. But I've had interviews where I gave answers that were "correct" to the assumptions and constraints I outlined but they didn't like them because they weren't the one from their rubric. I still remember a Google interview, in which we're supposed to "design to scale to big data", in which they wanted some fiddly array manipulation algorithm like this. I gave one that was O(n log n) but could be done in place with O(1) memory, and the interviewer said it was "incorrect" in favor of a much simpler O(n) one using dicts in Python that was O(n) memory. Had the interviewer specified O(n) memory was fine (not great for "big data" but ok) I would have given him the one liner that did it with dicts lol

      I guess my point is that interviewers should be flexible and view it as a dialogue rather than asking for the "right answer". I much prefer "identify the bug in this self contained code snippet and fix it" type problems that can be completed in <15-30 minutes personally, but Leetcode ones can be fine if you choose the right problems for the job.

  • Honestly in day to day programming I find data types & associated APIs are so so much more important than algorithms.

    I would rather work with a flexible data type with suboptimal performance than a brittle data type that maybe squeezes out some extra performance.

    Your example of in-place array mutation feels like a good example of such a thing. I feel like there should be a category of interviewing questions for "code-safety" not just performance.

> The point of these problems is to test your cleverness.

Last round I did at Meta it was clearly to test that you grinded their specific set of problems, over and over again, until you could reproduce them without thinking. It's clear because the interviewers are always a bit surprised when you answer with whatever is not the text-book approach on both leetcode and on the interview guide they studied.

Cleverness is definitely not high on the list of things they're looking for.

  • Cheekily using counting sort ended things the one and only time I agreed to interview with Meta. Definitely improved my inbox for a couple years though.

Bottom up dynamic programming algorithms require some cleverness.

All of the ones listed can be solved with a top down dynamic programing algorithm. Which just means "write recursive solution, add caching to memoize it".

For some of these, you can get cleverer. For example the coin change problem is better solved with an A* search.

Still, very few programmers will actually need these algorithms. The top thing we need is to recognize when we accidentally wrote a quadratic algorithm. A quick scan of https://accidentallyquadratic.tumblr.com/ shows that even good people on prominent projects make that mistake on a constant basis. So apparently being able to produce an algorithm on the test, doesn't translate to catching an algorithmic mistake in the wild.

When I interview with problem solving problems, the point is to understand how the candidate thinks, communicates, and decomposes problems. Critically, problem solving questions should have ways to progressively increase and decrease difficulty/complexity, so every candidate "gets a win" and no candidate "dunks the ball".

Interviewers learn nothing from an instant epiphany, and they learn next to nothing from someone being stumped.

Unfortunately, this is why we can't have nice things. Problem solving questions in interviews can be immensely useful tools that, sadly, are rarely usefully used.

  • > the point is to understand how the candidate thinks, communicates, and decomposes problems.

    100% and it's a shame that over time this has become completely lost knowledge, on both sides of the interview table, and "leetcode" is now seen as an arbitrary rote memorization hurdle/hazing ritual that software engineers have to pass to enter a lucrative FAANG career. Interviewees grind problems until they've memorized every question in the FAANG interview bank, and FAANG interviewers will watch a candidate spit out regurgitated code on a whiteboard in silence, shrug, and say "yep, they used the optimal dynamic programming solution, they pass."

    • If somebody writes the optimal algorithm that should be a negative unless their resume indicates they are writing that algorithm often. The only reason you should know any algorithm well enough to get it right is if your job is implementing the optimal version for every single language. Of course nobody maintains one algorithm in many different languages/libraries (say libc++, python, rust, ada, java - each has different maintainers), so I can safely safe the number is zero who should be able to implement your cleaver algorithm. Now if your cleaver algorithm is in the language standard library (or other library they often use) that should be able to call/use it, though even then I expect them to look up the syntax in most languages.

      1 reply →

  • > Critically, problem solving questions should have ways to progressively increase and decrease difficulty/complexity, so every candidate "gets a win" and no candidate "dunks the ball".

    Absolutely agree. When I interview, I start with a simple problem and add complexity as they go. Can they write X? Can they combine it with Y? Do they understand how Z is related?

    • Same. I'm never doing a fail/pass type interview. Instead I try to assess where the candidate is on the beginner/intermediate/expert axis and match that with the expectations of the role I'm interviewing for.

  • > the point is to understand how the candidate thinks, communicates, and decomposes problems

    Interviewers always say this, but consider: would you endorse a candidate who ultimately is unable to solve the problem you've presented them, even if they think, communicate, and decompose problems well? No interview in this industry prizes those things over getting the answer right.

    • Every interview I know is severely time limited. I don't care if you can solve the problem, so long as your are clearly making progress and have proven you could solve the problem if given longer.

      Now I give you problems I expect to take 20 minutes if you have never seen them before so you should at least solve 1. I have more than once realized someone was stuck on the wrong track and redirection efforts were not getting them to a good track so I switched to a different problem which they were then able to solve. I've also stopped people when they have 6 of 10 tests passing because it is clear they could get the rest passing but I wouldn't learn anything more so it wasn't worth wasting their time.

      In the real world I'm going to give people complex problems that will take days to solve.

  • Would a good answer be "I can do it as a constraint problem, but since I guess you are not asking for this, the solution is..." and then proceed as usual?

    • Id probably stop the candidate, dig into how they’d using constraint based solvers, and how they might expect that to fall apart. Applicability and judgment is worth way more than raw algorithmic questions.

      One way to think about this is:

      Is a fresh graduate more likely to provide a solid answer to this than a strategic-thinking seasoned engineer? If so, just be conscious of what your question is actually probing.

      And, yes, interview candidates are often shocked when I tell them that I’m fine with them using standard libraries or tools that fit the problem. It’s clear that the valley has turned interviewing into a dominance/superiority model, when it really should be a two-way street.

      We have to remember that the candidate is interviewing us, too. I’ve had a couple of interviews as the interviewee where the way the interview was conducted was why I said “no” to an offer (no interest in a counter, just a flat “no longer interested” to the recruiter, and, yes, that surprises recruiters, too).

      1 reply →

>The point of these problems is to test your cleverness.

No it's just memorization of 12 or so specific patterns. The stakes are too high that virtually everyone going in will not be staking passing on their own inherent problem solving ability. LeetCode has been so thoroughly gamified that it has lost all utility of differentiability beyond willingness to prepare.

  • Yeah, it tests if the candidate enjoys the programming-adjacent puzzle game of LeetCode, which is a perfectly decent game to play, but it is just a signal.

    If somebody grinds LeetCode while hating it, it signals they are really desperate for a job and willing to jump through hoops for you.

    If somebody actually enjoys this kind of stuff, that is probably a signal that they are a rare premium nerd and you should hire them. But the probably play Project Euler as well (is that still up?).

    If somebody figures out a one-trick to minmax their LeetCode score… I dunno, I guess it means they are aware of the game and want to solve it efficiently. That seems clever to me…

  • In defense of questions like this, “willingness to prepare” is a significant differentiator

    • But what is it differentiating? And is it really the best evidence of willingness to prepare? My MSc and BA on the topics, my open source contributions, two decades of industry experience... Those aren't evidence of not only willingness but execution of preparation?

      2 replies →

    • That they would ask me to prepare for that is a signal as well.

      In no case is it a useful signal on if I can do my job better than someone else. Some people like this type of problem and are good at it anyway which is a good signal compared to average - but there are also above average people who don't enjoy this type of problem and so don't practice it. Note that both cases the people I'm talking about did not memorize the problem and solution.

    • It also means "I don't have money for food, and at this point I am desperate".

    • That willingness to prepare doesn't reconcile with the realities of parenthood and all of the other responsibilities someone in their thirties may have. Consistently finding that time will be a huge ask, especially if you haven't worked on those problems in a while.

      2 replies →

No its not a measure of cleverness. Its about whether you can break down problems and apply common methods. Thats the entire job. Its a learnable skill and honestly resisting learning because of personal biases is a red flag in my book.

The point is to test whether or not you put in the time to sharpen common patterns and also to test your communication ability

  • Super common patterns like dynamic programming?

    • Yes. Common LC patterns such as 1D and 2D dynamic programming. I'm not defending leetcode style interviews, in fact I think they are actually bad, I'm simply stating their intent as observed by me.

      In my notes I have roughly 30 patterns to leetcode questions btw.