Comment by XCSme

19 hours ago

Wriring the code is not the challenge, but it's what was taking up most of the time. Not the typing itself, but also because I had to think of how to implement it.

Now I can just say "add 2FA" and in 5 minutes, while I test something else, it is done.

It also made iterations a lot faster, you can try something out, see how it feels, if it doesn't work, you can just trash all the code and start again.

But you still have to review that 2FA code and that involves thinking through the implementation, right?

  • No.

    Haven't typed a line of code or read any code for over 6 months now.

    And I used to love coding and be a competitive programmer, but this is how "coding" goes nowdays.

    I have a mental model of what it would do, and how it would work, and I ask questions to confirm things and tell it to watch for specific gotchas. Then simply test the feature myself a bit.

    Security-wise, I think the latest cyber models are better than me anyway at finding vulnerabilitates and pentesting such features.

    Plus 2FA is a very common pattern, so it likely has in the training dataset many really good implementations.

    • > Security-wise, I think the latest cyber models are better than me anyway at finding vulnerabilitates and pentesting such features.

      I don't doubt that, but they are equally good in making mistakes, over-engineering, or adding things you never asked for. They have all sorts of patterns in their training data from excellent to inadequate and I find them challenging to guide them consistently in one direction. Also with questions and tests, they can add something extra you didnt need and you dont know about, so your scrutinizing questions and test cases could miss that.

      At least for myself, I didnt find them reliable enough yet to do what you describe and just not look at the code at all.

      14 replies →

    • > Haven't typed a line of code or read any code for over 6 months now

      > I ask questions to confirm things

      Oh my.

      As someone who reads the code, I can tell you, asking questions to confirm things is inadequate. The models lie to me, daily.

      Every day I have two experiences:

      1. I’m blown away by what it can do

      2. I say, ”wait, you said this, but the code shows that, so you were just going to leave that endpoint without requiring any authentication??” and I get the “you’re absolutely right, that was my mistake, and you’re right to call it out” song and dance. Daily.

      It also adds all kinds of bloat to code, tests, and “documentation”. I’d say I spend ~30% of my dev time picking lines of code or documentation and asking, “why does this exist?” and “what would break if we deleted this line?” and then arguing with it and removing things.

      3 replies →

    • Current models love shoving in defaults where correct code would otherwise fall over with an exception.

      I dread to think what that means in security conscious code.

      1 reply →

    • Be brave and post a GitHub of your code that you haven't written a lick of and haven't read.

      EDIT: Come on? Won't post your code for everyone to see? Why not just put it all in a repo, client and server both?

      EDIT 2: Amazing. If you punch in notes on the keyboard for like 10 seconds then click the keyboard-icon button on the bottom right the website crashes

      EDIT 3: If you click the main CTA then click "Let's start" the website hangs and you need to manually refresh the page for the content to load

      16 replies →

I don't know what it built for you in 5 minutes, probably something that "works".

I have spent two weeks using opus just to write a plan/design for 2FA and iron it out until review (about 7 of them) doesn't flag it with 20+ problems (with security holes of various sizes), for which I had to guide it through to not turn it into a mess and whac-a-mole.

  • 2FA is quite simple, right?

    It's just a secret key that an autheticator uses to generate a time-based code, which the app can validate before completing a normal log-in flow.

    What model did you use?

    Astra xhigh on fast mode can probably indeed one-shot that in 5 minutes.

    Plus optionally QR image generator to easily add that key to the authenticator app.

    There are already many libraries doing 2FA, but implementing it in any language is quite trivial, right

    • > 2FA is quite simple, right?

      No?

      Aside from the fact that the implementation must be secure, you want for example to:

      - handle accounts that have lost their second factor in an way appropriate for your business - decide what to do with accounts who don't configure it. If e.g. you want to send them authentication codes via email or SMS that's another can of worms.

      Let alone the simple things such as making sure that your implementation works with the various TOTP apps

      1 reply →

    • Well yeah, and it wasn't, most of the pain was around rate limiting, lockouts and preventing exploits and making it work with this specific codebase.

      I agree that with a clean codebase it will be simpler, maybe couple of days, just running code review workflow takes 15-30 minutes and then decisions llm makes for each problem is often not good and lead it into overengineering rabbit hole, which means I have to think about each problem and prevent it from escalating.

      But yeah if "look, it sends the code and I can enter it to login" is enough validation then it can be made in 5 minutes, sure.

    • Any of the latest models can one shot a modern 2FA workflow given the right context in a sane codebase.

      The critical part is providing it a way to validate its work end to end. Without that, it's similar to asking a human to implement a feature with pen and paper.

      1 reply →

    • If you're not doing anything that matters, 2FA is easy. If you're doing something people actually care about, you're going to have to answer real world questions like "how do we provide support for this", "do we force certain roles to set up 2fa", "what if someone loses their phone", etc

      1 reply →

>Not the typing itself, but also because I had to think of how to implement it.

I believe "writing the code" means literally just "writing the code" not thinking how to implement it.

  • How can we write without thinking?

    If the problem and implementation is so well defined, and "determnistic", this means that LLMs should also be able to just "write code" from specs, without thinking.