Comment by latenightcoding

3 days ago

LLMs make me 10-20x more productive in frontend work which I barely do. But when it comes to low-level stuff (C/C++) I personally don't find it too useful. it just replaces my need to search stackoverflow.

edit: should have mentioned the low-level stuff I work on is mature code and a lot of times novel.

This is good if front end is something you just need to get through. It's terrible if your work is moving to involve a lot of frontend - you'll never pick up the skills yourself

As the fullstacker with a roughly 65/35 split BE/FE on the team who has to review this kinda stuff on the daily, there's nothing I dread more than a backender writing FE tickets and vice versa.

Just last week I had to review some monstrosity of a FE ticket written by one of our backenders, with the comment of "it's 90% there, should be good to takeover". I had to throw out pretty much everything and rewrite it from scratch. My solution was like 150 lines modified, whereas the monstrous output of the AI was non-functional, ugly, a performance nightmare and around 800 lines, with extremely unhelpful and generic commit messages to the tune of "Made things great!!1!1!!".

I can't even really blame them, the C-level craze and zeal for the AI shit is such that if you're not doing crap like this you get scrutinized and PIP'd.

At least frontenders usually have some humility and will tell you they have no clue if it's a good solution or not, while BEnders are always for some reason extremely dismissive of FE work (as can be seen in this very thread). It's truly baffling to me

Interesting, I find the exact opposite. Although to a much lesser extent (maybe 50% boost).

I ended shoehorned into backend dev in Ruby/Py/Java and don't find it improves my day to day a lot.

Specifically in C, it can bang out complicated but mostly common data-structures without fault where I would surely do one-off errors. I guess since I do C for hobby I tend to solve more interesting and complicated problems like generating a whole array of dynamic C-dispatchers from a UI-library spec in JSON that allows parsing and rendering a UI specified in YAML. Gemini pro even spat out a YAML-dialect parser after a few attempts/fixes.

Maybe it's a function of familiarity and problems you end using the AI for.

  • As in, it seems to be best at problems that you’re unfamiliar with in domains where you have trouble judging the quality?

    • >it seems to be best at problems that you’re unfamiliar with

      Yes.

      >in domains where you have trouble judging the quality

      Sure, possibly. Kind of like how you think the news is accurate until you read a story that's in your field.

      But not necessarily. Might just be more "I don't know how do to <basic task> in <domain that I don't spend a lot of time in>", and LLMs are good at doing basic tasks.

This is exactly my experience as well. I've had agents write a bit of backend code, always small parts. I'm lucky enough to be experienced enough with code I didn't write to be able to quickly debug it when it fails (and it always fails from the first run). Like using AI to write a report, it's good for outlines, but the details are always seemingly random as far as quality.

For frontend though? The stuff I really don't specialize in (despite some of my first html beginning on FrontPage 1997 back in 1997), it's a lifesaver. Just gotta be careful with prompts since so many front end frameworks are basically backend code at this point.

I've been hacking on some somewhat systemsy rust code, and I've used LLMs from a while back (early co-pilot about a year ago) on a bunch of C++ systems code.

In both of these cases, I found that just the smart auto-complete is a massive time-saver. In fact, it's more valuable to me than the interactive or agentic features.

Here's a snippet of some code that's in one of my recent buffers:

    // The instruction should be skipped if all of its named
    // outputs have been coalesced away.
    if ! self.should_keep_instr(instr) {
      return;
    }

    // Non-dropped should have a choice.
    let instr_choice =
      choices.maybe_instr_choice(instr_ref)
        .expect("No choice for instruction");
    self.pick_map.set_instr_choice(
      instr_ref,
      instr_choice.clone(),
    );

    // Incref all named def inputs to the PIR choice.
    instr_choice.visit_input_defs(|input_def| {
      self.def_incref(input_def);
    });

    // Decref all named def inputs to the SIR instr.
    instr.visit_inputs(
      |input_def| self.def_decref(input_def, sir_graph)
    );

The actual code _I_ wrote were the comments. The savings in not having to type out the syntax is pretty big. About 80% of the time in manual coding would have been that. Little typos, little adjustments to get the formatting right.

The other nice benefit is that I don't have to trust the LLM. I can evaluate each snippet right there and typically the machine does a good job of picking out syntactic style and semantics from the rest of the codebase and file and applying it to the completion.

The snippet, if it's not obvious, is from a bit of compiler backend code I'm working on. I would never have even _attempted_ to write a compiler backend in my spare time without this assistance.

For experienced devs, autocomplete is good enough for massive efficiency gains in dev speed.

I still haven't warmed to the agentic interfaces because I inherently don't trust the LLMs to produce correct code reliably, so I always end up reviewing it, and reviewing greenfield code is often more work than just writing it (esp now that autocomplete is so much more useful at making that writing faster).

  • What exact tool are you using for your smart auto-complete?

    • Whatever copilot defaults to doing on vscode these days. I didn't configure it very much - just did the common path setup to get it working.

It works with low-level C/C++ just fine as long as you rigorously include all relevant definitions in the context window, provide non-obvious context (like the lifecycle of some various objects) and keep your prompts focused.

Things like "apply this known algorithm to that project-specific data structure" work really well and save plenty of time. Things that require a gut feeling for how things are organized in memory don't work unless you are willing to babysit the model.

This feels like a parallel to the Gell-Mann amnesia effect.

Recently, my company has been investigating AI tools for coding. I know this sounds very late to the game, but we're a DoD consultancy and one not traditional associated with software development. So, for most of the people in the company, they are very impressed with the AI's output.

I, on the other hand, am a fairly recent addition to the company. I was specifically hired to be a "wildcard" in their usual operations. Which is too say, maybe 10 of us in a company of 3000 know what we're doing regarding software (but that's being generous because I don't really have visibility into half of the company). So, that means 99.7% of the company doesn't have the experience necessary to tell what good software development looks like.

The stuff the people using the AI are putting out is... better than what the MilOps analysts pressed into writing Python-scripts-with-delusions-of-grandeur were doing before, but by no means what I'd call quality software. I have pretty deep experience in both back end and front end. It's a step above "code written by smart people completely inexperienced in writing software that has to be maintained over a lifetime", but many steps below, "software that can successfully be maintained over a lifetime".

  • Well, that's what you'd expect from an LLM. They're not designed to give you the best solution. They're designed to give you the most likely solution. Which means that the results would be expected to be average, as "above average" solutions are unlikely by definition.

    You can tweak the prompt a bit to skew the probability distribution with careful prompting (LLMs that are told to claim to be math PHDs are better at math problems, for instance), but in the end all of those weights in the model are spent to encode the most probable outputs.

    So, it will be interesting to see how this plays out. If the average person using AI is able to produce above average code, then we could end up in a virtuous cycle where AI continuously improves with human help. On the other hand, if this just allows more low quality code to be written then the opposite happens and AI becomes more and more useless.

  • Before the industrial revolution a cabinetmaker would spend a significant amount of time advancing from apprentice to journeyman to master using only hand tools. Now master cabinetmakers that only use hand tools are exceedingly rare, most furniture is made with power tools and a related but largely different skillset.

    When it comes to software the entire reason maintainability is a goal is because writing and improving software is incredibly time consuming and requires a lot of skill. It requires so much skill and time that during my decades in industry I rarely found code I would consider quality. Furthermore the output from AI tools currently may have various drawbacks, but this technology is going to keep improving year over year for the foreseeable future.

    • Maintainable software is also more maintainable by AI. The required standards may be a bit different, for example there may be less emphasis on white space styling, but, for example, complexity in the form of subtle connections between different parts of a system is a burden for both humans and AI. AI isn't magic, it still has to reason, it fails on complexity beyond its ability to reason, and maintainable code is one that is easier to reason with.

Same. It’s amazing for frontend.

  • As a front-of-the-frontend guy, I think it's terrible with CSS and SVG and just okay with HTML.

    I work at a shop where we do all custom frontend work and it's just not up to the task. And, while it has chipped in on some accessibility features for me, I wouldn't trust it to do that unsupervised. Even semantic HTML is a mixed bag: if you point out something is a figure/figcaption it'll probably do it right, but I haven't found that it'll intuit these things and get it right on the first try.

    But I'd imagine if you don't care about the frontend looking original or even good, and you stick really closely to something like tailwind, it could output something good enough.

    And critically, I think a lot of times the hardest part of frontend work is starting, getting that first iteration out. LLMs are good for that. Actually got me over the hump on a little personal page I made a month or so ago and it was a massive help. Put out something that looked terrible but gave me what I needed to move forward.

  • It's astonishing. A bit scary actually. Can easily see the role of front-end slowly morphing into a single person team managing a set of AI tools. More of an architecture role.

  • Is this because they had the entire web to train on, code + output and semantics in every page?

    • I guess it’s because modern front-end “development” is mostly about copying huge amounts of pointless boilerplate and slightly modifying it, which LLMs are really good at.

    • It's moreso that a backend developer can now throw together a frontend and vice-versa without relying on a team member or needing to set aside time to internalize all the necessary concepts to just make that other part of the system work. I imagine even a full-stack developer will find benefits.

      4 replies →

    • I’m not sure how this was extended and refined but there are sure a lot of signs of open source code being used heavily (at least early on). It would make sense to test model fit with the web at large.