Comment by throwawaysleep
10 months ago
Depends on what you are looking for. I’ve turned half baked ideas into white papers for plenty of praise. I’ve used them to make my Jira tickets seem complicated and complete. I’ve used them to get praised for writing comprehensive documentation.
Part of my performance review is indirectly using bloat to seem sophisticated and thorough.
> comprehensive documentation
Documentation is an interesting use case. There are various kinds of documentation (reference, tutorial, architecture, etc.) and LLMs might be useful for things like
- repetitive formatting and summarization of APIs for reference
- tutorials which repeat the same information verbosely in an additive, logical sequence (though probably a human would be better)
- sample code (though human-written would probably be better)
The tasks that I expect might work well involve repetitive reformatting, repetitive expansion, and reduction.
I think they also might be useful for systems analysis, boiling down a large code base into various kinds of summaries and diagrams to describe data flow, computational structure, signaling, etc.
Still, there is probably no substitute for a Caroline Rose[1] type tech writer who carefully thinks about each API call and uses that understanding to identify design flaws.
[1] https://folklore.org/Inside_Macintosh.html?sort=date
Yes, but none of the current LLMs are even remotely useful doing that kind of work for even something moderately complex. I have a 2k LOC project that no LLM even "understands" *. They can't grasp what it is: It's a mostly react-compatible implementation of "hooks" to be used for a non-DOM application. Every code assistant thinks it's a project using React.
Any documentation they write at best re-states what is immediately obvious from the surrounding code (Useless: I need to explain why), or is some hallucination trying to pretend it's a React app.
To their credit they've slowly gotten better now that a lot of documentation already exists, but that was me doing the work for them. What I needed them to do was understand the project from existing code, then write documentation for me.
Though I guess once we're at the point AI is that good, we don't need to write any documentation anymore, since every dev can just generate it for themselves with their favorite AI and in the way they prefer to consume it.
* They'll pretend they understand by re-stating what is written in the README, then proceed to produce nonsense.
I've found "Claude 3.7 Sonnet (Thinking)" to be pretty good at moderately complex code bases, after going through the effort to get it to be thorough.
Without that effort it's a useless sycophant and is functionally extremely lazy (ie takes short cuts all the time).
Don't suppose you've tried that particular model, after getting it to be thorough?
Delivering a library with an llm to explain the api and idiomatic usage seems like an interesting use case.
I’d rather be homeless in Philadelphia than work where you work
This kind of "perf review" hacking works ~everywhere; how well it works correlates with how entrenched the organization is (i.e., how hard it is for new players to disrupt).
You don't have to play the game the same way to work there. But it helps to accept that others will play it, and manage your own expectations accordingly.
> This kind of "perf review" hacking works ~everywhere
I don't have tons of examples, but in my experience:
* This worked in toxic environments. They deserve it.
* This doesn't work in a functional environment, because they don't have those bullshit metrics.
If you have to rely on those tricks, it's time to look for another job.
9 replies →
But it helps to accept that others will play it
Feel for you or anyone surrounded by such others but it is most definitely not everywhere - that is used to justify your presence in a place of work you should not be
Would be nice to fix the performance reviews so we don't end up in a arms race of creating bloat until it becomes so unproductive it kills the host.
Over-fitting proxy measures is one of the scourges of modernity.
The only silver lining is if it becomes so wide spread and easy it loses the value of seeming sophisticated and thorough.
> creating bloat until it becomes so unproductive it kills the host
Maybe we should let/encourage this to happen. Maybe letting bloated zombie-like organisations bloat themselves to death would thin the herd somewhat, to make space for organisations that are less “broken”.
"But at what price?" is probably the right question here, and that'd be a case by case basis thing. ;)
2 replies →
I fully believe you and I am saddened by the reality of your situation.
At the same time, I strive really hard to influence the environment I am in so it does not value content bloat as a unit of productivity, so hopefully there are at least some places where people can have their sanity back!
If your organisation is such that you have to do this even though you are competent for your job, then they deserve it. They lose money because they do it wrong.
If your organisation is functional and you are abusing it by doing that, then you deserve to get fired.
...thinking about it, there are probably situations where making something more verbose makes it take less effort to read. I can see how an LLM might be useful in that situation.
[flagged]