Comment by lolinder
1 month ago
> I don't know what an "AI Engineer" is, but, is reading research papers actually necessary
Let's put it this way: if even half the people who call themselves "AI Engineers" would read the research in the field, we'd have a lot less hype and a lot more success in finding the actual useful applications of this technology. As is, most "AI Engineers" assume the same thing you do and consider "AI Engineering" to be "I know how to plug this black box into this other black box and return the result as JSON! Pay me!". Meanwhile most AI startups are doomed from the start because what they set out to do is known to be a bad fit.
> I know how to plug this black box into this other black box and return the result as JSON!
To be fair, most of software engineering is this.
Tbf- most of [any] engineering is like this.
But most 'engineering' is not engineering.
1 reply →
Okay, now take a slightly imbalanced stance: What is most software engineering?
I don't know, but if I say it's about working with things you don't fully understand people seem to trust me.
I kind of see this the opposite way...
Or rather, I guess I feel like it's a sign of the immaturity of the space that it is still kind of unclear (at least it is to me) how to build useful things without reading all the research papers.
To me, it seems like there is an uncanny valley between "people who are up on all the papers in this reading list" and "people who are just getting a feel for how these LLMs respond and slapping a UI on top".
Maybe it kind of reminds me of the CGI period of the web. The "research papers" side is maybe akin to all the people working on networking protocols and servers necessary to run the web, and the "slap a UI over the llm APIs" is akin to those of us slinging html and perl scripts.
You could make ok stuff that way, without needing to understand anything about TCP. But it still took a little while for a more professionalized layer to mature between those two extremes.
I feel like maybe generative AI is in the early days of that middle layer developing?
Even before the amazing achievement of LLM, there were millions of "ML engineers" on LinkedIn, per some stats about LinkedIn jobs. I'll bet a single digit percent of them could even derive the math of linear regression or every implemented a single ML algorithm from scratch. Not that it is wrong, mind you, but it means it's unlikely for half the "AI engineers" to read research papers.
A lot of people did a MOOC for the CV points.
> if even half the people who call themselves "AI Engineers" would read the research in the field, we'd have a lot less hype and a lot more success in finding the actual useful applications of this technology
As someone working in the area for a few years now (both on the product and research side), I strongly disagree. A shocking number of papers in this area are just flat out wrong. Universities/Research teams are churning out garbage with catchy titles at such a tremendous rate that reading all of these papers will likely leave one understanding less than if they read none.
The papers in this list are decent, but I wouldn't be shocked if the conclusions of a good number of them were ultimately either radically altered or outright inverted as we learn more about what's actually happening in LLMs.
The best AI engineers I've worked with are just out there experimenting and building stuff. A good AI engineer definitely has to be working closely to the model, if you're just calling an API you're not really an "AI Engineer" in my book. While most good AI engineers have likely accidentally read most of these paper through the course of their day job, they tend to be reading them with skepticism.
A great demonstration of this is the Stable Diffusion community. Hardly any of the innovation in that space is even properly documented (this, of course, is not ideal), much less used for flag planting on arXiv. But nonetheless the generative image AI scene is exploding in creativity, novel applications, and shocking improvements all with far less engineering/research resources devoted to the task than their peers in the LLM world.
Couldn't agree more with you. At the end of the day the people building the most successful products, are too be busy to be formalizing their experiments into research papers. While I have respect for academic researchers, I think their perspective is fundamentally very limited when it comes to AI engineering. The space is just too frothy.
Careful now. Don't want to upset the future generation of unemployed prompt engineers.
Investors can't tell the difference between hype and (future) results.