Comment by Frost1x
5 hours ago
I find this take off. In general, LLMs collapse a bit of all knowledge work, including many domain experts. I work with domain experts often and it’s usually the most difficult part of the process. Now, I can ask an LLM the questions I need to design the system and have an agentic system help me build parts. And it works, quite well.
The places it falls flat is domain expertise that isn’t well documented, like specific business processes. Knowing something will fail because X in dept A won’t like it and will undermine it (politics) or some silly process I didn’t know about forbids it.
So it’s not domain expertise, it’s idiosyncratic expertise that shines these days. Knowing where things deviate from a domain or the standard and being able to adapt around that. Years ago there’s a group I worked with and I was at the mercy of about 3 people I could ask questions to in order to make sure I was doing things correctly because digging into that domain was time prohibitive. Now, I can sit down one evening and dig fairly deep into a domain. I can understand common practice, approach, acronyms, nomenclature, so on. I can find other popular competing systems. I can rapidly figure out what I need.
The same is true for software, agents don’t collapse software they help people build fairly usable systems but they don’t find the expertise a senior level engineer has where designs or implementations may fail in practice. The idiosyncrasies of software and software systems, especially in your very specific set of constraints you need to operate in that may deviate from the rest of the world.
> I work with domain experts often and it’s usually the most difficult part of the process. Now, I can ask an LLM the questions
And you will not notice the obvious errors in its answers to even basic questions, because you are no expert and should have really consulted with one.
When talking about their own area of expertise, almost everybody immediately notices the glaring faults. But then they are very impressed and take LLMs as gospel when talking about anything else. I do not get it.
How would you know the agent is correct in domain expertise if you’re not possessing domain expertise yourself?
Shallow generic domain knowledge is not really a helpful domain expertise, is it.
Lets take banks - knowing how banks work in general means little for that one specific bank who is paying you, which has different portfolio than most, different processes because everything is homebrew over decades of doing business and they differentiate from rest of the market in many ways and thats their key proposition, different data infra, set of internal apps, protocols and so on. There is no general llm knowledge you can distill in few minutes that would help you much.
You need to know your specific company, no way around it. In fact, to be proficient in those deliveries, you need to know those internal politics, processes, their quirks and so on, 100x more than some code fast delivery since this is majority of any bigger project. You need some reputation to get things done effectively.
This, for some time, won't be something llm can massively improve, unless those companies change themselves.
>> You need to know your specific company, no way around it
Of course there are ways around it. For example you can build the foundation yourself (which will cover the 80% that is shared across every company in that industry) and give users the ability to build the rest themselves inside the software. This type of abstraction-based architecture is how platforms like Salesforce took off and became ubiquitous.