Comment by hn_throwaway_99
10 hours ago
While I agree that domain expertise has always been a moat, I believe the author is missing something critical: there is a big difference between being able to verify the output of a system is correct, and being able to tell a system how to generate the correct output to begin with.
Personal example: I had a software engineering colleague who was the best coder of financial management systems I've ever encountered. He gained these skills through years of in-the-trenches development. One of the things he told me, and that I also observed, was that the vast majority of financial experts (basically, the people in the accounting department of companies) had an extremely difficult time just telling him what the rules of any particular transaction should be. But what they could do was tell him whether the handling of any particular transaction was right or wrong. So often times he would sit down with these accounting folks and go through lots of example transactions he came up with, and from there he essentially built up the requirements spec.
In my experience, that is the primary difference between people I've known who are good software engineers and those who aren't: people who can specify the detailed rules of any system, vs. folks who take a "well, I know it when I see it" approach.
I have a strong suspicion that folks who have a high degree of domain expertise in a particular area will fail as software builders even in an agentic world because they will struggle to elucidate clearly the rules in their head that they've learned over years. As an analogy, it's kind of like asking a native speaker for the grammar rules of their language. Often times they can't, but they'll just say "well, that sounds wrong." They may be "domain experts" in their language, but they'd have a hell of a time prompting an AI system on how to grade a test for grammar correctness.
Anybody who has ever done programming professionally in the small scale knows this. Refining the requirements is the job.
In fact, I've never known an industry so keen on levelling its own moats as the software industry. We regularly invent things like 4GL, graphical programming and frameworks and engines such as Unity just to enable more people to do programming. People will happily teach programming for free in outreach programs (I am just one example). No other profession does this. Perhaps with the exception of mathematicians and other fields that are very close to programming.
I could teach an economist enough programming in a weeks that they could write an ERP module, but I could not learn enough economics in a week to write one. If the language was the barrier, we could invent a more effective one. We invent new languages weekly anyway. Having seen how quickly beginners can make things with Unity or Godot, I seriously doubt an LLM agent could improve much on this. Of course, if the job is writing yet another CRUD React app with a Java or Python backend, then sure, the LLM will be very effective. But compared to doing same app in something like Excel or MS Access? Not that much.
Everything everywhere does this. All human progress has been making common and accessible the rare and expensive. Universities, online courses, textbooks, etc all exist to make economics as accessible as possible, there’s just a really tight reward loop with programming so your rate of learning it is more tangible
I think you are underestimating how hard it is for average joe to learn programming basics. I remember a fellow in high school that just could not accept that = in programming is assignment not an equation (like in high school math)
I'd like to believe I have an inkling, having done a fair bit of teaching.
Still, imagine how hard other skills are to acquire. How much civil engineering can you learn in two weeks? How much violin playing? But you could absolutely get basic grasps on a general purpose programming language. With something specialized like Unity or Excel you would get tons of useful output.
The hurdle around assignment operator is as old as symbolic languages. That's why legends such as Niklaus Wirth wanted to use another operator, ":=", a notation that is still being used.
Anyway, it can be a hurdle, but one I find that most people get over pretty quickly.
1 reply →
You went to high school with Niklaus Wirth?
>We regularly invent things like 4GL, graphical programming and frameworks and engines such as Unity just to enable more people to do programming.
You're right on the money on this.
Earlier this month I went to visit a company for a complete demo prototype of a full one-to-one train simulator trainer mostly designed and programmed by a former civil engineer using Unity engine. According to the company, they could not do it if Unity engine (or similar) is not around because it will be prohibitively expensive to develop.
In a related news, Unity recently released AI eco-system namely Unity AI Suite [1].
[1] Unity AI Suite:
https://unity.com/features/ai
the 1 dimensional thinking that's perverse in the industry will be catastrophic.
we don't like doing the hard things e.g training juniors so they can be skilled seniors via good apprenticeship programs i.e on the job. now we r delegating to stochastic parrots.
in terms of systems thinking which is one skill you need to be a domain expertise - very few people are ever curious & are not willing or able to ask critical questions. hence the groupthink that's prevalent in the industry.
no wonder the quality of software never goes up - while the building blocks have gone up in quality. an analogy is like having super strong bricks but making brittle structures
I use LLMs daily for coding. That said, I agree with your sentiment. Super strong bricks, poorly put together, makes a bad building.
Another analogy I like is a beginner playing a $100k Stratavarius probably can't produce anything near a professional violinist playing a $50 violin.
Personally I use LLMs to level up my systems thinking. I describe the domain, I have it brainstorm some scalable solutions, I look them up, I bring them up to the team, and we discuss, and I implement. It's a great workflow imo.
>One of the things he told me, and that I also observed, was that the vast majority of financial experts (basically, the people in the accounting department of companies) had an extremely difficult time just telling him what the rules of any particular transaction should be
We have internalized more knowledge than we can explain sounds like the textbook definition of Polanyi's paradox:"Polanyi's paradox, named in honour of the British-Hungarian philosopher Michael Polanyi, is the theory that human knowledge of how the world functions and of our own capability are, to a large extent, beyond our explicit understanding" [0]
[0] https://en.wikipedia.org/wiki/Polanyi%27s_paradox
Thanks so much! I hadn't heard of Polanyi's paradox before but it is exactly what I was talking about. The Wikipedia entry even highlights the problem for AI-driven development:
> Polanyi's paradox has been widely considered to identify a major obstacle in the fields of AI and automation, since programming an automated task or system is difficult unless a complete and fully specific description of the procedure is available.
having worked on automation of accounting process, it has been a combination of ever evolving ruleset/edge scenarios for 10% cases plus standardised rules for 90% of the cases. i think the problem is that it is an evolving field. alot of times it borders law, which makes it murky and vague too.
cf https://en.wikipedia.org/wiki/Dual_process_theory as a causative agent behind the paradox.
I think this neatly captures the gap in the article; it’s thinking of domain expertise as static, rather than something that needs to be built/extracted.
It also goes the other way; A a good engineer can build experiments and discover the domain even without an oracle on the team; reading protocols or manuals or specs, for example. In other words, learn to be a domain expert themselves.
Personally I find the most efficient way to learn a concept to be building it; you are immediately faced with your blind spots. AI for sure helps improve cycle time on these discoveries if you use it that way; time writing boilerplate goes to ~zero and you can spend your time figuring out how to answer the real questions.
I do think that we as a profession need to learn new architectural and craftsmanship patterns to make it easier to learn from our code; concepts like Literate Programming which were too fussy for fast-moving teams may end up accelerating in the agentic world; I need to read a lot more code than I write now. But also things like Acceptance Test Driven Design; not new concepts, just newly increased in value.
WRT the native grammar, consider adjective order. Few native english speakers (me included) can off the cuff name the proper order, but everyone knows the "right" order.
https://dictionary.cambridge.org/us/grammar/british-grammar/...
Another one: tekamolo, for German and adverbs this time.
Temporal, kausal, Modal, Lokal
https://www.olesentuition.co.uk/single-post/tekamolo-how-to-...
Great example
I think the answer is in the second sentence of the article. The most valuable person in this new world (using the author's own phrasing from the penultimate paragraph) is the one who is good at "building a working model of the domain in [their] head".
Depending on the domain, this may either be the domain expert themselves, or someone else trained in formal logic, data structures and organizing information into coherent hierarchies. If this is neither the domain expert nor the programmer, then it must be someone in between. In the old world, this role was called "business analyst".
I'm this person, and I do find AI to be quite helpful, though I'm mostly just playing around at this point.
I'm the daughter and granddaughter of programmers, and I learned the basics of how to code as a kid. I'm good at it and have a knack for it, but I didn't want to do it for 8+ hours a day and then spend my nights on it as well, so I didn't pursue it as a career. I did an undergraduate degree in Linguistics, which has been really helpful for having an intuitive sense for what 'language as data' can accomplish and for a strong understanding of the difference between language as data and language as meaning. I studied formal logic systems. Then I did a graduate degree in Library Science and worked in libraries for a decade and a half.
I can organize and define systems very well, and I'm trained in how to wheedle information people don't consciously know out of them without them knowing I'm doing it. I've spent enough time around actual devs to understand where my limitations are and when to loop in someone who knows more to check my work, and when it's important for the work to be super accurate versus when I can learn by fucking around. (Front end and design? Fuck around! Database structure? Fuck around but with an exceptionally robust backup system kept outside of the AI tools' purview + don't fuck around in prod. Storing credentials and people's information? Ask someone.)
The problem companies are going to have is I'm very disinclined to work for them doing this, particularly if they want us because they think we're going to be cheaper. Most people who are in this category a.) could be devs and opted not to, and there's a reason for that and/or b.) are the children, cousins, etc. of programmers. We're not stupid: we know we're just as disposable as they're trying to make devs.
Having descended from a humanities social background and blundered into professional programming rather incidentally, a lot of this resonates with me.
I've frequently been credited as a person who can really string all the disparate elements of tacit knowledge together into a unified fabric in our particular subdomain, and helped a lot of people plug Swiss cheese gaps in their knowledge that way and come away with the feeling that it's all been tied together theoretically.
However, it's not immediately obvious to me how, in our LLM psychosis cultural moment, this facility shoots to the top of the value chain.
Having worked with domain experts I concur on the difficult time they have expressing the rules of their domains.
Once I built a little domain-specific language for them, that was tested against old jobs to see if they contradicted the past; it was a nifty project and since then I am convinced that DSLs are underrated as a way to encode expertise.
Can you give an example (even if contrived) of how that would have looked ? I’m very curious !
I had the experts write markdown files that contained the rules looked somewhat like:
## 1A Rule name
Some prose explaining the rules liking to official documentation.
``` if municipality and inhabitants > 10000 then functionA else functionB ```
Then a trivial parser would extract the rules, the DSL was then handled by Lark[1]. So pretty simple, but it made collaborating with experts easier as simulated results would also output some markdown they could read.
1. https://lark-parser.readthedocs.io/en/stable/
1 reply →
I am also very interested!
Scala to the rescue, then! :)
I think that you raise some excellent points here. I want to dive deeper into what is meant by domain knowledge -- as a I see it. This phrase comes to mind: "Those who know, do. Those that understand, teach." (Google search tells me that it is frequently misattributed to Aristotle.) People who cannot clearly articulate rules for their domain only "do". However, those that can clearly articulate will "teach". Thus, I would say true domain knowledge is the ability to teach that domain to others.
I largely agree, but want to note that there’s the counterpoint “Those who can, do; those who can’t, teach” (from George Bernard Shaw’s 1903 play Man and Superman), which also has some nugget of truth in it.
I agree with your point that people “from” the domain aren’t automatically equipped to start AI-building software in the domain, exactly because they often lack one of the most crucial software development skills — being able to (and having the desire to) describe a complex system with a finite set of deterministic rules.
But I don’t think we should be calling these people “domain experts”. I think we should reserve that name for the other group, for the people who truly and deeply understand the domain, the whys and whats and why nots.
That ship has sailed. Anyone who works in a field is now considered a domain expert (or "SME") even if they're nothing of the sort. There should ideally be another term that's a superset, but I doubt it would ever catch on.
You're doing a bit of a "true Scotsman fallacy" here.
It feels like without a bit of true Scotsmanship, the term “domain expert” doesn’t really mean anything.
1 reply →
Agree. If you can't explain it to someone else, you don't really understand it yourself. These people are practitioners not experts.
People you say are not experts are fully capable to explain whys and whats and why nots. In more details that you can absorb.
They will do that on examples tho. They will recognize and explain every single exception they see - but they are not able to list all exceptions up front. Because, that is highly unusual task
> But what they could do was tell him whether the handling of any particular transaction was right or wrong. So often times he would sit down with these accounting folks and go through lots of example transactions he came up with, and from there he essentially built up the requirements spec.
This combination of requirements is the business in most domains. Software or otherwise. Codefying different rules and executing them.
I have found this with (mechanical) engineers. They know what they want to see but don’t understand the underlying details which are more mathematical than the average engineer is able to work with. So the people working on engineering software are often physicists or applied mathematicians.
> So often times he would sit down with these accounting folks and go through lots of example transactions he came up with, and from there he essentially built up the requirements spec.
Right, so the spec is derived from examples, an interactive process that doesn’t require a programmer.
Interpolating between the examples still requires understanding of the domain for the interpolation to be a sensible one. And it’s an iterative feedback process: thinking about possible interpolations leads you to cases that need clarifications from the domain experts. The domain experts won’t come up with all relevant examples by themselves, and they typically aren’t as good at thinking about interpolations like a developer who is trained to always consider all possible cases; they don’t think in terms of models the way a developer does.
LLMs already have interactive discussions with me on the topics I engage with, including asking expansive questions, so I do not think this is beyond the realm of the technology.
The programmer skill is how to abstract the specs from all the examples. And then to formalize it. Actual coding is merely translation. And only beginners tend to focus on that.
Sounds very similar to what AI is good at.
2 replies →
Yes. The author stumbled on the p vs np problem but with humans. Verification is easy. Solving is the millennium prize question.
I agree that it has some resemblance, but the striking thing about NP-complete problems is that an efficient solution to any of them is an efficient solution to all of them, which makes it worth trying exceptionally hard to find one.
It could be that whatever lackluster expertise you can squeeze out of an LLM is good enough to discourage investment in the real thing since unlike NP-complete problems, expertise isn't generalizable.
Agreed. But there is a new dynamic coming to place. Who can execute faster now that we have AIs? I think the combination of (Good SWE using AIs) collaborating with (Domain Expert using AIs) is the winning team. Things are accelerating on all fronts, the frontier may be jagged but AIs is making progress on all capabilites.
This is the exasperating part about learning to speak Spanish using a textbook; you must guess the grammar rules because the textbook won't tell you. So, you use the English rule and hope and pray that it is the same in Spanish, and you'll be right the majority of the time but often wrong. Spanish textbooks written 100 years ago tell you the grammar rules and are more useful than recent textbooks.
While interesting, I rarely found knowing grammar rules beyond some very basic ones to help all that much for learning to speak a language, compared to language exposure and speaking practice.
> I have a strong suspicion that folks who have a high degree of domain expertise in a particular area will fail as software builders even in an agentic world because they will struggle to elucidate clearly the rules in their head that they've learned over years.
This is the part I disagree with.
In a non-agentic world, the expert and the programmer are two different people. If the expert finds a bug in the software, they have to describe that bug, send it over to a programmer to fix, wait for a new release and until that scenario occurs again, realize that their description was actually wrong, send a corrected description, rinse and repeat for a few iterations until the bug is finally fixed for good.
In a world of agents, the expert finds the bug, asks the agent to fix it, realizes that the fix is incorrect because of sloppy thinking, does a few iterations until the feature works correctly, and that's it. Bug fixed; with 10 minutes of work instead of a few weeks.
> realizes that the fix is incorrect because of sloppy thinking
If the domain expert doesn’t understand the generated code, they can only discover incorrect logic by specific examples (specific inputs), which is usually impossible to do exhaustively. The programmer, on the other hand, can see incorrect logic directly, generalized over all possible inputs and states. It’s the difference between testing and proving correctness.
> In a world of agents, the expert finds the bug, asks the agent to fix it, realizes that the fix is incorrect because of sloppy thinking, does a few iterations until the feature works correctly
That would require clearly (a) knowing what you want, and (b) expressing it unambiguously and in detail, including all edge cases. Essentially producing a spec.
Most people are not able to do either. Talking to an LLM does not change that.
but.. they can iterate real quick!
As long as the expert don't run of patience, he may be able to do that.
Consider finance sector -- the industry is powered by excel spreadsheets made by non-programmer having no clue how programming works.
1 reply →
I had the same problem in air travel industry. Our ticketing officers were never able to communicate how the ticket commission rules from our partners should be interpreted. I ended up adding a test framework, that allowed to add examples and counterexamples of existing flights to each rule, to verify that they are applied the way they'd assign commission by hand.
Domain expertise matters more than most bootcamp grads want to hear. Knowing how to phrase a query for a specific domain is half the battle.
You described reasoning by analogy, which two thirds of the population does.
Please explain explicitly what you are implying, because I don't get it
Who are the 1/3 of the population that does not reason by analogy?
This is basically the gap BDD tries to close I guess. It assumes that domain experts can't fully articulate the rules upfront.
I fully agree here. It's also not uncommon to have the reply for specific items be "Ahh, well that is a bit of an edge case see because of X and Y" and then you learn there are many many "special cases" to the point that they aren't that special anymore.
Some companies use product managers for this purpose. They translate the implicit domain knowledge of those experts into explicit requirements, mostly through interviewing.
> I have a strong suspicion that folks who have a high degree of domain expertise in a particular area will fail as software builders even in an agentic world because they will struggle to elucidate clearly the rules in their head that they've learned over years.
Maybe they need a build a hybrid expertise of "domain" and "software engineering". For example, robotic surgery requires expert surgeons to build sufficient expertise in robotics
Also, noticed a pretty high karma for a throwaway account.
Are we thinking about the wrong side here? Wouldn't the LLM be a super-intelligent financial expert as well as a super-intelligent coder?
It is neither.
I guess the flip side to that is the iterative conversation to develop these specifications is exactly what AI chatbots are good at.
It's also the main reason why very structured AI agent orchestration for software engineering modelled on rigid processes fails to really provide much value.
I suspect that the reason AI chatbots are not very good at developing such specifications is that they don’t build a domain model in their “head” — the only models they have are already hardcoded in their weights. A human learning a domain, on the other hand, forms new neural connections representing the model of the domain that they build in their head.