Comment by lukebuehler
2 months ago
It’s endlessly fascinating to read the AI transcript of an expert who _really_ knows how to cut to the chase. It just shows how much you can potentially squeeze out of these models. I’m also surprised to see that even Terrence Tao seems to use it in a way that resembles, in progression, how I use llms in my area of expertise (emphasis on progression and usage patterns, not absolute skill, obv I don’t match that): short pointed questions that goes all in on the jargon and machinery of the field and steers the llm hard (eg no softballs). I’ve noticed that llms switch their tone and meet you basically more or less on your level.
It reinforces how to "learn AI" is to first master the problem domain.
I can use AI for coding after decades of coding. I can't use it for theoretical physics because I can't evaluate the responses.
Correctamundo. That's EXACTLY what I just wrote about. And how we'll create the next generation of people who know how to evaluate responses when we're creating a generation of people who are increasingly reliant on LLMs to do the work makes for a strange paradox.
https://larsfaye.com/articles/ai-coding-will-prevent-experti...
It's so obvious to some and may never be considered by others.
If someone asks about AI I tell them step 1 is ask it how to do something you know all about. Step 2 is consider everything else you ask will be that inaccurate.
5 replies →
Good writeup, and certainly food for thought.
I wonder if this key point actually holds though:
>The skills to do so, however, are a function of someone who has experienced the friction and challenges over time that culminate in "good taste".
It's certainly sometimes true, but I don't think it's a general rule. Sometimes friction is just friction and sometimes you spend 1000 hours learning something that disappears and becomes obsolete or at least irrelevant to the goal.
Programmers used to need to know the instruction set of the CPU, assembly language and so on. Some still do but for most developers today that's not useful. Everything you know about 6800 assembly will not make your note-taking app any better.
I think we are in a state where AI tools so easily mimic what we used to do by hand that we think the friction is gone, but that's because we haven't raised the bar yet. One day we'll look at the Fable one-shot that's better than anything we ever made by hand ourselves and say "It could be even better".
And then the friction is back, just on a new level.
6 replies →
Great article… been worrying about this one quite a bit:
> If these tools demand expertise, yet the tools can actively circumvent the friction that cultivates expertise, then what is the path for one to become an expert so they can effectively use these tools?
I am not sure if I'm ignorant or have too much self esteem, but in cases where domain is not my expertise I try to cross challenge using different models to get less-biased and hopefully more correct insights.
4 replies →
The paradox is analogous to AI poisoning its own training data as more and more AI generated content is released on the Internet. Indeed, I see a hard terminus for both man and machine at some point.
2 replies →
Fully agree but ...
Industry reality is that for bespoke software solutions we have been running for decades on non-technical people straight out of a 5 day "boot camp" copy/pasting together "solutions" from SO, or "Sharepoint Configurators" cobbeling together a LoB process where is takes 3 minutes to get to the next screen with a 10% error rate etc.
Well put, great article.
Such a good read!
Nicely written!
More and more the skill of being able to ask the right question seems critical to me, and I don't know how one can do that without deeper and deeper domain expertise.
Asking the right questions has almost always been one of if not the most important skills. There are often an almost infinite list of possible approaches, but knowing the domain, you can get close to optimum very quickly. If you just rely on the LLM, it will give you the most common bog standard approach, including the usual bugs and usual quirks.
Yes, but sometimes the right question is "do I know enough about this subject to be asking questions about it?"
Succinctly put by Pablo Picasso's remark: "Computers are useless. They can only give you answers".
I agree. I (15+ years of coding) feel like I'm 100x more productive now using LLMs to code.
But I don't think a beginner would have the same experience. The AI still makes A LOT of stupid mistakes and decisions, but I catch them early enough (sometimes while it's still showing it's reasoning steps), stop the prompt, guide it on the right path, rinse and repeat.
Sometimes I am lazy and give the AI a broader prompt, let it do its thing, and then I come back to see that it spent 90% of the time working on some part/feature/implementation that was not really needed and that it over-engineered the solution.
I rarely write any line of code know or manually change any code, I tell the AI how to do it and what to watch out for. Many times it catches some edge-cases before I even haven to think about them. I do still feel like both me and the AI could miss some edge-cases now, because I'm thinking less about the implementation and what problems can arise, but I feel like 90% of "gotchas" are already engrained in my planning after so many years of coding and problem solving.
> I can use AI for coding after decades of coding. I can't use it for theoretical physics because I can't evaluate the responses.
That is what will happen though to future generations: they won't be able to use it for anything because none of them will have the "decades of coding" experience that you have had the privelege to have without AI.
They will have decades of experience with AI and they will be able to guide them by sniffing their hallucinations from single words.
8 replies →
That is probably not going to be the important difference.
You can't use it for theoretical physics because you have no meaningful question and there is no result that you can do anything with.
In contrast you don't need to be a coder to understand if your to-do list for cats works: You have an idea of what you want. You know the rough shape of what an app is and what it can do. You can put it in front of your can and look at it go. Or not.
There are still some interesting pitfalls though. A friend of mine vibe coded a small browser based app to track something personal over a long time period. Eventually he plans to share the app with others. It worked, but I was curious where it stored the data and he didn't know.
It turned out it was using LocalStorage, which is not entirely unreasonable, but there are obvious drawbacks (e.g if you move the file it might become inaccessible, if you switch machines there's no convenient way to transfer the data, there are all kinds of ways to lose it, etc..).
In this case "how is this storing my data" is a fundamental question with plenty of implications for your app over its lifetime, but most non-computer people don't think to ask it at all. These days many users enter CS programs without knowing how to manage files and folders on their computer, because even that is often abstracted away.
Depends on what you are building. Yes, a todo app is easy. What would the physics version be? Probably ask ”what are the acceleration of a failing object?”
But if you are building some complex data science statistical model and you don’t have any domain knowledge you won’t even know what to ask for.
> I can't use it for theoretical physics because I can't evaluate the responses.
I think pretty much the opposite. I can ask it to explain to me in ways I understand it. Even drill down the simplest of equations. Since llms have infinite patience. All I need to learn anything is patience.
I think the issue is that LLMs can be so confidently and convincingly wrong about anything.
And if we (LLM tool operators) don’t know the subject matter in question, we can’t easily distinguish what they are right or wrong about.
14 replies →
Indeed that’s the case but most of the programming problems are solved problems and unless you care how things are done someone with less/no knowledge can still make great use of programming to do what it needs to do.
Let’s not pretend you need to be programmer to use AI for programming like mathematicians need to solve math problems. Programming (most of the time)solves real problems rather than abstract constructs.
AI is already good enough to create the next “Facebook”(v1 and maybe v2 as well) without any real programmer. This makes the AI a big enabler and reduces the need for programmers vastly in the early stages of any business.
That being said as more online businesses will flourish these will require actual programmers after they get enough traction so the debate is still on if it will lead to massive layoffs in our industry.
That seems only partially true. I have no doubt it could crank out a UI like Facebook, but having it work for high volume is less well represented in the training data. Things that seems obvious to us, like sharding, using asynchronous messaging systems to provide more durability at the cost of instant consistency, traffic shedding, intelligent clients, conscious decisions on CAP properties in the light of user experience. etc., are not going to be requested by that non-technical founder.
Facebooks early years were dominated by a concern for users per engineer ratio; the rather florid style of LLMs suggest that they will generate so many systems of such a high complexity you will get Hadoop levels of non-application support needed - forget self healing, it’ll need constant LLM spend just to keep running at scale.
3 replies →
True, but I think it's the difference between doing it for a single $200 subscription vs spending $200 a day in tokens.
1 reply →
Yeah. I am a professor and I see a lot of other professors saying they are pivoting to "Teaching AI". I tell them I am pivoting harder to teaching fundamentals because using AI is easy. Its the knowledge which is hard.
> I can't use it for theoretical physics because I can't evaluate the responses.
A tool, even if it is a chisel, in the hands of a master sculptor would obviously result in a wildly different outcome.
But the AI can teach you to do theoretical physics, by guiding you through materials it selected for you, helping you with questions, etc.
Being good at "generating text" doesn't make you a good teacher. It was trained on internet users, not trained educators.
Most people posting on the internet, especially people who know about more complex subjects, are terrible teachers. Teaching is it's own skill
1 reply →
Yes but, AI can still be used for relatively common applications (coding basic pages), and furthermore you can build agentic loops to check more difficult code, or explain the conceptual order of the code, and I imagine over time the domains which will be considered “basic” for the AI will continue to grow, but perhaps so will the abilities of its users…
But someone who is not well versed in coding can evaluate the output simply by running the built app and testing it, which is an oversimplification and omits the tech debt and eventual issues that an experienced developer would catch or restrict the output.
Ah, yes, "experienced developers" see all the problems and resolve all the tech debt preemptively, which is why the state of software (even pre-LLM) was so great.
I recently prompted ChatGPT about theoretical physics because I wanted some examples of AI slop. It affirmed my hypothesis that dark energy results from the nonzero VEV of the Higgs field. Wow! I'm so smart! I solved dark energy!
Any actual physicist would probably be able to tell me why that's a category error. I don't know why because I'm not one. But there are actual mathematics underlying a statement like that and I'm 99% sure the maths don't work like that.
By the way I think that's why everyone thinks of so many weird physics ideas more than other fields. It's because things are explained in words that hide math, and you can make hypotheses in words that would be obviously nonsense at the level of maths. Like your boss asking why you don't just recompile the cloud.
My rule of thumb is: Never ask an LLM a question where you don't know what the right answer or wrong answer looks like.
Or in the words of Clickbait articles: MATHMATICIANS SHIVER OF THE IMPLICATIONS OF AI WILL MATHMATICIANS NO LONGER BE NEEDED.
to which betteridge's law of headlines says: NO.
That law only applies to headlines ending in a question mark.
1 reply →
[dead]
[dead]
I didn't understood anything about the thread, but reading Terrence's messages was weird because it looked exactly like the discussions I have with LLMs
I've mostly seen people trying to oneshot a result, while I'll quickly experienced that going through steps/discovery was more effective and more satisfying, since you can always steer it back in the right direction; while oneshotting is hit (and it kind feel like magic) or miss (and you'll have to rework your prompt).
I was struck in the same way but I think it makes sense in terms of a thinking partner.
It is still ultimately Terence that is steering things.
What is crazy to me is how few of other people's conversations like this I have actually read.
Tao is really great for this because the anti-AI crowd can't really chime in and take the thread in a pointless direction. It is hard to think of another human alive who can carry the weight of unassailable authority in the same way.
I encountered something fairly similar working with Claude a few days ago. For a current project I've been fairly hand-wavy with requirements since I was getting good results, but it seemed to be failing hard on some key points, so I started to be more strict with it. Even after the fails were resolved, I've noticed that Claude now behaves differently within that project, carefully checking and rechecking things up front and also looking to me for guidance more often. Mildly irritating, but if it works...
The most powerful way to use Fable in particular seems to be to pit it against Sol as a reviewer (or, I suppose, vice versa.) After going back and forth a half-dozen times, the result tends to be pretty much bulletproof.
What's somewhat disturbing is just how much Fable's code really does benefit from the review. It tends to leave a lot of low-hanging fruit, and you can see it getting kind of impatient when repeatedly called on it.
"We've now re-explored this question three times. While there were small differences along the way, one more would be pathological. Perhaps meditation or therapy would be more fitting."
This is basically how Fable told me to get therapy.
The problem with that approach, the code becomes over-complex, over-engineered and more often than not, it will take a specific point and iterate over it.
It's still what I do 90% of the case until I feel it's good enough for my usage.
What are you using to put Fable and Sol on each other?
1 reply →
The top comment has a counter example though https://x.com/DmitryRybin1/status/2079904005652893709
I think it's not even about the ability to steer the AI. Just the ability to ask the right questions
I am not impressed with that steering of the AI. In the case of Terry Tao, he clearly held a solid understanding of the problem domain and could also understand the outputs of the LLM at an expert level. He asked astute, topical questions that specifically probed areas of the problem that were either unclear or insufficiently specified, and ultimately ended up at a place where (presumably) he both understood the Jacobian Conjecture Counterexample exhaustively, and how it was derived. My impression is that through his learnings during the chat, he became sufficiently well-educated in this subdomain to apply some of these lessons to other mathematical pursuits.
In your counterexample, the human just repeatedly asked the model to "try harder". Not remotely the same.
I know but the point is that it worked too. You could argue one problem is harder than the other, I am not and expert so I wouldn't know. But at least it's clear the second result is also impressive, and didn't require much expertise. So it's wrong to say that you need expertise to get hard and impressive things done with AI.
Came to say the same thing. Experts in any field have a huge unlock from AI.
Is there a site with other good AI conversations like this?
Yes, the AI models are so useful in areas where you are an expert and can guide it in a useful way. I use multiple chats delay to help figure out client projects.
I think the questions are sufficiently suggestive to induce hallucinations. Still a lot of manual work to check all these claims.
For Professor Tao, I think that step is reading thru it once. Which is why the tool is more powerful for him than for us.
Well said! After a few years of LLM sycophancy convincing laypeople they've managed to make progress on or fully crack famous math problems it's satisfying to see a world-class mathematician really put it through its paces. It's all the more impressive that Tao seems satisfied with the conversation.
One of the prompting guides I came across early on was to ask your questions using vernacular of the domain not lay terms because the correct data you are looking for will be more likely to be around/near the expert terms rather than in the pop/lay/newbie terms next token wise.
Pretty spot on