Comment by czhu12
5 days ago
I'll personally attest: LLM's have been absolutely incredible to self learn new things post graduation. It used to be that if you got stuck on a concept, you're basically screwed. Unless it was common enough to show up in a well formed question on stack exchange, it was pretty much impossible, and the only thing you can really do is keep paving forward and hope at some point, it'll make sense to you.
Now, everyone basically has a personal TA, ready to go at all hours of the day.
I get the commentary that it makes learning too easy or shallow, but I doubt anyone would think that college students would learn better if we got rid of TA's.
>Now, everyone basically has a personal TA, ready to go at all hours of the day
This simply hasn't been my experience.
Its too shallow. The deeper I go, the less it seems to be useful. This happens quick for me.
Also, god forbid you're researching a complex and possibly controversial subject and you want it to find reputable sources or particularly academic ones.
I've found it excels at some things:
1) The broad overview of a topic
2) When I have a vague idea, it helps me narrow down the correct terminology for it
3) Providing examples of a particular category ("are there any examples of where v1 in the visual cortex develops in a disordered way?")
4) "Tell me the canonical textbooks in field X"
5) Posing math exercises
6) Free form branching--while talking about one topic, I want to shift to another that is distinct but related.
I agree they leave a lot to be desired when digging very deeply into a topic. And my biggest pet peeve is when they hallucinate fake references ("tell me papers that investigate this topic" will, for any sufficiently obscure topic, result in a bunch of very promising paper titles that are wholely invented).
These things are moving so quickly, but I teach a 2nd year combinatorics course, and about 3 months ago I tried th latest chatGPT and Deepseek -- they could answer very standard questions, but were wrong for more advanced questions, but often in quite subtle ways. I actually set a piece of homework "marking" chatGPT, which went well and students seemed to enjoy!
11 replies →
>When I have a vague idea, it helps me narrow down the correct terminology for it
so the opposite of Stack Overflow really, where if you have a vague idea your question gets deleted and you get reprimanded.
Maybe Stack Overflow could use AI for this, help you formulate a question in the way they want.
3 replies →
I find 2 invaluable for enhancing search, and combined with 1 & 4, it's a huge boost to self-learning.
outside of 5), I concur. It's good for discovery, as is Google for discovering topics while weighing on proper profesionally resources and articles for the learning.
It's too bad people are trying to substitute the latter with the chatGPT output itself. And I absolutely cannot trust any machine that is willing to lie to me rather than admit ignorance on a subject.
I’ve found the AI is particularly good at explaining AI, better than quite a lot of other coding tasks.
My core problem with LLMs is as you say; it's good for some simpler concepts, tasks, etc. but when you need to dive into more complex topics it will oversimplify, give you what you didn't ask for, or straight up lie by omission.
History is a great example, if you ask an LLM about a vaguely difficult period in history it will just give you one side and act like the other doesn't exist, or if there is another side, it will paint them in a very negative light which often is poorly substantiated; people don't just wake up and decide one day to be irrationally evil with no reason, if you believe that then you are a fool... although LLMs would agree with you more times than not since it's convenient.
The result of these things is a form of gatekeeping, give it a few years and basic knowledge will be almost impossible to find if it is deemed "not useful" whether that's an outdated technology that the LLM doesn't seem talked about very much anymore or a ideological issue that doesn't fall in line with TOS or common consensus.
A few weeks ago I was asking an LLM to offer anti-heliocentric arguments, from the perspective of an intelligent scientist. Although it initially started with what was almost a parody of writing from that period, with some prompting I got it to generate a strong rendition of anti-heliocentric arguments.
(On the other hand, it's very hard to get them to do it for topics that are currently politically charged. Less so for things that aren't in living memory: I've had success getting it to offer the Carthaginian perspective in the Punic Wars.)
7 replies →
The part about history perspectives sounds interesting. I haven't noticed this. Please post any concrete/specific examples you've encountered!
18 replies →
People _do_ just wake up one day and decide some piece of land should belong to them, or that they don't have enough money and can take yours, or they are just sick of looking at you and want to be rid of you. They will have some excuse or justification, but really they just want more than they have.
People _do_ just wake up and decide to be evil.
1 reply →
History in particular is rapidly approaching post-truth as a knowledge domain anyway.
There's no short-term incentive to ever be right about it (and it's easy to convince yourself of both short-term and long-term incentives, both self-interested and altruistic, to actively lie about it). Like, given the training corpus, could I do a better job? Not sure.
3 replies →
> people don't just wake up and decide one day to be irrationally evil with no reason, if you believe that then you are a fool
The problem with this, is that people sometimes really do, objectively, wake up and device to be irrationally evil. It’s not every day, and it’s not every single person — but it does happen routinely.
If you haven’t experienced this wrath yourself, I envy you. But for millions of people, this is their actual, 100% honest truthful lived reality. You can’t rationalize people out of their hate, because most people have no rational basis for their hate.
(see pretty much all racism, sexism, transphobia, etc)
2 replies →
> History is a great example, if you ask an LLM about a vaguely difficult period in history it will just give you one side and act like the other doesn't exist, or if there is another side, it will paint them in a very negative light which often is poorly substantiated
Which is why it's so terribly irresponsible to paint these """AI""" systems as impartial or neutral or anything of the sort, as has been done by hypesters and marketers for the past 3 years.
1 reply →
It's a floor raiser, not a ceiling raiser. It helps you get up to speed on general conventions and consensus on a topic, less so on going deep on controversial or highly specialized topics
That's the best and succinct description of using ChatGPT for this kind of things: it's a floor raiser, not a ceiling raiser.
1 reply →
I really think that 90% of such comments come from a lack of knowledge on how to use LLMs for research.
It's not a criticism, the landscape moves fast and it takes time to master and personalize a flow to use an LLM as a research assistant.
Start with something such as NotebookLM.
I use them and stay up to date reasonably. I have used NotebookLM, I have access to advanced models through my employer and personally, and I have done alot of research on LLMs and using them effectively.
They simply have limitations, especially on deep pointed subject matters where you want depth not breadth, and honestly I'm not sure why these limitations exist but I'm not working directly on these systems.
Talk to Gemini or ChatGPT about mental health things, thats a good example of what I'm talking about. As recently as two weeks ago my colleagues found that even when heavily tuned, they still managed to become 'pro suicide' if given certain lines of questioning.
And if we assume this is a knowledgable, technical community: how do you feel about the general populaces ability to use LLM's for research, without the skepticism needed to correct it?
> Also, god forbid you're researching a complex and possibly controversial subject and you want it to find reputable sources or particularly academic ones.
That's fine. Recognize the limits of LLMs and don't use them in those cases.
Yet that is something you should be doing regardless of the source. There are plenty of non-reputable sources in academic libraries and there are plenty of non-reputable sources from professionals in any given field. That is particularly true when dealing with controversial topics or historical sources.
IT can be beneficial for making your initial assessment, but you'll need to dig deeper for something meaningful. For example, I recently used Gemini's Deep Research to do some literature review on educational Color Theory in relation to PowerPoint presentations [1]. I know both areas rather well, but I wanted to have some links between the two for some research that I am currently doing.
I'd say that companies like Google and OpenAI are aware of the "reputable" concerns the Internet is expressing and addressing them. This tech is going to be, if not already is, very powerful for education.
[1] http://bit.ly/4mc4UHG
Taking a Gemini Deep Research output and feeding it to NotebookLM to create audio overviews is my current podcast go-to. Sometimes I do a quick Google and add in a few detailed but overly verbose documents or long form YouTube videos, and the result is better than 99% of the podcasts out there, including those by some academics.
1 reply →
Grandparent testimony of success, & parent testimony of frustration, are both just wispy random gossip when they don't specify which LLMs delivered the reported experiences.
The quality varies wildly across models & versions.
With humans, the statement "my tutor was great" and "my tutor was awful" reflect very little on "tutoring" in general, and are barely even responses to each other withou more specificity about the quality of tutor involved.
Same with AI models.
Latest OpenAI, Latest Gemini models, also tried with latest LLAMA but I didn’t expect much there.
I have no access to anthropic right now to compare that.
It’s an ongoing problem in my experience
Hmm. I have had pretty productive conversations with ChatGPT about non-linear optimization.
Granted, that's probably well-trodden ground, to which model developers are primed to pay attention, and I'm (a) a relative novice with (b) very strong math skills from another domain (computational physics). So Chuck and I are probably both set up for success.
What are some subjects that ChatGPT has given only shallow instruction on?
I'll tell you that I recently found it the best resource on the web for teaching me about the 30 Years War. I was reading a collection of primary source documents, and was able to interview ChatGPT about them.
Last week I used it to learn how to create and use Lehmer codes, and its explanation was perfect, and much easier to understand than, for example, Wikipedia.
I ask it about truck repair stuff all the time, and it is also great at that.
I don't think it's great at literary analysis, but for factual stuff it has only ever blown away my expectations at how useful it is.
It sounds like it is a good tool for getting you up to speed on a subject and you can leverage that newfound familiarity to better search for reputable sources on existing platforms like google scholar or arXiv.
I built a public tool a while back for some of my friends in grad school to support this sort of deep academic research use case. Sharing in case it is helpful: https://sturdystatistics.com/deepdive?search_type=external&q...
It is shallow. But as long as what you're asking it of is the kind of material covered in high school or college, it's fairly reliable.
This generation of AI doesn't yet have the knowledge depth of a seasoned university professor. It's the kind of teacher that you should, eventually, surpass.
I validate models in finance, and this is by far the best tool created for that purpose. I'd compare financial model validation to a Master's level task, where you're working with well established concepts, but at a deep, technical level. LLMs excel at that: ithey understand model assumptions, know what needs to be tested to ensure correctness, and can generate the necessary code and calculations to perform those tests. And finally, they can write the reports.
Model Validation groups are one of the targets for LLMs.
That’s one aspect of quantitative finance, and I agree. Elsewhere I noted that anything that is structured data + computation adjacent it has an easier time with, even excels in many cases.
It doesn’t cover the other aspects of finance, perhaps may be considered advanced (to a regular person at least) but less quantitative. Try having it reason out a “cigar butt” strategy and see if returns anything useful about companies that fit the mold from a prepared source.
Granted this isn’t quant finance modeling, but it’s a relatively easy thing as a human to do, and I didn’t find LLMs up to the task
The worst is when it's confidently wrong about things... Thankfully, this occurance is becoming less & less common -- or at least, it's boundary is beyond my subject matter expertise.
> Its too shallow. The deeper I go, the less it seems to be useful. This happens quick for me.
You must be using a free model like GPT-4o (or the equivalent from another provider)?
I find that o3 is consistently able to go deeper than me in anything I'm a nonexpert in, and usually can keep up with me in those areas where I am an expert.
If that's not the case for you I'd be very curious to see a full conversation transcript (in chatgpt you can share these directly from the UI).
I have access to the highest tier paid versions of ChatGPT and Google Gemini, I've tried different models, tuning things like size of context windows etc.
I know it has nothing to do with this. I simply hit a wall eventually.
I unfortunately am not at liberty to share the chats though. They're work related (I very recently ended up at a place where we do thorny research).
A simple one though, is researching Israel - Palestine relations since 1948. It starts off okay (usually) but it goes off the rails eventually with bad sourcing, fictitious sourcing, and/or hallucinations. Sometimes I actually hit a wall where it repeats itself over and over and I suspect its because the information is simply not captured by the model.
FWIW, if these models had live & historic access to Reuters and Bloomberg terminals I think they might be better at a range of tasks I find them inadequate for, maybe.
1 reply →
I have found that being very specific and asking things like "can you tell me what another perspective might be, such that I can understand potential counter-arguments might be, and how people with other views might see this topic?" can be helpful when dealing with complex/nuanced/contentious subjects. Likewise with regard to "reputable" sources.
This can happen if you use the free model and not a paid deep research model. You can use a gpt model and ask things like , "how many moons does Jupiter have?" But if you want to ask, "can you go on the web a research the affects that chamical a has had on our water supply a cite sources?", you will need to use a deep research model.
Why not do the research yourself rather than risk it misinterpreting? I FAFO'd repeatedly with that, and it is just horribly unreliable.
This is where feeding in extra context matters. Paste in text that shows up from a google search, textbooks preferred, to get in depth answers.
No one builds multi shot search tools because they eat tokens like no ones business, but I've deployed them internal to a company with rave reviews at the cost of $200 per seat per day.
> and you want it to find reputable sources
Ask it for sources. The two things where LLMs excel is by filling the sources on some claim you give it (lots will be made up, but there isn't anything better out there) and by giving you queries you can search for some description you give it.
Also, Perplexity.ai cites its sources by default.
It often invents sources. At least for me.
Try to red team blue team with it
Blue team you throw out concepts and have it steelman them
Red team you can literally throw any kind of stress test at your idea
Alternate like this and you will learn
A great prompt is “give me the top 10 xyz things” and then you can explore
Back when I was in 2006 I used Wikipedia to prepare for job interviews :)
Can you give a specific example where at certain depth it has stopped becoming useful?
> god forbid you're researching a complex and possibly controversial subject and you want it to find reputable sources
If you're really researching something complex/controversial, there may not be any
“The deeper I go, the less it seems to be useful. This happens quick for me. Also, god forbid you're researching a complex and possibly controversial subject and you want it to find reputable sources or particularly academic ones.”
These things also apply to humans. A year or so ago I thought I’d finally learn more about the Israeli/Palestinians conflict. Turns out literally every source that was recommended to me by some reputable source was considered completely non-credible by another reputable one.
That said I’ve found ChatGPT to be quite good at math and programming and I can go pretty deep at both. I can definitely trip it into mistakes (eg it seems to use calculations to “intuit” its way around sometimes and you can find dev cases where the calls will lead it the wrong directions), but I also know enough to know how to keep it on rails.
> learn more about the Israeli/Palestinians
> to be quite good at math and programming
Since LLMs are essentially summarizing relevant content, this makes sense. In "objective" fields like math and CS, the vast majority of content aligns, and LLMs are fantastic at distilling the relevant portions you ask about. When there is no consensus, they can usually tell you that ("this is nuanced topic with many perspectives...", etc), but they can't help you resolve the truth because, from their perspective, the only truth is the content.
Israel / Palestine is a collision between two internally valid and mutually exclusive worldviews. It's kind of a given that there will be two camps who consider the other non-reputable.
FWIW, the /r/AskHistorians booklist is pretty helpful.
https://www.reddit.com/r/AskHistorians/wiki/books/middleeast...
2 replies →
> Turns out literally every source that was recommended to me by some reputable source was considered completely non-credible by another reputable one.
That’s the single most important lesson by the way, that this conflict just has two different, mutually exclusive perspectives, and no objective truth (none that could be recovered FWIW). Either you accept the ambiguity, or you end up siding with one party over the other.
3 replies →
Re: conflicts and politics etc.
I've anecdotally found that real world things like these tend to be nuanced, and that sources (especially on the internet) are disincentivised in various ways from actually showing nuance. This leads to "side-taking" and a lack of "middle-ground" nuanced sources, when the reality lies somewhere in the middle.
Might be linked to the phenomenon where in an environment where people "take sides", those who display moderate opinions are simply ostracized by both sides.
Curious to hear people's thoughts and disagreements on this.
1 reply →
This is the part where you actually need to think and wonder if AI is the right tool in this particular purpose. Unfortunately you can't completely turn your brain off just yet.
What is "it"? Be specific: are you using some obsolete and/or free model? What specific prompt(s) convinced you that there was no way forward?
>Its too shallow. The deeper I go, the less it seems to be useful. This happens quick for me.
If its a subject you are just learning how can you possibly evaluate this?
If you're a math-y person trying to get up to speed in some other math-y field you can discern useless LLM output pretty quickly even as a relative novice.
Falling apart under pointed questioning, saying obviously false things, etc.
It's easy to recognize that something is wrong if it's wrong enough.
If we have custom trained LLMs per subject doesn't that solve the problem. The shallow problem seems really easy to solve
Can you share some examples?
Try doing deep research on the Israel - Palestine relations. That’s a good baseline. You’ll find it starts spitting out really useless stuff fast, or will try to give sources that don’t exist or are not reputable.
It's not a doctoral adviser.
Human interlocutors have similar issues.
When ChatGPT came out it was like I had the old Google back.
Learning a new programming language used to be mediated with lots of useful trips to Google to understand how some particular bit worked, but Google stopped being useful for that years ago. Even if the content you're looking for exists, it's buried.
And the old ChatGPT was nothing compared to what we have today, nowadays reasoning models will eat through math problems no problem when this was a major limitation in the past.
I don't buy it. Open AI doesn't come close to passing my credibility check. I don't believe their metrics.
2 replies →
I've learnt Rust in 12 weeks with a study plan that ChatGPT designed for me, catering to my needs and encouraging me to take notes and write articles. This way of learning allowed me to publish https://rustaceo.es for Spanish speakers made from my own notes.
I think the potential in this regard is limitless.
I learned Rust in a couple of weeks by reading the book.
Yeah regardless of time taken the study plan for Rust already exists (https://doc.rust-lang.org/book/). You don't need ChatGPT to regurgitate it to you.
2 replies →
But I agree though, I am getting insane value out of LLMs.
Doubtful. Unless you have very low standards of "learn".
2 replies →
Now this is a ringing endorsement. Specific stuff you learned, and actual proof of the outcome.
(Only thing missing is the model(s) you used).
I'd tend to assume the null hypothesis, that if they were capable of learning it, they'd have likely done fine without the AI writing some sort of lesson plan for them.
The psychic reader near me has been in business for a long time. People are very convinced they've helped them. Logically, it had to have been their own efforts though.
Standard ChatGPT 4o.
yes Chat GPT has helped me learn about actix web a framework similar to FastAPI in rust.
Absolutely. I used to have a lot of weird IPv6 issues in my home network I didn't understand. ChatGPT helped me to dump some traffic with tcpdump and explained what was happening on the network.
In the process it helped me to learn many details about RA and NDP (Router Advertisments/Neighbor Discovery Protocol, which mostly replace DHCP and ARP from IPv4).
It made me realize that my WiFi mesh routers do quite a lot of things to prevent broadcast loops on the network, and that all my weird issues could be attributed to one cheap mesh repeater. So I replaced it and now everything works like a charm.
I had this setup for 5 years and was never able to figure out what was going on there, although I really tried.
Would you say you were using the LLM as a tutor or as tech support, in that instance?
Probably both. I think ChatGPT wouldn't have found the issue by itself. But I noticed some specific things, asked for some tutoring and then it helped my to find the issues. It was a team effort, either of "us" alone wouldn't have finished the job. ChatGPT had some really wrong ideas in the process.
As somebody who has done both tech support, and lectured a couple of semesters at a business school on a technical topic... they're not that far removed from each other, it's just context and audience changes. The work is pretty similar.
So why not have tech support that teaches you, or a tutor that helps with you with a specific example problem you're having?
Providing you don't just rely on training data and can reduce hallucinations, this is the angle of attack that is likely the killer app some people are already seeing.
Vibe coding is nonsense because it's not teaching you to maintain and extend that application when the LLM runs out of steam. Use it to help you fix your problem in a way that you understand and can learn from? Rocket fuel to my mind. We're maybe not far away...
My rule with LLMs has been "if a shitty* answer fast gets you somewhere, the LLMs are the right tool," and that's where I've seen them for learning, too. There are times when I'm reading a paper, and there's a concept mentioned that I don't know - I could either divert onto a full Google search to try to find a reasonable summary, or I can ask ChatGPT and get a quick answer. For load-bearing concepts or knowledge, yes, I need to put the time in to actually research and learn a concept accurately and fully, but for things tangential to my actual current interests or for things I'm just looking at for a hobby, a shitty answer fast is exactly what I want.
I think this is the same thing with vibe coding, AI art, etc. - if you want something good, it's not the right tool for the job. If your alternative is "nothing," and "literally anything at all" will do, man, they're game changers.
* Please don't overindex on "shitty" - "If you don't need something verifiably high-quality"
I agree. I recently bought a broken Rolex and asked GPT for a list of tools I should get on Amazon to work on it.
I tried using YouTube to find walk through guides for how to approach the repair as a complete n00b and only found videos for unrelated problems.
But I described my issues and took photos to GPT O3-Pro and it was able to guide me and tell me what to watch out for.
I completed the repair (very proud of myself) and even though it failed a day later (I guess I didn’t re-seat well enough) I still feel far more confident opening it and trying again than I did at the start.
Cost of broken watch + $200 pro mode << Cost of working watch.
what was broken on it?
> the only thing you can really do is keep paving forward and hope at some point, it'll make sense to you.
I find it odd that someone who has been to college would see this as a _bad_ way to learn something.
"Keep paving forward" can sometimes be fruitful, and at other times be an absolutely massive waste of time.
I'm not sold on LLMs being a replacement, but post-secondary was certainly enriched by having other people to ask questions to, people to bounce ideas off of, people that can say "that was done 15 years ago, check out X", etc.
There were times where I thought I had a great idea, but it was based on an incorrect conclusion that I had come to. It was helpful for that to be pointed out to me. I could have spent many months "paving forward", to no benefit, but instead someone saved me from banging my head on a wall.
In college sometimes asking the right question in class or in a discussion section led by a graduate student or in a study group would help me understand something. Sometimes comments from a grader on a paper would point out something I had missed. While having the diligence to keep at it until you understand is valuable, the advantage of college over just a pile of textbooks is in part that there are other resources that can help you learn.
Imagine you're in college, have to learn calculus, and you can't afford a textbook (nor can find a free one), and the professor has a thick accent and makes many mistakes.
Sure, you could pave forward, but realistically, you'll get much farther with either a good textbook or a good teacher, or both.
In college you can ask people who know the answer. It's not until PhD level that you have to struggle without readily available answers.
The main difference in college was that there were office hours
I share your experience and view in that regard! There is so much criticism of LLMs and some of it is fair, like the problem of hallucinations, but that weakness can be reframed as a learning opportunity. It's like discussing a subject with a personal scientist who may at certain times test you, by making claims that may be simplistic or outright wrong, to keep the student skeptical and check if they are actually paying attention.
This requires a student to be actually interested in what they are learning tho, for others, who blindly trust its output, it can have adverse effects like the illusion of having understood a concept while they might have even mislearned it.
I agree... spent last weekend chatting with an LLM, filling in knowledge gaps I had on the electromagnetic spectrum. It does an amazing job educating you on known unknowns, but I think being able to know how to ask the right questions is key. I don't know how it would do with unknown unknowns, which is where I think books really shine and are still a preferable learning method.
"It used to be that if you got stuck on a concept, you're basically screwed."
There seems to be a gap in problem solving abilities here...the process of breaking down concepts into easier to understand concepts and then recompiling has been around since forever...it is just easier to find those relationships now. To say it was impossible to learn concepts you are stuck on is a little alarming.
> It used to be that if you got stuck on a concept, you're basically screwed
No, not really.
> Unless it was common enough to show up in a well formed question on stack exchange, it was pretty much impossible, and the only thing you can really do is keep paving forward and hope at some point, it'll make sense to you.
Your experience isn't universal. Some students learned how to do research in school.
"Screwed" = spending hours sifting through poorly-written, vaguely-related documents to find a needle in a haystack. Why would I want to continue doing that?
> "Screwed" = spending hours sifting through poorly-written, vaguely-related documents to find a needle in a haystack.
From the parent comment:
> it was pretty much impossible ... hope at some point, it'll make sense to you
Not sure where you are getting the additional context for what they meant by "screwed", but I am not seeing it.
1 reply →
I do a lot of research and independent learning. The way I translated “screwed” was “4-6 hours to unravel the issue”. And half the time the issue is just a misunderstanding.
It’s exciting when I discover I can’t replicate something that is stated authoritatively… which turns out to be controversial. That’s rare, though. I bet ChatGPT knows it’s controversial, too, but that wouldn’t be as much fun.
Like a car can be "beyond economical repair", a problem can be not worth the time (and uncertainty) or fixing. Especially from subjective judgement with incomplete information etc
As you say, your experience isn't universal, and we all have different modes of learning that work best for us.
They should have focused on social skills too I think
You should always check. I've seen LLM's being wrong (and obstinate) on topics which are one step separated from common knowledge.
I had to post the source code to win the dispute, so to speak.
Now think of all the times you didn't already know enough to go and find the real answer.
Ever read mainstream news reporting on something you actually know about? Notice how it's always wrong? I'm sure there's a name for this phenomenon. It sounds like exactly the same thing.
Why would you try to convince an LLM of anything?
Often you want to proceed further based on a common understanding, so it’s an attempt to establish that common understanding.
Well, not exactly convince. I was curious what will happen.
If you are curious it was a question about the behavior of Kafka producer interceptors when an exception is thrown.
But I agree that it is hard to resist the temptation to treat LLM's as a pear.
I don’t know what subject you are learning but for circuit design I have failed to get any response out of LLMs that’s not straight from a well known text book chapter that I have already read
It definitely depends heavily on how well represented the subject is on the internet at large. Pretty much every question I've asked it about SystemVerilog it gets wrong, but it can be very helpful about quite complex things about random C questions, for example why I might get undefined symbol errors with `inline` functions in C but only in debug mode.
On the other hand it told me you can't execute programs when evaluating a Makefile and you trivially can. It's very hit and miss. When it misses it's rather frustrating. When it hits it can save you literally hours.
A "TA" which has only the knowledge which is "common enough to show up in a well formed question on stack exchange"...
And which just makes things up (with the same tone and confidence!) at random and unpredictable times.
Yeah apart from that it's just like a knowledgeable TA.
LLMs are to learning what self driving cars are to transportation. They take you to the destination most of the time. But the problem is that if you use them too much your brain (your legs) undergoes metaphorical atrophy and when you are faced in the position of having to do it on your own, you are worse than you would be had you spent the time using your brain (legs). Learning is great but learning to learn is the real skilset. You don't develop that if you are always getting spoonfed.
This is one of the challenges I see with self-driving cars. Driving requires a high level of cognitive processing to handle changing conditions and potential hazards. So when you drive most of your brain is engaged. The impact self-driving cars are going to have on mental stimulation, situational awareness, and even long-term cognitive health could be bigger than we think, especially if people stop engaging in tasks that keep those parts of the brain active. That said, I love the idea of my car driving me around the city while I play video games.
Regarding LLMs, they can also stimulate thinking if used right.
> It used to be that if you got stuck on a concept, you're basically screwed.
Given that humanity has been able to go from living in caves to sending spaceships to the moon without LLMs, let me express some doubt about that.
Even without going further, software engineering isn't new and people have been stuck on concepts and have managed to get unstuck without LLMs for decades.
What you gain in instant knowledge with LLMs, you lose in learning how to get unstuck, how to persevere, how to innovate, etc.
Depending on context, I would advise you to be extremely careful. Modern LLMs are Gell‑Mann Amnesia to the square. Once you watched a LLM butcher a topic you know extremely well, it is spooky how much authority they still project on the next interaction.
IMO your problem is the same as many people these days: you don't own any books and refuse to get them.
> It used to be that if you got stuck on a concept, you're basically screwed. Unless it was common enough to show up in a well formed question on stack exchange,
It’s called basic research skills - don’t they teach this anymore in high school, let alone college? How ever did we get by with nothing but an encyclopedia or a library catalog?
Something is lost as well if you do 'research' by just asking an LLM. On the path to finding your answer in the encyclopedia or academic papers, etc. you discover so many things you weren't specifically looking for. Even if you don't fully absorb everything there's a good chance the memory will be triggered later when needed: "Didn't I read about this somewhere?".
Yep, this is why I just don’t enjoy or get much value from exploring new topics with LLMs. Living in the Reddit factoid/listicle/TikTok explainer internet age my goal for years (going back well before ChatGPT hit the scene) has been to seek out high quality literature or academic papers for the subjects I’m interested in.
I find it so much more intellectually stimulating then most of what I find online. Reading e.g. a 600 page book about some specific historical event gives me so much more perspective and exposure to different aspects I never would have thought to ask about on my own, or would have been elided when clipped into a few sentence summary.
I have gotten some value out of asking for book recommendations from LLMs, mostly as a starting point I can use to prune a list of 10 books down into a 2 or 3 after doing some of my research on each suggestion. But talking to a chatbot to learn about a subject just doesn’t do anything for me for anything deeper than basic Q&A where I simply need a (hopefully) correct answer and nothing more.
LLMs hallucinate too much and too frequently for me to put any trust in their (in)ability to help with research.
Its a little disingenuous to say that, most of us would have never gotten by with literally just a library catalog and encyclopedia. Needing a community to learn something in is needed to learn almost anything difficult and this has always been the case. That's not just about fundamentally difficult problems but also about simple misunderstandings.
If you don't have access to a community like that learning stuff in a technical field can be practically impossible. Having an llm to ask infinite silly/dumb/stupid questions can be super helpful and save you days of being stuck on silly things, even though it's not perfect.
Wait until you waste days down a hallucination-induced LLM rabbit hole.
> most of us would have never gotten by with literally just a library catalog and encyclopedia.
I meant the opposite, perhaps I phrased it poorly. Back in the day we would get by and learn new shit by looking for books on the topic and reading them (they have useful indices and tables of contents to zero in on what you need and not have to read the entire book). An encyclopedia was (is? Wikipedia anyone?) a good way to get an overview of a topic and the basics before diving into a more specialized book.
> LLM's have been absolutely incredible to self learn new things post graduation.
I haven't tested them on many things. But in the past 3 weeks I tried to vibe code a little bit VHDL. On the one hand it was a fun journey, I could experiment a lot and just iterated fast. But if I was someone who had no idea about hardware design, then this trash would've guided me the wrong way in numerous situations. I can't even count how many times it has built me latches instead of clocked registers (latches bad, if you don't know about it) and that's just one thing. Yes I know there ain't much out there (compared to python and javascript) about HDLs, even less regarding VHDL. But damn, no no no. Not for learning. never. If you know what you're doing and you have some fundamental knowledge about the topic, then it might help to get further, but not for the absolute essentials, that will backfire hard.
LLM's are useful because they can recommend several famous/well-known books (or even chapters of books) that are relevant to a particular topic. Then you can also use the LLM to illuminate the inevitable points of confusion and shortcomings in those books while you're reading and synthesizing them.
Pre-LLM, even finding the ~5 textbooks with ~3 chapters each that decently covered the material I want was itself a nontrivial problem. Now that problem is greatly eased.
> they can recommend several famous/well-known books
They can recommend many unknown books as well, as language models are known to reference resources that do not exist.
5 replies →
Everything you state was available in the net. Did the people grow more informed? So far practice suggests the opposite conclusion[0]. I hope for the best, but the state of the world so far doesn't justify it...
[0] https://time.com/7295195/ai-chatgpt-google-learning-school/
I was recently researching and repairing an older machine with a 2020 Intel Gen 9 CPU and a certain socket motherboard, and AI made it so much easier and pleasant to find information and present answers about various generations and sockets and compatibility, I felt like I didn't deserve this kind of tool. LLMs are not great for some things, but amazing for others.
I use it to refresh some engineering maths I have forgotten (ODE, numerical schemas, solving linear equations, data sciences algorithms, etc) and the explanations are most of the time great and usually 2 or 3 prompts give me a good overview and explain the tricky details.
I also use it to remember some python stuff. In rust, it is less good: makes mistakes.
In those two domains, at that level, it's really good.
It could help students I think.
Maybe TAs are a good metaphor. Back in college, the classmates who went to TAs for help multiple times every week, really didn't get the material. I literally never went to a TA for help in my life, and learned the material much better by really figuring it out myself, "the hard way" (the only way?). These were math, EE, and CS courses.
how are you checking its correctness if you're learning the topic?
The same way you check if you learn in any other ways? Cross referencing, asking online, trying it out, etc.
We're giving this to children who inherently don't have those skills.
This is important, as benchmarks indicate we aren't at a level where a LLM can truly be relied upon to teach topics across the board.
It is hard to verify information that you are unfamiliar with. It would be like learning from a message board. Can you really trust what is being said?
What is the solution? Toss out thousands of years of tested pedagogy which shows that most people learn by trying things, asking questions, and working through problems with assistance and instead tell everyone to read a textbook by themselves and learn through osmosis?
So what if the LLM is wrong about something. Human teachers are wrong about things, you are wrong about things, I am wrong about things. We figure it out when it doesn't work the way we thought and adjust our thinking. We aren't learning how to operate experimental nuclear reactors here, where messing up results in half a country getting irradiated. We are learning things for fun, hobbies, and self-betterment.
>we aren't at a level where a LLM can truly be relied upon to teach topics across the board.
You can replace "LLM" here with "human" and it remains true.
Anyone who has gone to post-secondary has had a teacher that relied on outdated information, or filled in gaps with their own theories, etc. Dealing with that is a large portion of what "learning" is.
I'm not convinced about the efficacy of LLMs in teaching/studying. But it's foolish to think that humans don't suffer from the same reliability issue as LLMs, at least to a similar degree.
1 reply →
If it's coding you can compile or test your program. For other things you can go to primary sources
> It used to be that if you got stuck on a concept, you're basically screwed.
We were able to learn before LLMs.
Libraries are not a new thing. FidoNet, USENET, IRC, forums, local study/user groups. You have access to all of Wikipedia. Offline, if you want.
I learned how to code using the library in the 90s.
I think it's accurate to say that if I had to do that again, I'm basically screwed.
Asking the LLM is a vastly superior experience.
I had to learn what my local library had, not what I wanted. And it was an incredible slog.
IRC groups is another example--I've been there. One or two topics have great IRC channels. The rest have idle bots and hostile gatekeepers.
The LLM makes a happy path to most topics, not just a couple.
>Asking the LLM is a vastly superior experience.
Not to be overly argumentative, but I disagree, if you're looking for a deep and ongoing process, LLMs fall down, because they can't remember anything and can't build upon itself in that way. You end up having to repeat alot of stuff. They also don't have good course correction (that is, if you're going down the wrong path, it doesn't alert you, as I've experienced)
It also can give you really bad content depending on what you're trying to learn.
I think for things that represent themselves as a form of highly structured data, like programming languages, there's good attunement there, but you start talking about trying to dig around about advanced finance, political topics, economics, or complex medical conditions the quality falls off fast, if its there at all
11 replies →
Agreed, I'd add to the statement, "you're basically screwed, often, without investing a ton of time (e.g. weekends)"
Figuring out 'make' errors when I was bad at C on microcontrollers a decade ago? (still am) Careful pondering of possible meanings of words... trial and error tweaks of code and recompiling in hopes that I was just off by a tiny thing, but 2 hours later and 30 attempts later, and realizing I'd done a bad job of tracking what I'd tried and hadn't? Well, made me better at being careful at triaging issues. But it wasn't something I was enthusiastic to pick back up the next weekend, or for the next idea I had.
Revisiting that combination of hardware/code a decade later and having it go much faster with ChatGPT... that was fun.
Are we really comparing this research to just writing and having a good answer in a couple of seconds?
Like, I agree with you and I believe those things will resist and will always be important, but it doesn't really compare in this case.
Last week I was in the nature and I saw a cute bird that I didn't know. I asked an AI and got the correct answer in 10 seconds. Of course I would find the answer at the library or by looking at proper niche sites, but I would not have done it because I simply didn't care that much. It's a stupid example but I hope it makes the point
There's a gigantic difference between outsourcing your brain to generative AI (LLMs, Stable Diffusion, ..) and pattern recognition that recognises songs, birds, plants or health issues.
It’s not an or/either situation.
> We were able to learn before LLMs.
We were able to learn before the invention of writing, too!
It's one more step on the path to A Young Lady's Illustrated Primer. Still a long way to go, but it's a burden off my shoulders to be able to ask stupid questions without judgment or assumptions.
>Now, everyone basically has a personal TA, ready to go at all hours of the day.
And that's a bad thing. Nothing can replace the work in learning, the moments where you don't understand it and have to think until it hurts and until you understand. Anything that bypasses this (including, for uni students, leaning too heavily on generous TAs) results in a kind of learning theatre, where the student thinks they've developed an understanding, but hasn't.
Experienced learners already have the discipline to use LLMs without asking too much of them, the same way they learned not to look up the answer in the back of the textbook until arriving at their own solution.
[dead]
I'll personally attest anecdotes mean little in sound arguments.
When I got stuck on a concept, I wasn't screwed: I read more; books if necessary. StackExchange wasn't my only source.
LLMs are not like TAs, personal or not, in the same way they're not humans. So it then follows we can actually contemplate not using LLMs in formal teaching environments.
Sometimes you don't have tens of hours to spend on a single problem you can not figure out.
I agree. We are talking about technical, mathy stuff, right?
As long as you can tell that you don’t deeply understand something that you just read, they are incredible TAs.
The trick is going to be to impart this metacognitive skill on the average student. I am hopeful we will figure it out in the top 50 universities.
I've found LLMs to be great in summarizing non-controversial non-technical bodies of knowledge. For example - the facts in the long swings of regional histories. You have to ask for nuance and countervailing viewpoints, though you'll get them if they're in there.
I have been very skeptical of AI. But getting unstuck when studying. Its a huge help. This is the first I see the benifit with AI. I take a picture of a formula and ask chatgpt to explain the steps.
I really don't get it. Literally the only thing you need to do research is know what term to look up and you get at a bunch of info written by real humans
>Unless it was common enough to show up in a well formed question on stack exchange, it was pretty much impossible
sorry but if you've gone to university, in particular at a time when internet access was already ubiquitous, surely you must have been capable to find an answer to a programming problem by consulting documentation, manual, or tutorials which exist on almost any topic.
I'm not saying the chatbot interface is necessarily bad, it might be more engaging, but it literally does not present you with information you couldn't have found yourself.
If someone has a computer science degree and tells me without stack exchange they can't find solutions to basic problems that is a red flag. That's like the article about the people posted here who couldn't program when their LLM credits ran out
Yes. Learning assistance is one of the few use cases of IA that I have had success with.
> I'll personally attest: LLM's have been absolutely incredible to self learn new things post graduation.
How do you know when it's bullshitting you though?
All the same ways I know when Internet comments, outdated books, superstitions, and other humans are bullshitting me.
Sometimes right away, something sounds wrong. Sometimes when I try to apply the knowledge and discover a problem. Sometimes never, I believe many incorrect things even today.
When you Google the new term it gives you and you get good results, you know it wasn't made up.
Since when was it acceptable to only ever look at a single source?
That’s the neat part, you don’t!
Same way you know for humans?
But an LLM isn't a human, with a human you can read body language or look up their past body of work. How do you do his with against an LLM
1 reply →
I haven't used LLMs too much for study yet, so maybe they really are force multipliers, but I completely disagree with your assessment of self-directed learning pre-llm, the paving forward part isn't so dire.
The internet, and esp. stack exchange is a horrible place to learn concepts. For basic operational stuff, sure that works, but one should mostly be picking up concepts form books and other long form content. When you get stuck it's time to do three things:
Incorporate a new source that covers the same material in a different way, or at least from a different author.
Sit down with the concept and write about it and actively try to reformulate it and everything you do/don't understand in your own words.
Take a pause and come back later.
Usually one of these three strategies does the trick, no llm required. Obviously these approaches require time that using an LLM wouldn't. I have a suspicion doing it this way will also make it stick in long term memory better, but that's just a hunch.
I'm curious what you've used it to learn
Nah I'm calling BS, for me self-learning after college is either Just Do It(tm) trial-and-error, blogs, or hitting the nonfiction section of the library.
You can always ask in stack exchange, IRC or forums.
Closed: duplicate
Closed: RTFM, dumbass
<No activity for 8 years, until some random person shows up and asks "Hey did you figure it out?">
My favourite moment was when I tried to figure a specific software issue out that had to do with obscure hardware and after hours I found one forum post detailing the solution with zero replies. And it turns out I wrote it myself, years prior and had forgotten about it.
2 replies →
Or even worse, you ask an "xyz" question in the "xyz" StackExchange, then immediately get flagged as off-topic
"Nevermind I figured it out"
On IRC> Newb: I need help with <thing>. Does anyone have any experience with this?
J. Random Hacker: Why are you doing it like that?
Newb: I have <xyz> constraint in my case that necessitates this.
J. Random Hacker: This is a stupid way to do it. I'm not going to help you.
This is the way to go.