Comment by jacobedawson

> Learning something online 5 years ago often involved trawling incorrect, outdated or hostile content and attempting to piece together mental models without the chance to receive immediate feedback on intuition or ask follow up questions. This is leaps and bounds ahead of that experience.

But now, you're wondering if the answer the AI gave you is correct or something it hallucinated. Every time I find myself putting factual questions to AIs, it doesn't take long for it to give me a wrong answer. And inevitably, when one raises this, one is told that the newest, super-duper, just released model addresses this, for the low-low cost of $EYEWATERINGSUM per month.

But worse than this, if you push back on an AI, it will fold faster than a used tissue in a puddle. It won't defend an answer it gave. This isn't a quality that you want in a teacher.

So, while AIs are useful tools in guiding learning, they're not magical, and a healthy dose of scepticism is essential. Arguably, that applies to traditional learning methods too, but that's another story.

The fear of asking stupid questions is real, especially if one has had a bad experience with humiliating teachers or professors. I just recently saw a video of a professor subtly shaming and humiliating his students for answering questions to his own online quiz. He teaches at a prestigious institution and has a book that has a very good reputation. I stopped watching his video lectures.

> Adding in a mode that doesn't just dump an answer but works to take you through the material step-by-step is magical

Except these systems will still confidently lie to you.

The other day I noticed that DuckDuckGo has an Easter egg where it will change its logo based on what you've searched for. If you search for James Bond or Indiana Jones or Darth Vader or Shrek or Jack Sparrow, the logo will change to a version based on that character.

If I ask Copilot if DuckDuckGo changes its logo based on what you've searched for, Copilot tells me that no it doesn't. If I contradict Copilot and say that DuckDuckGo does indeed change its logo, Copilot tells me I'm absolutely right and that if I search for "cat" the DuckDuckGo logo will change to look like a cat. It doesn't.

Copilot clearly doesn't know the answer to this quite straightforward question. Instead of lying to me, it should simply say it doesn't know.

Consider the adoption of conventional technology in the classroom. The US has spent billions on new hardware and software for education, and yet there has been no improvement in learning outcomes.

This is where the skepticism arises. Before we spend another $100 billion on something that ended up being worthless, we should first prove that it’s actually useful. So far, that hasn’t conclusively been demonstrated.

I agree with all that you say. It’s an incredible time indeed. Just one thing I can’t wrap my mind around is privacy. We all seem to be asking sometimes stupid and some times incredibly personal questions to these llms. Questions that we may not even speak out loud from embarrassment or shame or other such emotions to even our closest people. How are these companies using our data ? More importantly what are you all doing to protect yourself from misuse of your information? Or is it if you want to use it you have to give up such privacy and uncomfortableness ?

> Learning something online 5 years ago often involved trawling incorrect, outdated or hostile content and attempting to piece together mental models without the chance to receive immediate feedback on intuition or ask follow up questions.

That trained and sharpened invaluable skills involving critical thinking and grit.

LLMs, by design, are peak Duning-Kruegers, which means they can be any good of a study partner for basic introductory lessons and topics. Yet they still require handholding and thorough verification, because LLMs will spit out factually incorrect information with confidence and will fold on correct answers when prodded. Yet the novice does not posses the skill to handhold the LLM. I think there's a word for that, but chadgbt is down for me today.

Furthermore, forgetting curve is a thing and therefore having to piece information together repetitively, preferably in a structured manner, leads to a much better information retention. People love to claim how fast they are "learning" (more like consuming tiktoks) from podcasts at 2x speed and LLMs, but are unable to recite whatever was presented few hours later.

Third, there was a paper circulating even here on HN that showed that use of LLMs literally hinder brain activation.

In my experience asking questions to Claude, the amount of incorrect information it gives is on a completely different scale in comparison to traditional sources. And the information often sounds completely plausible too. When using a text book, I would usually not Google every single piece of new information to verify it independently, but with Claude, doing that is absolutely necessary. At this point I only use Claude as a stepping stone to get ideas on what to Google because it is giving me false information so often. That is the only "effective" usage I have found for it, which is obviously much less useful than a good old-fashioned textbook or online course.

Admittedly I have less experience with ChatGPT, but those experiences were equally bad.

>I'm puzzled (but not surprised) by the standard HN resistance & skepticism

The good: it can objectively help you to zoom forward in areas where you don’t have a quick way forward.

The bad: it can objectively give you terrible advice.

It depends on how you sum that up on balance.

Example: I wanted a way forward to program a chrome extension which I had zero knowledge of. It helped in an amazing way.

Example: I am keep trying to use it in work situations where I have lots of context already. It performs better than nothing but often worse than nothing.

Mixed bag, that’s all. Nothing to argue about.

HN is resistant because at the end of the day, these are LLMs. They cannot and do not think. They generate plausible responses. Try this in your favorite LLM: "Suppose you're on a game show trying to win a car. There are three doors, one with a car and two with goats. You pick a door. The host then gives you the option to switch doors. What is the best strategy in this situation?" The LLM will recognize this as SIMILAR to the Monty Hall problem and tell you to always switch. I just reproduced this with ChatGPT.

But this is completely wrong! In the Monty Hall problem, the host has to reveal a door with a goat behind it for you to gain the benefit of switching. I have to point this out for the LLM to get it right. It did not reason about the problem I gave it, it spat out the most likely response given the "shape" of the problem.

This is why shrugging and saying "well humans get things wrong too" is off base. The problem is that the LLM is not thinking, period. So it cannot create a mental model of your understanding of a subject, it is taking your text and generating the next message in a conversation. This means that the more niche the topic (or your particular misunderstanding), the less useful it will get.

It's quite boring to listen to people praising AI (worshipping it, putting it on a pedastal, etc). Those who best understand the potential of it aren't doing that. Instead they're talking about various specific things that are good or bad, and they don't go out of the way to lick AI's boots, but when they're asked they acknowledge that they're fans of AI or bullish on it. You're probably misreading a lot of resistance & skepticism on HN.

> I'm puzzled (but not surprised) by the standard HN resistance & skepticism.

It happens with many technological advancements historically. And in this case there are people trying hard to manufacture outrage about LLMs.

Yeah, I've been a game-dev forever and had never built a web-app in my life (even in college) I recently completed my 1st web-app contract, and gpt was my teacher. I have no problem asking stupid questions, tbh asking stupid questions is a sign of intelligence imo. But where is there to even ask these days? Stack Overflow may as well not exist.

The freedom to ask "dumb" questions without judgment is huge, and it's something even the best classrooms struggle to provide consistently

20 years ago I used to hang out in IRC channels where I learnt so much. I wasn't afraid of asking stupid questions. These bots are pale imitation of that.

I've learnt a great many things online, but I've also learnt a great many more from books, other people and my own experience. You just have to be selective. Some online tutorials are excellent, for example the Golang and Rust tutorials. But for other things books are better.

What you are missing is the people. We used to have IRC and forums where you could discuss things in great depth. Now that's gone and the web is owned by big tech and governments you're happy to accept a bot instead. It's sad really.

I know some Spanish - close to B1. I find ChatGPT to be a much better way to study than the standard language apps. I can create custom lessons, ask questions about language nuances etc. I can also have it speak the sentences and practice pronunciation.

> Learning something online 5 years ago often involved trawling incorrect, outdated or hostile content

What's funny is tha LLMs got trained on datasets that includes all that incorrect, outdated or hostile content.

> Should we trust the information at face value without verifying from other sources? Of course not, that's part of the learning process.

It mostly isn't, the point of the good learning process is to invest time into verifying "once" and then add verified facts to the learning material so that learners can spend that time learning the material instead of verifying everything again.

Learning to verify is also important, but it's a different skill that doesn't need to be practiced literally every time you learn something else.

Otherwise you significantly increase the costs of the learning process.

> An underrated quality of LLMs as study partner is that you can ask "stupid" questions without fear of embarrassment.

Not underrated at all. Lots of people were happy to abandon Stack Overflow for this exact reason.

> Adding in a mode that doesn't just dump an answer but works to take you through the material step-by-step is magical

I'd be curious to know how much this significantly differs from just a custom academically minded GPT with an appropriately tuned system prompt.

https://chatgpt.com/gpts

>Should we trust the information at face value without verifying from other sources? Of course not, that's part of the learning process. Will some (most?) people rely on it lazily without using it effectively? Certainly, and this technology won't help or hinder them any more than a good old fashioned textbook.

Not true if we make the assumption that most books from publishing houses with good reputation are verified for errors. Good books maybe dated but they don't contain made up things.

Skepticism is great, it means less competition. I'm forcing everyone around me to use it.

>Learning something online 5 years ago often involved trawling incorrect, outdated or hostile content and attempting to piece together mental models without the chance to receive immediate feedback on intuition or ask follow up questions. This is leaps and bounds ahead of that experience.

Researching online properly requires cross referencing, seeing different approaches, and understanding various strenghts, weaknesses, and biases among such sources.

And that's for objective information, like math and science. I thought Grok's uhh... "update" shows enough of the dangers when we resort to a billionaire controlled oracle as a authoritative resource.

>Will some (most?) people rely on it lazily without using it effectively? Certainly, and this technology won't help or hinder them any more than a good old fashioned textbook.

I don't think facilitating bad habits like lazy study is an effective argument.And I don't really subscribe to this ineviability angle either: https://tomrenner.com/posts/llm-inevitabilism/

A lot of the comments have to do with how does one use these things to speed up learning. I've tried a few things. A couple of them are prompts: 1. Make me a tutorial on ... 2. Make probes to quiz me along the way ...

I think the trick is to look at the references that the model shows you. e.g. o3 with web search will give you lots of references. 90% of the time just reading those tells me of the model and I are aligned.

For example the other day I was figuring out why using SQL alchemy Sessions and PyTest async might I get the "Connection was attached to different loop" error. Now If you started using o3 to give you a solution you would take a long time because there would be small mistakes it would make in the code and You would spend a lot of time trying to fix it. Better way to use 03 then was to ask it to give you debugging statements (session listeners attached to Sqlalchemy sessions) and understand by reading code output, what was going on. Much faster.

Once it(and I) started looking at the debugging statements the error became clear: the session/connections where leaking to different event loop, a loop_scope= param needed to be specified for all fixtures. O3 did not provide a correct solution for the code but I could, but it's help.was crucial in writing a fuck ton of debugging code and getting clues.

I also asked o3 to make a bunch of probe questions to test me, for example it said something like: try changing the loop_scope module to function, what do you expect the loopid and transaction id to be for this test?

I learned More than I realized about ORMs and how it can be used to structure transactions and structuring async PyTest tests.

One thing I'm trying these days is to have it create a memory palace from all the stuff I have in my house and link it to a new concept I'm learning and put it into an anki decks.

Firstly, I think skepticism is a healthy trait. It's OK to be a skeptic. I'm glad there are a lot of skeptics because skepticism is the foundation of inquiry, including scientific inquiry. What if it's not actually Zeus throwing those lightning bolts at us? What if the heliocentric model is correct? What if you actually can't get AIDS by hugging someone who's HIV positive? All great questions, all in opposition to the conventional (and in some cases "expert") wisdom of their time.

Now in regards to LLMs, I use them almost every day, so does my team, and I also do a bit of postmortem and reflection on what was accomplished with them. So, skeptical in some regards, but certainly not behaving like a Luddite.

The main issue I have with all the proselytization about them, is that I think people compare getting answers from an LLM to getting answers from Google circa 2022-present. Everyone became so used to just asking Google questions, and then Google started getting worse every year; we have pretty solid evidence that Google's results have deteriorated significantly over time. So I think that when people say the LLM is amazing for getting info, they're comparing it to a low baseline. Yeah maybe the LLM's periodically incorrect answers are better than Google - but are you sure they're not better than just RTFM'ing? (Obviously, it all depends on the inquiry.)

The second, related issue I have is that we are starting to see evidence that the LLM inspires more trust than it deserves due to its humanlike interface. I recently started to track how often Github Copilot gives me a bad or wrong answer, and it's at least 50% of the time. It "feels" great though because I can tell it that it's wrong, give it half the answer, and then it often completes the rest and is very polite and nice in the process. So is this really a productivity win or is it just good feels? There was a study posted on HN recently where they found the LLM actually decreases the productivity of an expert developer.

So I mean I'll continue to use this thing but I'll also continue to be a skeptic, and this also feels like kinda where my head was with Meta's social media products 10 years ago, before I eventually realized the best thing for my mental health was to delete all of them. I don't question the potential of the tech, but I do question the direction that Big Tech may take it, because they're literal repeat offenders at this point.

> Certainly, and this technology won't help or hinder them any more than a good old fashioned textbook.

Except that the textbook was probably QA’d by a human for accuracy (at least any intro college textbook, more specialized texts may not have).

Matters less when you have background in the subject (which is why it’s often okay to use LLMs as a search replacement) but it’s nice not having a voice in the back of your head saying “yeah, but what if this is all nonsense”.

There is no skepticism. LLMs are fundamentally lossy and as a result they’ll always give some wrong result/response somewhere. If they are connected to a data source, this can reduce the error rate but not eliminate it.

I use LLMs but only for things that I have a good understanding of.

I think both sides seem to have the same issues with the other. One side is sceptical that the other is getting good use from LLMs, and the other suggests they're just not using it correctly.

Both sides think the other is either exaggerating or just not using the tool correctly.

What both sides should do is show evidence in the form of chat extracts or videos. There are a number from the pro-LLM side, but obviously selection bias applies here. It would be interesting if the anti-LLM side started to post more negative examples (real chat extracts or videos).

> we have access to incredible tools like this

At what cost? Are you considering all the externalities? What do you think will happen when Altman (and their investors) decides to start collecting their paychecks?

It's not just "stupid" questions.

In my experience, most educational resources are either slightly too basic or slightly too advanced, particularly when you're trying to understand some new and unfamiliar concept. Lecturers, Youtubers and textbook authors have to make something that works for everybody, which means they might omit information you don't yet know while teaching you things you already understand. This is where LLMs shine, if there's a particular gap in your knowledge, LLMs can help you fill it, getting you unstuck.

>I'm puzzled (but not surprised) by the standard HN resistance & skepticism

Thinking back, I believe the change from enthusiasm to misanthropy (mis[ai]thropy?) happened around the time, and in increasing proportion to, it became a viable replacement for some of the labor performed by software devs.

Before that, the tone was more like "The fact is, if 80% of your job or 80% of its quality can be automated, it shouldn't be a job anymore."

I am just surprised they used an example requiring calculation/math. In the field the results are very much mixed. Otherwise it of course is a big help.

Knowing myself it perhaps wasn't that bad that I didn't have such tools, depends on the topic. I couldn't imagine ever writing a thesis without an LLM anymore.

There might not be any stupid questions, but there's plenty of perfectly confident stupid answers.

https://www.reddit.com/r/LibreWolf/s/Wqc8XGKT5h

Yeah. I’ll take this over the “you’re doing it wrong” condescension of comp.lang.lisp, or the Debian mailing list. Don’t even get me started on the systemd channels back in the day.

On the flip, I prefer the human touch of the Kotlin, Python, and Elixir channels.

>Learning something online 5 years ago often involved trawling incorrect, outdated or hostile content

Leanring what is like that? MIT open courseware has been available for like 10 years with anything you could want to learn in college

Textbooks are all easily pirated

> Should we trust the information at face value without verifying from other sources? Of course not, that's part of the learning process.

People who are learning a new topic are precisely the people least able to do this.

A friend of mine used chatgpt to try to learn calculus. It gave her an example...with constants changed in such a way that the problem was completely different (in the way that 1/x^2 is a totally different integration problem than 1/(x^2 + 1)). It then proceeded to work the problem incorrectly (ironically enough, in exactly the way that I'd expect a calculus student who doesn't really understand algebra to do it incorrectly), produced a wrong answer, and merrily went on to explain to her how to arrive at that wrong answer.

The last time I tried to use an LLM to analyze a question I didn't know the answer to (analyze a list of states to which I couldn't detect an obvious pattern), it gave me an incorrect answer that (a) did not apply to six of the listed states, (b) DID apply to six states that were NOT listed, even though I asked it for an exclusive property, (c) miscounted the elements of the list, and (d) provided no less than eight consecutive completely-false explanations on followup, only four of which it caught itself, before finally giving up.

I'm all for expanding your horizons and having new interfaces to information, but reliability is especially important when you're learning (because otherwise you build on broken foundations). If it fails at problems this simple, I certainly don't trust it to teach me anything in fields where I can't easily dissect bullshit. In principle, I don't think it's impossible for AI to get there; in practice, it doesn't seem to be.

Another quality is that everything is written. To me having a text support to discuss and the discussion recorded in text format is one of the strongest support someone can get when learning.

> Learning something online 5 years ago often involved trawling incorrect, outdated or hostile content

Also using OpenAI as a tutor means trawling incorrect content.

I'd share a little bit experience about learning from human teachers.

Here in my country, English is not you'll hear in everyday conversation. Native English speakers account to a tiny percentage of population. Our language doesn't resemble English at all. However, English is a required subject in our mandatory education system. I believe this situation is quite typical across many Asian countries.

As you might imagine, most English teachers in public schools are not native speakers. And they, just like other language learners, make mistakes that native speakers won't make without even realizing what's wrong. This creates a cycle enforcing non-standard English pragmatics in the classroom.

Teachers are not to blame. Becoming fluent and proficient enough in a second language to handle questions students spontaneously throw to you takes years, if not decades of immersion. It's an unrealistic expectation for an average public school teacher.

The result is rich parents either send their kids to private schools or have extra classes taught by native speakers after school. Poorer but smart kids realize the education system is broken and learn their second language from Youtube.

-

What's my point?

When it comes to math/science, in my experience, the current LLMs act similarly to the teachers in public school mentioned above. And they're worse in history/economics. If you're familiar with the subject already, it's easy to spot LLM's errors and gather the useful bits from their blather. But if you're just a student, it can easily become a case of blind-leading-the-blind.

It doesn't make LLMs completely useless in learning (just like I won't call public school teachers 'completely useless', that's rude!). But I believe in the current form they should only play a rather minor role in the student's learning journey.

HN’s fear is the same job security fear we’ve been seen since the beginning of all this. You’ll see this on programming subs on Reddit as well.

Spot on. You can even ask the LLM to ground itself with the content you provide it.

on hn i find most people here to be high iq low eq

high iq enough that they really find holes in the capabilities of LLMs in their industries

low eq enough that they only interpret it on their own experiences instead of seeing how other people's quality of life have improved

> A tireless, capable, well-versed assistant

Correction: a tireless, capable, well-versed, sycophantic assistant that is often prone to inventing absolute bullshit.

> ...is an autodidact's dream

Not so sure about that, see above.

It does go both ways. You can ask stupid questions without fear of embarrassment or ruined reputation, and it can respond with stupid answers without fear of embarrassment or ruined reputation.

It can confidently spew completely wrong information and there's no way to tell when it's doing that. There's a real risk that it will teach you a complete lie based on how it "thinks" something should work, and unlearning that lie will be much harder than just learning the truth initially

> Learning something online 5 years ago often involved trawling incorrect, outdated or hostile content

... your "AI" is also trained on the above incorrect, outdated or hostile content ...

Agrée, it would have been a godsend for those of us who were not as fast as the other and were eventually left over in usual schooling system.

Beside there isn’t any of the usual drawback with privacy because no one care if OpenAI learn about some bullshit you were told to learn.

This is a dream I agree. Detractors are always left behind.

> An underrated quality of LLMs as study partner is that you can ask "stupid" questions without fear of embarrassment.

Even more important for me, as someone who did ask questions but less and less over time, is this: with GPTs I no longer have to the see passive-aggressive banner saying

> This question exists for historical reasons, not because it’s a good question."

all the time on other peoples questions, and typically on the best questions with the most useful answers there were.

As much as I have mixed feelings about where AI is heading, I’ll say this: I’m genuinely relieved I don’t need to rely on Stack Overflow anymore.

It is also deeply ironic how stackoverflow alienated a lot of users in the name of inclusion (the Monica case) but all the time they themselves were the ones who really made people like me uncomfortable.

Fear causes defensive behavior.

[dead]