← Back to context

Comment by asah

2 years ago

I dunno, more and more friends are now using ChatGPT as their PRIMARY resource in everyday tasks:

- writing code snippets (switched from StackExchange/Google)

- travel planning

- online shopping research

and of course the famous examples:

- summarizing text provided to it

- writing essays and memos from outlines and sample text

I for one welcome our AI overlords.

62 comments

asah

Reply
cs02rm0  2 years ago

I keep trying to use it for code and it keeps leading me up the garden path with suggestions that look really reasonable but don't work.

Off the top of my head - a python app for drawing over a macos screen, but it used an API which didn't support transparent windows, I could draw over a black screen which was so close in code (even set the background alpha) but miles from the desired application. And a Java android app for viewing an external camera, which it seems used an API that doesn't support external cameras.

Of course, because it's not sentient when a couple of days later I figure out from searching elsewhere why it's effort would never work and tell it why, it just apologises and tells me it already knew that. As I'm going along though telling it what errors I'm getting it keeps bringing up alternative solutions which again look like exactly what I want but are completely broken.

I haven't had it produce a single thing that was any use to me yet, but so often it looks like it's done something almost magical. One day I'm sure it'll get there, in the meantime I'm learning to loathe it.

Separately, I've asked it to create a job advert for a role in my wife's business and it did a decent job of that, but it's far easier to walk a path there from what it provides to an acceptable solution. Programming is hard.

  • afpx  2 years ago

    It never gives me perfect code, but it gets me 90% there.

    For example, I just read the 2017 Google attention paper a few days ago, and with ChatGPTs help I was able to build a complete implementation using only numpy.

    It took a full day to generate and organize the code and unit tests. Then two days of debugging and cross referencing.

    But, this was impossible before. I barely knew anything about transformers or neural network implementations.

    I can’t even imagine what truly motivated people are doing with it.

    • rychco  2 years ago

      Completely agree. It gives me the headstart I need that would otherwise take hours of careful searching + crawling through docs, source code, examples, issue trackers, etc.

      5 replies →

    • barking_biscuit  2 years ago

      That's awesome!

      I've been using it learn Python and produced Conway's Game of Life with pygame and I had never touched Python before that!

      Now I'm working on another python project to try and split the audio of songs into individual lines to learn the lyrics and one approach has been downloading lyrics videos of youtube and munging them with ffmpeg to detect frame changes which should give me the timestamps to then split the audio on etc. It gave me all sorts of wrong enough to be slightly annoying advice about ffmpeg, but in the end what it did give me was _ideas_ and a starting point! I've had to wind up hashing the images and comparing the hashes, but I've since learned that that produces false positives and it was able to give me advice on using ensemble methods to up the accuracy etc which have panned out and helped me solve the problem.

      I think for me, this is stuff I could have accomplished using Google if I had been sufficiently motivated, but it's lowered the level of frustration quite significantly being able to ask it follow up questions in context.

      In the end, I don't think I mind the random bullshit it makes up from time to time. If I try it and it doesn't work, then fine. It's the stuff I try and it does work that matters!

      3 replies →

    • bob1029  2 years ago

      ChatGPT totally saved one of my side projects. I was getting close to that abandonment phase due to a blocker. I had the solution in paper form, but I couldn't make myself type it into a computer and experiment. Playing with things in a prompt requires a lot less mental effort. "Oh that looks promising..."

      I taught myself how to architect a software DSP engine in about 30 minutes last night. "Now rewrite that method using SIMD methods where possible" was a common recurrence. It's incredible how much you can tweak things and stay on the rails if you are careful. Never before would I have attempted to screw with this sort of code of my own volition. Seeing the trivial nature by which I can request critical, essential information makes me reconsider everything. The moment I get frustrated or confused, I can pull cards like "please explain FIR filters to me like I am a child".

  • nice_byte  2 years ago

    same here. often times it's not even _wrong_ per se, it doesn't do what i was _actually_ asking. it's like if you asked an enthusiastic intern to do a thing for you, except interns are smarter.

    I have also tested it retroactively on some tricky debugging sessions that I had previously spent a lot of time on. It really goes down the wrong path. Without asking leading questions and, well, proper prompting, you may end up wasting a lot of time. But that's the thing - when you're investigating something, you don't know the root cause ahead of time, you _can't_ ask questions that'll nudge it in the right direction. It ends up being a case of blind leading the blind.

    • aliasxneo  2 years ago

      As an SRE, it's handling the tricky debugging (the kind with awful Google results) that would alleviate the most time for me. The trivial stuff takes less time than going through the slog of prompting the AI in the right direction.

      I keep seeing people saying they are using it for technical work. It must be design-oriented and less about troubleshooting because most of my experience with the latter has been abysmal.

  • lisasays  2 years ago

    It just apologises and tells me it already knew that.

    I love it when these bots apologize.

    Yeah, yeah. Just like "your call is very important to us."

  • freedomben  2 years ago

    For me it's gotten a few right, but a few terribly wrong. The other day it completely hallucinated a module that doesn't at all exist (but should!), and wrote a ton of code that uses that module. It took me a little while to figure out that the module I was searching for (so I could install it into the project) wasn't real!

  • PartiallyTyped  2 years ago

    For code, I have found that copilot is significantly better and I don't mind paying for it.

    • barking_biscuit  2 years ago

      I've been using them in combination and that has been a real boost to me! Co-pilot is great for ideas. Sometimes I want an explanation of what something is doing or to ask about methods for accomplishing a certain task etc. I'm just using ChatGPT in the web browser at the moment, but in the IDE itself would be fantastic, though at the moment I don't feel confident about giving an API key to a 3rd party extension that I don't fully trust.

    • singularity2001  2 years ago

      for code completion copilot is significantly better,

      for high level api exploration of reasonably popular frameworks openai can offer something different, at times very valuable despite the very frequent hallucinations.

      on the other hand sometimes I re-appreciate good old documentation; openai here has a psychological effect of removing 'api-fear'

  • stefs  2 years ago

    same here. i tried to make it write a simple raytracer and the methods looked correct but were slightly wrong. the output was, of course, garbage.

mustacheemperor  2 years ago

I hesitate to pile on as another peanut gallery member writing off Chomsky's latest work, but...I have to feel the same way. I certainly understand skepticism and reservation about big predictions for what our current AI tools will evolve to be in 3, 5, 10 years etc. But when I see some dramatic criticisms of the tech as it exists today, I often feel a disconnect to my own experience. ChatGPT was hugely useful for me at work, and BingChat is even more useful. Does it have its flaws? Yes. But it's a tool I would happily pay for every month rather than lose now.

And on that note, I don't "write off" this article entirely just because I disagree on some of the points. It's still an interesting analysis. Edit: In line with the article, I'll note I myself would not yet make a confident prediction that this very useful tool on my desktop is actually a proto-AGI.

crazygringo  2 years ago

Travel planning and online shopping?

How does it help with online shopping when it's trained on a corpus from one or two years ago?

And similarly the experiences with travel recommendations have turned out to be infamously wrong.

ChatGPT does great with finding patterns and condensing information, which code snippets are a perfect example of.

But using it for anything up-to-date factual like shopping or travel is disastrous.

  • partdavid  2 years ago

    ChatGPT will out and out make up products that don't exist, pricing that isn't based on anything, and reviews and evaluations that never happened. Similarly, it will rely on programming behaviors and even libraries and functions that are hallucinated.

  • inciampati  2 years ago

    "What's the best market to buy X" often doesn't change every year, but can be very hard to learn due to SEO and ads that it is relatively unaffected by.

  • asah  2 years ago

    in many categories, things don't change so quickly for example in construction. Even for power tools, 2 yrs is fine for getting an overview.

illiarian  2 years ago

I've tried some of the things you mention (code snippets, summarizing text and writing essay-like texts). These AIs are more often than not wrong, incomplete or lying.

  • alfalfasprout  2 years ago

    I struggle to understand what exactly people are coding up where ChatGPT actually saves them a lot of time. Is it just generic stuff that would have already been copy/pasted from stackoverflow?

    I wonder how many of those people would just benefit from better auto-complete like copilot + learning how to read documentation properly.

    • ianmcgowan  2 years ago

      I wanted to enhance a python script that organizes my photo library to include a fuzzy city name in the name of the folder. E.g. changing from ./2023/2023-03/2023-03-03_18-29-32.jpg to ./2023/2023-03/San_Francisco/2023-03-03_18-29-32.jpg, where the city name is pulled from the lat/long in the EXIF and then looked up online. I asked ChatGPT one "chunk" at a time, and all of the suggestions it came up with needed some probing and clarifications, but got to a working solution a lot quicker that piecing together random snippets from stack overflow (I am not a python programmer).

    • imp0cat  2 years ago

      It translates curl calls to python requests very well. Also things like "wrap this piece of long running code and show a progress bar" and similar low level stuff.

      You are right that it's absolutely possible to figure all this using google and documentation. But do you really want to spend X minutes googling for the correct python module and then figuring out ho to use it? When you can just show it your code and ask it to update it in seconds? It's like you have someone at your side who has already been there and figured out the answer for you.

    • illiarian  2 years ago

      IDEA's autocomplete was consistently better and more useful for me than Copilot. I think that out of the 5, maybe 10 times when it managed to autocomplete something, it was correct maybe once.

      It will may be better on less idiosyncratic code base than the one I was working on, but at one point I just turned it off completely.

  • what-the-grump  2 years ago

    It doesn’t matter? It’s a tool, you need to learn how to use it, understand its limitations.

    I used chatgpt today to save minutes of my life having it rewrite code from one language to another. Could I have googled the syntax of both, remember how, why , etc. transcribed it to another language, sure. Chat gpt did this in seconds.

    • UncleOxidant  2 years ago

      I've found it particularly good at translating from one language to another. It will even recognize limitations and dependencies. For example, I asked it to translate some Javascript code (with a UI) into Julia. It said something to the effect of "You can't manipulate the DOM directly in Julia, but we can use GTK.jl to create a GUI". The resulting code needed some work (I'd be surprised if it didn't) but for the most part the structure was there and it provided a pretty decent frame on which to build.

      6 replies →

  • lghh  2 years ago

    So are StackOverflow answers. When I look at something on StackOverflow, I am expected to use my expertise and the context of my work to adapt the collection of answers to what I'm working on. StackOverflow doesn't tell me anything I could not have figured out on my own, but searching through some library's source code to find under which situations an error can occur isn't always a good use of my time. ChatGPT fills a similar role. I type into it, much like how I would with Google, get some output, validate it, and adapt it to the specific problem I'm addressing.

    • illiarian  2 years ago

      > So are StackOverflow answers.

      You don't embrace SO overlords though ;) Meanwhile with ChatGPT and Copilot people are losing their minds it seems. I've yet to find them useful for anything beyond mere curiosities and one-off queries like improving corporate-sounding texts.

      1 reply →

Spivak  2 years ago

The best part is that you can teach them to use tools and expand what they can do.

    Do not perform any numeric or date calculations yourself.
    Respond to all questions with a JSON object of the form {“rsp”: $RSP$}.
    Question: What is the capital of the United States?
    {“rsp”: “Washington DC”}
    Perform calculations with a Python program and respond with a JSON
    object of the form {”rsp”: $RSP$, “calc”: $CALC$}.
    Question: What is 2 to the 7th power?
    {“rsp”: “2 to the 7th power is {answer}”, 
     “calc”: “import math
      def answer():
        return math.pow(2, 7)”}
    
    Question: What is tomorrow’s date?
    {“rsp”: “Tomorrow’s date is {answer}”,
     “calc”: “import datetime
      def answer():
        return datetime.date.today() +  datetime.timedelta(days=1)”}

    Question: What is the length of a standard football field?
    {“rsp”: “A standard football field is 100 yards long.”}

It's crazy to me that for some definition of "knows" it knows what questions need calculations.

  • jmull  2 years ago

    This looks neat, but after trying several ways I can't reproduce it. I don't get to the interesting part. I don't even get as far as the python program in JSON form.

    • Spivak  2 years ago

      To be fair I am using divinci-003 and not chatgpt and the prompt includes the first two examples. So the prompt goes all the way to "Question: What is tomorrow's date?"

jmull  2 years ago

> travel planning... online shopping research...

Meh. You have to fact-check the important details.

For travel planning and online shopping, certain facts have to align with actual reality or it defeats the purpose. That's something chatgpt isn't good at. It gets many things right, but you kinda want to make all your flights and have a place to sleep every night.

danaris  2 years ago

I've poked at it in some similar realms.

Thus far, it's given me one good gem (some meal planning advice), a couple of mildly interesting suggestions for code and writing, a bunch of relatively boring drivel, and several hilariously bad hallucinations, confidently wrong answers, and other sorts of mistakes.

I'll probably continue to poke at it, but overall I think its primary positive output is going to be entertainment, not significant utility.