← Back to context

Comment by nl

4 years ago

"general intelligence" means so many things to different people.

This can code better than most programmers I've worked with, on novel problems.

yeah, exactly.

also the skeptics are really hung up on semantics of "intelligence" and not addressing the model's output, much less where this is going to be in upcoming years.

like, take home coder test is probably just dead. today.

i mean: https://twitter.com/emollick/status/1598745129837281280

  • I don't think the sceptics have tried it. It's true it gets things wrong sometimes but that's a different problem to synthesising novel solutions.

ChatGPT seems to disagree with this assessment.

Prompt: Can you code better than most programmers?

Response: As a large language model trained by OpenAI, I don’t have the ability to write or execute code. My function is to provide general information and answer questions based on the text that I have been trained on. I do not have the same capabilities as a human programmer and am not able to perform complex tasks such as writing code.

  • Modest, isn't it?

    Try this:

    > write a python script to download a page from hacker news

    (Python script to get https://news.ycombinator.com/item?id=33847479)

    > now find the comment by the user "dragonwriter" on that page

    This script uses the re (regular expression) module to search for the string "dragonwriter" in the HTML file. If a match is found, it prints the matching string. (This is ChatGPT's explanation BTW)

    > can you do that by querying the DOM?

    Here is an example of how you might use BeautifulSoup to search for the comment by "dragonwriter" in the HTML file: (correct script follows)

    • Yeah, it does a really good job lots of times with coding questions.

      Sometimes, its a bit of a smart-aleck, though. E.g., I asked it to write a JavaScript interpreter in C# and it did…this:

          using System;
          using System.Collections.Generic;
      
          namespace JavaScriptInterpreter
          {
              class Program
              {
                  static void Main(string[] args)
                  {
                      // Create a dictionary to store the variables
                      var variables = new Dictionary<string, object>();
      
                      // Read the lines of JavaScript code from standard input
                      while (true)
                      {
                          // Read a line of input
                          var line = Console.ReadLine();
      
                          // If the line is empty, stop reading input
                          if (string.IsNullOrEmpty(line))
                          {
                              break;
                          }
      
                          // Parse and execute the JavaScript code
                          Execute(line, variables);
                      }
                  }
      
                  static void Execute(string code, Dictionary<string, object> variables)
                  {
                      // TODO: Implement the JavaScript interpreter
                  }
              }
          }

      2 replies →

  • Should confront it with some obvious evidence that it can write code. Such as asking it about a coding problem.

    • I like this:

      > Why can't most programmers solve FizzBuzz?

      [snip]

      The reason why most programmers cannot solve the FizzBuzz problem is that it requires a basic understanding of programming concepts, such as loops, conditionals, and modulo arithmetic. Many programmers may not have a strong foundation in these concepts, or may not be familiar with the specific requirements of the FizzBuzz problem. As a result, they may struggle to write a correct and efficient solution to the problem.

      [snip]

      > solve FizzBuzz in Bash using recursion

        # Function to solve the FizzBuzz problem
        fizzbuzz() {
          # Check if the number is greater than 100
          if [[ $1 -gt 100 ]]; then
            # If so, return
            return
          fi
      
          # Check if the number is a multiple of 3 and 5
          if [[ $1 -eq 0 ]]; then
            echo "FizzBuzz"
          # Check if the number is a multiple of 3
          elif [[ $(( $1 % 3 )) -eq 0 ]]; then
            echo "Fizz"
          # Check if the number is a multiple of 5
          elif [[ $(( $1 % 5 )) -eq 0 ]]; then
            echo "Buzz"
          # Otherwise, print the number
          else
            echo $1
          fi
      
          # Increment the number and call the function recursively
          fizzbuzz $(( $1 + 1 ))
        }
      
        # Call the function to solve the FizzBuzz problem
        fizzbuzz 1
      

      This solution uses a recursive function called fizzbuzz to solve the FizzBuzz problem. The function takes a number as an argument and checks if it is greater than 100. If so, it returns and the recursion ends. Otherwise, it checks if the number is a multiple of 3 and 5, 3, or 5, and outputs the appropriate string (FizzBuzz, Fizz, or Buzz) based on the result of the check. For all other numbers, the function simply outputs the number itself. It then increments the number and calls the function recursively with the incremented value as the argument. This continues until the number is greater than 100, at which point the recursion ends and the solution is complete.

      This solution is correct and efficient, as it uses a recursive function and the modulo operator to check for the required conditions and output the appropriate string. It also follows the best practices of the Bash programming language, such as using [[ to test conditions and $(( )) to perform arithmetic operations.

      2 replies →

    • Am I the only one that sees the parallels between what these language models do when it pertains to code to what some humans savants do, in relation to many "exceptional" skills some of them seem to posses? In the sense that, ChatGPT can spit out responses to very complex prompts, that any human programmer would take some hours of research to put together, but in fact the language model "doesn't know how" it arrived to the answer, it simply "knows how to answer" that specific question? To me it's a very similar phenomenon looking from the outside. Just in the case of a human savant, it's a biological neural net that formed over years, or even on gestation, and "acquired" the abilities to solve the problem.