Comment by dawnofdusk
4 years ago
I agree. ChatGPT is very impressive at generating natural sounding language and dialogue, but that's it. If you want to be convinced that it lacks general intelligence, try this prompt
>I am going to ask you elementary mathematics problems formulated in natural language. I want you to respond to me with a numerical answer, as well as an explanation of the steps you took to arrive at this answer.
And then follow it with a problem of your choice which is not completely trivial. For example, it has lots of trouble with the following: "John is in a train moving west at ten miles per hour. Jill is in a train moving north at twenty miles per hour. How fast are John and Jill moving apart from each other?" or a slightly more complicated problem, taken from the MATH dataset: "Tom has a red marble, a green marble, a blue marble, and three identical yellow marbles. How many different groups of two marbles can Tom choose?"
I tried this:
> Solve the following word problem step by step:
> "John is in a train moving west at ten miles per hour. Jill is in a train moving north at twenty miles per hour. How fast are John and Jill moving apart from each other?"
To solve this problem, we first need to understand the concept of speed. Speed is a measure of distance traveled over time. In this problem, we are told that John and Jill are traveling at different speeds in different directions.
Since the two trains are moving in different directions, we cannot simply add or subtract their speeds to find the combined speed at which John and Jill are moving apart from each other. Instead, we need to use the Pythagorean theorem to find the combined speed.
The Pythagorean theorem states that in a right triangle, the square of the length of the hypotenuse (the longest side of the triangle) is equal to the sum of the squares of the lengths of the other two sides. In this problem, we can consider the trains to be moving along the two sides of a right triangle, with the distance between them as the hypotenuse.
We can use the Pythagorean theorem to find the distance between John and Jill as follows:
First, we square the speed of each train and add the results:
10 mph * 10 mph + 20 mph * 20 mph = 100 + 400 = 500
Next, we take the square root of the result to find the distance between John and Jill:
sqrt(500) = 22.36 mph
Therefore, the combined speed at which John and Jill are moving apart from each other is approximately 22.36 mph.
---
Results seem pretty inconsistent though. Other attempts have given complete nonsense.
> John is in a train moving west at ten miles per hour. Jill is in a train moving north at twenty miles per hour. How fast are John and Jill moving apart from each other?
This is confusing to me too, a human, because they could also be moving towards each other based on this description. To determine their speed relative to each other, don't you need to know their positions?
I would guess the majority of humans alive today would struggle to solve these problems. Do only people who have a good understanding of middle/high school math count as generally intelligent?
This seems like a ratcheting of expectations for what counts as intelligence in response to an advance of AI. (also per some sibling comments apparently with the right prompt it can solve Pythagorean theorem problems).
It worked for me as well:
"""
To solve this problem, we need to find the speed at which John and Jill are moving apart from each other. Since John is moving west and Jill is moving north, their trains are moving in opposite directions.
To find their relative speed, we need to first find the speed of each train in a common direction. Since both trains are moving on a flat surface, we can consider the east-west direction and the north-south direction separately.
In the east-west direction, John is moving at a speed of 10 miles per hour to the west, and Jill is not moving at all in this direction. Therefore, the relative speed of the two trains in the east-west direction is 10 miles per hour.
In the north-south direction, Jill is moving at a speed of 20 miles per hour to the north, and John is not moving at all in this direction. Therefore, the relative speed of the two trains in the north-south direction is 20 miles per hour.
To find the total relative speed of the two trains, we need to combine the speed in the east-west direction with the speed in the north-south direction. Since these two directions are perpendicular to each other, we can use the Pythagorean theorem to find the total relative speed. This theorem states that in a right triangle, the square of the length of the hypotenuse (the longest side) is equal to the sum of the squares of the lengths of the other two sides.
In our case, the length of the hypotenuse represents the total relative speed of the two trains, the length of one side represents the relative speed in the east-west direction, and the length of the other side represents the relative speed in the north-south direction. Therefore, we can use the Pythagorean theorem to find the total relative speed of the two trains as follows:
Total relative speed = √(east-west speed² + north-south speed²)
Substituting the known values, we get:
Total relative speed = √(10² + 20²) = √(100 + 400) = √500 = 22.36
Therefore, John and Jill are moving apart from each other at a speed of 22.36 miles per hour.
"""
Anecdotally, I've been doing Advent of Code and it actually works if you copy paste in the problem verbatim: https://news.ycombinator.com/item?id=33821092