← Back to context

Comment by vsnf

1 year ago

> ensure that numbers from one to ten as written as words and numbers greater ten as digits in the given text

I can’t fault llms for not knowing what to do here because I, a human, have no idea what on earth this means.

Given the text "1,2,3,4,5,6,7,8,9,10,11,12" it should result in "one, two, three, four, five, six, seven, eight, nine, ten, 11, 12"

or at least that's my understanding of the prompt

“Ten” is a word, “10” are digits.

I’m not a native English speaker, how would you write it?

FWIW the LLMs get it right many times, but fail other times.

  • I couldn't understand the original wording either, but after reading one of the sibling comments that explains it, it suddenly made sense.

    I think you left out a few words that most English writers would include. So instead of:

    > "ensure that numbers from one to ten as written as words and numbers greater ten as digits in the given text",

    something like the following might be better for most people:

    > "ensure that the numbers from one to ten are written as words, and the numbers greater ten are written using numerical digits in the given text"

    There are multiple ways to write this, so other people may have better versions.

    I'm not an English grammar expert, so I cannot explain to you why the addition of those extra words helps with the clarity of that sentence.

    • Much better, but still missing "than" after "greater", which seems kind of critical.

      "Using" is important as a number greater than ten can't be written as a digit, but can be written using digits ("with" would be just as good). Repeating "written" makes it clearer that there are two instructions.

      2 replies →

  • If your not a native English speaker, why are you even expecting the LLM to understand even 80% of the time?

    Just ask it in your own native language.

    • First of all, the texts the rule has to be applied to are written in English. Second, I believe English is by far (by far) the most prevalent language in the training dataset for those models, so I’d expect it to work better at this kind of task.

      And third, I’m not the only one working on this problem, there are others that are native speakers, and as my initial message stated, there have been many variations of the prompt. None work for all cases.

      And lastly, how would you rewrite my sample prompt? Which BTW bad a typo (unrelated to my English skills) that I’ve now fixed.

      6 replies →

It's a simple prescriptive rule in English. If you are writing about a small number, like less than ten, spell it out. For example: "According to a survey, nine out of ten people agree."

But if you are writing about a large number, particularly one with a lot of different digits, prefer writing the digits: "A mile is 5,280 feet." Compare that to: "A mile is five thousand, two hundred, and eighty feet."

I think he mean that numbers less or equal than ten are written as words, and others are written as numbers.

Given the many reaponses, it would be fun to aee if llm beat humans on understanding the sentence ahah

  • to me the main problem is that it should read "numbers greater than ten." I asked Gemini to rephrase it and Gemini produced correct English with the intended meaning:

    > Change all numbers between one and ten to words, and write numbers eleven and above as digits in the text.

    It even used eleven rather than ten which sounds like counting.

> > ensure that numbers from one to ten as written as words and numbers greater ten as digits in the given text

There are two blue, one red, and 15 green m&ms in this bag.