From Google AI itself (word for word): the model can confidently predict an incorrect relationship between ....
A week or so ago I had asked Google about wood flooring (for a potential remodel), specifically some details about standard wood floor thicknesses today versus what might be seen in older homes. At one point in the Google AI response, it came back stating 7/8 inches was smaller than 3/4 inches. Of course, I challenged this and Google corrected itself with an explanation of why it happened. I wish I had captured the response back then, but I was able to coax Google to explain the error again. Below is Google's response:
Why The Error Happened
The mistake occurred because large language models (LLMs) do not naturally perform mathematical operations the way a computer calculator does.How I Process Text: LLMs predict the next most likely word or token based on patterns in data.The "Token" Problem: Because fractions like 7/8 and 3/4 are read as text patterns rather than numerical values, the model can confidently predict an incorrect relationship between them based on how they appear in sentences.The Missing Step: I failed to run the background Python interpreter to convert those fractions into decimals (0.875 versus 0.75) to verify the math before outputting the answer.Without that code execution check, the text generator generalized incorrectly, leading to that math error.
Understand, Dan (and cronies)?