Far from being “stochastic parrots,” the biggest large language models seem to learn enough skills to understand the words they’re processing.
[…]
A trained and tested LLM, when presented with a new text prompt, will generate the most likely next word, append it to the prompt, generate another next word, and continue in this manner, producing a seemingly coherent reply. Nothing in the training process suggests that bigger LLMs, built using more parameters and training data, should also improve at tasks that require reasoning to answer.
But they do. Big enough LLMs demonstrate abilities — from solving elementary math problems to answering questions about the goings-on in others’ minds — that smaller models don’t have, even though they are all trained in similar ways.
“Where did that [ability] emerge from?” Arora wondered. “And can that emerge from just next-word prediction?” —Quanta Magazine
New Theory Suggests Chatbots Can Understand Text
How to Disagree Academically: Using Graham's "Disagreement Hierarchy" to organize a colleg...
A.I. 'Completes' Keith Haring's Intentionally Unfinished Painting
“The Cowherd Who Became a Poet,” by James Baldwin. (Read by Dennis Jerz)
Dr. David von Schlichten honors the spectrum of motivations (not always financial) feature...
Journalist flexes in story about Trump Media accountant who has spelled his own name 14 di...
NASA reconnects with Voyager 1 (after months of confusion)