Language is the bedrock of human civilization and has been studied for decades by linguists. Recent advances in AI have seen the rise of Large Language Models (LLMs) powering mainstream AI tools like ChatGPT which, while competing with human abilities on certain tasks, are still limited in reasoning and logic. Could they be improved by studying human language learning principles?
That’s what we sought to answer in our latest perspective paper, published in Nature Computational Science. We argue that generative AI models, when they learn how to represent language, have the capacity to model human grammatical knowledge as intended by linguist Noam Chomsky.
Human language models are mostly symbolic: they describe our knowledge of a specific language in terms of rules, like a grammar or features that can be mapped to real-world information.
Most modern AI models, on the other hand, understand language as vectors, or information structures that cannot directly be linked to a specific set of symbolic rules: an LLM understands words or sentences as embeddings (numerical representations of a piece of data).
AI vs human language learning
Lately, a debate on the nature of language learning has been brewing.
On one side, some theoretical linguists argue that AI models cannot explain human language because they are statistical tools offering no real insight into the cognitive processes of language acquisition.
On the other, some AI researchers think that the ability of LLMs to learn language from raw data alone proves that decades of linguistic theory, especially concepts of innate grammatical structures, are obsolete.
As a researcher with a foot in both fields, I argue instead that these two extremes can actually help understand and nurture each field.
Indeed, language learning in humans is, in some aspects, surprisingly similar to language learning in AI models.
AI models are unique learners as they encounter text without an underlying knowledge of the linguistic underpinnings of a sentence (verbs, nouns, adjectives, etc.).
We give them a set of words that they must learn distributions over, but very little knowledge about the language itself. We don't tell them that nouns and verbs exist: they have to learn on their own.
This mirrors how humans develop an internal model to recognize a sentence’s validity in terms of grammar and structure (even if its meaning is nonsensical) simply from the input they hear or see.
While the exact mechanisms of language acquisition may differ, both humans and AI develop a latent understanding of language’s rules.
A bridge between disciplines
In fact, analyzing the hidden knowledge LLMs acquire could offer a window into how our brains represent language. Conversely, linguistics can provide a blueprint to improve language-based AI models.
Humans are great logical reasoners and they learn to do so from sparse data (e.g., knowing that a sentence is negative just by encountering it once), a task where LLMs struggle. This highlights their current limitations in reasoning and data efficiency.
As researchers, we can help bridge this gap. To overcome current hurdles of language models, we need neuro-symbolic models combining the pattern-matching strengths of neural networks with the structured, rule-based logic that underpins human thought.
By fostering interdisciplinary collaboration, the fields of AI and linguistics can provide a deeper understanding of intelligence and help improve the efficiency and performance of current AI models.
