DeepAI AI Chat
Log In Sign Up

Large Linguistic Models: Analyzing theoretical linguistic abilities of LLMs

05/01/2023
by   Gašper Beguš, et al.
45

The performance of large language models (LLMs) has recently improved to the point where the models can generate valid and coherent meta-linguistic analyses of data. This paper illustrates a vast potential for analyses of the meta-linguistic abilities of large language models. LLMs are primarily trained on language data in the form of text; analyzing their meta-linguistic abilities is informative both for our understanding of the general capabilities of LLMs as well as for models of linguistics. In this paper, we propose several types of experiments and prompt designs that allow us to analyze the ability of GPT-4 to generate meta-linguistic analyses. We focus on three linguistics subfields with formalisms that allow for a detailed analysis of GPT-4's theoretical capabilities: theoretical syntax, phonology, and semantics. We identify types of experiments, provide general guidelines, discuss limitations, and offer future directions for this research program.

READ FULL TEXT

page 6

page 7

page 9

page 10

page 12

page 15

page 16

page 17

02/28/2023

Beyond the limitations of any imaginable mechanism: large language models and psycholinguistics

Large language models are not detailed models of human linguistic proces...
04/28/2023

Are Emergent Abilities of Large Language Models a Mirage?

Recent work claims that large language models display emergent abilities...
08/18/2020

Password Guessers Under a Microscope: An In-Depth Analysis to Inform Deployments

Password guessers are instrumental for assessing the strength of passwor...
02/23/2023

Testing AI performance on less frequent aspects of language reveals insensitivity to underlying meaning

Advances in computational methods and big data availability have recentl...
05/16/2022

What GPT Knows About Who is Who

Coreference resolution – which is a crucial task for understanding disco...
02/19/2022

Is there an aesthetic component of language?

Speakers of all human languages make use of grammatical devices to expre...
01/16/2023

Dissociating language and thought in large language models: a cognitive perspective

Today's large language models (LLMs) routinely generate coherent, gramma...