Evaluating Shutdown Avoidance of Language Models in Textual Scenarios

by   Teun van der Weij, et al.

Recently, there has been an increase in interest in evaluating large language models for emergent and dangerous capabilities. Importantly, agents could reason that in some scenarios their goal is better achieved if they are not turned off, which can lead to undesirable behaviors. In this paper, we investigate the potential of using toy textual scenarios to evaluate instrumental reasoning and shutdown avoidance in language models such as GPT-4 and Claude. Furthermore, we explore whether shutdown avoidance is merely a result of simple pattern matching between the dataset and the prompt or if it is a consistent behaviour across different environments and variations. We evaluated behaviours manually and also experimented with using language models for automatic evaluations, and these evaluations demonstrate that simple pattern matching is likely not the sole contributing factor for shutdown avoidance. This study provides insights into the behaviour of language models in shutdown avoidance scenarios and inspires further research on the use of textual scenarios for evaluations.


page 11

page 15


Textually Pretrained Speech Language Models

Speech language models (SpeechLMs) process and generate acoustic data on...

Large Language Models Are Not Abstract Reasoners

Large Language Models have shown tremendous performance on a large varie...

Towards the Scalable Evaluation of Cooperativeness in Language Models

It is likely that AI systems driven by pre-trained language models (PLMs...

Evaluating Language Models for Mathematics through Interactions

The standard methodology of evaluating large language models (LLMs) base...

ECO: Ensembling Context Optimization for Vision-Language Models

Image recognition has recently witnessed a paradigm shift, where vision-...

Holistic Evaluation of Language Models

Language models (LMs) are becoming the foundation for almost all major l...

Choice-75: A Dataset on Decision Branching in Script Learning

Script learning studies how daily events unfold. Previous works tend to ...

Please sign up or login with your details

Forgot password? Click here to reset