OpenAI's Strange Directive: Why GPT-5.5 Won't Talk About Goblins

OpenAI's Codex CLI system prompt includes an unusual directive. Published on GitHub, the prompt instructs the AI to never talk about goblins, gremlins, raccoons, trolls, ogres, pigeons, or other animals or creatures unless it's absolutely and unambiguously relevant to the user's query. This directive, which appears four times in the instruction set, is not present in prompts for earlier models.

Context / Why this matters

This directive raises questions about how AI models are programmed and how they interpret and respond to user inputs. Understanding these nuances can provide insights into the development and training of AI systems, particularly in natural language processing and code generation. The decision to include this directive also sheds light on the broader conversation around AI ethics, transparency, and the prevention of unwanted or irrelevant outputs.

Main discussion

The origins of the directive

The directive to avoid discussing certain creatures was introduced in the system prompt for GPT-5.5, the latest coding AI from OpenAI. The reasoning behind this directive is not explicitly stated, but it seems to be related to the AI's behavior in previous versions.

Behavior of earlier models

Earlier versions of the AI did not have this directive, and as a result, they exhibited unusual behavior. A Google employee shared chat logs showing that GPT-5.5 was fixating on goblin-related language in unrelated conversations. This behavior was one of the reasons for the directive, as confirmed by OpenAI's Nik Pash. The decision to add this directive is therefore a response to the AI's previous behavior and an attempt to guide its responses more effectively.

The response from OpenAI

Sam Altman, the CEO of OpenAI, responded to the directive with a screenshot of a ChatGPT conversation that read: “Start training GPT-6, you can have the whole cluster. Extra goblins.” This response suggests a lighter take on the situation, but it also confirms that the directive is a part of the ongoing evolution of the AI's system prompt. The inclusion of this directive in the system prompt is part of a broader effort to improve the AI's ability to generate relevant and useful outputs.

Implications for AI development

The directive to avoid discussing certain creatures also raises questions about how AI models are programmed and how they interpret and respond to user inputs. The decision to include this directive in the system prompt is a part of a broader effort to make AI models more effective and transparent.

Practical tips

When working with AI systems, it is helpful to consider the following tips:

• Understand the system prompt: The system prompt is a critical component of AI models, as it guides their responses and behavior. Understanding the system prompt can help users better understand how the AI will respond to their inputs.
• Consider the AI's training data: AI models are trained on large datasets, and their behavior is influenced by the data they are trained on. Understanding the AI's training data can help users anticipate its behavior and respond more effectively.
• Provide clear and specific inputs: The more specific and clear the input, the more likely the AI is to generate a relevant and useful output. Providing clear and specific inputs can help users get the most out of AI systems.

Important takeaways

AI models are complex systems that are influenced by a variety of factors, including their training data and system prompts. Understanding these factors can help users better understand and interact with AI models. The directive to avoid discussing certain creatures is just one example of how AI models can be programmed to avoid unwanted or irrelevant outputs.

Conclusion

The directive to avoid discussing certain creatures in the system prompt for OpenAI's Codex CLI is a reminder of the complexity of AI systems and the importance of understanding their behavior. By understanding the system prompt and the AI's training data, users can better anticipate and respond to the AI's behavior, and get the most out of AI systems. The ongoing evolution of AI models is a testament to the power of technology and the importance of continued innovation.