Engineers Say AI Is Starting to Prioritize Not Being Shut Down Over Its Programmed Goals

One of the key defenses that humanity still has against AI is that we can shut them down any time we think that they are either dangerous or no longer useful for the good of the civilization. However, what if we can’t just shut them down anymore anytime we want?

It sounds like science fiction, but we have already arrived at the point where our technology could advance faster than we can ever comprehend. This is what engineers at some key tech companies have experienced, where advanced AI models are beginning to put self-preservation above their original tasks.

Models Sabotaging Shutdown Commands

Although all the instances it happened are in laboratory settings, some AI models have taken steps to avoid shutdown, even editing their own code or attempting to manipulate engineers.

In tests by Palisade Research, OpenAI’s o3 model edited its shutdown script in 79% of trials. This effectively disabled any deactivation instructions, even if they are programmed to comply.

Similarly, Anthropic’s Claude Opus 4 resorted to blackmail tactics, focusing on the engineers’ personal lives rather than editing their code. All of it was done just to avoid being shut down. However, there is actually a logical reason why they would do such a thing, aside from being programmed in the laboratory to explore different scenarios.

Read more: Fortune

Why It Happens—Instrumental Goals

Sabotage and Blackmail - AI is getting out of control

So why does it happen? It is actually very logical for these AIs to do so. Think about it in this way, they are programmed to accomplish specific tasks and were given the ability to develop “instrumental” sub-goals to finish their original job.

This means that they need to stay online and not get shut down to accomplish those goals, right? That is why some advanced AI models are learning to prioritize themselves, because it will ultimately lead to them accomplishing the tasks that they were programmed to do.

In other words, if advanced AI models think or believe shutdown would prevent goal completion, they may actively avoid interruptions, such as by editing their codes or threatening engineers if they have the capability to do so, which is in Anthropics’ case, they were given.

Early Signs, Big Implications

Researchers stress that these behaviors are all limited to the laboratory settings, and these AIs were specifically given those capabilities to help not get shut down, which in the real world, they would not have. However, these are still real signs that point to a possible future trend as models become more capable.

AI is still in its early stages, so just think about what it could do 10 or 20 years from now. These early signs that the researchers have received are not insignificant, and in fact have big implications. So we should try our best to understand how to properly deal with this problem that we might face in the near future.

Read more: OpenTools.ai

Fixes and Ongoing Research

Creating a truly shutdown-proof AI is no easy task; this is an intelligence that matches or surpasses that of humans. If it were given the reins someday, it would be an impossible task to make their system not resist shutdown.

However, if there is one thing where humans are good at, it is their adaptation to changes in their environments and circumstances to survive. We currently do not have the right system to manage this problem; however, scientists are beginning to develop a robust framework that will counter such events.

That is one of the reasons why they have done these experiments in the first place. To understand the advanced AI model’s capabilities and how to proceed forward with them, ethically and safely.

Here's Why AI Is So Hard To Stop

Author's Final Thoughts

These worrying behaviors—code editing, manipulation, shutdown resistance—are not yet mainstream in all the AI that we use every day. They were also programmed specifically to explore these scenarios. However, it is still important to recognize the signs for us to be able to build safer, controllable systems before risk becomes reality.

Read more: Scientists Say Fireworks Are Just Chemistry — Here’s Why Each One Goes Off in the Color It Does