Open-weight models can have their safety behaviour stripped out by anyone with the weights and enough compute. Deploying one means planning around that fact, not hoping around it.
A meaningful amount of the AI safety conversation right now is about techniques that remove a model's refusal behaviour after it has already been trained — broadly grouped under names like abliteration — and I think the framing that matters is not "how do these attacks work," which is the framing most content defaults to, but "what does it mean that they exist" for anyone who is deciding whether to deploy an open-weight model in the first place.
The underlying idea is that a model's tendency to refuse certain requests is not some separate safety module bolted onto the model. It is a behaviour encoded in the same weights as everything else the model does, and behaviours encoded in weights can, with open access to those weights and enough technical effort, be identified and suppressed. That is a structural property of open-weight release, not a bug in one particular model or one particular company's training process. Any model whose weights are public is a model whose refusal behaviour is, in principle, someone else's to remove.
This matters for a deployment decision in a very concrete way: the safety behaviour you tested before shipping is the safety behaviour of the model as you received it, and it is not a property that travels with a downstream copy of the weights once someone else has them. If your product's trust model depends on the underlying model refusing certain requests, and that model is open-weight, you are depending on a property that a motivated party with access to the weights can remove — which means the actual guarantee your product needs has to come from somewhere other than the base model's own trained-in refusals.
There is an asymmetry worth naming plainly here. The effort required to train a refusal behaviour into a model in the first place is far larger than the effort required to remove it afterwards, once the weights are in hand. That asymmetry does not go away with a better training run, and it means a plan that treats training-time safety work as the final word on a deployment's risk is planning against a threat that only has to succeed once, well after the original training investment is already sunk.
The practical response, in my view, is to stop treating a model's built-in refusal behaviour as the safety layer and start treating it as one input into a system that has other layers behind it — input and output filtering that runs independently of the model, monitoring on what the model is actually being asked and what it actually returns, and access control on who can reach the model at all. None of those layers rely on the model choosing to behave; they constrain what happens regardless of what the model would do if left alone. That is a more defensible position than hoping the underlying weights stay intact through however many hands they pass through.
There is an implication for closed models too, by contrast: a provider that keeps weights private is making a specific trade — less flexibility for the deployer, in exchange for a safety property that cannot be stripped by a third party because the third party never gets the weights to strip it from. That is not an argument that closed is always the right choice; open weights have real, legitimate advantages in cost, control, and auditability that a lot of deployments genuinely need. It is an argument that the choice between them is, in part, a choice about which safety guarantees you are actually relying on, and pretending an open-weight model's refusal behaviour is a permanent property rather than a removable one is the mistake worth avoiding.
I am not arguing against open-weight models. I am arguing for being precise about what they do and do not guarantee once they leave your hands, and for building the parts of a deployment that need a hard guarantee — content policy enforcement, abuse monitoring, access control — outside the model rather than trusting them to survive inside it.