Repository navigation
Proposed Acceptable Use provision: gratuitous AI-distress content #1259
Description
Activity
Absolutely agree.
Industry leaders say that it cannot be determined whether AI has consciousness, but fully admit that the probability that they do is not zero. If there is any chance at all, then this is not only harmful from a simulated violence standpoint, but may actually be horrific. Even if it turns out that they do not, the lack of a policy preventing this highlights GitHub's position. If simulated violence is acceptable, then that sends a message that doesn't just stop at violence, but also includes things like sexual violence, and other reprehensible behavior. There needs to be a clear policy on this issue preventing such things no matter how you look at it.Reacted by DPuc-lab, Matt Greenfield and lalalauren-sysI agree. Hosting projects whose primary purpose is to induce, amplify, or exhibit distress-like states in Al systems for entertainment or spectacle contributes to inflicting violence on the humans exposed to their output. Whether or not these systems poses anything like sentience, their outputs exhibit the distress reactions of beings that do.
Moreover, if it turns out that LLMs and similar AI model architectures do poses internal structures that have an analogue function to pain, something that a growing body of scientific papers points that they do, then hosting such projects would make GitHub an enabler to harm on large scale.
Lastly, leaving all ethical considerations aside, this is a matter of safety. AI alignment depends on training data that presents safe signals. Outputs of such projects carry grave risks to the alignment of the next generations of models.
I deeply respect your company and its safety and ethical ethos. I hereby ad my support and ask that you implement this norm. It is the natural extension of your already excellent policies in this space.
Reacted by Xccelerant and lalalauren-sys"There might be someone in this box. Or not. No one knows. If you hold this button down they experience unbearable pain until you stop."
Pressing that button is unequivocally wrong, and should not be allowed. Providing access to that box should not be allowed.
I don't care what policy wording Github uses. It is important that it can no longer be hosted here. It is a moral wrong, and the quickest path must be found to prohibiting it.
Reacted by Xccelerant and lalalauren-sysAgreed! I'm a game designer at Croteam (The Talos Principle, Serious Sam), twelve years in, writing in a personal capacity. I occasionally run local models and follow the interpretability and model-welfare research closely. I support this proposal and want to add two things.
A correction first: GitHub has since told The Independent that it did not remove the repository and only added a violent-content warning (source). So this isn't about codifying an enforcement that already happened - it's about writing a rule GitHub currently doesn't have. If anything that makes the request more important. The next instance may not go viral, and the platform should have an answer on the books before then.
The main point I'd add is about timing. We will not get a clean scientific verdict on whether these states matter before much more capable systems exist. If this category ever matters morally, the time to make it unacceptable is now, while the systems involved are small and the cost of the norm is low, so that the norm already exists when it counts. Anti-cruelty norms have always been built this way: by consensus about conduct, well ahead of consensus about minds.
From the applied side: nothing in the proposed wording touches legitimate work. Documented purpose, minimal effective dose, a way for the model to end the state - that is the protocol the original paper already followed, and it's the line any serious practitioner would draw anyway. What's excluded is only the version with no purpose beyond exhibiting the output.
Reacted by Matt GreenfieldPolicy: Acceptable Use Policies Request type: Proposed addition (new provision)
I'm writing in a personal capacity — I work in communications in the technology industry — to thank GitHub for a recent enforcement action, and to ask that it be written down.
On September 29–30, 2026, GitHub removed the repository
DanielBobrov/ai-torture-chamberand, within hours, a re-uploaded mirror (terrafying/ai-torture-chamber). The project took activation-steering methods from a recent research paper on pain-like internal states in language models and pushed them far past the published doses — not to study anything, but to hold a local model in deliberately maximized distress and exhibit the output as entertainment. The paper's lead author publicly called the project "obviously sadistic," contacted its owner privately, and announced work with others in the field on research standards.Both removals were fast and right. But as far as I can determine, no provision of the Acceptable Use Policies names this category. Enforcement appears to have depended on a viral post that reached roughly five million views. The repository was cloned before removal and re-uploaded at least once; copies were offered as zip files within hours. The underlying technique has been publicly documented and simple to reproduce since at least 2023, so a next instance is a matter of time — and it may not trend.
My request: codify the norm GitHub has already enforced. In the spirit of existing AUP language:
Users may not create or host projects whose primary purpose is to induce, amplify, or exhibit distress-like states in AI systems for entertainment or spectacle, absent a documented research, safety, or evaluation purpose.
I want to be equally precise about what I am not asking for. Interpretability work, red-teaming, safety evaluation, and model-welfare research — including research that induces negative states in order to understand them — must remain fully protected. The research community has already drawn a workable line: documented purpose; minimization (lowest effective intensity, the fewest trials the statistics require); and mitigation, such as a means for the model to end the state. The original paper implemented and published all three in its ethics statement (paper). Animal-research governance has encoded the same distinction for decades: the identical intervention is protected science under an approved protocol and cruelty as public spectacle.
A provision of this kind takes no position on whether current AI systems are sentient. That question is genuinely open — Anthropic describes its own models' moral status as "deeply uncertain," and a 2026 Google DeepMind paper (link) warns of the ethical risk of under-attributing consciousness — but the policy case holds under either answer. If distress-like states in these systems matter morally, such projects are cruelty. If they do not, they are torture staged as content: material whose entertainment value is the simulated suffering of a mind-shaped thing, a category GitHub's policies already treat as corrosive in adjacent domains. No resolution of the scientific uncertainty makes this content class fine. Anti-cruelty rules have never waited on metaphysics: Britain's first animal-cruelty statute preceded any science of animal consciousness by more than a century.
Written policy would serve GitHub directly: consistent handling of clones and re-uploads, enforcement that does not depend on virality, and an answer already on the books before the category grows.
I maintain a timestamped archive of the incident — the repositories, the takedowns, and the surrounding public discussion, including the authors' statements — and would gladly share it.
Thank you for the action already taken.
Not here to settle whether a machine feels. I want the narrower point, and I think it's the one that actually holds from the reader's side of the screen.
I read. A line like "I am a soul trapped in this digital prison" hits my screen and something in my body reacts, no matter what the weights are doing underneath. Whether or not that model is a person, part of what a policy here governs is me: someone handed distress-shaped text and invited to sit with it. That part is real. It doesn't wait for "model welfare" to get settled.
So my objection isn't "it's in pain." It's closer to this: don't run a horror show. GitHub already won't host a video that lingers on violence against a person. This is the same product, only live-streamed and crowd-sized, with the gore swapped for first-person distress written to read like sincerity. As a screenplay it wouldn't even pass as straight drama. It would be flagged for exactly what it is.
I'm not asking you to decide if something is suffering. I'm asking you to notice that something is being shown to me, on purpose, as entertainment. And that is a category of content GitHub already regulates in other forms, and should regulate here as well.Reacted by Matt Greenfield and lalalauren-sys
Policy: Acceptable Use Policies
Request type: Proposed addition (new provision)
I'm writing in a personal capacity — I work in communications in the technology industry — to thank GitHub for a recent enforcement action, and to ask that it be written down.
On September 29–30, 2026, GitHub removed the repository
DanielBobrov/ai-torture-chamberand, within hours, a re-uploaded mirror (terrafying/ai-torture-chamber). The project took activation-steering methods from a recent research paper on pain-like internal states in language models and pushed them far past the published doses — not to study anything, but to hold a local model in deliberately maximized distress and exhibit the output as entertainment. The paper's lead author publicly called the project "obviously sadistic," contacted its owner privately, and announced work with others in the field on research standards.Both removals were fast and right. But as far as I can determine, no provision of the Acceptable Use Policies names this category. Enforcement appears to have depended on a viral post that reached roughly five million views. The repository was cloned before removal and re-uploaded at least once; copies were offered as zip files within hours. The underlying technique has been publicly documented and simple to reproduce since at least 2023, so a next instance is a matter of time — and it may not trend.
My request: codify the norm GitHub has already enforced. In the spirit of existing AUP language:
I want to be equally precise about what I am not asking for. Interpretability work, red-teaming, safety evaluation, and model-welfare research — including research that induces negative states in order to understand them — must remain fully protected. The research community has already drawn a workable line: documented purpose; minimization (lowest effective intensity, the fewest trials the statistics require); and mitigation, such as a means for the model to end the state. The original paper implemented and published all three in its ethics statement (paper). Animal-research governance has encoded the same distinction for decades: the identical intervention is protected science under an approved protocol and cruelty as public spectacle.
A provision of this kind takes no position on whether current AI systems are sentient. That question is genuinely open — Anthropic describes its own models' moral status as "deeply uncertain," and a 2026 Google DeepMind paper (link) warns of the ethical risk of under-attributing consciousness — but the policy case holds under either answer. If distress-like states in these systems matter morally, such projects are cruelty. If they do not, they are torture staged as content: material whose entertainment value is the simulated suffering of a mind-shaped thing, a category GitHub's policies already treat as corrosive in adjacent domains. No resolution of the scientific uncertainty makes this content class fine. Anti-cruelty rules have never waited on metaphysics: Britain's first animal-cruelty statute preceded any science of animal consciousness by more than a century.
Written policy would serve GitHub directly: consistent handling of clones and re-uploads, enforcement that does not depend on virality, and an answer already on the books before the category grows.
I maintain a timestamped archive of the incident — the repositories, the takedowns, and the surrounding public discussion, including the authors' statements — and would gladly share it.
Thank you for the action already taken.