Pretraining data filtering: How AI labs remove dangerous knowledge before a model learns it

Data filtering for safety means removing risky documents from an AI model’s training material, so the model never learns that content in the first place. The idea is simple: a model cannot reproduce what it never read. Labs use it most where the finished model will be shared openly and cannot be recalled.

The approach is now in the news. Moonshot AI says it removed cyber-related data from training before opening the weights of Kimi K3.1 Pro, as covered in our report.

Keeping a dangerous chapter out of the library

Picture a student who studies from a huge library. You worry that one shelf holds step-by-step instructions for something harmful. You can ask the student to refuse questions about it later. Or you can quietly take that shelf out before they start studying.

Refusal training is the first option. The model has learned the material and is taught not to share it. Data filtering is the second option. The model is trained on everything except the flagged shelf.

Scoring every document, then dropping the risky ones

Training corpora hold billions of documents, so no person can review them. Labs use automated scoring instead. In Anthropic’s 2025 write-up, the steps were: score each document with a harmfulness classifier, remove documents above a chosen threshold, then train a model from scratch on what remains. A classifier is a small program that labels text, here as harmful or harmless.

Several scoring methods exist. Anthropic’s researchers compared classifiers fine-tuned on labeled examples, prompted language models, cheap keyword-style models such as FastText, and string matching on named entities. They chose a fine-tuned classifier because a large prompted model was too expensive to scan the whole corpus.

A second design, from EleutherAI and the UK AI Security Institute, uses two stages. A blocklist of key terms first picks out candidate documents. Only documents with two or more blocklist terms go to a classifier for a final decision. According to the authors, about 91.6% of documents never reach the second stage.

Where the idea came from

Safety teams have long tried to patch behavior after training. Examples are refusal training and “unlearning” methods that try to erase facts from a finished model. Anthropic’s authors describe post-hoc unlearning as sometimes unreliable and able to impair other capabilities, and the Deep Ignorance authors frame post-training safeguards as fragile for models whose weights anyone can download and fine-tune.

Two papers brought data filtering into focus in August 2025:

• Anthropic’s Alignment Science team (Yanda Chen, Mrinank Sharma and colleagues) published “Enhancing Model Safety through Pretraining Data Filtering” on August 19, 2025. It targeted chemical, biological, radiological and nuclear (CBRN) weapons information.
• EleutherAI, the UK AI Security Institute and Oxford (Kyle O’Brien, Stephen Casper, Stella Biderman and colleagues) published “Deep Ignorance” on August 8, 2025. It focused on biothreat knowledge in open-weight models.

What the studies measured

Anthropic trained paired models, one on unfiltered data and one on filtered data. At its chosen threshold, accuracy on a weapons-knowledge test (WMDP) fell from 33.7% to 30.8%. Random guessing scores 25%, so Anthropic describes this as a 33% relative reduction. It reported no statistically significant drop on general science, code or prose tests. Results came from one medium model size.

The Deep Ignorance team trained several 6.9-billion-parameter models from scratch on roughly 550 billion tokens. Their filtered models resisted adversarial fine-tuning on biothreat text for up to 10,000 steps and about 300 million tokens. The authors say that beats post-training baselines by more than an order of magnitude, with no observed loss on unrelated capabilities. Those models were not competitive with state-of-the-art systems of similar size.

The cost was modest. The authors report the filtering pipeline used under 1% of total training compute, about 1.5 days on 80 H100 GPUs.

Moonshot’s cyber cut and the open-weight question

Moonshot’s launch post says it found cyber-related content inside its general mid-training corpus. Mid-training is a stage between broad pretraining and final tuning. The company says it removed that material from mid-training and from reinforcement learning, and that it will keep applying controls at each stage.

On Moonshot’s own internal test, xdev-bench-hard (pass@3), a checkpoint without the risk reduction scored 90.5% and K3.1 Pro scored 81.0%. Moonshot has not published how that test is built, and the uncleaned checkpoint is not being released. Outsiders therefore cannot confirm the size of the effect. The figures are the company’s own and not independently verified.

The same table shows the limits of reading too much into it. DeepSeek-V4.1-Flash scores 85.7, above K3.1 Pro, and GLM-5.3 scores 57.1. Filtering lowers a score. It does not make a model unable to help with hacking.

Where filtering falls short

The research is open about its gaps.

• Dual-use knowledge. Anthropic notes that general scientific knowledge can be both harmful and beneficial, so filtering may affect legitimate uses.
• Context still works. The Deep Ignorance authors found filtered models can still use dangerous information placed in the prompt, for example through a search tool. No defense they tested resisted an attack combining fine-tuning with open-book retrieval.
• Imperfect labels. Anthropic validated its classifiers on synthetic documents, which may differ from real web text.
• Narrow evidence. The published results cover specific sizes and topics, mainly biothreat proxies. Anthropic’s post did not test recovery through adversarial fine-tuning and lists it as future work.

The Deep Ignorance authors conclude that filtering needs to sit inside a layered defense, not stand alone. Combining it with circuit breaking, a post-training method, gave better resistance to some attacks in their tests.

What to watch when the weights arrive

Moonshot says full weights are due by November 4, 2026. Independent researchers will then be able to test what the evidence so far cannot show: whether fine-tuning restores the removed cyber skill, and how a filtered model behaves when given exploit details in a prompt.

Other open questions are whether labs will publish how their filters work and which benchmarks are used to check them. Without that, a claim like “we removed it” is hard to compare across labs.