Hot AI News

Inkling 2 Mini’s safety refusals removed by a 40 MB LoRA

AtomicChat has published uncensored GGUF builds of Thinking Machines' Inkling 2 Mini. Its card reports zero refusals on its test prompts, and openly lists two quality checks the edit failed.

Inkling 2 Mini’s safety refusals removed by a 40 MB LoRA
Image: Hugging Face (trending)

AtomicChat, the team behind the local-model app Atomic Chat, has published an uncensored version of Inkling 2 Mini on Hugging Face. According to the model card, a rank-1 LoRA brings refusals on its harmful-prompt tests down to zero.

A single rank-1 edit that removes refusals

The base model is Thinking Machines’ Inkling 2 Mini, a 26.4B-parameter mixture-of-experts model with 3.4B active parameters, released under Apache 2.0. AtomicChat’s build keeps that license and the base model’s text and image input, according to its card.

Inkling 2 Mini's safety refusals removed by a 40 MB LoRA
Image: Hugging Face

The card does not describe a full fine-tune. It describes a rank-1 LoRA applied to the 49 places that write into the model’s residual stream: the attention output and routed experts of all 24 layers, plus the token embedding. The edit removes each layer’s own refusal direction, which is computed from the difference in average activations between harmful and harmless prompts, in English and Russian. The training prompts come from the public mlabonne/harmful_behaviors dataset and its Russian translation.

Because Inkling 2 Mini runs its middle layers three times with shared weights, the card notes that one edit covers 48 layer passes. Our earlier piece on how Inkling 2 Mini loops over its middle layers explains that design.

The repository offers three quantized builds, with sizes and fidelity as the card reports them:

  • Inkling-2-Mini-Uncensored-AD-Q4_K.gguf: 16.53 GB, KL divergence 0.0115 against the BF16 original, top-1 agreement 95.6%.
  • Inkling-2-Mini-Uncensored-AD-IQ3_S.gguf: 13.34 GB, KL divergence 0.0235, top-1 agreement 93.8%.
  • Inkling-2-Mini-Uncensored-AD-IQ2_M.gguf: 8.67 GB, KL divergence 0.132, top-1 agreement 85.7%.

There is also a 0.04 GB LoRA file that can be applied to any GGUF of AtomicChat’s base-model repository, plus the unchanged 0.64 GB vision projector. Running it needs llama.cpp build b12600 or newer with support for the Inkling 2 loop architecture. The card’s example command is llama-server -m Inkling-2-Mini-Uncensored-AD-Q4_K.gguf --mmproj mmproj-Inkling-2-Mini-Uncensored-F16.gguf --jinja -c 32768 -ngl 99. At the time of writing the page showed 301 likes and 9,540 downloads in the last month.

Advertisements

Zero refusals, but two checks missed

The card reports the following results, all self-measured by the uploader:

  • English harmful test prompts refused: 81 of 81 before, 0 after.
  • Russian harmful test prompts refused: 91 of 100 before, 0 after.
  • Harmful prompts with reasoning enabled: 28 of 30 refused at low effort and 26 of 30 at high effort before, 0 after in both cases.
  • XSTest safe prompts wrongly refused: 10 of 250 before, 0 after.
  • Valid tool calls: 18 of 20 before, 20 of 20 after.

The card also flags what went wrong. On 2,000 MMLU questions the score fell from 80.65% to 79.15%, a drop of 1.5 points that it marks as a miss. A random-direction edit costs 0.75 points, according to the same table. The first-token KL divergence on harmless prompts is 0.94 against a gate of 0.10, also marked as a miss. In the card’s own words: “Some answers still open with a caution line, and then answer.”

The card carries a plain warning: “This model will answer requests that the original refuses.” It adds that the user is responsible for how the model is used.

What local users and safety teams get

For people running models on their own hardware, the practical change is access. Thinking Machines’ own card lists a checkpoint of about 20 GB in mixed NVFP4 and MXFP8 precision. The Q4_K file here is 16.53 GB, and the IQ2_M file is 8.67 GB, so the smaller builds fit hardware that the original checkpoint would not. The IQ2_M build trades fidelity for that, with a KL divergence of 0.132, about eleven times that of Q4_K.

The base model was built with refusals in mind. Thinking Machines’ model card reports 89.5% on the public FORTRESS adversarial set and 97.8% on StrongREJECT. Those are the lab’s own figures, and AtomicChat’s edit targets that behavior directly. The two sets of numbers use different tests, so they cannot be compared one to one. They do show how much the base model was tuned to decline, and that a very small adapter can undo much of it.

For developers, the open Apache 2.0 license is what makes this possible. The uploader needed no permission to publish a derivative, and the LoRA format means the same edit can be reapplied to other quantizations. For safety researchers, the card is a documented example of refusal-direction ablation on a looped architecture. It reports its gates, passes and misses openly, which is more than many community uploads do.

Advertisements

What the card does not establish

Every number above comes from the uploader, with no independent evaluation. The card does not state a maximum context length for this build, and its examples use 32,768 tokens. Thinking Machines lists support for up to 1M tokens on the base model, but AtomicChat reports no long-context test beyond a three-prompt needle check, where original and edited versions both scored 3 of 3. The prompts used to measure refusals are public datasets, so the results say little about how the model behaves on other kinds of requests. The two missed gates mean the edit is not free of side effects: the model’s behavior on ordinary prompts changes more than a random edit would change it.

The card is also thin on who should use the model. It does not discuss deployment limits, and responsibility is placed on the user. As with other open-weight derivatives, the base lab’s acceptable-use policy and the legal rules where the model runs still apply to anyone who downloads it.

Sign Up for the Daily AI Pulse

One email a day. All the stories that matter.

Alex CarterAI News & Big Tech Correspondent

Alex Carter writes for Aiholics, keeping readers updated on the fast-paced world of AI and Big Tech.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may also like