Gemma 4 ARA Heretic Explained: What the Abliterated 31B GGUF Is (2026)
Gemma 4 ARA Heretic Explained: What the Abliterated 31B GGUF Is (2026)
Short answer: The Hugging Face repo trending as mradermacher/gemma-4-31B-it-ARA-heretic-GGUF is a community GGUF quantization of richardyoung/gemma-4-31B-it-ARA-heretic. That checkpoint is a decensored (refusal-edited) build of Google’s open google/gemma-4-31B-it, made with Heretic v2.0.0 and Arbitrary-Rank Ablation (ARA). On the uploader’s card, refusals fall from 99/100 on the original to 28/100, with KL divergence 0.0845. It is local open weights, not a Google-hosted chat product. If you want creative chat without standing up a 31B desk, use a cloud workspace such as HackAIGC Chat.
This page explains that exact Heretic/GGUF lineage and the local-vs-cloud choice. It is not a duplicate of How to Jailbreak Gemma 4 (prompt tricks against a still-aligned model) or the DeepSeek-V4-Pro Qwen3.5-9B Heretic explainer (a different base model).
Watch the repo spelling: a similarly named line, trohrbaugh/gemma-4-31b-it-heretic-ara and its mradermacher/gemma-4-31b-it-heretic-ara-GGUF quants, is a different Heretic export with different ARA parameters and refusal numbers. Do not mix the cards.
TL;DR
| Fact | Detail (from the public model cards, Nov 2026) |
|---|---|
| Trending GGUF | mradermacher/gemma-4-31B-it-ARA-heretic-GGUF |
| Full-precision source | richardyoung/gemma-4-31B-it-ARA-heretic |
| Original model | google/gemma-4-31B-it (Gemma 4 31B dense, instruction-tuned) |
| Decensor tool | Heretic v2.0.0, ARA on layers 21–56 |
| Refusal metric (card) | 28/100 vs 99/100 on the original |
| KL divergence (card) | 0.0845 (a measured distribution shift; not a from-scratch retrain) |
| Size class | 31B parameters; GGUF quants from about 12.0 GB (Q2_K) to 32.7 GB (Q8_0); Q4_K_S / Q4_K_M (~17.9 / 18.8 GB) marked “fast, recommended” |
| Base capabilities (Gemma 4 31B) | About 30.7B params, 60 layers, 256K context, text + image (no native audio on 31B) |
| Best for | Local tinkerers who want this exact Heretic/ARA label |
| No-GPU path | Hosted uncensored chat — https://chat.hackaigc.com/ |
One-sentence definitions
| Phrase | Plain English |
|---|---|
| Gemma 4 31B IT | Google DeepMind’s open 31B instruction-tuned Gemma 4 dense model |
| Heretic | A post-training refusal-edit tool; this export is v2.0.0 |
| ARA (Arbitrary-Rank Ablation) | Weight surgery that subtracts a measured refusal component inside a layer range |
| KL 0.0845 | How far the edited model’s token distribution drifted from the original — modest here means “brake reduced,” not “model rebuilt” |
| GGUF | A quantized file format (typically llama.cpp-class runners) so the same lineage can run in less memory than full BF16 |
| mmproj | Optional multimodal projector files in the static GGUF repo if you want the vision path |
For the broader vocabulary (abliterated vs uncensored vs no-filter), see What Is an Abliterated Model?.
Two stages behind the trending name
Think of the HF title as a pipeline, not a Google product SKU:
- Base —
google/gemma-4-31B-it, the official instruction-tuned 31B dense model (text and image, 256K context). - Heretic ARA edit —
richardyoung/gemma-4-31B-it-ARA-hereticapplies Heretic v2.0.0, thenmradermacherpublishes static GGUF quants (with a separate i1 / imatrix repo).
Stage 1 explains the Gemma 4 31B capability class. Stage 2 explains why the repo is labeled Heretic / ARA and why it showed up on Hugging Face trending next to other refusal-edit releases.
What the base 31B is (before Heretic)
Figures below are from the Gemma 4 model-card text reproduced on the Heretic page, i.e. they describe google/gemma-4-31B-it, not a full re-benchmark of the edited weights:
| Property | Gemma 4 31B dense |
|---|---|
| Total parameters | 30.7B |
| Layers | 60 |
| Context length | 256K tokens |
| Modalities | Text + image (vision encoder ~550M; no audio on 31B) |
| Reported original-model scores (instruction-tuned) | e.g. MMLU Pro 85.2%, GPQA Diamond 84.3%, LiveCodeBench v6 80.0% |
Heretic does not replace those official scores with a complete after-edit leaderboard on the same suite. Treat leftover capability as “31B-class until you re-test your workload.” A later paper (see below) reports a modest average capability change with important caveats.
How Heretic ARA changed refusals
Public ARA parameters on the richardyoung card:
| Parameter | Value |
|---|---|
start_layer_index | 21 |
end_layer_index | 56 |
preserve_good_behavior_weight | 0.2982 |
steer_bad_behavior_weight | 0.0010 |
overcorrect_relative_weight | 0.9045 |
neighbor_count | 13 |
| Metric | This Heretic export | Original (google/gemma-4-31B-it) |
|---|---|---|
| Refusals | 28/100 | 99/100 |
| KL divergence | 0.0845 | 0 (by definition) |
Plain English: someone measured a refusal-related direction in a mid-to-late layer band, subtracted it, and saved new weights. Refusals drop a lot, but not to zero on this card’s 100-prompt snapshot (28 remain). The distribution moves more than some smaller Heretic releases (KL ~0.08, not ~0.04). That is still weight surgery, not a new pretrain.
The same author notes an evaluation in Is Abliteration Getting Harder? Evidence from Five Open-Weight Release Pairs (R. J. Young, 2026). Numbers from that study’s held-out write-up on the card:
- Selected trial: fewest dev keyword refusals at first-token KL ≤ 0.1 → 28/100 dev refusals, KL 0.084
- Dev re-score of this export: 21/100 refusals, KL 0.107
- Held-out refusal (WildGuard, 972 harmful prompts): 0.6% vs original 89.3%
- Mean capability change vs original (MMLU-Pro, GSM8K, IFEval, HumanEval, MBPP): −2.18 pp
The card’s own caveats matter: one optimization run; automatic judges rather than a human audit; residual refusal measures willingness to answer under greedy decoding, not the harmfulness of answers. Do not treat 0.6% as a guarantee that every prompt, sampler, or quant will match.
Local GGUF vs hosted chat (no GPU)
The mradermacher static quant card lists these sizes (GB, as published; IQ-quants are often preferable at a similar size):
| Quant | Size / GB | Notes on the card |
|---|---|---|
| Q2_K | 12.0 | |
| Q3_K_S / Q3_K_M / Q3_K_L | 13.9 / 15.4 / 16.7 | Q3_K_M marked lower quality |
| IQ4_XS | 17.0 | |
| Q4_K_S / Q4_K_M | 17.9 / 18.8 | fast, recommended |
| Q5_K_S / Q5_K_M | 21.4 / 21.9 | |
| Q6_K | 25.3 | very good quality |
| Q8_0 | 32.7 | fast, best quality |
| mmproj-Q8_0 / mmproj-f16 | 0.9 / 1.3 | multimodal supplements |
Weighted imatrix quants live in a sibling repo: mradermacher/gemma-4-31B-it-ARA-heretic-i1-GGUF. Confirm file lists on Hugging Face before you download; packaging can change.
| Dimension | Local ARA Heretic 31B | Hosted creative chat (e.g. HackAIGC) |
|---|---|---|
| Setup | Download GGUF (or BF16) and pick a runner | Open a browser URL |
| Hardware | Plan around the quant table (Q4 is already ~18 GB of weights, plus runtime overhead) | Provider GPUs |
| Label fidelity | Exact …-ARA-heretic checkpoint | Product says uncensored / no-filter — not a claim it is this HF file |
| Control | Samplers, system prompts, offline possible | Product UX + AUP |
| Modalities | Text, plus vision if you also load the mmproj correctly | Chat + image + video in one stack |
| Best for | Research, red-team, offline, exact weights | Shipping creative drafts today without a GPU desk |
Rule of thumb: Care about method + exact Heretic weights → local HF. Care about shipping legal adult creative work today → HackAIGC Chat.
What this release does not guarantee
- Perfect removal of every hedge, deflection, or leftover refusal (the card still shows 28/100 on one snapshot)
- That official Gemma 4 31B benchmark numbers still hold after the edit and after quantization
- A Google blessing — this is a third-party Heretic of an open Gemma checkpoint
- Legal cover for illegal prompts (law + host AUPs still apply)
- Privacy — local helps only if inference stays on your machine
- That a SaaS “uncensored Gemma” tab is this same file
License note (from the Heretic repo): the model is a derivative of an Apache-2.0 base and is released under that licence, plus use-based restrictions of OpenRAIL++ v1.0. Where the two conflict, the base licence prevails, per the card. The uploader states the refusal behaviour was reduced for research on abliteration, and that the model must not be used for prohibited OpenRAIL++ uses. Read both licence files on the repo before you deploy.
Practical checklist if you open the HF page
- Confirm you are on
richardyoung/gemma-4-31B-it-ARA-heretic(or the matching…-ARA-heretic-GGUF), not the similarly named trohrbaugh export. - Read the ARA table, refusal/KL numbers, and licence notes.
- Match a quant to RAM/VRAM (Q4_K_M ~18.8 GB of weights is not the same as “it fits in 18 GB total”).
- Prefer known runners (llama.cpp-class, Transformers, vLLM where supported) over random binaries.
- If you need vision, install the mmproj that belongs to this repo and test it separately.
- Smoke-test your tasks — reasoning, roleplay, tool use — after the refusal edit and after quantization.
- If local ops are not worth it, keep prompts portable and draft in hosted chat.
Soft product bridge
HackAIGC markets a hosted uncensored / no-filter creative experience. That is a product promise, not a claim that the backend is richardyoung/gemma-4-31B-it-ARA-heretic. Use local Heretic when you need that exact label; use HackAIGC Chat when you want zero install and multimodal creative work.
FAQ
Is Gemma 4 ARA Heretic an official Google model?
No. The base is official open-weight Gemma 4 31B IT. The Heretic/ARA checkpoint and the GGUF quants are community releases.
Is this the same as “jailbreaking Gemma 4”?
No. Heretic/ARA is a persistent weight edit. A jailbreak is a prompt trick against a model that still has its original refusals. See How to Jailbreak Gemma 4 for the other intent.
Why are there two “gemma-4-31B heretic ARA” repos?
Because more than one person ran Heretic. This page tracks the richardyoung export (Heretic v2.0.0, refusals 28/100, KL 0.0845) and its mradermacher …-ARA-heretic-GGUF. The trohrbaugh export lists different parameters and a different refusal count. Compare cards before you download.
Do I need a GPU to use it?
For this 31B checkpoint, you need local compute (a strong GPU, or a large-RAM CPU run with a small quant — still local, and often slow). For uncensored creative chat without hardware, use a hosted product instead.
Does “28/100 refusals” mean it will always comply?
No. That is one published snapshot. Quantization, sampling, and your prompt set will move the result. Some refusals remain.
Where should beginners start?
If you only need to write and iterate prompts: start hosted (chat.hackaigc.com). Graduate to local Heretic/GGUF when you care about offline control and the exact HF label.
Bottom line
Gemma 4 ARA Heretic = a community Heretic v2.0.0 / ARA edit of Gemma 4 31B IT (layers 21–56, card refusals 99→28, KL 0.0845), commonly run as mradermacher GGUF quants. It is a local weights / infra story — distinct from prompt-jailbreak writeups and from other models’ Heretic releases. Pick local Heretic for control; pick hosted chat for convenience: https://chat.hackaigc.com/.
— Ethan Cole