How does exl2 calibration interact with abliterated weights?

[behrnt-slatgng]

Disclosure first: I build Grunz, a hosted chat + coding agent running abliterated open weights. Cloud, not local, so I'm not pitching anything here. This is a question about exl2 specifically that I haven't found an answer to.

The question

exl2 quantization uses a calibration dataset to decide bit allocation across layers. That's most of why it does well at a given bpw.

What I can't work out is how that interacts with an abliterated model.

Abliteration works by identifying a refusal direction in activation space and projecting it out of the weights. So by construction, the weights of an abliterated model have already had a specific structure deliberately flattened before quantization ever sees them.

That raises two things I don't know the answer to:

1. Does the calibration set choice matter more on an abliterated model than a clean one?

Bit allocation is driven by what the calibration data activates. If the refusal direction has been removed, the activation statistics on any calibration corpus containing refusal-adjacent content are going to look different from the base model's. Whether that meaningfully changes the allocation, or is lost in the noise, I have no idea.

2. Can calibration partially undo abliteration?

This is the one I actually care about. If quantization is fitting the weights to reproduce calibration-set activations as closely as possible, and the calibration set contains content where the base model would have refused, is there a path by which quantization nudges some of the removed behaviour back? Or is that structurally impossible because the direction is gone from the weights and calibration only allocates precision rather than reintroducing directions?

My instinct says the second — you can't recover a direction that isn't there — but I don't trust my instinct on this and I haven't found anyone addressing it directly.

Why I'm asking rather than testing

I can observe the downstream symptom but not attribute it. What I see consistently is that abliterated models degrade instruction-following and output-format adherence before they degrade knowledge — the model still knows the material, it just stops reliably respecting the instruct template, prefill, stop sequences and structured-output contracts. And that degradation appears to get worse faster as bpw drops than it does on base models.

That's consistent with "abliteration damage and quant damage compound," but it's equally consistent with "I'm using the wrong calibration set for these models," and I can't separate those two from the outside.

What would settle it

Quantizing a base model and its abliterated counterpart to the same bpw with the same calibration set, then measuring format-compliance rate rather than perplexity — perplexity on a generic corpus barely moves while format compliance falls off a cliff, so the usual metric says the model is fine when the application says it isn't.

Has anyone here done anything like that, or is there a reason the framing is wrong? Happy to run it and report back if nobody has, I'd just rather not redo existing work or publish a badly-designed test.