Small Models Replicate Large Model Chain of Thought via Jailbreaking

Research demonstrates that weaker models can reproduce the encrypted reasoning of larger counterparts like Claude and GPT, undermining current anti-distillation protections.

Woofun AI reports that recent research indicates large model manufacturers' attempts to prevent capability distillation by hiding complete chains of thought may be ineffective. By feeding Opus's encrypted chain of thought to a weaker model, Haiku, and subsequently jailbreaking Haiku, researchers enabled the smaller model to directly reproduce Opus's original inference. This method was demonstrated on Claude, GPT, and Gemini. The discovery builds on work from May by cryptographer Matthew Green, who found that encrypted chains of thought could be replayed across sessions, though no stable reading method existed at the time.

The new paper utilizes the manufacturer's own models to 'decrypt' the data rather than cracking the encryption algorithm. In March, John X. Morris and others showed that detailed synthetic inference could be reconstructed from answers and summaries alone. The study also tested Kimi K3, finding that its inferences aligned more closely with Opus when fed few inference tokens, and that original fragments from Claude and GPT were extracted up to six orders of magnitude more easily from Kimi K3 than other models. The authors emphasize this proves method feasibility, not actual distillation occurrence.