THE MECHANISM
How does arithmetic coding hide a message in text?
Follow the path from secret bits to language-model token choices, and back again.
A language model supplies choices
A generative language model assigns probabilities to the next token given the text so far. Linguistic steganography uses those choices to carry an additional message. The visible text is the cover; the hidden message is recovered by someone who knows the coding procedure and the same language model.
Neural Linguistic Steganography, by Zachary M. Ziegler*, Yuntian Deng*, and Alexander M. Rush (2019; * equal contribution), combines arithmetic coding with a large pretrained language model. Its contribution includes the coding method, analysis of the generated distribution, capacity and text-quality experiments, and a demonstration that turns natural-language messages into unrelated cover text. STEGASURAS is the accompanying interactive demo.
Earlier approaches assigned bits to vocabulary bins or used Huffman codes. Arithmetic coding instead keeps a single interval that is refined as tokens are generated. This allows token selection to track a probability distribution more closely than a coarse assignment of a fixed number of bits to each token. The 2019 paper compares these approaches under a shared GPT-2 model. Read Sections 2–4 of the original paper.
A small, idealized example
Imagine four possible next tokens with probabilities 1/2, 1/4, 1/8, and 1/8. Split the interval from 0 to 1 into pieces with those widths. Treat a stream of secret bits as the binary digits of a number inside the interval.
The bitstream beginning 0.101… falls inside B’s interval, so the sender emits B. The next language-model distribution depends on B and the earlier context. The coder subdivides B’s interval using that new distribution and selects another token. As the interval narrows, more message bits become determined.
If the message bits behave like uniform random bits, a region with half the interval’s width is selected half the time in the ideal construction. This is the connection between message coding and distribution matching. In this toy example, B’s interval fixes the first two binary digits to 10; subsequent choices reveal further bits. Real implementations use finite-precision integer arithmetic and renormalization rather than retaining an indefinitely precise real number.
Why the receiver can recover the message
The receiver reads the cover tokens and reconstructs the same probability partitions in the same order. Each observed token selects the same subinterval used by the sender. Bits common to every number in the narrowed interval can then be recovered.
That agreement depends on the complete procedure: model and weights, context, tokenization, candidate selection, probability transformations, token ordering, numerical precision, and message framing. A matching model name alone is not enough. The original implementation provides the concrete arithmetic-coding and decoding procedure.
Steganography and encryption answer different questions. Encryption conceals a message’s contents. Steganography tries to conceal that a message is being carried at all. An arithmetic coder by itself does not provide a general confidentiality guarantee; its distributional behavior also depends on the input-bit assumptions.
Where the ideal model meets an implementation
Temperature and vocabulary truncation alter the distribution used for generation. Restricting choices can improve human judgments of text while moving the generated distribution away from the original model. The original paper measures that tension. Self-adjusting arithmetic coding subsequently replaces a fixed candidate-set size with an adaptive one, explicitly building on the 2019 method.
There is a separate text-interface problem: the generated token sequence may not be the sequence obtained by converting the visible text back to tokens. Yan and Murawaki (2025) study this inconsistency and use the 2019 arithmetic-coding method throughout their steganography experiments. Coding and tokenization reliability are distinct layers that must work together for successful extraction.
Continue with capacity, detectability, and security to see what a fluent example or a successful round trip does—and does not—establish.