Papers · explanations · citations

# Neural Linguistic
Steganography

Hidden messages. Generated language.

How can a language model encode a message in ordinary-looking text? Explore the methods, security assumptions, and connections to watermarking and communication between AI agents.

[Explore 10 papers](https://steganography.live/reading-list/papers/) · [Start with the guide →](https://steganography.live/reading-list/guides/arithmetic-coding/)

## One text, two readings

A secret message as bits **↓**Language-model probabilities + coding **↓**Text for a reader · bits for a decoder

The sender and receiver share the decoding procedure. A third-party observer sees the generated text. What that observer can distinguish depends on the distribution and threat model.
[Try the original STEGASURAS demo →](https://steganography.live/)

## Three ways into the literature

Follow the mechanism, the guarantee, or the connection.

01 / THE MECHANISM

### [How does arithmetic coding hide a message in text?](https://steganography.live/reading-list/guides/arithmetic-coding/)

Follow the path from secret bits to language-model token choices, and back again.

02 / THE GUARANTEE

### [Capacity, detectability, and security are different measurements](https://steganography.live/reading-list/guides/capacity-and-security/)

Read steganography results by asking what is measured, which distribution is protected, and what the observer can do.

03 / THE CONNECTIONS

### [From hidden messages to watermarks and agent communication](https://steganography.live/reading-list/guides/watermarking-and-agents/)

Shared token-level ideas connect these fields, while their objectives and evidence remain distinct.

The original project

## Neural Linguistic
Steganography

First arXiv submission · September 3, 2019
Published · EMNLP-IJCNLP, November 2019

### [Neural Linguistic Steganography](https://steganography.live/reading-list/papers/neural-linguistic-steganography/)

Zachary M. Ziegler*, Yuntian Deng*, Alexander M. Rush

* Equal contribution.

Combines arithmetic coding with large pretrained language models to encode secret information into generated text while closely matching the model's distribution. GPT-2 experiments evaluate information capacity, statistical divergence, and human judgments of contextual continuations, and demonstrate encoding natural-language messages into unrelated cover text.

The paper develops neural language-model steganography through arithmetic coding, with analysis of the induced distribution and experiments on message capacity and generated text. STEGASURAS is the interactive demonstration accompanying this work.

[Published paper](https://aclanthology.org/D19-1115/) · [Original preprint](https://arxiv.org/abs/1909.01496v1) · [Code](https://github.com/harvardnlp/NeuralSteganography) · [Demo](https://steganography.live/) · [Citations](https://steganography.live/reading-list/papers/neural-linguistic-steganography/#citations)

## A short reading path

[Distribution and security](https://steganography.live/reading-list/papers/dai-cai-2019/) → [Neural Linguistic Steganography](https://steganography.live/reading-list/papers/neural-linguistic-steganography/) → [Self-adjusting arithmetic coding](https://steganography.live/reading-list/papers/saac/) → [Minimum entropy coupling](https://steganography.live/reading-list/papers/minimum-entropy-coupling/) → [Tokenization reliability](https://steganography.live/reading-list/papers/tokenization/)

Then follow the connection to [watermarking](https://steganography.live/reading-list/papers/llm-watermark/) and [agent communication](https://steganography.live/reading-list/papers/secret-collusion/). A citation, a shared mechanism, and a direct extension are different relationships; each paper note explains the connection supported by its sources.

[Browse the full bibliography →](https://steganography.live/reading-list/papers/) · [Download citations and reading notes →](https://steganography.live/reading-list/data/)