Cryptography has become a linchpin of modern society, but while effective at concealing the content of communication cryptography reveals that communication is taking place at all. Steganography, on the other hand, deals with concealing information in an innocent cover signal such that an observer would not even know that any communication was taking place. Natural language is a desirable cover signal given its everyday occurrence, but traditionally it has been challenging to encode large amounts of information in text without sacrificing quality.
In our work, "Neural Linguistic Steganography," we show how the combination of arithmetic coding, a powerful entropy coding algorithm, with large modern language models (GPT-2) allows for a practical steganography system that fools both statistical steganalysis and human detection. We call the approach STEGASURAS: STEGanography via Arithmetic coding and Strong neURAl modelS. Code is available on github. This website offers a demo of the approach, targeted specifically for relatively long (>1 sentence) natural language secret messages:
Encrypt - Generates a continuation (cover text) of some natural language context which encodes an unrelated natural language secret message. Encryption from the secret message into cover text consists of two parts. 1. The message text is decoded into bits via arithmetic coding using the LM with an empty context. 2. The bits are encoded into cover text via arithmetic coding using the LM and the given context.
Decrypt - Reads previously generated cover text and the context used to generate it to recover the previously encoded secret message. Decryption consists of two parts. 1. The cover text is decoded into bits via arithmetic coding using the LM with the given context. 2. The bits are encoded into the reconstructed secret message via arithmetic coding using the LM and an empty context.
NOTE: Because GPT-2 uses byte-pair-encoding, which is non-reversible, there is a small chance that the generated cover text will be interpreted as different tokens while decrypting than when the text was generated during encryption. For example, the encoder might select the tokens “lead” followed by “ing” but the byte-pair-encoding may interpret the word as the token “leading”. This would lead to a decoding error. This issue is only due to the form of BPE used by GPT-2, and is unrelated to the reversibility of the entropy coding techniques more generally.
