Entropy

An entropy encoder is a data-encoding method that achieves lossless compression by encoding a message with its "wasted" or extra information removed. In other words, entropy encoding removes information that was not necessary in the first place to accurately represent the message. A high degree of entropy implies a message with a great deal of wasted information; English text encoded in ASCII is an example of a message type that has very high entropy. Already-compressed messages, such as JPEG graphics or ZIP archives, have very little entropy and do not benefit from further attempts at entropy encoding.

English text encoded in ASCII has a high degree of entropy because every character is encoded using the same number of bits, eight. It is a known fact that the letters E, L, N, R, S, and T occur at a considerably higher frequency than most other letters in English text. If a way could be found to encode just these letters with four bits, then the new encoding would be smaller, would contain all the original information, and would have less entropy. ASCII uses a fixed number of bits for a reason, however: it is easy, since one always deals with a fixed number of bits to represent each possible glyph or character. How would an encoding scheme that used four bits for the above letters be able to distinguish between the four-bit codes and eight-bit codes? This seemingly difficult problem is solved using what is known as a "prefix-free variable-length" encoding.

In such an encoding, any number of bits can be used to represent any glyph, and glyphs not present in the message are simply not encoded. However, in order to be able to recover the information, no bit pattern that encodes a glyph is allowed to be the prefix of any other encoding bit pattern. This allows the encoded bitstream to be read bit by bit, and whenever a set of bits that represents a glyph is encountered, that glyph can be decoded. If the prefix-free constraint were not enforced, then such a decoding would be impossible.

Consider the text AAAAABCD. Using ASCII, encoding this would require 64 bits. If instead we encode A with the bit pattern 00, B with 01, C with 10, and D with 11, then we can encode this text in only 16 bits; the resulting bit pattern would be 0000000000011011. This is still a fixed-length encoding, however; we are using two bits per glyph instead of eight. Since the glyph A occurs with greater frequency, could we do better by encoding it with fewer bits? In fact we can, but in order to maintain a prefix-free encoding, some of the other bit patterns will become longer than two bits. An optimal encoding is to encode A with 0, B with 10, C with 110, and D with 111. (This is clearly not the only optimal encoding, since the encodings for B, C, and D could be interchanged freely without increasing the size of the final encoded message.) Using this encoding, the message encodes in only 13 bits to 0000010110111, a compression ratio of 4.9 to 1 (that is, each bit in the final encoded message represents as much information as did 4.9 bits in the original encoding). Reading through this bit pattern from left to right, you will see that the prefix-free encoding makes it simple to decode this into the original text even though the codes have varying bit lengths.

As a second example, consider the text THE CAT IN THE HAT. In this text, the letter T and the space character both occur with the highest frequency, so they will clearly have the shortest encoding bit patterns in an optimal encoding. The letters C, I, and N occur only once, however, so they will have the longest codes. There are many possible sets of prefix-free variable-length bit patterns that would yield the optimal encoding, that is, that would allow the text to be encoded in the fewest number of bits. One such optimal encoding encodes the space with 00, A with 100, C with 1110, E with 1111, H with 110, I with 1010, N with 1011, and T with 01. This optimal encoding therefore requires only 51 bits, compared to the 144 that would be necessary with 8-bit ASCII encoding, a compression ratio of 2.8 to 1.

The input contains a list of text strings, one per line. Each text string consists only of uppercase alphanumeric characters and underscores (underscores stand in place of spaces). The end of the input is signalled by a line containing only the word END; this line must not be processed.

For each input text string, print three values on one line, separated by single spaces: the length in bits of the 8-bit ASCII encoding, the length in bits of an optimal prefix-free variable-length encoding, and the compression ratio rounded to one decimal place. The compression ratio is the ASCII bit length divided by the optimal bit length, rounded to exactly one digit after the decimal point.