The bits between the lambdas

Abstract

For the programmer, storage media are usually assumed to have a minimum atomic unit of transfer of one byte. However, sometimes it is useful to have an even finer storage granularity of one bit, for instance in order to compress data.This paper describes an API in the lazy functional language Haskell for treating storage media as arbitrary-length streams of bits without byte-alignment constraints. So far as possible, storage media are treated uniformly. In particular, bit-stream memory and binary files share the same API -- a new and useful abstraction over memory management and file management. This uniformity of access leads to a novel technique for lazy random-access to files in a purely functional manner. We also describe a technique for automatically deriving compressed binary representations of user-defined data structures, whose operations provide both in-heap data compression and convenient high-level binary I/O.Of many possible applications, we illustrate the processing of Huffman-encoded image data, and a bibliographic information system which uses lazy B-trees for efficient storage management.