A data structure for a sequence of string accesses in external memory

Abstract

We introduce a new paradigm for querying strings in external memory, suited to the execution of sequences of operations. Formally, given a dictionary of n strings S 1 , …, S n , we aim at supporting a search sequence for m not necessarily distinct strings T 1 , T 2 , …, T m , as well as inserting and deleting individual strings. The dictionary is stored on disk, where each access to a disk page fetches B items, the cost of an operation is the number of pages accessed (I/Os), and efficiency must be attained on entire sequences of string operations rather than on individual ones. Our approach relies on a novel and conceptually simple self-adjusting data structure (SASL) based on skip lists, that is also interesting per se . The search for the whole sequence T 1 , T 2 , …, T m can be done in an expected number of I/Os: O (∑ j =1 m | T j |/ B + ∑ i =1 n n ( n i log B m / n i )), where each T j may or may not be present in the dictionary, and n i is the number of times S i is queried (i.e., the number of T j s equal to S i ). Moreover, inserting or deleting a string S i takes an expected amortized number O (| S i |/ B + log B n ) of I/Os. The term ∑ j =1 m | T j |/ B in the search formula is a lower bound for reading the input, and the term ∑ i =1 n n i log B m / n i (entropy of the query sequence) is a standard information-theoretic lower bound. We regard this result as the static optimality theorem for external-memory string access , as compared to Sleator and Tarjan's classical theorem for numerical dictionaries [Sleator and Tarjan 1985]. Finally, we reformulate the search bound if a cache is available, taking advantage of common prefixes among the strings examined in the search.