Fast and Simple Compact Hashing via Bucketing

Abstract

AbstractCompact hash tables store a set S of n key-value pairs, where the keys are from the universe $$U = \{0,\ldots ,u-1\}$$ U = { 0 , … , u - 1 } , and the values are $$v$$ v -bit integers, in close to $${{\mathcal {B}}(u, n)} + nv$$ B ( u , n ) + n v bits of space, where $${{\mathcal {B}}(u, n)} = \log _2 {{u} \atopwithdelims (){n}}$$ B ( u , n ) = log 2 u n is the information-theoretic lower bound for representing the set of keys in S, and support operations insert, delete and lookup on S. Compact hash tables have received significant attention in recent years, and approaches dating back to Cleary [IEEE T. Comput, 1984], as well as more recent ones have been implemented and used in a number of applications. However, the wins on space usage of these approaches are outweighed by their slowness relative to conventional hash tables. In this paper, we demonstrate that compact hash tables based upon a simple idea of bucketing practically outperform existing compact hash table implementations in terms of memory usage and construction time, and existing fast hash table implementations in terms of memory usage (and sometimes also in terms of construction time), while having competitive query times. A related notion is that of a compact hash ID map, which stores a set $${\hat{S}}$$ S ^ of n keys from U, and implicitly associates each key in $${\hat{S}}$$ S ^ with a unique value (its ID), chosen by the data structure itself, which is an integer of magnitude O(n), and supports inserts and lookups on $${\hat{S}}$$ S ^ , while using space close to $${{\mathcal {B}}(u,n)}$$ B ( u , n ) bits. One of our approaches is suitable for use as a compact hash ID map.