Evidence of Decreasing Internet Entropy: The Lack of Redundancy in DNS Resolution by Major Websites and Services

Abstract

The Internet, and the Web built on top of it, were intended to support an “entropic” physical and logical network map (Zittrain, 2013). That is, they have been designed to allow servers to be spread anywhere in the world in an ad hoc and evolving fashion, rather than a centralized one. Arbitrary distance among, and number of, servers causes no particular architectural problems, and indeed ensures that problems experienced by one data source remain unlinked to others. A Web page can be assembled from any number of upstream sources, through the use of various URLs, each pointing to a different location. To a user, the page looks unified. Over time, however, there are signs that the hosting and finding of Internet services has become more centralized. We explore and document one possible dimension of this centralization. We analyze the extent to which the Internet’s global domain name resolution (DNS) system has preserved its distributed resilience given the rise of cloud-based hosting and infrastructure. We offer evidence of the dramatic concentration of the DNS hosting market in the hands of a small number of cloud service providers over a period spanning from 2011-2018. In addition, we examine changes in domains’ tendency to “diversify” their pool of nameservers – how frequently domains employ DNS management services from multiple providers rather than just one provider. Throughout the paper, we use the catastrophic October 2016 attack on Dyn, a major DNS hosting provider, to illustrate the cybersecurity consequences of our analysis.