Managing Big Data: Cloud Computing and Co-Location Centers

Abstract

The oil and gas industry is among the top sectors contributing to the proliferation of a phenomenon known as “big data.” A recent McKinsey Global Institute report defines big data as “datasets whose size is beyond the ability of typical database software tools to capture, store, manage, and analyze.” This translates into datasets on the order of terabytes and petabytes (Table 1) in size, with the assumption that “as technology advances over time, the size of datasets that qualify as big data will also increase.” Dataset increase follows what is known as Moore’s Law, first described by Intel co-founder Gordon Moore in the 1960s, which states that the number of transistors that can be placed on an integrated circuit doubles approximately every two years. This has tended to mean that the amount of computing power that can be purchased for the same amount of money doubles about every two years. Managing big data calls for capitalizing on economies of scale in order to achieve efficiencies that ensure ongoing data security and virtual 100% accessibility. This is where cloud computing enters the picture. The term “cloud” conjures up the image of vast amorphous forms that have a metaphysical, even mystical, relation to solid, three-dimensional substance. However, this vagueness is misleading. In reality, all data and the entire internet must reside on servers that in turn are housed somewhere. Among the largest members of the cloud community are familiar companies such as Amazon.com, Apple, Facebook, Google, Hewlett-Packard, IBM, Microsoft, Twitter, and Yahoo, whose services are used by all industries, including oil and gas. Signaling a trend that cuts across all industries, these companies have consolidated their information on huge arrays of servers—servers capable of increasingly high-speed processing—housed in gigantic data centers ranging in size from 50,000 sq ft to half a million or more sq ft. These nine companies among them own or lease 47 data centers, most in the US.