To quantify data center costs, we consider a data center housing on the order of 50,000 servers that would be built based on currently well-understood techniques, using good quality, highly available equipment. Table 1 provides a rough guide to associated costs. Costs are amortized, i.e., one time purchases are amortized over reasonable lifetimes, assuming a 5% cost of money. By amortizing, we obtain a common cost run rate metric that we can apply to both one time purchases (e.g., for servers) and ongoing expenses (e.g., for power).
While networking is not the largest cost category, this paper will argue that networking and systems innovation is the key to reducing costs and getting the most out of each dollar invested.
First and foremost, the leading cost in the enterprise is operational staff. In the data center, such costs are so small (under 5% due to automation), that we safely omit them from Table 1. In a well-run enterprise, a typical ratio of IT staff members to servers is 1:100. Automation is partial [25], and human error is the cause of a large fraction of performance impacting problems [21]. In cloud service data centers, automation is a mandatory requirement of scale, and it is accordingly a foundational principle of design [20]. In a well run data center, a typical ratio of staff members to servers is 1:1000. Automated, recovery-oriented computing techniques cope successfully with the vast majority of problems that arise [20, 12].
Scale Out. Enterprises often optimize for physical space and number of devices, consolidating workload onto a small number of high-price “scale-up” hardware devices and servers. Cloud service data centers “scale-out” — distributing workload over large numbers of low cost servers and hardware.
Massive data analysis applications (e.g., computing the web search index) are a natural fit for a mega data center, where some problems require huge amounts of fast RAM, others require massive numbers of CPU cycles, and still others require massive disk I/O bandwidth. These problems typically call for extensive communication between servers, so the speed of the computation would drop as the propagation delay between servers increases. Further, the dollar cost of communication would go up if the servers were spread out across multiple data centers separated by long distance links, as the market price for these far exceeds the cost of intra-building links
By infrastructure, we mean facilities dedicated to consistent power delivery and to evacuating heat. In some sense, infrastructure is the overhead of cloud services data centers. As Table 1 indicates, the aggregate cost is substantial. As depicted in Figure 1, drawing power from the utility leads to capital investments in large scale generators, transformers, and Uninterruptible Power Supply (UPS) systems. These are not commodity parts — for some the time between order and delivery is 8 months or more. With typical infrastructure cost of $200M, 5% cost of money, and 15 year amortization, the cost of infrastructure comes to $18.4 million/year.
One area of innovation that is impacted by networking is the idea of running the data center hotter — literally reducing the amount of cooling to save money on cooling equipment and the power it consumes. Initial experiments show that equipment failure rates increase with temperature, so the research challenge becomes determining what and how to harden. For example, the network may have to become more resilient and more mesh-like.
The capital cost of networking gear for data centers is a significant fraction of the cost of networking, and is concentrated primarily in switches, routers, and load balancers. The remaining networking costs are concentrated in wide area networking: (1) peering, where traffic is handed off to the Internet Service Providers that deliver packets to end users, (2) the inter-data center links carrying traffic between geographically distributed data centers, and (3) regional facilities (backhaul, metro-area connectivity, co-location space) needed to reach wide area network interconnection sites. The value of the wide area network is shared across the data centers, and its total cost exceeds the cost of networking within any one data center. Back-of-the-envelope calculations for wide area newtwork [sic, lol] cost are difficult, as the costs defy a simple breakdown into quantities such as fiber miles or traffic volumes.
It is worth noting that some other optimizations have less potential for impact in cloud service DCs. Consider reducing power draw in internal data center networking equipment. Well over half the power used by network equipment is consumed by the top of rack switches — while drawing less power per device than other gear, they are far greater in number. A top of rack switch draws ∼60W, while supporting 20 to 40 servers, each drawing ∼200W. The result is cumulative network power draw is a small fraction of the total data center power draw, and economizing on network power draw provides little relative impact. Similarly, improving power distribution efficiency (e.g., using a more efficient UPS than the one considered above) will have relatively low impact, as power distribution is already fairly efficient.
A single data center network supports two types of traffic: (a) traffic flowing between external end systems and internal servers, and (b) traffic flowing between internal servers. A given application typically involves both of these traffic types. In Search applications, for example, internal traffic dominates – building and synchronizing instances of the index. In Video download applications, external traffic dominates
A different opportunity to get more work for each dollar invested in the data center stems from shaping resource consumption – a form of yield management. Designing mechanisms to implement economic incentives that encourage efficient behavior is a rich area for study and impact. Without reasonable incentives, customers (in particular, internal customers), have little to drive them to modulate their demand, leading to a vicious cycle of facilities procurement, followed by a lengthy period of highly bursty load and low utilization. Of top importance are the problems of trough filling and server allocation during times of shortage.
Trough filling: Periods of peak usage of network and power are relatively expensive to a data center – both resources are typically charged based on 95th percentiles of usage, meaning that the cost is determined by the height of the peaks and not by the total area under the curve of usage across time. Thus, a large peak to valley ratio in the temporal usage pattern is inefficient, as the “troughs” in the usage curve can be filled at little additional cost. There are many “bin packing” opportunities to manage services to smooth resource consumption, at many levels of granularity. For example, ensuring leased/committed capacity with fixed minimum cost is always used is a safe way to improve efficiency. By setting prices that vary with resource availability, and by incenting service developers to differentiate demands by urgency for execution, workload can be shifted from peaks to troughs.
Speed and latency matter. There is substantial empirical evidence suggesting that performance directly impacts revenue [22]. For example, Google reported 20% revenue loss due to a specific experiment that increased the time to display search results by as little as 500 msecs. Amazon reported a 1% sales decrease for an additional delay of as little as 100 msecs. This creates a strong motivation for geographically distributing data centers around the world to reduce speed-of-light delays, but it also opens the door to additional opportunities and commensurate research challenges: determining where to place data centers; how big to make them; and using the geographic diversity of data centers as a source of redundancy to improve system availability.
The third factor is network cost. One would like to place data centers as close to the users as possible while minimizing the cost and latency of transferring data between various data centers. One challenge is to find an optimal balance between performance and cost while placing micro data centers near (e.g., within tens of milliseconds) major population centers and fiber hotels supporting access to low cost Internet peering, and access to low cost dedicated or leased lines between data centers
A more sophisticated optimization would also take into account the dependencies of the services offered from the data centers. For example, an email service may depend on an authentication service, an ad insertion service, and a buddy list maintenance service; these dependencies may call for intense and/or low latency communications. Services are often created in tiers of server pools. It is possible, for example, to decompose some services into a front end tier and a back end tier, where the front ends are mapped in micro data centers to minimize latency, and the back end to mega data centers to leverage greater resources. Several other non-technical factors contribute to deciding where to place data centers and how large to make them, including tax policies, and the target markets for the services being hosted.