High Throughput Computing Data Center Architecture
Abstract
In the last few decades, data center（DC）technologies have kept evolving from DC 1.0（tightly-coupled silos）to DC 2.0（computer virtualization)to enhance data processing capability. Emerging big data analysis based business raises highly-diversified and time-varied demand for DCs. Due to the limitations on throughput, resource utilization, manageability and energy efficiency, current DC 2.0 shows its incompetence to provide higher throughput and seamless integration of heterogeneous resources for different big data applications. By rethinking the demand for big data applications, Huawei proposes a high throughput computing data center architecture（HTC-DC）。Based on resource disaggregation and interface-unified interconnects, HTC-DC is enabled with PB-level data processing capability, intelligent manageability, high scalability and high energy efficiency. With competitive features, HTC-DC can be a promising candidate for DC3.0.
Contents
- Era of Big Data: New Data Center Architecture in Need
1、Needs on Big Data Processing
2、DC Evolution: Limitations and Strategies
3、Huawei’s Vision on Future DC
- DC3.0: Huawei HTC-DC
1、HTC-DC Overview
2、Key Features
- Summary
ERA OF BIG DATA: NEW DATA CENTER ARCHITECTURE IN NEED
Needs on Big Data Processing
During the past few years, applications which are based on big data analysis have emerged, enriching human life with more real-time and intelligent interactions. Such applications have proven themselves to become the next wave of mainstream of online services. As the era of big data approaches, higher and higher demand on data processing capability has been raised. Being the major facilities to support highly varied big data processing tasks, future data centers（DCs）are expected to meet the following big data requirements（Figure 1）：
- PB/s-level data processing capability ensuring aggregated high-throughput computing, storage and networking;
- Adaptability to highly-varied run-time resource demands;
- Continuous availability providing 24x7 large-scaled service coverage, and supporting high-concurrency access;
- Rapid deployment allowing quick deployment and resource configuration for emerging applications.
DC Evolution: Limitations and Strategies
DC technologies in the last decade have been evolved（Figure 2）from DC 1.0（with tightly-coupled silos）to current DC 2.0（with computer virtualization）。 Although data processing capability of DCs have been significantly enhanced, due to the limitations on throughput, resource utilization, manageability and energy efficiency, current DC 2.0 shows its incompetence to meet the demands of the future:
Figure 2.DC Evolution
- Throughput : Compared with technological improvement in computational capability of processors, improvement in I/O access performance has long been lagged behind. With the fact that computing within conventional DC architecture largely involves data movement between storage and CPU/memory via I/O ports, it is challenging for current DC architecture to provide PB-level high throughput for big data applications. The problem of I/O gap is resulted from low-speed characteristics of conventional transmission and storage mediums, and also from inefficient architecture design and data access mechanisms.
To meet the requirement of future high throughput data processing capability, adopting new transmission technology（e.g. optical interconnects）and new storage medium can be feasible solutions. But a more fundamental approach is to re-design DC architecture as well as data access mechanisms for computing. If data access in computing process can avoid using conventional I/O mechanism, but use ultra-high-bandwidth network to serve as the new I/O functionality, DC throughput can be significantly improved.
