In discussions with both communication service providers and internet content providers (ICPs), data center interconnect (DCI) is the hottest segment in transport networking, with annual capacity growth exceeding 50 percent. In parallel, data centers themselves are evolving. Underpinning the computational needs of artificial intelligence (AI) and machine learning (ML), GPUs like the recently announced Nvidia Blackwell GPU are projected to overtake CPUs inside data centers by 2027 [1]. But GPUs also need more power than CPUs. In fact, a single Blackwell GPU rack (72 GPUs) can draw as much as 120 kilowatts. To reduce the impact on power grids and local real estate markets and to be closer to end users for latency-sensitive applications, new data centers are increasingly smaller, modular, and distributed. In the Chicago area alone, 25 new data centers are being planned – requiring an estimated 5 gigawatts of incremental power demand.
So, how can we cost-effectively address growing inter-data center connectivity demands for both hyperscale data centers and increasingly smaller, modular, and distributed ones? The answer is threefold: we need to go faster, expand the fiber, and simplify capacity expansion with compact modularity.
Go faster with terabit wavelengths
Leading coherent optical engines are evolving in two directions simultaneously: 1) embedded, sled-based optical engines with sophisticated transmission schemes that maximize capacity-reach and spectral efficiency and 2) smaller, lower-power pluggables that can reach 1,000 km or more.
While today’s 800G embedded engines deliver enormous value for DCI applications, we are moving into the terabit era with the development of 1.2+ Tb/s engines that can transmit 1.2 Tb/s wavelengths hundreds of km and 800G waves up to 3,000 km. Due to their high capacity-reach, embedded optical engines are ideal for long-distance DCI connectivity, including across continents or oceans with subsea connectivity. Embedded engines are also ideal where fiber is scarce and spectral efficiency matters most. As an example, data center operators that lease fiber can utilize embedded optical engines to maximize data transmission over a single fiber pair and thus avoid the incremental costs associated with leasing incremental fibers.
400G coherent pluggables are widely available today, including 400ZR, which supports fixed DCI applications up to 120 km. 400G ZR+ pluggables offer more advanced functionality, including increased programmability and better optical performance for metro-regional and some long-distance connectivity. 800G coherent pluggables are under development for delivery in early 2025. This latest generation of coherent pluggables based upon 3-nm digital signal processor technology expands capacity-reach significantly, with the ability to deliver 800G wavelengths up to 2,000 km. With such capabilities in small QSFP-DD or OSFP packages, IP over DWDM (IPoDWDM) will continue to be realized in some DCI applications with pluggables being deployed directly into routers and switches.
More spectrum with Super C
For more than 10 years, we have used the 4.8 THz extended C-band spectrum for DCI and other fiber optic transmission applications. Occasionally, especially with ICPs and for DCI applications, the adjacent 4.8 THz L-band spectrum has been added for a combined 9.6 THz C+L delivery. But, with advancements in optical line system components like amplifiers and wavelength-selective switches, we can now cost-effectively increase the transmission spectrum from 4.8 THz to 6.1 THz to create Super C. A similar approach can be applied to creating a Super L transmission band. For a small incremental line system infrastructure cost, we can realize 27 percent incremental spectrum, enabling up to 50 Tb/s transmission capacity per band. Super C and Super L transmission are cost-effective ways to squeeze more out of existing fiber and to keep up with DCI capacity demands.
Figure 1: C-band spectrum evolution. Credit: Infinera.
But expanding the transmission spectrum does not just benefit embedded engines. Super C expansion also benefits network operators using coherent pluggables in their DCI deployments. While coherent pluggables continue to make huge strides in terms of performance, they are generally less spectrally efficient than their sled-based, embedded optical counterparts. With coherent pluggables, one generally accepts the trade-off that pluggables are less spectrally efficient with reduced total fiber transmission capacity in return for reduced power, space, and cost. However, by combining a Super C optical line system with Super C-capable coherent pluggables, service providers can reclaim any reduction in total fiber capacity. As seen in Figure 2, 800G Super C pluggable optics exceed the total fiber transmission capacity of the latest C-band-only embedded optical engines. By combining next-generation coherent pluggables and Super C-band transmission, network operators can leverage the benefits of smaller space and power utilization without sacrificing total fiber transmission capacity.
Figure 2: Total fiber transmission capacity vs. distance. Credit: Infinera.
Pay as you grow with compact modularity
If we are going utilize a combination of embedded and pluggable engines in the terabit era and we are going to expand the optical spectrum to include Super C-band, we need a cost-effective, scalable approach to match the transmission capacity to the data center connectivity needs today while also accommodating rapid growth tomorrow. Today’s next-generation compact modular optical platforms support a mix-and-match of optical line system functionality and embedded and pluggable optical engines. Network operators can start with a single 1RU, 2RU, or 3RU chassis and stack them as needed, matching cost to capacity while minimizing complexity. By supporting both line system and optical engine functionality in a common platform, network operators can cost-effectively support small, medium, and large DCI capacity requirements while minimizing the amount of equipment required versus dedicated per-function alternatives. As capacity demand grows, additional pluggables, sleds, and chassis can be easily added – all managed as a single network entity for operational simplicity.
Bringing it all together
Data centers are evolving and optical transport solutions that support their interconnect must evolve with them. With wavelengths approaching 1 terabit per second in pluggables (and exceeding 1 terabit in embedded) and with Super C expanding the transmission spectrum, we are enabling network operators to deliver more capacity per fiber for DCI use cases than ever before. By utilizing compact modular optical platforms with Lego-like modularity, we are also enabling network operators to mix and match functionality and to tailor the connectivity solution to the capacity needed today and tomorrow – across town, across continents, and even across oceans.
References
[1] OMDIA, May 2023, AI Processors for Cloud and Data Center Forecast Report
