The presence of Ethernet in our lives paved the way for the emergence of the Internet of Things (IoT). Ethernet has connected everything around us and beyond, from smart homes and businesses to industries, schools and governments. This specification is even found in our vehicles and facilitates communication between internal devices. Ethernet has enabled high-performance data centers, accelerated industrial processes and commerce, and can be found in homes worldwide. Despite advances in Ethernet technology, with the rise of 800G Ethernet and the standardization of 1.6T Ethernet, high-speed Ethernet beyond 100G remains a rarity in edge computing. This article examines how 100G Ethernet enables edge computing and describes applications and design challenges for IP designers.
Speed requirements for edge computing
“The Edge” refers to any data source that ultimately ends up in a data center or cloud processing paradigm. Examples are cameras and sensors, mobile devices, many types of vehicles, routers and switches, and even smart devices with processing and data collection/sharing capabilities. Although it may seem counterintuitive, the edge is both fuzzy and dynamic – when data aggregation or processing occurs at this perimeter, we speak of edge computing. The proliferation of edge devices has seen meteoric growth with machines, sensors and gauges, mobile and wearable devices, and the continued adoption of AI in transportation, home, and metropolitan technologies. According to Vantage Market Research, “The global edge computing market is valued at US$7.1 billion in 2021 and is projected to reach US$49.6 billion by 2028, at a compound annual growth rate (CAGR) of 38.2 % over the forecast period 2022-2028.” The devices involved can have many form factors and architectures, but let’s consider a single server as representative of them.
Fig. 1: Pathways to the cloud from the edge.
Servers typically use a shared PCIe bus to attach network interface cards (NICs), and computers running PCIe 3.0 are the first generation with a bus fast enough at 8 GT/s per lane to support 100G Ethernet adapters over a x16 connection (unidirectional 16 GB/s or 128 GB/s). With PCIe 4.0, an 8-lane slot supports a 100G adapter at full speed. That’s a sweet spot for today’s machines, since x8 slots are typically available on a PCIe bus. Even with the upcoming generation of PCIe 5.0/CXL 1.1 or 2.0 systems, a 100G data rate will fit comfortably on a shared PCIe bus unless designers try to use parallel computation with maximum bandwidth and minimum latency for interprocess communication ( IPC) to speed up. , as designers need for HPC clusters.
Table 1: PCIe speeds as a function of version and lane count (total BW shown is bi-directional)
Edge devices are generally designed to preprocess, compress, and reduce the amount of data that needs to be transferred upstream. Even if you have the required amount of post-processed data to fully utilize the 100G data rate at the single server connection, all traffic to the data center must be aggregated through a concentrated set of routers and switches. Furthermore, these architectures could not serve too many simultaneous connections at full bandwidth unless they have uplinks that are significant multiples of the speeds of the individual ports. For example, a 32-port 100G Ethernet switch must send all traffic upstream. Link Aggregation Control Protocol (LACP) can be used to aggregate multiple ports for a connection, but even this protocol is limited to eight ports for any given connection. Using LACP with a fixed-radius switch quickly drives up infrastructure and cabling costs by rapidly reducing the number of downstream links the device can provide. Wi-Fi connections are all well below 1Gbps individually, and even 5G cellular theoretically peaks at 20Gbps, so at the aggregation level 100G serves these markets well.
Automotive applications rarely require more than 10 to 25G Ethernet within the vehicle, but require many of the optional Quality of Service (QoS) and time-sensitive network features not yet found in higher-speed Ethernet specifications. If you share a network between vehicle control systems like brakes and an entertainment system, it’s important to prioritize vehicle control, even if your kids are watching an engaging video. Time-sensitive network capabilities, soon to be supported at 100G, enable support for aggregation on industrial floors, audio-visual, security, healthcare, and even high-end automotive applications at the edge!
Another benefit that 100G Ethernet offers over its higher-speed counterparts is support for all required and many optional features specified by the IEEE standards, such as:
All required functions of the base standard IEEE 802.3/802.3ba IEEE 802.3 standards for 10/25/40/50/100G Ethernet systems IEEE 802.3br parameters for Interspersing Express Traffic IEEE 802.1 TSN functions IEEE 1588 Precision Clock Synchronization Protocol IEEE 802.1 -Qav for Audio Video (AV) traffic Energy Efficient Ethernet (EEE) compliant with IEEE 802.3az
100G Ethernet is currently the fastest Ethernet speed that can be sustained over a single lane. The third generation of 100G Ethernet using a single 100 Gb/s lane was released in December 2022 as IEEE 802.3ck, along with 200G and 400G Ethernet using two and four of those lanes, respectively, and is referred to as 100GBASE CR for Twinax up supports up to 2m and 100GBASE-KR for electrical backplanes. Using multi-lane architectures, the 100GBASE-ZR standard can support 100G Ethernet more than 80km over a single wavelength dense wavelength division multiplexing (DWDM) system! For lower-cost options, a four-lane configuration using 25G NRZ SerDes provides a reliable transport medium.
Security is important for all network environments, but especially important at the edge, where 100G Ethernet fully supports MACsec – also known as IEEE 802.1AE. MACsec is a hardware-level encryption mechanism that protects and secures data by ensuring compliance with privacy laws and preventing data theft. MACsec can also prevent rogue devices from connecting to a network, which is a critical safeguard for an edge environment that is both unmanaged and unmonitored. Each link in an Ethernet network (host-to-host, host-to-switch, or switch-to-switch) traverses both encrypted and unencrypted traffic when control over that encryption is imposed at higher layers, but once MACsec for when a connection is enabled, all traffic is turned on, and that connection is hidden from prying eyes.
Finally, at the forefront of high-speed Ethernet technology, the cost per port is increasing dramatically. When you add the cost of cabling for ultra-high-speed Ethernet for edge devices, it just gets that much more expensive. These factors combine to make 100G the perfect top-end game for all but the most modern computing applications, which in turn has led to the creation of a huge market for 100G Ethernet products at both consumer and professional levels – switches and Routers, NICs, and cables, and the competition has helped keep the price point for edge deployments manageable.
If you are developing products like NICs, switches and/or routers for the edge market, Synopsys offers a complete solution for 100G Ethernet IP: MAC, PCS and a full range of PHY options along with verification IP, software development and IP prototyping kits . Beyond the edge, Synopsys also offers high-speed Ethernet IP up to 800G today, and we are working with the various standards groups to enable 1.6T in the future.
