10GBASE-T vs SFP+: Which one is suitable for 10G Data Center Cabling?

When designing a new network architecture based on 10GB Ethernet, we face the challenge of choosing the right equipment to achieve maximum performance and support the future demands of complex network applications.

There are two options for 10Gb Ethernet interconnection: 10GBASE-T and SFP+ solutions (SFP+ and DAC/AOC). 10GBASE-T copper cable modules can span network links of up to 100 meters using cat 6a/cat 7 cables. SFP+ optical devices will support distances of up to 300 meters on multimode fiber and up to 80 kilometers on single-mode fiber.

What are the differences?

SFP+ fiber offers lower latency and cost, and the power consumption of SFP+ solutions is also significantly lower, with the power consumption of 10GBASE-T being approximately three to four times that of the SFP+ solution. Moreover, 1Gb SFP transceivers can be inserted into SFP+ ports, functioning at a speed of 1Gb and linking through optical cables to conventional ports. They can also be plugged into SFP modules that are compatible with 1GBase-T, establishing connections at lower speeds with traditional ports.

However, 10GBASE-T copper cabling provides effective backward compatibility with standard copper network equipment, making optimal use of existing copper infrastructure wiring. Additionally, 10GBASE-T is backward compatible with 1G ports, and many low-bandwidth devices still use 1G ports. Compared to SFP+ solutions for small enterprises, 10GBASE-T is generally more cost-effective and easier to deploy.

Conclusion

In comparison, if scalability and flexibility are crucial for small enterprise applications, then 10GBASE-T cabling is the better choice. However, if power efficiency and lower latency are paramount, then 10G SFP+ cabling is clearly the winner.

Click to explore a more detailed purchasing guide: 10GBASE-T vs SFP+ Fiber vs SFP+ DAC: Which to Choose for 10GbE Data Center Cabling? | FS Community

How FS Can Help

Whether it’s a 10G copper cabling solution or a 10G fiber cabling solution, FS can customize it according to your needs. Click to register now and promptly enjoy your exclusive solution design.

Time-to-Link Test for 1000BASE-T and 10GBASE-T

Background

This post is composed on the basis of the physical layer (PHY) behavior assessment of 1000BASE-T and 10GBASE-T. In order to understand the test results and the meaning of this discussion, some terminologies have to be introduced first.

The Meaning of Time-to-Link

Time-to-link (TTL) is a system performance standard that characterizes and measures the PHY behavior through autonegotiation (AN) and 1G/10GBASE-T startup sequences (correspond to training). It is one of the two primary performance measures (the other is bit error rate) used to characterize BASE-T PHY link rate interoperability.

For Ethernet over twisted pair, autonegotiation is defined in clause 28 of IEEE 802.3. It is a procedure by which two connected devices choose common transmission parameters. In this process, the link partner firstly share their capabilities, such as speed, duplex mode, and flow control, and then choose the highest performance transmission mode they both support.

Since servers networking drivers must meet the third party certifications, the TTL standard used to measure link interoperability becomes rather important. Otherwise, long TTLs (>6s) can lead to device certification failures.

How to Measure the Link Interoperability?

There are several representative link interoperability metrics associated with TTL. Their meanings are explained as follows:

TTL: time to achieve link after link initiate event.

Link attempts number: number of attempts made to resolve Master/Slave status for each link. Within a link, one link partner is designated as the master timing source for transmitted signals in both directions. One partner is Master and one partner is Slave.

Link drops number: number of link drops observed after link is established.

Clock recovery: Some digital data streams, especially high-speed serial data streams, such as Ethernet, are sent without an accompanying clock signal. The receiver generates a clock from an approximate frequency reference, and then phase-aligns the clock to the transitions in the data stream with a phase-locked loop (PLL). This is one method of performing a process commonly known as clock and data recovery (CDR). Here it is also called Master/Slave resolution.

TTL distribution: percentage of links by link time.

Speed downshift/downgrade: resolved speed if other than 10Gbps.

Presentation and Analysis of the Results

Totally 1550 link tests are performed, and the results are:

  • 1,050 out of 1,550 tests, or 67% of the total number of link tests, achieved a link state in 7s or less (green slice).
  • 499 out of 1,550 tests, or 32% of the total number of link tests, achieved a link state somewhere between 7s and 15s (blue slice).
  • 1 out of 1,550 tests, or < 1 % (actually 0.15%) of the total number of link tests, achieved a link state longer than 15s (exactly 16.4s; yellow splice, actually it should be smaller than presented in the pie chart).

TTL % of total trials pie chart

Source: http://www.ieee802.org

Characterizing TTL behavior

Cumulative percentage (%) TTL is the distribution of measured link times as a percentage of total measured link time. Total link time recorded for all 1,550 tests is 10,837,835ms or about 3h 0min 38sec. The measured link time and cumulative percentage of each result is recorded in following table and chart:

Cumulative percentage TTL

Source: http://www.ieee802.org

TTL behavior

Source: http://www.ieee802.org

TTL Distribution and Master/Salve Resolution by Channel Length

In this part, the example of 10GBASE-T TTL measured from 2m to 115m channels (9790 links) will be given. The average TTL across 2m to 100m is 7.5s; the average time in autonegotiation is 5s; the average time in training is 2.6s. The following two charts illustrate the TTL distribution and clock recovery results by channel lengths from 2m to 115m.

TTL distribution by channel length

Source: http://www.ieee802.org

clock recovery distribution by channel length

Source: http://www.ieee802.org

According to the charts, we can see that there is an apparent loop timing trend towards Master preference with increasing channel length. And very long TTLs (>15s) at >100m channels are associated with downshits to 1Gb link speed.

AN & Training Times for 1000BASE-T and 10GBASE-T

Measured autonegotiation and training times from 1550 1Gb links for 10GBASE-T device to 1000BASE-T link partner, and 10GBASE-T device to 10GBASE-T link partner are respectively:

AN & traning times and TLL

Conclusion

From the test results on 1000BASE-T and 10GBASE-T, user TTL experience of 1000BASE-T installed over Cat5e cable or better is between 3s and 4s, and 10GBASE-T installed over Cat6a or better is about 7s, or longer in some cases. And the measured autonegotiation times for 1000BASE-T and 10GBASE-T are comparable. And for future 2.5/5GBASE-T, it is highly desirable that their autonegotiation and startup times can be improved, and that total TTL be minimized, so as to be more aligned with end-users’ expectations and requirements.

Appendix: AN & Training Times for 1000BASE-T and 10GBASE-T

1G AN time ditribution

1G traning time ditribution

10G AN time ditribution

10G traning time ditribution

Source: http://www.ieee802.org

10GbE Interconnect Solutions Overview

New sophisticated networking services, coupled with the increase of Internet users push the Internet traffic to an even higher point, driving the need for increased bandwidth consequently. One Ethernet technology—10 Gigabit Ethernet (GbE) is adequate for such bandwidth demand, and has become widely available due to the competitive price and performance, as well as its simplified cabling structure.

Several cable and interconnect solutions are available for 10GbE, the choice of which depends on the maximum interconnect distance, power budget and heat consumption, signal latency, network reliability, component adaptability to future requirements, cost. Here cost includes more than what we call the equipment interface and cable cost, but more often the labor cost. Thus, choosing a 10GbE interconnect solution requires careful evaluation of each option against the specific applications. This text aims to introduce two main 10GbE interconnect solutions: fiber optics and copper.

Fiber Optics Solution

Fiber optic cables include single-mode fiber (SMF) and multi-mode fiber (MMF). MMF is larger in diameter than that of single-mode, thus portions of the light beam follow different paths as they bounce back and forth between the walls of the fiber, leading to the possible distorted signal when reach the other end of the cable. The amount of distortion increases with the length of the cable. The light beam follows a single path through thinner single-mode cable, so the amount of distortion is much lower.

fiber optics solution: SMF & MMF

The typical 10GBASE port type that uses MMF is 10GBASE-SR which uses 850nm lasers. When used with OM3 MMF, 10GBASE-SR can support 300m-connection distances, and when with OM4 MMF, 400m link length is possible through 10GBASE-SR SFP+ transceiver.

10GBASE-LR (eg. E10GSFPLR), 10GBASE-ER and 10GBASE-ZR are all specified to work via SMF. SMF can carry signals up to 80km, so it is more often used in wide-area networks. But since SMF requires a more expensive laser light source than MMF does, SMF is replaced by MMF when the required connection distance is not so long.

Copper Solution

10GBASE-CX4, SFP+ Direct Attach (DAC) and 10GBASE-T are all specified to operate through copper medium.

  • 10GBASE-CX4

Being the first 10GbE copper solution standardized by the IEEE as 802.3ak in 2002, 10GBase-CX4 uses four cables, each carrying 2.5gigabits of data. It is specified to work up to a distance of 15m. Although 10GBase-CX4 provides an extremely cost-effective method to connect equipment within that 15m-distance, its bulky weight and big size of the CX4 connector prohibited higher switch densities required for large scale deployment. Besides, large diameter cables are purchased in fixed lengths, causing problems in managing cable slack. What’s more, the space isn’t sufficient enough to handle these large cables.

  • SFP+ DAC

SFP+ Direct Attach Cable (DAC), or called 10GSFP+Cu, is a copper 10GBASE twin-axial cable, connected directly into an SFP+ housing. It comes in either an active or passive twin-axial cable assembly. This solution provides a low-cost and low energy-consuming interconnect with a flexible cabling length, typically 1 to 7m (passive versions) or up to 15m (active versions) in length. Below is the SFP+ to SFP+ passive copper cable assembly with 1m length, 487655-B21, a HP compatible 10GbE cabling product.

SFP+ to SFP+ passive copper cable assembly, 1m link length

  • 10GBASE-T

10GBASE-T, known as IEEE 802.3an-2006, utilizes twisted pair cables and RJ-45 connectors over distances up to 100m. Cat 6 and Cat 6a are recommended, with the former reaching the full length at 100m, and the latter at 55m. In a word, 10GBASE-T permits operations over 4-connector structured 4-pair twisted-pair copper cabling for all supported distances within 100m. Besides, 10GBASE-T cabling solution is backward-compatible with 1000BASE-T switch infrastructures, keeping costs down while offering an easy migration path from 1GbE to 10GbE.

Conclusion

In summary, two main media options are available for 10GbE interconnect: copper and fiber optics, including 10GBASE-CX4, SFP+ DAC, 10GBASE-T, 10GBASE-SR, 10GBASE-LR, 10GBASE-ER, 10GBASE-ZR, and so on. Fiberstore offers all these 10GBASE SFP+ modules and cables for your 10GbE deployment, which are quality-assured and cost-effective, like E10GSFPLR and 487655-B21 mentioned above. For more information about 10GbE interconnect solutions, you can visit Fiberstore.

Why Choose 10GBASE-T Interface for 10GbE Infrastructure?

The increasing availability of virtualization applications and unified networking infrastructure puts extreme input/output (I/O) demands on 1 Gigabit Ethernet (GbE), making data centers facing bandwidth challenges. Deploying 10GbE infrastructure can address these problems by delivering greater bandwidth, simplifying network, and lowering power consumption.

Well, the deployment of 10GbE requires cost-effective solution. In general, there are several 10GbE interfaces to choose from, including CX4, SFP+ fiber, SFP+ Direct Attach Copper (DAC), and 10GBASE-T. As for CX4, it’s an older technology that does not meet high density requirements. Although most deployment chooses SFP+ fiber (eg. F5-UPG-SFP+-R) solution, fiber is in no case cost-effective. Besides, SFP+ DAC is limited by its short reach. In such a case, 10GBASE-T is selected as the less power-consuming and cost-saving solution for 10GbE. This article details at what are the reasons that drive the 10GBASE-T to become the suitable 10GbE media option.

Firstly, let’s figure out what is 10GBASE-T. 10GBASE-T, or IEEE 802.3an-2006, is a standard released in 2006 to provide 10Gbit/s connections over unshielded or shielded twisted pair cables, with distances up to 100 meters (330 ft) with RJ45 connectors. 10GBASE-T cable infrastructure can also be used for 1000BASE-T, allowing a gradual upgrade from 1000BASE-T using auto-negotiation to select which speed to use.

10GBASE-T, CAT6 and CAT6A CablingListed below are several reasons why 10GBASE-T become the 10GbE media option.

Flexibility in Reach

Like other copper network implementations using BASE-T standards, 10GBASE-T works for link lengths up to 100 meters, giving network designers a far greater level of flexibility in connecting devices in the data center. Able to realize flexible reach, 10GBASE-T can accommodate either top of the rack, middle of row, or end of the row network topologies, making server placement even more easy and convenient.

Backward Compatibility

10GBASE-T is backward-compatible with existing 1GbE networks, meaning that it can be deployed based on existing 1GbE switch infrastructures in data centers that are cabled with CAT6 and CAT6A (or above) cabling. In other words, when migrating from 1GbE to 10GbE, 10GBASE-T provides an easy path, saving cost.

Reduction in Power Consumption

In widespread deployment of 10GbE networks using 10GBASE-T interface, one challenge lies in the fact that the early physical layer interface chips (PHYs) consumed too much power. The original gigabit chips were roughly 6.5 Watts per port. With technology improvements, the chips improved from one generation to the next, leading to less 1 W per port for 1GbE interfaces. It’s the same with 10GBASET. And owing to the manufacturing processes, the 10GBASE-T reduction in power consumption has been made possible. The figure below shows the relationship between power consumption and wavelength.

power consumption vs. wavelength

When 10GBASE-T adapters were first introduced in 2008, they required 25 W of power for a single port, and later, power has been reduced thanks to the successive generations of developing newer and smaller process technologies. The latest 10GBASE-T adapters require less than 6 W per port,which makes 10GBASE-T suitable for motherboard integration and high-density switches.

Latency

Depending on packet size, latency for 10GBASE-T ranges from just over 2 µs to less than 4 µs—a much tighter latency range. For Ethernet packet sizes of 512 bytes or larger, 10GBASE-T’s overall throughput offers an advantage over 1000BASE-T. Latency for 10GBASE-T is more than three times lower than 1000BASE-T with larger packet sizes. For those enterprise applications that have been operating for years with 1000BASE-T latency, 10GBASE-T latency only makes things better. Many products designed for Local Area Network (LAN) purposely add small amounts of latency to reduce power consumption or CPU overhead.

Broad use of 10GBASE-T interface simplifies data center infrastructures, making it easier to manage server connectivity while delivering the bandwidth needed for heavily virtualized servers and I/O-intensive applications. As the cost continues to fall, and new technological processes further lower power consumption, all these make 10GBASE-T suitable for integration on server motherboards.

Conclusion

10GBASE-T offers the flexible reach, and its backward compatibility with existing 1GbE networks makes it the ideal cost-effective media option for 10GbE infrastructure. As a professional fiber optic product manufacturer and supplier, Fiberstore provides countless 10GBASE-T transceivers for 10GbE applications. Of course, besides 10GBASE-T, other 10GBASE standard transceivers also available in Fiberstore, such as 10GBASE-ER SFP+ (J9153A). For more information about 10GbE interfaces, you can visit Fiberstore.

Data Center 10 Gigabit Ethernet Cabling Options

With the dramatic growth in data center throughput, the usage and demand for higher-performance servers, storage and interconnects have also increased. As a result, the expansion of higher speed Ethernet solutions, especially 10 and 40 Gigabit Ethernet has been ongoing. For 10 Gigabit Ethernet solution, selecting the appropriate 10-gigabit physical media is a challenge, because 10GbE is offered in two broad categories: optical and copper. This article will introduce both optical and copper cabling options for 10 Gigabit Ethernet.

Fiber Optic Cables

Two general types of fiber optic cables are available: single-mode fiber and multimode fiber.

Single-mode Fiber (SMF), typically with an optical core of approximately 9 μm (microns), has lower modal dispersion than multimode fiber. It is able to support distances of at least 10 kilometers, depending on transmission speed, transceivers and the buffer credits allocated in the switches.

Multimode Fiber (MMF), with an optical core of either 50 μm or 62.5 μm, can support distances up to 600 meters, depending on transmission speed and transceivers.

When planning data center cabling requirements, be sure to consider that a service life of 15-20 years can be expected for fiber optic cabling. Thus the cable chosen should support legacy, current and emerging data rates.

10GBASE-SR — a port type for multimode fiber, 10GBASE-SR cable is the most common type for fiber optic 10GbE cable. It is able to support an SFP+ connector with an optical transceiver rated for 10GbE transmission speed. 10GBASE-SR cable is known as “short reach” fiber optic cable.

10GBASE-LR — a port type for single-mode fiber, 10GBASE-LR cable is the “long reach” fiber optic cable. It is able to support a link length of 10 kilometers.

OM3 and OM4 are multimode cables that are “laser optimized” and support 10GbE applications. The transmission distance can be up to 300 m and 400 m respectively.

Copper Cables

Common forms of 10GbE copper cables are as follows:

10GBASE-CR — the most common type of copper 10GbE cable, 10GBASE-CR cable uses an attached SFP+ connector and it is also known as a SFP+ Direct Attach Copper (DAC). This fits into the same form factor connector and housing as the fiber optic cables with SFP+ connectors. Many 10GbE switches accept cables with SFP+ connectors, which support both copper and fiber optic cables.

Passive and Active DAC — passive copper connections are common with many interfaces. As the transfer rates increase, passive copper does not provide the distance needed and takes up too much physical space. So the industry is moving towards an active copper type of interface for higher speed connections. Active copper connections include components that boost the signal, reduce the noise and work with smaller gauge cables, improving signal distance, cable flexibility and airflow.

10GBASE-T — 10GBASE-T cables are Cat6a (category 6 augmented). Supporting the higher frequencies required for 10GbE transmission, category 6a is required to reach the distance of 100 meters (330 feet). Cables must be certified to at least 500 MHz to ensure 10GBASE-T compliance. Cat 6 cables may work in 10GBASE-T deployments up to 55 meters (180 feet) depending on the quality of installation. Some 10GbE switches support 10GBASE-T (RJ45) connectors.

When to Use Different Type of 10GbE Cables

To summarize, currently the most common types of 10GbE cables use SFP+ connectors.

  • For short distances, such as within a rack or to a nearby rack, use DAC with SFP+ connectors, also known as 10GBASE-CR.
  • For mid-range distances, use laser optimized multimode fiber cables, either OM3 or OM4, with SFP+ connectors.
  • For long-range distances, use single-mode fiber optic cables, also known as 10GBASE-LR.