A card can physically fit into a PCIe x16 slot and still fail to deliver its expected performance if the slot is electrically limited to x8, the server cannot provide enough airflow, or the required driver and protocol stack are unavailable.
The Slot Is Not the Whole Specification
A server PCIe slot has at least three separate characteristics: physical size, electrical lane width, and PCIe generation.
These characteristics are related but not interchangeable. A slot may be physically long enough for an x16 card but electrically wired as x8 or x4. A server may also expose different PCIe generations through different risers or CPU sockets.
| Server Attribute | Why It Matters |
|---|---|
| Physical slot length | Determines whether the card can be inserted |
| Electrical lane width | Determines how many PCIe lanes are actually available |
| PCIe generation | Determines the signaling rate per lane |
| CPU or NUMA connection | Can affect latency and memory locality |
| Riser model | May change slot width, direction, and supported cards |
| BIOS configuration | May affect enumeration, bifurcation, and link negotiation |
| Power and airflow | Determines whether the card can operate reliably |
This is why “the server has an x16 slot” is not enough information for a compatibility decision. The server model, riser, slot wiring, and adapter requirements must be reviewed together.
Send the server model, PCIe slot or riser, network speed, protocol mode, and cable type.
PCIe Generation and Lane Width
PCIe Gen3, Gen4, and Gen5 provide different signaling rates per lane. A newer generation can provide more bandwidth without increasing the lane count, while a wider link can provide more aggregate bandwidth within the same generation.
The practical question is not whether the card fits in an x16 slot. The question is whether the server can provide the PCIe generation, electrical lane width, power, and cooling that the adapter requires.
| PCIe Interface | Signaling Rate per Lane | Typical Selection Concern |
|---|---|---|
| PCIe Gen3 | 8 GT/s | May be sufficient for lower-speed adapters |
| PCIe Gen4 | 16 GT/s | Common for modern 25G to 200G server adapters |
| PCIe Gen5 | 32 GT/s | Used by newer high-speed and AI-oriented adapters |
| x8 | 8 lanes | May be sufficient for selected 25G or 100G cards |
| x16 | 16 lanes | Common for high-throughput dual-port or 200G/400G cards |
These are link signaling values, not direct application throughput figures. Protocol overhead, traffic patterns, CPU performance, memory bandwidth, and adapter architecture still affect real throughput.
NVIDIA documentation provides a useful example with ConnectX-6: a PCIe x16 card can support up to 100GbE on a Gen3 link or up to 200GbE on a Gen4 link, while an x8 ConnectX-6 card can support up to 100Gb/s in a Gen4 slot. The exact result depends on the adapter model.
Network Speed and Connector Type
The network-side connector is just as important as the PCIe-side interface. A PCIe card with QSFP56 ports is not automatically compatible with every QSFP56 optic or cable.
| Network Speed | Common Adapter Connector | Typical Use |
|---|---|---|
| 1/10GbE | RJ45 or SFP+ | Enterprise access and server uplinks |
| 25GbE | SFP28 | Modern server access and storage |
| 100GbE | QSFP28 | Data center leaf, storage, and uplinks |
| 200GbE | QSFP56 or QSFP112 | AI and high-performance data centers |
| 400GbE | QSFP112 or OSFP | AI fabric and high-density spine/leaf networks |
Before selecting the card, verify:
- Ethernet or InfiniBand mode.
- Supported network speed.
- DAC, AOC, or optical module type.
- FEC requirements.
- Breakout support.
- Switch-side port compatibility.
- Cable length and thermal requirements.
A dual-port 100GbE adapter may be appropriate for a storage server or leaf connection, while a quad-port 25/50GbE adapter may be more suitable when one server needs several independent links.
Ethernet, InfiniBand, and the Two Bandwidth Ceilings
Ethernet or InfiniBand?
Many NVIDIA Mellanox adapters support both Ethernet and InfiniBand, but the choice affects the complete solution.
Before ordering, confirm:
- Which protocol the switch fabric uses.
- Whether the adapter is configured for Ethernet or InfiniBand.
- Whether the application requires RDMA.
- Whether the deployment uses RoCE or native InfiniBand.
- Which drivers and firmware are required.
- Whether the selected cables and optics support the chosen mode.
A card intended for an InfiniBand fabric should not be treated as a drop-in Ethernet adapter without checking its exact product documentation and software environment.
The Two Bandwidth Ceilings
A server network adapter is constrained by two separate bandwidth ceilings.
| Bandwidth Ceiling | What It Includes | What to Verify |
|---|---|---|
| Network-side ceiling | Adapter ports, cables, optics, and switch ports | Port speed, connector, cable, FEC, and breakout |
| PCIe-side ceiling | PCIe generation, lane width, and host bus | Gen3/4/5, x8/x16 wiring, riser, and negotiated link |
If the PCIe link is too narrow or too old, the adapter may still appear online, but the host bus can limit aggregate traffic. This is especially important for dual-port adapters, high-speed storage traffic, RDMA workloads, AI servers, and simultaneous full-rate traffic on multiple ports.
ConnectX-6 and ConnectX-7: What to Check
The right question is not simply whether ConnectX-7 is newer than ConnectX-6. The more useful question is whether the server and workload can use the additional capability.
When ConnectX-6 May Fit
- The server provides PCIe Gen3 or Gen4.
- The target speed is 25G, 50G, 100G, or a supported 200G configuration.
- The existing software environment already uses ConnectX-6 drivers.
- The workload needs Ethernet, RoCE, RDMA, or storage acceleration without requiring the newest PCIe interface.
For example, the MCX623106AC-CDAT is a dual-port QSFP56 ConnectX-6 Dx adapter with PCIe Gen4 x16 positioning. Its selection still depends on the server slot wiring, bracket, airflow, cable type, and required security and software features.
When ConnectX-7 May Fit
- The server provides PCIe Gen4 or Gen5.
- The deployment requires higher throughput or newer port configurations.
- The workload is an AI, HPC, storage, or high-density data center fabric.
- The server can meet the adapter’s power and airflow requirements.
- The software stack supports the intended Ethernet or InfiniBand mode.
NVIDIA specifies that some ConnectX-7 adapter cards are designed for data-center servers and require suitable power and airflow. They should not be selected as ordinary desktop expansion cards without checking the server environment.
The MCX713104AC-ADAT, for example, should be evaluated by its exact port configuration, PCIe interface, bracket, firmware, and server requirements rather than by the ConnectX-7 name alone.
Server Compatibility and Installation Checks
Server and Slot Information
Before ordering a PCIe network card, collect the following information:
- Server manufacturer and complete model.
- CPU generation.
- PCIe slot number.
- PCIe generation.
- Electrical lane width.
- Riser or backplane model.
- CPU or NUMA association.
- BIOS version.
- Bifurcation requirements.
Adapter and Network Information
- Exact adapter PID or OPN.
- Ethernet or InfiniBand mode.
- Number of ports.
- Port speed.
- PCIe generation and lane requirement.
- Card height and length.
- Bracket type.
- Power consumption.
- Required airflow.
- Switch model and port speed.
- SFP28, QSFP28, QSFP56, QSFP112, or OSFP requirement.
- DAC, AOC, or optical module requirement.
- Breakout and FEC requirements.
Software Information
- Operating system.
- Kernel or hypervisor version.
- NVIDIA driver or OFED package.
- Firmware requirement.
- SR-IOV or virtual-function support.
- DPDK, RDMA, RoCE, or NVMe over Fabrics requirements.
Common Installation and Performance Problems
| Problem | Possible Cause | First Check |
|---|---|---|
| The card is not detected | Slot, riser, BIOS, power, driver, or firmware issue | Check server documentation, BIOS, and operating-system detection |
| The card negotiates fewer PCIe lanes | Physical x16 slot is electrically x8 or x4 | Check riser, slot wiring, BIOS, and negotiated link width |
| Throughput is below expectation | PCIe bottleneck, CPU, NUMA, FEC, cable, or driver issue | Check PCIe link, driver, firmware, and network counters |
| The adapter overheats | Insufficient server airflow or high-power cable configuration | Check fan profile, airflow direction, and thermal requirements |
| Ethernet or InfiniBand does not operate as expected | Protocol, firmware, driver, or switch-fabric mismatch | Confirm the exact SKU and intended protocol mode |
A complete list of available NVIDIA and Mellanox network adapters can be used as a starting point, but the server model and target workload must be included before a specific card is selected.
Frequently Asked Questions
Final Takeaway
A PCIe network card should be selected as part of a server platform, not as an isolated component.
The most reliable selection process is:
- Confirm the server model and riser.
- Check the slot’s actual electrical lane width.
- Match PCIe generation and adapter requirements.
- Confirm network speed and connector type.
- Decide between Ethernet and InfiniBand.
- Verify power, airflow, and physical clearance.
- Check drivers, firmware, and operating-system support.
- Match the cable, optics, FEC, and switch port.
- Confirm the expected performance with the complete configuration.
A card can fit into a server and still be the wrong choice. The correct adapter is the one whose PCIe interface, network ports, protocol mode, thermal requirements, and software stack all match the server and the workload.