Infiniband

InfiniBand, also called System I/O, is a point-to-point bidirectional serial link that, in storage area networks, connects computers to fabric switches. In addition, it has been used as an interconnect inside computer chassis.

It supports signaling rates of 10, 20, and 40 Gbps, and, as with PCI Express, links can be put in parallel for additional bandwidth. For physical transmission, it supports board-level connection, both active and passive copper (up to 30 meters pending speeds) and fiber-optic cabling (up to 10km)

The InfiniBand Trade association sees it as complementary to Ethernet and Fibre Channel technologies, which they see as appropriate to feed an InfiniBand core switching fabric. It has lower latency than an Ethernet of the same signaling speed. Case-by-case analysis, however, will is required to tell if InfiniBand of a 10 Gbps signaling rate, for example, is more cost-effective than a 40 Gbps Ethernet fabric.

History

It came from the merger of two technologies. Compaq, IBM, and Hewlett-Packard developed the first, Future I/O. Tandem's ServerNet was the ancestor of the Compaq technology. The other half of the merger came from the Next Generation I/O team of Intel, Microsoft and Sun.

InfiniBand was initially deployed as a high performance computing interconnect, bur it was always envisioned as a "system area network", interconnecting computers, network devices, and storage arrays in data centers.

Software

The Open Fabrics Alliance has specified software stacks for it. Some vendors say they can achieve better performance, in the highly specialized supercomputing environment, by using proprietary extensions.

Topologies

For HPC clusters, Fat Tree is the most common topology, but others use torus or mesh topologies, especially when interconnecting thousands of processors.