So, you grabbed a few Infiniband cards for cheap off eBay and are wondering if they’re any good?
Here’s a quick way to throw some data through them and see. You need two cards and two Linux boxes. In the examples below, I’m using Ubuntu Linux and the interface name of the IB card is simply
iperf is already installed (
apt-get install iperf).
On both boxes, we’ll use IPoIB (IP over Infiniband) to assign a couple temporary IPs and
iperf to run a performance test. It’s important to put the cards into
connected mode and set a large MTU:
On the first box, put
iperf into server mode:
On the second box, throw data at your first one (
-P 2 means to use 2 threads):
So, there you have it. The cards in the above example are pushing a healthy 25 Gbps. It’ll be even faster if using pure Infiniband applications (rather than IPoIB, since more processing is done in the Infiniband hardware, rather than CPUs having to shuffle the TCP/IP stack, among other factors).