Ask HN: Looking for bare metal hosting company without virtualized NICs
I've been using spot Amazon c5n metal instances to do DPDK NIC I/O development work. About $1.20/hr/machine. Spot instances are awesome for development work, and on the whole, using Amazon is painless.
See https://github.com/rodgarrison/reinvent
However, AWS ENA NICs are virtualized. They are not bonafide plugged-into-the-PCI-bus cards. Therefore the data pathway to/from the NIC is not direct like it would be at home with the NIC right on the motherboard or in the PCI bus. For a cloud company probably virtualized is the only way to go.
Non-virtualized is what I have in mind for the application I have in mind. To that end, anybody know of a decent HW hosting company provides:
- Some kind of Xeon based processors - regular run of the mill COTS NICs (not virtualized) - Bonus: provides or could provide SolarFlare NICs
In this technical retrospective (2022), @scrubs shares practical architectural lessons learned from running, scaling, or migrating production software under lean constraints.
HN engineers debated the tradeoffs between cloud convenience and raw server economics, database indexing strategies, and resisting premature architectural complexity.
Disclaimer: I'm at AWS, but not on the EC2 or networking teams, I'm not experienced with their stuff, and they'd be rightly upset if you took me as an authority.
That said, SR-IOV may obviate some or all of your concerns. The hypervisor is bypassed for most purposes, kinda like the way DPDK etc bypass the kernel. Microsoft has a nice diagram at https://docs.microsoft.com/en-us/windows-hardware/drivers/network/overview-of-sr-iov-data-paths
What if you just look for “dedicated servers”? Pretty much all of them will fit your needs, whereas “bare metal” will only bring up buzzword hosts.
Shops like nforce or FDCservers might fit your needs better than the super overpriced options most people are listing here, and there are hundreds of other options.
Hi there,
You might check out some smaller providers, look more for dedicated servers vs bare metal and that'll get what you need.
I'm not aware of anyone who sells SolarFlare nics directly but maybe check with knownhost.com as they might be able to do something custom for you.
I cannot imagine trying to do DPDK I/O development in any sort of IaaS. Aren’t you going to have to physically mess around with the cables and Traffic generators all the time? It seems like it would be extremely hard to find one company that offers the NICs you want to test.
What is it that you actually need? Latency? Jitter?
The "good" news is that most bare metal hosts like Equinix and SoftLayer haven't caught up to Nitro so they use traditional NICs.
And if you can use the OFED API, have you tried the Elastic Fabric Adapter?