OpenVPN and 10Gbps ?

[Kanade]

Hello there,

the backbone of our AS is currently at 10Gbps, we got aroud 8-9Gbps in download/upload with 5Gbps guaranteed.
We connected a server to the backbone and this server is currently connected in 10Gigabit Ehernet Full Duplex. But we encounter a small problem.

We want to connect several data centers on the same IP block (yes, IPv4 is expensive, we only have a /24 IPv4 and a /29 IPv6 as LIR at RIPE, so we need to use all of our datacenters servers on the same IP block), and for that, we have chosen to configure an OpenVPN in layer 2. It's efficient, it works perfectly, the only problem is that... the routed speed does not exceed 1Gbps.
We tested everything: changing the encryption used, ciphers, OpenVPN versions on the client/server side, checking the hardware (we mainly use 10-Gigabit Ethernet, and more rarely SFP+), there is really nothing abnormal. Moreover, if I do an iperf between two sites, I exceed 5Gbps without any problem.

Conclusion, I think of a software limitation on the side of OpenVPN. Our configuration is totally classic, there is nothing exceptional about it, but we would like to know if anyone has already encountered this problem.

On the other hand, if it turns out that OpenVPN is limited to layer 2 transfers of 1Gbps maximum, which VPN software would you recommend? I was told about Wireguard but it seems to be layer 3, so it doesn't suit us. And we would preferably like it to be encrypted.

So, if anyone has anything to add...

 

[SharedGrid]

What CPU are you using in this server to run OpenVPN? The encryption and decryption of OpenVPN is very CPU heavy. Do you have any monitoring capability to see how the hardware is performing while trying to push as much bandwidth as possible through the server?

If CPU is OK, you could look into using multiple OpenVPN tunnels. Obviously this means you would need to put something together to handle routing over the parallel connections.

Personally I would be looking to use something else rather than a OpenVPN implementation between servers. Do you own the equipment which connects the uplinks? Depending on what you're using, you could see if it's possible to implement some sort of stretched vlan feature?

 

[skhron]

OpenVPN is not the best software for high speed tunnels. Also, if encryption is a must, IPsec will be definitely a better option in a long-term, having the hardware offload on high end NIC, for example Mellanox ConnectX-6 Dx with crypto unit.

If you don't need any encryption, plain GRETAP (L2) tunnel will be better, however I would like to recommend you to use plain GRE (L3) with DMVPN feature, which is also called NVGRE and is offloadable to mid-range NIC as well.

The cheapest NIC won't have neither of offload, thus you may eventually encounter packet loss due to inefficient usage of a single core of your CPU, not multiple simultaneously as it is supposed to.

 

[Greenhost.cloud]

It seems like OpenVPN may have a software limitation for layer 2 transfers above 1Gbps. Have you considered looking into other VPN software options that may better suit your needs, such as Wireguard?

 

[SharedGrid]

Have you considered looking into other VPN software options that may better suit your needs, such as Wireguard?

It has already been mentioned by the OP that Wireguard is layer 3 only.