IPsec bandwidth perf on APU4C4

classic Classic list List threaded Threaded
18 messages Options
Reply | Threaded
Open this post in threaded view
|

IPsec bandwidth perf on APU4C4

mabi
Hi,

I am currently testing a PC Engines APU4C4 with OpenBSD 6.5 and iked for an IPsec tunnnel between two sites which both have 1 Gbit/s uplink.

Bypassing the IPsec tunnel I get around 500 Mbit/s of bandwidth throughput which is quite satisfying. The bandwidth throughput over my IPsec tunnel achieves a max of 80 Mbit/s which I was sort of expecting with the default encryption settings (auth hmac-sha2-256 enc aes-256).

In order to increase bandwidth throughput over my IPsec tunnel I wanted to know what you guys think is a good compromise between performance and security? I was thinking for example of changing the encryption cipher to aes-128 instead of aes-256 and maybe blowfish? What would you recommend?

Anything else I should be looking at? maybe like a hardware crypto accellerator miniPCI card compatible with the APU4 and OpenBSD?

Cheers,
Mabi





Reply | Threaded
Open this post in threaded view
|

Re: IPsec bandwidth perf on APU4C4

Christian Weisgerber
On 2019-06-10, mabi <[hidden email]> wrote:

> Bypassing the IPsec tunnel I get around 500 Mbit/s of bandwidth throughput which is quite satisfying. The bandwidth throughput over my IPsec tunnel achieves a max of 80 Mbit/s which I was sort of expecting with the default encryption settings (auth hmac-sha2-256 enc aes-256).

It helps to understand that the authentication algorithm can require
as much or more CPU than the encryption.  HMAC-SHA2 is expensive.
On hardware that has AES-NI support, like the APU2 family, AES-GCM
is generally the fastest encryption/authentication combo.

> In order to increase bandwidth throughput over my IPsec tunnel I wanted to know what you guys think is a good compromise between performance and security? I was thinking for example of changing the encryption cipher to aes-128 instead of aes-256 and maybe blowfish? What would you recommend?

AES-128 is good enough, although on the APU2 family with AES-NI it
seems to be only marginally faster than AES-256.

Don't use Blowfish.  It's obsolete.  And its reputation for speed
precedes the introduction of AES.

> Anything else I should be looking at? maybe like a hardware crypto accellerator miniPCI card compatible with the APU4 and OpenBSD?

No, that was 15 years ago.

--
Christian "naddy" Weisgerber                          [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: IPsec bandwidth perf on APU4C4

mabi
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Monday, June 10, 2019 4:49 PM, Christian Weisgerber <[hidden email]> wrote:

> It helps to understand that the authentication algorithm can require
> as much or more CPU than the encryption. HMAC-SHA2 is expensive.
> On hardware that has AES-NI support, like the APU2 family, AES-GCM
> is generally the fastest encryption/authentication combo.

Thanks for the tip regarding the cpu cost of the authentication algorithm. Now I was wondering how do you use the AES-GCM combo? I  can't find any auth or enc parameters mentioning that combo.

Reply | Threaded
Open this post in threaded view
|

Re: IPsec bandwidth perf on APU4C4

Christian Weisgerber
mabi:

> Thanks for the tip regarding the cpu cost of the authentication algorithm. Now I was wondering how do you use the AES-GCM combo? I  can't find any auth or enc parameters mentioning that combo.

enc aes-128-gcm         etc.

--
Christian "naddy" Weisgerber                          [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: IPsec bandwidth perf on APU4C4

mabi
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Monday, June 10, 2019 6:00 PM, Christian Weisgerber <[hidden email]> wrote:

> enc aes-128-gcm etc.

That part for the "enc" parameter makes sense to me but what about the "auth" parameter? Would you keep the default hmac-sha2-256? or which combination with the "enc aes-128-gcm" would be a good fit?

Reply | Threaded
Open this post in threaded view
|

Re: IPsec bandwidth perf on APU4C4

Christian Weisgerber
mabi:

> > enc aes-128-gcm etc.
>
> That part for the "enc" parameter makes sense to me but what about the "auth" parameter?

No "auth".  AES-GCM is an authenticated encryption algorithm, i.e.,
it handles both encryption and authentication at the same time.
Specifying an additional "auth" algorithm doesn't make sense.

--
Christian "naddy" Weisgerber                          [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: IPsec bandwidth perf on APU4C4

mabi
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Monday, June 10, 2019 7:09 PM, Christian Weisgerber <[hidden email]> wrote:

> No "auth". AES-GCM is an authenticated encryption algorithm, i.e.,
> it handles both encryption and authentication at the same time.
> Specifying an additional "auth" algorithm doesn't make sense.

Ahh now it all makes sense with the word "combo", thanks for precising this!

Reply | Threaded
Open this post in threaded view
|

Re: IPsec bandwidth perf on APU4C4

mabi
In reply to this post by Christian Weisgerber
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Monday, June 10, 2019 7:09 PM, Christian Weisgerber <[hidden email]> wrote:

> No "auth". AES-GCM is an authenticated encryption algorithm, i.e.,
> it handles both encryption and authentication at the same time.
> Specifying an additional "auth" algorithm doesn't make sense.

Last question hopefully... Reading the iked.conf man page I conclude that all I need for that is to add to my ikev2 config is the following additional parameter:

childsa enc aes-128-gcm

is this correct?

Reply | Threaded
Open this post in threaded view
|

Re: IPsec bandwidth perf on APU4C4

Christian Weisgerber
mabi:

> Last question hopefully... Reading the iked.conf man page I conclude that all I need for that is to add to my ikev2 config is the following additional parameter:
>
> childsa enc aes-128-gcm

Correct.

--
Christian "naddy" Weisgerber                          [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: IPsec bandwidth perf on APU4C4

mabi
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Tuesday, June 11, 2019 1:04 PM, Christian Weisgerber <[hidden email]> wrote:

> > childsa enc aes-128-gcm
>
> Correct.

For reference I now changed the childsa encryption cipher to aes-128-gcm and get 93 Mbit/s throughput instead of the 80 Mbit/s I saw with aes-256.

Better than nothing but still not quite optimal so I was wondering if anyone had already achieved better IPsec site-to-site bandwidth throughput with a PC Engines APU4 box?

I have a very simple site-2-site IPsec connection which basically is just the following config in my iked.conf file:

ikev2 active esp from $local_ip to $remote_ip local $local_ip peer $remote_ip childsa enc aes-128-gcm srcid $local_ip dstid $remote_ip
ikev2 active esp from $local_network to $remote_network local $local_ip peer $remote_ip childsa enc aes-128-gcm srcid $local_ip dstid $remote_ip

Cheers,
Mabi

Reply | Threaded
Open this post in threaded view
|

Re: IPsec bandwidth perf on APU4C4

Daniel Gracia-2
Those look like reasonable numbers for the given scenario. Improving
your IPsec bandwidth would take more horsepower than an APU box.
Improving site-to-site encrypted VPN speed, asuming two APU boxes,
would require switching from IPsec to something like a WireGuard VPN,
available on -current as a package, but I'm not quite sure how much
performance would be attainable on OpenBSD. Heard >500Mbps on
APU3/Debian combo[1], but once again, don't believe everything you
read on Internet.

Regards and good luck!

[1] https://teklager.se/en/knowledge-base/apu2-vpn-performance/

El mar., 11 jun. 2019 a las 18:10, mabi (<[hidden email]>) escribió:

>
> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> On Tuesday, June 11, 2019 1:04 PM, Christian Weisgerber <[hidden email]> wrote:
>
> > > childsa enc aes-128-gcm
> >
> > Correct.
>
> For reference I now changed the childsa encryption cipher to aes-128-gcm and get 93 Mbit/s throughput instead of the 80 Mbit/s I saw with aes-256.
>
> Better than nothing but still not quite optimal so I was wondering if anyone had already achieved better IPsec site-to-site bandwidth throughput with a PC Engines APU4 box?
>
> I have a very simple site-2-site IPsec connection which basically is just the following config in my iked.conf file:
>
> ikev2 active esp from $local_ip to $remote_ip local $local_ip peer $remote_ip childsa enc aes-128-gcm srcid $local_ip dstid $remote_ip
> ikev2 active esp from $local_network to $remote_network local $local_ip peer $remote_ip childsa enc aes-128-gcm srcid $local_ip dstid $remote_ip
>
> Cheers,
> Mabi
>

Reply | Threaded
Open this post in threaded view
|

Re: IPsec bandwidth perf on APU4C4

mabi
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Wednesday, June 12, 2019 11:34 AM, Daniel Gracia <[hidden email]> wrote:

> Those look like reasonable numbers for the given scenario. Improving
> your IPsec bandwidth would take more horsepower than an APU box.
> Improving site-to-site encrypted VPN speed, asuming two APU boxes,
> would require switching from IPsec to something like a WireGuard VPN,
> available on -current as a package, but I'm not quite sure how much
> performance would be attainable on OpenBSD. Heard >500Mbps on
> APU3/Debian combo[1], but once again, don't believe everything you
> read on Internet.

Interesting article thanks Daniel. As you mention I am also under the impression that with my config I have maxed out that APU4 box.

It's quite hard to believe that on the same box WireGuard with Linux achieves 5x more throughput, even considered that it would fully use all 4 cores, so I do take these numbers cautiously.

For now I'll stick to OpenBSD with iked, it's rock stable, easy to configure and "just" works.

Reply | Threaded
Open this post in threaded view
|

Re: IPsec bandwidth perf on APU4C4

Stuart Henderson
If you're on an old BIOS revision for the APU (more than a couple of
months old), try updating, they have enabled "core performance boost"
which increases speed of a single core if the others are not under
heavy load.

I haven't done network benchmarks but there is a noticable improvement
in some other things (md5 -tt goes from 12 -> 9 seconds).

To update BIOS from OpenBSD, pkg_add flashrom and download the BIOS
version for your board (https://pcengines.github.io/). Go to serial
console and reboot in single-user mode (boot -s), mount -a, and run
"flashrom --programmer internal -w apuX_vXXX.rom". Then reboot back
as normal.

If you'd like to compare benchmarks, the feature can be toggled
from the setup menu in BIOS.

https://blog.3mdeb.com/2019/2019-02-14-enabling-cpb-on-pcengines-apu2/


Reply | Threaded
Open this post in threaded view
|

Re: IPsec bandwidth perf on APU4C4

Stuart Henderson
On 2019-06-12, Stuart Henderson <[hidden email]> wrote:

> If you're on an old BIOS revision for the APU (more than a couple of
> months old), try updating, they have enabled "core performance boost"
> which increases speed of a single core if the others are not under
> heavy load.
>
> I haven't done network benchmarks but there is a noticable improvement
> in some other things (md5 -tt goes from 12 -> 9 seconds).
>
> To update BIOS from OpenBSD, pkg_add flashrom and download the BIOS
> version for your board (https://pcengines.github.io/). Go to serial
> console and reboot in single-user mode (boot -s), mount -a, and run
> "flashrom --programmer internal -w apuX_vXXX.rom". Then reboot back
> as normal.

Apologies I should have said: it should be power cycled at this point,
not just a standard reboot.

> If you'd like to compare benchmarks, the feature can be toggled
> from the setup menu in BIOS.
>
> https://blog.3mdeb.com/2019/2019-02-14-enabling-cpb-on-pcengines-apu2/
>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: IPsec bandwidth perf on APU4C4

mabi
In reply to this post by Stuart Henderson
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Wednesday, June 12, 2019 10:26 PM, Stuart Henderson <[hidden email]> wrote:

> If you're on an old BIOS revision for the APU (more than a couple of
> months old), try updating, they have enabled "core performance boost"
> which increases speed of a single core if the others are not under
> heavy load.
>
> I haven't done network benchmarks but there is a noticable improvement
> in some other things (md5 -tt goes from 12 -> 9 seconds).
>
> To update BIOS from OpenBSD, pkg_add flashrom and download the BIOS
> version for your board (https://pcengines.github.io/). Go to serial
> console and reboot in single-user mode (boot -s), mount -a, and run
> "flashrom --programmer internal -w apuX_vXXX.rom". Then reboot back
> as normal.
>
> If you'd like to compare benchmarks, the feature can be toggled
> from the setup menu in BIOS.
>
> https://blog.3mdeb.com/2019/2019-02-14-enabling-cpb-on-pcengines-apu2/

Thanks Stuart for the hint, that sounds fantastic. I bought my APU4 recently so it has a few months old BIOS (v4.0.24 to be precise) and based on the change log it also seems to include that "core performance boost". I need to reboot and check the BIOS settings first see if this new setting is enabled or not by default. I have the feeling it is not enabled. Anyway I think I will upgrade the BIOS to the latest v4.9.0.6.

Will keep you posted as soon as I check this but right no I can't reboot the box.

Reply | Threaded
Open this post in threaded view
|

Re: IPsec bandwidth perf on APU4C4

Stuart Henderson
On 2019/06/13 20:08, mabi wrote:

> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> On Wednesday, June 12, 2019 10:26 PM, Stuart Henderson <[hidden email]> wrote:
>
> > If you're on an old BIOS revision for the APU (more than a couple of
> > months old), try updating, they have enabled "core performance boost"
> > which increases speed of a single core if the others are not under
> > heavy load.
> >
> > I haven't done network benchmarks but there is a noticable improvement
> > in some other things (md5 -tt goes from 12 -> 9 seconds).
> >
> > To update BIOS from OpenBSD, pkg_add flashrom and download the BIOS
> > version for your board (https://pcengines.github.io/). Go to serial
> > console and reboot in single-user mode (boot -s), mount -a, and run
> > "flashrom --programmer internal -w apuX_vXXX.rom". Then reboot back
> > as normal.
> >
> > If you'd like to compare benchmarks, the feature can be toggled
> > from the setup menu in BIOS.
> >
> > https://blog.3mdeb.com/2019/2019-02-14-enabling-cpb-on-pcengines-apu2/
>
> Thanks Stuart for the hint, that sounds fantastic. I bought my APU4 recently so it has a few months old BIOS (v4.0.24 to be precise) and based on the change log it also seems to include that "core performance boost". I need to reboot and check the BIOS settings first see if this new setting is enabled or not by default. I have the feeling it is not enabled. Anyway I think I will upgrade the BIOS to the latest v4.9.0.6.
>
> Will keep you posted as soon as I check this but right no I can't reboot the box.

4.9.0.6 does have it enabled by default. I'm not sure about the 4.0.x releases
and don't want to reboot mine to check now either :)

Reply | Threaded
Open this post in threaded view
|

Re: IPsec bandwidth perf on APU4C4

Oliver Marugg
On 13 Jun 2019, at 22:46, Stuart Henderson wrote:

> On 2019/06/13 20:08, mabi wrote:
>> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
>> On Wednesday, June 12, 2019 10:26 PM, Stuart Henderson
>> <[hidden email]> wrote:
>>
>>> If you're on an old BIOS revision for the APU (more than a couple of
>>> months old), try updating, they have enabled "core performance
>>> boost"
>>> which increases speed of a single core if the others are not under
>>> heavy load.
>>>
>>> I haven't done network benchmarks but there is a noticable
>>> improvement
>>> in some other things (md5 -tt goes from 12 -> 9 seconds).
>>>
>>> To update BIOS from OpenBSD, pkg_add flashrom and download the BIOS
>>> version for your board (https://pcengines.github.io/). Go to serial
>>> console and reboot in single-user mode (boot -s), mount -a, and run
>>> "flashrom --programmer internal -w apuX_vXXX.rom". Then reboot back
>>> as normal.
>>>
>>> If you'd like to compare benchmarks, the feature can be toggled
>>> from the setup menu in BIOS.
>>>
>>> https://blog.3mdeb.com/2019/2019-02-14-enabling-cpb-on-pcengines-apu2/
>>
>> Thanks Stuart for the hint, that sounds fantastic. I bought my APU4
>> recently so it has a few months old BIOS (v4.0.24 to be precise) and
>> based on the change log it also seems to include that "core
>> performance boost". I need to reboot and check the BIOS settings
>> first see if this new setting is enabled or not by default. I have
>> the feeling it is not enabled. Anyway I think I will upgrade the BIOS
>> to the latest v4.9.0.6.
>>
>> Will keep you posted as soon as I check this but right no I can't
>> reboot the box.
>
> 4.9.0.6 does have it enabled by default. I'm not sure about the 4.0.x
> releases
> and don't want to reboot mine to check now either :)

Beginning with BIOS 4.0.24 CPB is activated for the legacy 4.0.x BIOS
(according to pcengines.github.io). My APU2s and APU3s are running
smooth with 4.9.0.6 with 6.5-syspatched.

Reply | Threaded
Open this post in threaded view
|

Re: IPsec bandwidth perf on APU4C4

mabi
In reply to this post by Stuart Henderson
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Thursday, June 13, 2019 10:46 PM, Stuart Henderson <[hidden email]> wrote:

> 4.9.0.6 does have it enabled by default. I'm not sure about the 4.0.x releases
> and don't want to reboot mine to check now either :)

Finally managed to reboot my firewall box and so I can confirm that on my previous firmware (v4.0.24) the boost option was already enabled by default. I now upgraded to v4.9.0.6 but unfortunately as that boost option was already enabled I do not see any further improvements.

For reference here is the output of a "md5 -tt":

MD5 time trial.  Processing 100000 10000-byte blocks...
Digest = 766a2bb5d24bddae466c572bcabca3ee
Time   = 9.690000 seconds
Speed  = 103199174.406605 bytes/second