OpenBSD-based ISP

classic Classic list List threaded Threaded
17 messages Options
Reply | Threaded
Open this post in threaded view
|

OpenBSD-based ISP

Juan Guillermo Narvaez
Hello everyone!

I'm relative new using OpenBSD, I have just 4 years using this OS for dhcp
servers.
Today I have the mission of implement this OS in a cablemodem headend, in
my first try I get negative results with this rules:

*pass all flags S/SA*

*#LAN*
*match out log on bge0 inet from 192.168.254.0/24 <http://192.168.254.0/24>
to any nat-to 200.91.35.55*
*pass on bge0 inet from 192.168.254.0/24 <http://192.168.254.0/24> to any
flags S/SA*
*#CPE Network*
*match out on bge0 inet from 172.21.0.0/19 <http://172.21.0.0/19> to any
nat-to 200.91.35.55*
*pass on bge0 inet from 172.21.0.0/19 <http://172.21.0.0/19> to any flags
S/SA*

This is a basic PF that I use for this try, the CPE network has 900 active
customers.
When I put the whole customer network traffic through my OpenBSD router the
traffic tend to fall slowly and the LAN network is really slow too. I read
about a lot of 'tweaks' the high performance configurations but I think
that OpenBSD can handle 400mbps without tweaking.

I'm wrong?
What am I doing bad?

Thank you!




--
J. Guillermo Narvaez
@_aran0id
Reply | Threaded
Open this post in threaded view
|

Re: OpenBSD-based ISP

James Shupe-4
Have you raised states? 10K is the default I believe, the most likely
culprit.

On 8/16/2017 12:55 PM, Juan Guillermo Narvaez wrote:

> Hello everyone!
>
> I'm relative new using OpenBSD, I have just 4 years using this OS for dhcp
> servers.
> Today I have the mission of implement this OS in a cablemodem headend, in
> my first try I get negative results with this rules:
>
> *pass all flags S/SA*
>
> *#LAN*
> *match out log on bge0 inet from 192.168.254.0/24 <http://192.168.254.0/24>
> to any nat-to 200.91.35.55*
> *pass on bge0 inet from 192.168.254.0/24 <http://192.168.254.0/24> to any
> flags S/SA*
> *#CPE Network*
> *match out on bge0 inet from 172.21.0.0/19 <http://172.21.0.0/19> to any
> nat-to 200.91.35.55*
> *pass on bge0 inet from 172.21.0.0/19 <http://172.21.0.0/19> to any flags
> S/SA*
>
> This is a basic PF that I use for this try, the CPE network has 900 active
> customers.
> When I put the whole customer network traffic through my OpenBSD router the
> traffic tend to fall slowly and the LAN network is really slow too. I read
> about a lot of 'tweaks' the high performance configurations but I think
> that OpenBSD can handle 400mbps without tweaking.
>
> I'm wrong?
> What am I doing bad?
>
> Thank you!
>
>
>
>

--
James Shupe, HermeTek
developer/ engineer
BSD/ Linux support & hosting
[hidden email] | www.hermetek.com
Office 5127922525 | Mobile 5122846350


Reply | Threaded
Open this post in threaded view
|

Re: OpenBSD-based ISP

Juan Guillermo Narvaez
Thanks James, now I'm trying with 3K customers and 1M states.

I will comments my results to the list when a finish.

Guillermo.

On Wed, Aug 16, 2017 at 4:01 PM, James Shupe <[hidden email]> wrote:

> Have you raised states? 10K is the default I believe, the most likely
> culprit.
>
> On 8/16/2017 12:55 PM, Juan Guillermo Narvaez wrote:
> > Hello everyone!
> >
> > I'm relative new using OpenBSD, I have just 4 years using this OS for
> dhcp
> > servers.
> > Today I have the mission of implement this OS in a cablemodem headend, in
> > my first try I get negative results with this rules:
> >
> > *pass all flags S/SA*
> >
> > *#LAN*
> > *match out log on bge0 inet from 192.168.254.0/24 <
> http://192.168.254.0/24>
> > to any nat-to 200.91.35.55*
> > *pass on bge0 inet from 192.168.254.0/24 <http://192.168.254.0/24> to
> any
> > flags S/SA*
> > *#CPE Network*
> > *match out on bge0 inet from 172.21.0.0/19 <http://172.21.0.0/19> to any
> > nat-to 200.91.35.55*
> > *pass on bge0 inet from 172.21.0.0/19 <http://172.21.0.0/19> to any
> flags
> > S/SA*
> >
> > This is a basic PF that I use for this try, the CPE network has 900
> active
> > customers.
> > When I put the whole customer network traffic through my OpenBSD router
> the
> > traffic tend to fall slowly and the LAN network is really slow too. I
> read
> > about a lot of 'tweaks' the high performance configurations but I think
> > that OpenBSD can handle 400mbps without tweaking.
> >
> > I'm wrong?
> > What am I doing bad?
> >
> > Thank you!
> >
> >
> >
> >
>
> --
> James Shupe, HermeTek
> developer/ engineer
> BSD/ Linux support & hosting
> [hidden email] | www.hermetek.com
> Office 5127922525 | Mobile 5122846350
>
>
>


--
J. Guillermo Narvaez
@_aran0id
Reply | Threaded
Open this post in threaded view
|

Re: OpenBSD-based ISP

Robert Blacquiere-7
Just some more pointers? Please correct me if I am saying some thing
wrong.

Maybe also good to look at cpu interupts. I'me not sure how good if_bge
today are. I found them in the past "slowly" eating interrupts when
passing lot of small sized traffic. How is your avarage packet size?

I could blast 1000 mbit on if_em interfaces but only 400mbit when very
small packets. So also a thing to check. Also check interface drops.

Regards

Robert

On Wed, Aug 16, 2017 at 04:34:50PM -0300, Juan Guillermo Narvaez wrote:

> Thanks James, now I'm trying with 3K customers and 1M states.
>
> I will comments my results to the list when a finish.
>
> Guillermo.
>
> On Wed, Aug 16, 2017 at 4:01 PM, James Shupe <[hidden email]> wrote:
>
> > Have you raised states? 10K is the default I believe, the most likely
> > culprit.
> >
> > On 8/16/2017 12:55 PM, Juan Guillermo Narvaez wrote:
> > > Hello everyone!
> > >
> > > I'm relative new using OpenBSD, I have just 4 years using this OS for
> > dhcp
> > > servers.
> > > Today I have the mission of implement this OS in a cablemodem headend, in
> > > my first try I get negative results with this rules:
> > >
> > > *pass all flags S/SA*
> > >
> > > *#LAN*
> > > *match out log on bge0 inet from 192.168.254.0/24 <
> > http://192.168.254.0/24>
> > > to any nat-to 200.91.35.55*
> > > *pass on bge0 inet from 192.168.254.0/24 <http://192.168.254.0/24> to
> > any
> > > flags S/SA*
> > > *#CPE Network*
> > > *match out on bge0 inet from 172.21.0.0/19 <http://172.21.0.0/19> to any
> > > nat-to 200.91.35.55*
> > > *pass on bge0 inet from 172.21.0.0/19 <http://172.21.0.0/19> to any
> > flags
> > > S/SA*
> > >
> > > This is a basic PF that I use for this try, the CPE network has 900
> > active
> > > customers.
> > > When I put the whole customer network traffic through my OpenBSD router
> > the
> > > traffic tend to fall slowly and the LAN network is really slow too. I
> > read
> > > about a lot of 'tweaks' the high performance configurations but I think
> > > that OpenBSD can handle 400mbps without tweaking.
> > >
> > > I'm wrong?
> > > What am I doing bad?
> > >
> > > Thank you!
> > >
> > >
> > >
> > >
> >
> > --
> > James Shupe, HermeTek
> > developer/ engineer
> > BSD/ Linux support & hosting
> > [hidden email] | www.hermetek.com
> > Office 5127922525 | Mobile 5122846350
> >
> >
> >
>
>
> --
> J. Guillermo Narvaez
> @_aran0id

Reply | Threaded
Open this post in threaded view
|

Re: OpenBSD-based ISP

Hrvoje Popovski
In reply to this post by Juan Guillermo Narvaez
On 16.8.2017. 19:55, Juan Guillermo Narvaez wrote:

> Hello everyone!
>
> I'm relative new using OpenBSD, I have just 4 years using this OS for dhcp
> servers.
> Today I have the mission of implement this OS in a cablemodem headend, in
> my first try I get negative results with this rules:
>
> *pass all flags S/SA*
>
> *#LAN*
> *match out log on bge0 inet from 192.168.254.0/24 <http://192.168.254.0/24>
> to any nat-to 200.91.35.55*
> *pass on bge0 inet from 192.168.254.0/24 <http://192.168.254.0/24> to any
> flags S/SA*
> *#CPE Network*
> *match out on bge0 inet from 172.21.0.0/19 <http://172.21.0.0/19> to any
> nat-to 200.91.35.55*
> *pass on bge0 inet from 172.21.0.0/19 <http://172.21.0.0/19> to any flags
> S/SA*
>
> This is a basic PF that I use for this try, the CPE network has 900 active
> customers.
> When I put the whole customer network traffic through my OpenBSD router the
> traffic tend to fall slowly and the LAN network is really slow too. I read
> about a lot of 'tweaks' the high performance configurations but I think
> that OpenBSD can handle 400mbps without tweaking.
>
> I'm wrong?
> What am I doing bad?
>
> Thank you!
>
>
>
>

could you send dmesg, cat /etc/sysctl.conf and sysctl | grep ifq

i'm having 2 old Dell R610 with 2 x E5630 cpu and bcm5709 nic's in very
standard pf,carp,pfsync,pflow setup and on top of that i'm logging
everything. boxes are doing cca 100k states and having around 2k hosts
behind them ... of course that i'm running -current :)

Reply | Threaded
Open this post in threaded view
|

Re: OpenBSD-based ISP

Stuart Henderson
In reply to this post by Juan Guillermo Narvaez
On 2017-08-16, Juan Guillermo Narvaez <[hidden email]> wrote:
> *match out on bge0 inet from 172.21.0.0/19 <http://172.21.0.0/19> to any
> nat-to 200.91.35.55*

natting a whole /19 to a single address, especially with the default port range
50001-65535, isn't going to work well.

I'd suggest at least using a dedicated IP (not used for services or locally
sourced connections) with "port 1024:65535", if not multiple IPs.

As already mentioned, check your state limit. Also check sysctl net.inet.ip.ifq,
if there are drops you may need to increase the queue size.


Reply | Threaded
Open this post in threaded view
|

Re: OpenBSD-based ISP

Juan Guillermo Narvaez
In reply to this post by Hrvoje Popovski
# sysctl | grep ifq
net.inet.ip.ifq.len=0
net.inet.ip.ifq.maxlen=1024
net.inet.ip.ifq.drops=46068291
net.inet6.ip6.ifq.len=0
net.inet6.ip6.ifq.maxlen=256
net.inet6.ip6.ifq.drops=0

# cat sysctl.conf
net.inet.ip.forwarding=1
kern.bufcachepercent=90
net.ip.ifq.maxlen=1024



On Wed, Aug 16, 2017 at 5:06 PM, Hrvoje Popovski <[hidden email]> wrote:

> On 16.8.2017. 19:55, Juan Guillermo Narvaez wrote:
> > Hello everyone!
> >
> > I'm relative new using OpenBSD, I have just 4 years using this OS for
> dhcp
> > servers.
> > Today I have the mission of implement this OS in a cablemodem headend, in
> > my first try I get negative results with this rules:
> >
> > *pass all flags S/SA*
> >
> > *#LAN*
> > *match out log on bge0 inet from 192.168.254.0/24 <
> http://192.168.254.0/24>
> > to any nat-to 200.91.35.55*
> > *pass on bge0 inet from 192.168.254.0/24 <http://192.168.254.0/24> to
> any
> > flags S/SA*
> > *#CPE Network*
> > *match out on bge0 inet from 172.21.0.0/19 <http://172.21.0.0/19> to any
> > nat-to 200.91.35.55*
> > *pass on bge0 inet from 172.21.0.0/19 <http://172.21.0.0/19> to any
> flags
> > S/SA*
> >
> > This is a basic PF that I use for this try, the CPE network has 900
> active
> > customers.
> > When I put the whole customer network traffic through my OpenBSD router
> the
> > traffic tend to fall slowly and the LAN network is really slow too. I
> read
> > about a lot of 'tweaks' the high performance configurations but I think
> > that OpenBSD can handle 400mbps without tweaking.
> >
> > I'm wrong?
> > What am I doing bad?
> >
> > Thank you!
> >
> >
> >
> >
>
> could you send dmesg, cat /etc/sysctl.conf and sysctl | grep ifq
>
> i'm having 2 old Dell R610 with 2 x E5630 cpu and bcm5709 nic's in very
> standard pf,carp,pfsync,pflow setup and on top of that i'm logging
> everything. boxes are doing cca 100k states and having around 2k hosts
> behind them ... of course that i'm running -current :)
>
>

--
J. Guillermo Narvaez
@_aran0id

dmesg.tar.xz (6K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: OpenBSD-based ISP

Chris Cappuccio
Juan Guillermo Narvaez [[hidden email]] wrote:
> # sysctl | grep ifq
> net.inet.ip.ifq.len=0
> net.inet.ip.ifq.maxlen=1024
> net.inet.ip.ifq.drops=46068291
> net.inet6.ip6.ifq.len=0
> net.inet6.ip6.ifq.maxlen=256
> net.inet6.ip6.ifq.drops=0
>

The drops are high. You probably want a higher maxlen. I use 8192 on busy
forwarding boxes.

> # cat sysctl.conf
> net.inet.ip.forwarding=1
> kern.bufcachepercent=90
> net.ip.ifq.maxlen=1024
>

You want net.inet.ip.ifq.maxlen=8192 not 'net.ip.ifq.maxlen=1024'

Reply | Threaded
Open this post in threaded view
|

Re: OpenBSD-based ISP

Hrvoje Popovski
On 17.8.2017. 17:13, Chris Cappuccio wrote:

> Juan Guillermo Narvaez [[hidden email]] wrote:
>> # sysctl | grep ifq
>> net.inet.ip.ifq.len=0
>> net.inet.ip.ifq.maxlen=1024
>> net.inet.ip.ifq.drops=46068291
>> net.inet6.ip6.ifq.len=0
>> net.inet6.ip6.ifq.maxlen=256
>> net.inet6.ip6.ifq.drops=0
>>
>
> The drops are high. You probably want a higher maxlen. I use 8192 on busy
> forwarding boxes.
>
>> # cat sysctl.conf
>> net.inet.ip.forwarding=1
>> kern.bufcachepercent=90
>> net.ip.ifq.maxlen=1024
>>
>
> You want net.inet.ip.ifq.maxlen=8192 not 'net.ip.ifq.maxlen=1024'
>

besides what chris told you maybe you could silence pf logging... your
dmesg is full of pf logs, maybe you have pf debuging enabled?

please send cat /var/run/dmesg.boot inline just to see which version of
openbsd your running and on which hardware ...

and set your pf states to some big number.. set limit states 100000 or
something like that ..

and of course run at least openbsd 6.1 or if you brave enough run
-current ....

just side note, openbsd on E5-2643 v2 @ 3.50GHz from around February
2017 had plain forwarding performance of 1.4Mpps and openbsd from today
on same box can forward cca 1.7Mpps ...





Reply | Threaded
Open this post in threaded view
|

Re: OpenBSD-based ISP

Juan Guillermo Narvaez
This is the dmesg.boot.

In pf.conf:
set debug notice

On Thu, Aug 17, 2017 at 3:46 PM, Hrvoje Popovski <[hidden email]> wrote:

> On 17.8.2017. 17:13, Chris Cappuccio wrote:
> > Juan Guillermo Narvaez [[hidden email]] wrote:
> >> # sysctl | grep ifq
> >> net.inet.ip.ifq.len=0
> >> net.inet.ip.ifq.maxlen=1024
> >> net.inet.ip.ifq.drops=46068291
> >> net.inet6.ip6.ifq.len=0
> >> net.inet6.ip6.ifq.maxlen=256
> >> net.inet6.ip6.ifq.drops=0
> >>
> >
> > The drops are high. You probably want a higher maxlen. I use 8192 on busy
> > forwarding boxes.
> >
> >> # cat sysctl.conf
> >> net.inet.ip.forwarding=1
> >> kern.bufcachepercent=90
> >> net.ip.ifq.maxlen=1024
> >>
> >
> > You want net.inet.ip.ifq.maxlen=8192 not 'net.ip.ifq.maxlen=1024'
> >
>
> besides what chris told you maybe you could silence pf logging... your
> dmesg is full of pf logs, maybe you have pf debuging enabled?
>
> please send cat /var/run/dmesg.boot inline just to see which version of
> openbsd your running and on which hardware ...
>
> and set your pf states to some big number.. set limit states 100000 or
> something like that ..
>
> and of course run at least openbsd 6.1 or if you brave enough run
> -current ....
>
> just side note, openbsd on E5-2643 v2 @ 3.50GHz from around February
> 2017 had plain forwarding performance of 1.4Mpps and openbsd from today
> on same box can forward cca 1.7Mpps ...
>
>
>
>
>
>

--
J. Guillermo Narvaez
@_aran0id

dmesgb.tar.bz2 (12K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: OpenBSD-based ISP

Hrvoje Popovski
On 17.8.2017. 21:23, Juan Guillermo Narvaez wrote:
> This is the dmesg.boot.

nice box with nice cpu and interfaces ... :)

if you can, disable Hyper Threading ..

> In pf.conf:
> set debug notice

default is error

when you do all that what people have told you, i would be interested if
you see some performance improvement?



> On Thu, Aug 17, 2017 at 3:46 PM, Hrvoje Popovski <[hidden email]> wrote:
>
>> On 17.8.2017. 17:13, Chris Cappuccio wrote:
>>> Juan Guillermo Narvaez [[hidden email]] wrote:
>>>> # sysctl | grep ifq
>>>> net.inet.ip.ifq.len=0
>>>> net.inet.ip.ifq.maxlen=1024
>>>> net.inet.ip.ifq.drops=46068291
>>>> net.inet6.ip6.ifq.len=0
>>>> net.inet6.ip6.ifq.maxlen=256
>>>> net.inet6.ip6.ifq.drops=0
>>>>
>>>
>>> The drops are high. You probably want a higher maxlen. I use 8192 on busy
>>> forwarding boxes.
>>>
>>>> # cat sysctl.conf
>>>> net.inet.ip.forwarding=1
>>>> kern.bufcachepercent=90
>>>> net.ip.ifq.maxlen=1024
>>>>
>>>
>>> You want net.inet.ip.ifq.maxlen=8192 not 'net.ip.ifq.maxlen=1024'
>>>
>>
>> besides what chris told you maybe you could silence pf logging... your
>> dmesg is full of pf logs, maybe you have pf debuging enabled?
>>
>> please send cat /var/run/dmesg.boot inline just to see which version of
>> openbsd your running and on which hardware ...
>>
>> and set your pf states to some big number.. set limit states 100000 or
>> something like that ..
>>
>> and of course run at least openbsd 6.1 or if you brave enough run
>> -current ....
>>
>> just side note, openbsd on E5-2643 v2 @ 3.50GHz from around February
>> 2017 had plain forwarding performance of 1.4Mpps and openbsd from today
>> on same box can forward cca 1.7Mpps ...
>>
>>
>>
>>
>>
>>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: OpenBSD-based ISP

Juan Guillermo Narvaez
Sure Hrvoje, I'm applying every config and looking the performance
improvement. I will post my final configuration when finish.

Thanks!

On Thu, Aug 17, 2017 at 4:45 PM, Hrvoje Popovski <[hidden email]> wrote:

> On 17.8.2017. 21:23, Juan Guillermo Narvaez wrote:
> > This is the dmesg.boot.
>
> nice box with nice cpu and interfaces ... :)
>
> if you can, disable Hyper Threading ..
>
> > In pf.conf:
> > set debug notice
>
> default is error
>
> when you do all that what people have told you, i would be interested if
> you see some performance improvement?
>
>
>
> > On Thu, Aug 17, 2017 at 3:46 PM, Hrvoje Popovski <[hidden email]> wrote:
> >
> >> On 17.8.2017. 17:13, Chris Cappuccio wrote:
> >>> Juan Guillermo Narvaez [[hidden email]] wrote:
> >>>> # sysctl | grep ifq
> >>>> net.inet.ip.ifq.len=0
> >>>> net.inet.ip.ifq.maxlen=1024
> >>>> net.inet.ip.ifq.drops=46068291
> >>>> net.inet6.ip6.ifq.len=0
> >>>> net.inet6.ip6.ifq.maxlen=256
> >>>> net.inet6.ip6.ifq.drops=0
> >>>>
> >>>
> >>> The drops are high. You probably want a higher maxlen. I use 8192 on
> busy
> >>> forwarding boxes.
> >>>
> >>>> # cat sysctl.conf
> >>>> net.inet.ip.forwarding=1
> >>>> kern.bufcachepercent=90
> >>>> net.ip.ifq.maxlen=1024
> >>>>
> >>>
> >>> You want net.inet.ip.ifq.maxlen=8192 not 'net.ip.ifq.maxlen=1024'
> >>>
> >>
> >> besides what chris told you maybe you could silence pf logging... your
> >> dmesg is full of pf logs, maybe you have pf debuging enabled?
> >>
> >> please send cat /var/run/dmesg.boot inline just to see which version of
> >> openbsd your running and on which hardware ...
> >>
> >> and set your pf states to some big number.. set limit states 100000 or
> >> something like that ..
> >>
> >> and of course run at least openbsd 6.1 or if you brave enough run
> >> -current ....
> >>
> >> just side note, openbsd on E5-2643 v2 @ 3.50GHz from around February
> >> 2017 had plain forwarding performance of 1.4Mpps and openbsd from today
> >> on same box can forward cca 1.7Mpps ...
> >>
> >>
> >>
> >>
> >>
> >>
> >
> >
>
>


--
J. Guillermo Narvaez
@_aran0id
Reply | Threaded
Open this post in threaded view
|

Re: OpenBSD-based ISP

Hrvoje Popovski
On 17.8.2017. 21:56, Juan Guillermo Narvaez wrote:
> Sure Hrvoje, I'm applying every config and looking the performance
> improvement. I will post my final configuration when finish.
>
> Thanks!
>

If you do not filter anything on the internal interfaces in pf.conf you
could skip them

set skip on { lo bge1 vlan123 vlan124 }


> On Thu, Aug 17, 2017 at 4:45 PM, Hrvoje Popovski <[hidden email]> wrote:
>
>> On 17.8.2017. 21:23, Juan Guillermo Narvaez wrote:
>>> This is the dmesg.boot.
>>
>> nice box with nice cpu and interfaces ... :)
>>
>> if you can, disable Hyper Threading ..
>>
>>> In pf.conf:
>>> set debug notice
>>
>> default is error
>>
>> when you do all that what people have told you, i would be interested if
>> you see some performance improvement?
>>
>>
>>
>>> On Thu, Aug 17, 2017 at 3:46 PM, Hrvoje Popovski <[hidden email]> wrote:
>>>
>>>> On 17.8.2017. 17:13, Chris Cappuccio wrote:
>>>>> Juan Guillermo Narvaez [[hidden email]] wrote:
>>>>>> # sysctl | grep ifq
>>>>>> net.inet.ip.ifq.len=0
>>>>>> net.inet.ip.ifq.maxlen=1024
>>>>>> net.inet.ip.ifq.drops=46068291
>>>>>> net.inet6.ip6.ifq.len=0
>>>>>> net.inet6.ip6.ifq.maxlen=256
>>>>>> net.inet6.ip6.ifq.drops=0
>>>>>>
>>>>>
>>>>> The drops are high. You probably want a higher maxlen. I use 8192 on
>> busy
>>>>> forwarding boxes.
>>>>>
>>>>>> # cat sysctl.conf
>>>>>> net.inet.ip.forwarding=1
>>>>>> kern.bufcachepercent=90
>>>>>> net.ip.ifq.maxlen=1024
>>>>>>
>>>>>
>>>>> You want net.inet.ip.ifq.maxlen=8192 not 'net.ip.ifq.maxlen=1024'
>>>>>
>>>>
>>>> besides what chris told you maybe you could silence pf logging... your
>>>> dmesg is full of pf logs, maybe you have pf debuging enabled?
>>>>
>>>> please send cat /var/run/dmesg.boot inline just to see which version of
>>>> openbsd your running and on which hardware ...
>>>>
>>>> and set your pf states to some big number.. set limit states 100000 or
>>>> something like that ..
>>>>
>>>> and of course run at least openbsd 6.1 or if you brave enough run
>>>> -current ....
>>>>
>>>> just side note, openbsd on E5-2643 v2 @ 3.50GHz from around February
>>>> 2017 had plain forwarding performance of 1.4Mpps and openbsd from today
>>>> on same box can forward cca 1.7Mpps ...
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: OpenBSD-based ISP

Juan Guillermo Narvaez
In reply to this post by Stuart Henderson
Stuart,

Where I can set the port range of NAT?

Greetings

On Thu, Aug 17, 2017 at 5:04 AM, Stuart Henderson <[hidden email]>
wrote:

> On 2017-08-16, Juan Guillermo Narvaez <[hidden email]> wrote:
> > *match out on bge0 inet from 172.21.0.0/19 <http://172.21.0.0/19> to any
> > nat-to 200.91.35.55*
>
> natting a whole /19 to a single address, especially with the default port
> range
> 50001-65535, isn't going to work well.
>
> I'd suggest at least using a dedicated IP (not used for services or locally
> sourced connections) with "port 1024:65535", if not multiple IPs.
>
> As already mentioned, check your state limit. Also check sysctl
> net.inet.ip.ifq,
> if there are drops you may need to increase the queue size.
>
>
>


--
J. Guillermo Narvaez
@_aran0id
Reply | Threaded
Open this post in threaded view
|

Re: OpenBSD-based ISP

Stuart Henderson
In reply to this post by Hrvoje Popovski
On 2017-08-17, Hrvoje Popovski <[hidden email]> wrote:

> On 17.8.2017. 21:23, Juan Guillermo Narvaez wrote:
>> This is the dmesg.boot.
>
> nice box with nice cpu and interfaces ... :)
>
> if you can, disable Hyper Threading ..
>
>> In pf.conf:
>> set debug notice
>
> default is error

You might not need to change the queue length after this is fixed. The
higher-than-default debug options are for debugging, and will reduce
performance.

Reply | Threaded
Open this post in threaded view
|

Re: OpenBSD-based ISP

Stuart Henderson
In reply to this post by Juan Guillermo Narvaez
On 2017-08-17, Juan Guillermo Narvaez <[hidden email]> wrote:
> Stuart,
>
> Where I can set the port range of NAT?

pf.conf. "nat-to $address port $low:$high"

Reply | Threaded
Open this post in threaded view
|

Re: OpenBSD-based ISP

Juan Guillermo Narvaez
In reply to this post by Hrvoje Popovski
Hello everyone!

From the last email of this thread (August 17 2017) I'm running 2 OpenBSD
servers with 4x1G interfaces each one, configured with 2 trunk of 2G,
routing and making NAT to more than 3000 customers each one.

Thank you for the help!

On Thu, Aug 17, 2017 at 4:45 PM, Hrvoje Popovski <[hidden email]> wrote:

> On 17.8.2017. 21:23, Juan Guillermo Narvaez wrote:
> > This is the dmesg.boot.
>
> nice box with nice cpu and interfaces ... :)
>
> if you can, disable Hyper Threading ..
>
> > In pf.conf:
> > set debug notice
>
> default is error
>
> when you do all that what people have told you, i would be interested if
> you see some performance improvement?
>
>
>
> > On Thu, Aug 17, 2017 at 3:46 PM, Hrvoje Popovski <[hidden email]> wrote:
> >
> >> On 17.8.2017. 17:13, Chris Cappuccio wrote:
> >>> Juan Guillermo Narvaez [[hidden email]] wrote:
> >>>> # sysctl | grep ifq
> >>>> net.inet.ip.ifq.len=0
> >>>> net.inet.ip.ifq.maxlen=1024
> >>>> net.inet.ip.ifq.drops=46068291
> >>>> net.inet6.ip6.ifq.len=0
> >>>> net.inet6.ip6.ifq.maxlen=256
> >>>> net.inet6.ip6.ifq.drops=0
> >>>>
> >>>
> >>> The drops are high. You probably want a higher maxlen. I use 8192 on
> busy
> >>> forwarding boxes.
> >>>
> >>>> # cat sysctl.conf
> >>>> net.inet.ip.forwarding=1
> >>>> kern.bufcachepercent=90
> >>>> net.ip.ifq.maxlen=1024
> >>>>
> >>>
> >>> You want net.inet.ip.ifq.maxlen=8192 not 'net.ip.ifq.maxlen=1024'
> >>>
> >>
> >> besides what chris told you maybe you could silence pf logging... your
> >> dmesg is full of pf logs, maybe you have pf debuging enabled?
> >>
> >> please send cat /var/run/dmesg.boot inline just to see which version of
> >> openbsd your running and on which hardware ...
> >>
> >> and set your pf states to some big number.. set limit states 100000 or
> >> something like that ..
> >>
> >> and of course run at least openbsd 6.1 or if you brave enough run
> >> -current ....
> >>
> >> just side note, openbsd on E5-2643 v2 @ 3.50GHz from around February
> >> 2017 had plain forwarding performance of 1.4Mpps and openbsd from today
> >> on same box can forward cca 1.7Mpps ...
> >>
> >>
> >>
> >>
> >>
> >>
> >
> >
>
>


--
J. Guillermo Narvaez
Public Key: http://www.nrvz.net/pk/jgnrvz.asc