Pflow granularity

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Pflow granularity

BARDOU Pierre
Hello,

I sat up NetFlow reporting on a PF firewall, but there seems to be a flaw in the implementation : only global statistics about the flow are given (start time, end time, IP/port source, IP/port dest, bits in both ways, ...). So as an example if somebody establishes an sftp connexion, downloads a file @10 Mbps for 2 mins, then waits 2 min and ends the connexion, all I will see in the netflow report is a 5 Mbps flow, and I will never know that my 10 Mbps link was saturated.

I saw questions about this were already posted on misc@ :
http://openbsd.7691.n7.nabble.com/pflow-packets-before-state-expires-td233952.html

Some diff were even posted :
http://marc.info/?l=openbsd-misc&m=124661838923498&w=2

But it seems they never made their way to the base system.

Is there any way to break-up long flows in fragments, like the Cisco command "ip flow-cache timeout active" does ?

--
Cordialement,
 
Pierre BARDOU
Ingénieur réseau - P2I Infrastructure
05 67 69 71 84

MiPih
12, rue Michel Labrousse - BP93668
31036 TOULOUSE Cedex 1
www.mipih.fr

 Avant d'imprimer cet e-mail, pensons à l'environnement

Reply | Threaded
Open this post in threaded view
|

Re: Pflow granularity

Andy Lemin
I think you might have to try softflowd instead of the built-in sflowd..

These guys had the same problem and moved to softflowd to allow them to
analyse DDOS traffic with netflow..

https://ripe68.ripe.net/presentations/276-DDoS.pdf

Cheers, Andy.


On Mon 02 Jun 2014 14:38:33 BST, BARDOU Pierre wrote:

> Hello,
>
> I sat up NetFlow reporting on a PF firewall, but there seems to be a flaw in the implementation : only global statistics about the flow are given (start time, end time, IP/port source, IP/port dest, bits in both ways, ...). So as an example if somebody establishes an sftp connexion, downloads a file @10 Mbps for 2 mins, then waits 2 min and ends the connexion, all I will see in the netflow report is a 5 Mbps flow, and I will never know that my 10 Mbps link was saturated.
>
> I saw questions about this were already posted on misc@ :
> http://openbsd.7691.n7.nabble.com/pflow-packets-before-state-expires-td233952.html
>
> Some diff were even posted :
> http://marc.info/?l=openbsd-misc&m=124661838923498&w=2
>
> But it seems they never made their way to the base system.
>
> Is there any way to break-up long flows in fragments, like the Cisco command "ip flow-cache timeout active" does ?
>
> --
> Cordialement,
>
> Pierre BARDOU
> Ingénieur réseau - P2I Infrastructure
> 05 67 69 71 84
>
> MiPih
> 12, rue Michel Labrousse - BP93668
> 31036 TOULOUSE Cedex 1
> www.mipih.fr
>
>   Avant d'imprimer cet e-mail, pensons à l'environnement

Reply | Threaded
Open this post in threaded view
|

Re: Pflow granularity

BARDOU Pierre
Hello,

Many thanks for the idea, I didn't knew about softflowd.

But I wonder if it is "production ready" :
* It seems there are no new developments : https://code.google.com/p/softflowd/source/list
* The TODO list is quite long, and has not moved since 2007.
* The counters are not 64 bit, thus flows are limited to 2 Gb
* There is no multiple interface support, all flows are exported with IfIndex 0

I am testing it anyway, it gives me correct graphs with -t maxlife=60.
It's really sad that pflow doesn't have such an option, it would be perfect.

--
Cordialement,
Pierre BARDOU


-----Message d'origine-----
De : Andy [mailto:[hidden email]]
Envoyé : lundi 2 juin 2014 18:01
À : BARDOU Pierre
Cc : [hidden email]
Objet : Re: Pflow granularity

I think you might have to try softflowd instead of the built-in sflowd..

These guys had the same problem and moved to softflowd to allow them to analyse DDOS traffic with netflow..

https://ripe68.ripe.net/presentations/276-DDoS.pdf

Cheers, Andy.


On Mon 02 Jun 2014 14:38:33 BST, BARDOU Pierre wrote:

> Hello,
>
> I sat up NetFlow reporting on a PF firewall, but there seems to be a flaw in the implementation : only global statistics about the flow are given (start time, end time, IP/port source, IP/port dest, bits in both ways, ...). So as an example if somebody establishes an sftp connexion, downloads a file @10 Mbps for 2 mins, then waits 2 min and ends the connexion, all I will see in the netflow report is a 5 Mbps flow, and I will never know that my 10 Mbps link was saturated.
>
> I saw questions about this were already posted on misc@ :
> http://openbsd.7691.n7.nabble.com/pflow-packets-before-state-expires-t
> d233952.html
>
> Some diff were even posted :
> http://marc.info/?l=openbsd-misc&m=124661838923498&w=2
>
> But it seems they never made their way to the base system.
>
> Is there any way to break-up long flows in fragments, like the Cisco command "ip flow-cache timeout active" does ?
>
> --
> Cordialement,
>
> Pierre BARDOU
> Ingénieur réseau - P2I Infrastructure
> 05 67 69 71 84
>
> MiPih
> 12, rue Michel Labrousse - BP93668
> 31036 TOULOUSE Cedex 1
> www.mipih.fr
>
>   Avant d'imprimer cet e-mail, pensons à l'environnement

Reply | Threaded
Open this post in threaded view
|

Re: Pflow granularity

Stuart Henderson
In reply to this post by Andy Lemin
On 2014-06-02, Andy <[hidden email]> wrote:
> I think you might have to try softflowd instead of the built-in sflowd..
>
> These guys had the same problem and moved to softflowd to allow them to
> analyse DDOS traffic with netflow..
>
> https://ripe68.ripe.net/presentations/276-DDoS.pdf

see also the video from UKNOF28, though my understanding was that a
big part of the reason for softflowd was to capture stats from blocked
packets.

Reply | Threaded
Open this post in threaded view
|

Re: Pflow granularity

Tristan PILAT
2014-06-04 16:37 GMT+02:00 Stuart Henderson <[hidden email]>:

> On 2014-06-02, Andy <[hidden email]> wrote:
> > I think you might have to try softflowd instead of the built-in sflowd..
> >
> > These guys had the same problem and moved to softflowd to allow them to
> > analyse DDOS traffic with netflow..
> >
> > https://ripe68.ripe.net/presentations/276-DDoS.pdf
>
> see also the video from UKNOF28, though my understanding was that a
> big part of the reason for softflowd was to capture stats from blocked
> packets.
>
> I noticed the same problems in my reports

Why this diff was not imported ?
http://marc.info/?l=openbsd-misc&m=124661838923498&w=2

After all, that was a great idea.

Reply | Threaded
Open this post in threaded view
|

Re: Pflow granularity

Sebastian Benoit
Tristan PILAT([hidden email]) on 2014.06.24 11:04:35 +0200:

> 2014-06-04 16:37 GMT+02:00 Stuart Henderson <[hidden email]>:
>
> > On 2014-06-02, Andy <[hidden email]> wrote:
> > > I think you might have to try softflowd instead of the built-in sflowd..
> > >
> > > These guys had the same problem and moved to softflowd to allow them to
> > > analyse DDOS traffic with netflow..
> > >
> > > https://ripe68.ripe.net/presentations/276-DDoS.pdf
> >
> > see also the video from UKNOF28, though my understanding was that a
> > big part of the reason for softflowd was to capture stats from blocked
> > packets.
> >
> > I noticed the same problems in my reports
>
> Why this diff was not imported ?

you'll have to ask joerg. :)

however right now some people are working on something similar.

Reply | Threaded
Open this post in threaded view
|

Re: Pflow granularity

Tristan PILAT
2014-06-24 13:50 GMT+02:00 Sebastian Benoit <[hidden email]>:

> Tristan PILAT([hidden email]) on 2014.06.24 11:04:35 +0200:
> > I noticed the same problems in my reports
> >
> > Why this diff was not imported ?
>
> you'll have to ask joerg. :)
>
> however right now some people are working on something similar.
>

Very happy to read that :)

Looking forward to know more about that.