kernel/5215: RLIMIT_RSS appears busted - the kernel does not enforce it.

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

kernel/5215: RLIMIT_RSS appears busted - the kernel does not enforce it.

Bob Beck-4
>Number:         5215
>Category:       kernel
>Synopsis:       Setting RLIMIT_RSS appears to have no effect
>Confidential:   yes
>Severity:       serious
>Priority:       medium
>Responsible:    bugs
>State:          open
>Quarter:        
>Keywords:      
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Fri Aug 25 19:30:01 GMT 2006
>Closed-Date:
>Last-Modified:
>Originator:     SysAdmin
>Release:        3.9, 4.0-current
>Organization:
net
>Environment:
        4.0-current and 3.9, tested on both i386 and amd64
       
        System      : OpenBSD 4.0
        Architecture: OpenBSD.i386
        Machine     : i386
>Description:
        setting RLIMIT_RSS appears to have no effect. a process can then happilyexceed the RSS limit and is not killed.
>How-To-Repeat:
#include <sys/types.h>
#include <sys/time.h>
#include <sys/resource.h>
#include <stdio.h>
#include <stdlib.h>
#include <errno.h>

int main(int, char **);
void print_limit();
void setup_limit();

int
main(int argc, char **argv){
        unsigned char *r[32];
        int n;
        size_t s, mem;

        print_limit();
        setup_limit();
        print_limit();
       
        mem = 0;
        for(n = 31 ; n >= 0  ; n--){
                r[n] = 0; s = 1 << n;
                printf("malloc(0x%x) ",s); fflush(stdout);
                if ((r[n] = malloc(s)) != NULL){
                        mem += s;
                        printf("succeeded - total 0x%x\n", mem);
                        memset(r[n], 13, s);
                        sleep(2);
                } else
                        perror("malloc");
        }
       
        print_limit();
        printf("memory use: 0x%x (%f M)\n", mem,(float)(mem/(1024*1024)));
        printf("hit [enter]\n"); fgetc(stdin);
        for(n=31 ; n==0 ; n--)
                if ((void *)r[n])
                        free((void *)r[n]);
}

void setup_limit(){
        struct rlimit rlp;

        printf("\nSetting Resource Limits:\n");
        rlp.rlim_cur = rlp.rlim_max = 0;
        if (setrlimit(RLIMIT_CORE, &rlp) < 0)
                perror("setrlimit(RLIMIT_CORE, 0)");
        else
                printf("core = 0x%08x\n", rlp.rlim_max);

        rlp.rlim_cur =  rlp.rlim_max = 0x1000000;
        if (setrlimit(RLIMIT_RSS, &rlp) < 0)
                perror("setrlimit(RLIMIT_RSS, 0x1000000)");
        else
                printf("rss = 0x%x\n", rlp.rlim_max);

        rlp.rlim_cur = rlp.rlim_max = RLIM_INFINITY;
        if (setrlimit(RLIMIT_DATA, &rlp) < 0)
                perror("setrlimit(RLIMIT_DATA, RLIM_INFINITY)");
        else
                printf("date = 0x%x\n", rlp.rlim_max);

        return;
}

void print_limit(){
        struct rlimit rlp;

        printf("\nGetting Resource Limits:\n");
        getrlimit(RLIMIT_DATA, &rlp);
        printf("\tRLIMIT_DATA = 0x%x\n", rlp.rlim_cur);
        getrlimit(RLIMIT_RSS, &rlp);
        printf("\tRLIMIT_RSS  = 0x%x\n", rlp.rlim_cur);
        getrlimit(RLIMIT_CORE, &rlp);
        printf("\tRLIMIT_CORE = 0x%x\n", rlp.rlim_cur);
        fflush(stdout);

        return;
}

>Fix:
        Don't know yet. chasing.


>Release-Note:
>Audit-Trail:
>Unformatted:

Reply | Threaded
Open this post in threaded view
|

Re: kernel/5215: RLIMIT_RSS appears busted - the kernel does not enforce it.

beck-7
The following reply was made to PR kernel/5215; it has been noted by GNATS.

From: Bob Beck <[hidden email]>
To: [hidden email], [hidden email]
Cc: [hidden email]
Subject: Re: kernel/5215: RLIMIT_RSS appears busted - the kernel does not enforce it.
Date: Fri, 25 Aug 2006 15:14:09 -0600

 This appears to be a pseudo workaround, but I'm not sure I like it, as
 I'm not sure I'm happy with it being in the scheduler, although I
 can't think of a better place to put the check at the moment, and
 I'm really not sure about the SIGTERM first. should it be SIGSEGV
 like what happens when you exceed the stack limit? SIGABRT? Some
 other signal? This needs discussion. bleah.
 
  -Bob
 
 
 Index: sched_bsd.c
 ===================================================================
 RCS file: /cvs/src/sys/kern/sched_bsd.c,v
 retrieving revision 1.5
 diff -u -r1.5 sched_bsd.c
 --- sched_bsd.c 2005/06/17 22:33:34 1.5
 +++ sched_bsd.c 2006/08/25 21:11:17
 @@ -490,6 +490,20 @@
  }
 
  /*
 + * Check if the process exceeds its maximum resident set size
 + * limit. If over max, kill it.
 + */
 + rlim = &p->p_rlimit[RLIMIT_RSS];
 + if ((rlim_t)(vm_resident_count(p->p_vmspace) * PAGE_SIZE)
 +    >= rlim->rlim_cur) {
 + if ((rlim_t)(vm_resident_count(p->p_vmspace) * PAGE_SIZE)
 +    >= rlim->rlim_max)
 + psignal(p, SIGKILL);
 + else
 + psignal(p, SIGTERM); /* XXX SIGABRT? SIGSEGV? */
 + }
 +
 + /*
  * Process is about to yield the CPU; clear the appropriate
  * scheduling flags.
  */

Reply | Threaded
Open this post in threaded view
|

Re: kernel/5215: RLIMIT_RSS appears busted - the kernel does not enforce it.

Tobias Weingartner-2
On Friday, August 25, Bob Beck wrote:
>
>  This appears to be a pseudo workaround, but I'm not sure I like it, as
>  I'm not sure I'm happy with it being in the scheduler, although I
>  can't think of a better place to put the check at the moment, and
>  I'm really not sure about the SIGTERM first. should it be SIGSEGV
>  like what happens when you exceed the stack limit? SIGABRT? Some
>  other signal? This needs discussion. bleah.

I'm not sure I like this at all... I think it would be "righter" if you
started paging the offending process out if it exceeds its RSS.  But
that has other implications (I/O bandwidth, etc).

--Toby.

Reply | Threaded
Open this post in threaded view
|

Re: kernel/5215: RLIMIT_RSS appears busted - the kernel does not enforce it.

Otto Moerbeek
On Sat, 26 Aug 2006, Tobias Weingartner wrote:

> On Friday, August 25, Bob Beck wrote:
> >
> >  This appears to be a pseudo workaround, but I'm not sure I like it, as
> >  I'm not sure I'm happy with it being in the scheduler, although I
> >  can't think of a better place to put the check at the moment, and
> >  I'm really not sure about the SIGTERM first. should it be SIGSEGV
> >  like what happens when you exceed the stack limit? SIGABRT? Some
> >  other signal? This needs discussion. bleah.
>
> I'm not sure I like this at all... I think it would be "righter" if you
> started paging the offending process out if it exceeds its RSS.  But
> that has other implications (I/O bandwidth, etc).

My memory says this it how it was done in 4.3BSD, i.e. rss was never a
hard limit, just a hint to the pageout daemon.

        -Otto

Reply | Threaded
Open this post in threaded view
|

Re: kernel/5215: RLIMIT_RSS appears busted - the kernel does not enforce it.

Ted Unangst-2
On 8/26/06, Otto Moerbeek <[hidden email]> wrote:

> On Sat, 26 Aug 2006, Tobias Weingartner wrote:
>
> > On Friday, August 25, Bob Beck wrote:
> > >
> > >  This appears to be a pseudo workaround, but I'm not sure I like it, as
> > >  I'm not sure I'm happy with it being in the scheduler, although I
> > >  can't think of a better place to put the check at the moment, and
> > >  I'm really not sure about the SIGTERM first. should it be SIGSEGV
> > >  like what happens when you exceed the stack limit? SIGABRT? Some
> > >  other signal? This needs discussion. bleah.
> >
> > I'm not sure I like this at all... I think it would be "righter" if you
> > started paging the offending process out if it exceeds its RSS.  But
> > that has other implications (I/O bandwidth, etc).
>
> My memory says this it how it was done in 4.3BSD, i.e. rss was never a
> hard limit, just a hint to the pageout daemon.

that sounds more useful.  we already have limits on [virtual] memory.
it can be really hard for a program to control it's physical memory
usage if the limit is less than virtual.

Reply | Threaded
Open this post in threaded view
|

Re: kernel/5215: RLIMIT_RSS appears busted - the kernel does not enforce it.

Chris Kuethe-2
In reply to this post by Bob Beck-4
The following reply was made to PR kernel/5215; it has been noted by GNATS.

From: Chris Kuethe <[hidden email]>
To: Bob Beck <[hidden email]>
Cc: [hidden email], [hidden email]
Subject: Re: kernel/5215: RLIMIT_RSS appears busted - the kernel does not enforce it.
Date: Mon, 28 Aug 2006 10:12:59 -0600 (MDT)

 On Fri, 25 Aug 2006, Bob Beck wrote:
 
 >
 > This appears to be a pseudo workaround, but I'm not sure I like it, as
 > I'm not sure I'm happy with it being in the scheduler, although I
 > can't think of a better place to put the check at the moment, and
 > I'm really not sure about the SIGTERM first. should it be SIGSEGV
 > like what happens when you exceed the stack limit? SIGABRT? Some
 > other signal? This needs discussion. bleah.
 
 SIGTERM is not right - some programs (like httpd) print messages when
 they quit. Perhaps SEGV is the right message. When running out of memory
 the last thing we want is for the program to allocate more on the way
 out.
 
 Put another way: the use of TERM *is* causing problems. We're trying
 it with SEGV.
 
 CK
 
 --
 Chris Kuethe, GCIA: Secure Systems Specialist - U of A AICT
        office: 157 General Services Bldg.    +1.780.492.8135
                chris.kuethe@[pyxis.cns.]ualberta.ca
 
       GDB has a 'break' feature; why doesn't it have 'fix' too?