Reminder about the X Aperture

classic Classic list List threaded Threaded
38 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Reminder about the X Aperture

Theo de Raadt
I would like to educate people of something which many are not aware
of -- how X works on a modern machine.

Some of our architectures use a tricky and horrid thing to allow X to
run.  This is due to modern PC video card architecture containing a
large quantity of PURE EVIL.  To get around this evil the X developers
have done some rather expedient things, such as directly accessing the
cards via IO registers, directly from userland.  It is hard to see how
they could have done other -- that is how much evil the cards contain.
Most operating systems make accessing these cards trivially easy for X
to do this, but OpenBSD creates a small security barrier through the
use of an "aperture driver", called xf86(4)

        http://www.openbsd.org/cgi-bin/man.cgi?query=xf86

This device exists on i386, amd64, alpha, cats, macppc, and sparc64.
(Other architectures do not need such a thing, since they have less evil).

So let's say X wants to use the "aperture".  Permission to use it is
controlled by the following sysctl(8) variable:

        # sysctl -a machdep.allowaperture
        machdep.allowaperture=0

The three possible values are 0 (aperture disabled), 1 (small window
for very old video cards), or 2 (large window for modern video cards
which have more evil in them).  If you are running X on one of the
architectures listed above, you will have it set to 1 or 2.

The aperture setting cannot be changed once the system has booted
multiuser because the system securelevel locks it.  The initial
setting of this variable however comes from a line in /etc/sysctl.conf.
You will find a line like this (ie. 2, for a fancy video card):

        machdep.allowaperture=2        # See xf86(4)

If you had a machine that was not running X you might see either of
the following (# is a comment character, so that is why these are the
same).

        #machdep.allowaperture=2        # See xf86(4)
        machdep.allowaperture=0         # See xf86(4)

The kernel default is 0.... but for a few releases the OpenBSD install
script has had the question:

        Do you expect to run the X Window System? [yes]

And if you answered "yes" (or just hit return), /etc/sysctl.conf was
changed, so that the setting became "2".

Well, recently we have changed our minds, because we still feel that
the aperture is too dangerous.  And the vendors keep finding creative
ways to squeeze more and more evil into their video cards!

Please be aware that other operating systems don't even have an
aperture device, because they simply let root processes talk to the
video cards (via /dev/mem).  Their X servers also run entirely as
root, while ours is now privilege seperated and running jailed as user
_x11.  Even so, our privilege seperated X server is talking directly
to the IO registers of a video card with much evil in it.  And many
newer video cards are very smart, capable, and thus dangerous. So we
have concerns.

Therefore, after 3.9, that default for the install script question is
being changed to "no".

If you are not using X we recommend ensuring that the aperture is closed.
Please edit /etc/sysctl.conf, change to machdep.allowaperture=0, and
reboot.

Reply | Threaded
Open this post in threaded view
|

Re: Reminder about the X Aperture

Shane J Pearson
Hi Theo,

On 2006.03.14, at 9:41 PM, Theo de Raadt wrote:

> Well, recently we have changed our minds, because we still feel that
> the aperture is too dangerous.  And the vendors keep finding creative
> ways to squeeze more and more evil into their video cards!
>
> Please be aware that other operating systems don't even have an
> aperture device, because they simply let root processes talk to the
> video cards (via /dev/mem).  Their X servers also run entirely as
> root, while ours is now privilege seperated and running jailed as user
> _x11.  Even so, our privilege seperated X server is talking directly
> to the IO registers of a video card with much evil in it.  And many
> newer video cards are very smart, capable, and thus dangerous. So we
> have concerns.

Are these new programable cards capable of reading main memory, which  
OpenBSD would not be able to prevent if machdep.allowaperture were  
set to something other than 0?


Shane

Reply | Threaded
Open this post in threaded view
|

Re: Reminder about the X Aperture

Robert Jacobs-2
In reply to this post by Theo de Raadt
>Therefore, after 3.9, that default for the install script question is
>being changed to "no".

I am sure this will at least double the number of "I installed OpenBSD
and X11 won't work" questions on this mailing list. But it sounds like
a good change in the interest of security.


Thanks,
Robert

Reply | Threaded
Open this post in threaded view
|

Re: Reminder about the X Aperture

Hannah Schroeter
Hi!

On Tue, Mar 14, 2006 at 02:11:13PM +0000, Robert Jacobs wrote:
>>Therefore, after 3.9, that default for the install script question is
>>being changed to "no".

>I am sure this will at least double the number of "I installed OpenBSD
>and X11 won't work" questions on this mailing list. But it sounds like
>a good change in the interest of security.

Just hit them with a copy of the FAQ, specifically question 8.2.

>Thanks,
>Robert

Kind regards,

Hannah.

Reply | Threaded
Open this post in threaded view
|

Re: Reminder about the X Aperture

mvdeventer
In reply to this post by Theo de Raadt
Maybe the team should consider this for the OpenBSD 4.0 artwork.

Maybe with a tagline like "The Admin who could not read" or "Annie get
your Glasses".

OR, (in light of so many users who expect list members to hold their
hands) it could say something about the value of man pages.

I'm sure any new user who sees that on his new CD jewel case will think
twice before posting silly questions.

> -----Original Message-----
> From: Robert Jacobs [mailto:[hidden email]]
> Sent: 14 March 2006 04:11 PM
> To: [hidden email]
> Subject: Re: Reminder about the X Aperture
>
>
> >Therefore, after 3.9, that default for the install script question is
> >being changed to "no".
>
> I am sure this will at least double the number of "I installed OpenBSD
> and X11 won't work" questions on this mailing list. But it sounds like
> a good change in the interest of security.
>
>
> Thanks,
> Robert

[demime 1.01d removed an attachment of type application/x-pkcs7-signature which had a name of smime.p7s]

Reply | Threaded
Open this post in threaded view
|

Re: Reminder about the X Aperture

Theo de Raadt
In reply to this post by Shane J Pearson
> Are these new programable cards capable of reading main memory, which  
> OpenBSD would not be able to prevent if machdep.allowaperture were  
> set to something other than 0?

Yes, they have DMA engines.  If the privilege seperate X server has a
bug, it can still wiggle the IO registers of the card to do DMA to
physical addresses, entirely bypassing system security.

Reply | Threaded
Open this post in threaded view
|

Re: Reminder about the X Aperture

J.C. Roberts-2
In reply to this post by mvdeventer
On Tue, 14 Mar 2006 16:37:17 +0200, "Marius Van Deventer - Umzimkulu"
<[hidden email]> wrote:

>Maybe the team should consider this for the OpenBSD 4.0 artwork.
>
>Maybe with a tagline like "The Admin who could not read" or "Annie get
>your Glasses".
>
>OR, (in light of so many users who expect list members to hold their
>hands) it could say something about the value of man pages.
>
>I'm sure any new user who sees that on his new CD jewel case will think
>twice before posting silly questions.

Maybe I'm just a bit too jaded but...

These days, you see computer security mentioned on the nightly news, yet
there's never any mention of correctness or quality. The result has been
obvious; people have flocked to OpenBSD in hopes of attaining this
supposed "security" thing but they never realized there is a lot of work
and learning required.

The often used OpenBSD phrase "Secure By Default" actually encourages
the lazy attitudes and lack of learning. Worse yet, "Secure By Default"
is fairly misleading since systems are always secured by knowledge,
effort and dedication.

Though he dumbed down the details a lot, before Theo's post on this
thread, how many people had any clue how dangerous X and/or video
drivers (particularly closed source blob drivers) really are? More
importantly, how many people would extend the effort to try solving the
problem?

If a slogan was used that is less buzzword compliant, less inviting and
less misleading, the situation might improve or at least potential users
would be forewarned about the study and effort required.

Personally, I lean towards "Difficult By Default" but probably because
it also applies to my personality. ;-)

kind regards,
jcr

Reply | Threaded
Open this post in threaded view
|

Re: Reminder about the X Aperture

Shane J Pearson
In reply to this post by Theo de Raadt
Thanks Theo,

On 2006.03.15, at 5:22 AM, Theo de Raadt wrote:

>> Are these new programable cards capable of reading main memory, which
>> OpenBSD would not be able to prevent if machdep.allowaperture were
>> set to something other than 0?
>
> Yes, they have DMA engines.  If the privilege seperate X server has a
> bug, it can still wiggle the IO registers of the card to do DMA to
> physical addresses, entirely bypassing system security.

Wow. As if running a binary blob was not bad enough, video card  
binary blobs are suddenly found to be all-powerful.


Shane

Reply | Threaded
Open this post in threaded view
|

Re: Reminder about the X Aperture

Theo de Raadt
> >> Are these new programable cards capable of reading main memory, which
> >> OpenBSD would not be able to prevent if machdep.allowaperture were
> >> set to something other than 0?
> >
> > Yes, they have DMA engines.  If the privilege seperate X server has a
> > bug, it can still wiggle the IO registers of the card to do DMA to
> > physical addresses, entirely bypassing system security.
>
> Wow. As if running a binary blob was not bad enough, video card  
> binary blobs are suddenly found to be all-powerful.

This issue is not about binary blobs for video cards.

Reply | Threaded
Open this post in threaded view
|

Re: Reminder about the X Aperture

Darrin Chandler
In reply to this post by J.C. Roberts-2
J.C. Roberts wrote:

>These days, you see computer security mentioned on the nightly news, yet
>there's never any mention of correctness or quality. The result has been
>obvious; people have flocked to OpenBSD in hopes of attaining this
>supposed "security" thing but they never realized there is a lot of work
>and learning required.
>
>The often used OpenBSD phrase "Secure By Default" actually encourages
>the lazy attitudes and lack of learning. Worse yet, "Secure By Default"
>is fairly misleading since systems are always secured by knowledge,
>effort and dedication.
>  
>
I don't think "Secure by Default" is a bad thing. Neither perceptually
nor in practice. I really like the ability to bring up an OpenBSD box on
a public IP without much concern that it'll get hacked in 30 minutes.

Installing things, even most packages, takes reading and learning. And
that's as it should be. Opening up ports should take *some*
understanding of what you're getting into. Other oz make it too easy to
install services, and encourage the use of webmin, all to the detriment
of the users.

>Though he dumbed down the details a lot, before Theo's post on this
>thread, how many people had any clue how dangerous X and/or video
>drivers (particularly closed source blob drivers) really are? More
>importantly, how many people would extend the effort to try solving the
>problem?
>  
>

I was less aware than I should have been.

>If a slogan was used that is less buzzword compliant, less inviting and
>less misleading, the situation might improve or at least potential users
>would be forewarned about the study and effort required.
>
>Personally, I lean towards "Difficult By Default" but probably because
>it also applies to my personality. ;-)
>  
>

It's not that difficult. It's just not point and click (thank goodness).
The faq, the man pages, and this list all encourage reading, learning,
and understanding what the hell you're doing. I don't see any conflict
whatsoever in that and in Secure by Default.

--
Darrin Chandler            |  Phoenix BSD Users Group
[hidden email]   |  http://bsd.phoenix.az.us/
http://www.stilyagin.com/  |

Reply | Threaded
Open this post in threaded view
|

Re: Reminder about the X Aperture

Andrew Ng
I agreed too. Anyone who choose to use OpenBSD should have a basic
understanding that no system is 100% secure. Even if there is, people
can still attack the weakest link(human) with social engineering.
OpenBSD and other projects allow us a choice against vendors who care
about making more $ than producing secure and reliable products. Nothing
wrong with their approach, I might do likewise if in their shoes.

On Tue, 14 Mar 2006 17:50:31 -0700, "Darrin Chandler"
<[hidden email]> said:

> J.C. Roberts wrote:
>
> >These days, you see computer security mentioned on the nightly news, yet
> >there's never any mention of correctness or quality. The result has been
> >obvious; people have flocked to OpenBSD in hopes of attaining this
> >supposed "security" thing but they never realized there is a lot of work
> >and learning required.
> >
> >The often used OpenBSD phrase "Secure By Default" actually encourages
> >the lazy attitudes and lack of learning. Worse yet, "Secure By Default"
> >is fairly misleading since systems are always secured by knowledge,
> >effort and dedication.
> >  
> >
> I don't think "Secure by Default" is a bad thing. Neither perceptually
> nor in practice. I really like the ability to bring up an OpenBSD box on
> a public IP without much concern that it'll get hacked in 30 minutes.
>
> Installing things, even most packages, takes reading and learning. And
> that's as it should be. Opening up ports should take *some*
> understanding of what you're getting into. Other oz make it too easy to
> install services, and encourage the use of webmin, all to the detriment
> of the users.
>
> >Though he dumbed down the details a lot, before Theo's post on this
> >thread, how many people had any clue how dangerous X and/or video
> >drivers (particularly closed source blob drivers) really are? More
> >importantly, how many people would extend the effort to try solving the
> >problem?
> >  
> >
>
> I was less aware than I should have been.
>
> >If a slogan was used that is less buzzword compliant, less inviting and
> >less misleading, the situation might improve or at least potential users
> >would be forewarned about the study and effort required.
> >
> >Personally, I lean towards "Difficult By Default" but probably because
> >it also applies to my personality. ;-)
> >  
> >
>
> It's not that difficult. It's just not point and click (thank goodness).
> The faq, the man pages, and this list all encourage reading, learning,
> and understanding what the hell you're doing. I don't see any conflict
> whatsoever in that and in Secure by Default.
>
> --
> Darrin Chandler            |  Phoenix BSD Users Group
> [hidden email]   |  http://bsd.phoenix.az.us/
> http://www.stilyagin.com/  |
>
--
  Andrew Ng
  [hidden email]

--
http://www.fastmail.fm - Or how I learned to stop worrying and
                          love email again

Reply | Threaded
Open this post in threaded view
|

Re: Reminder about the X Aperture

JR Dalrymple
In reply to this post by J.C. Roberts-2
J.C. Roberts wrote:

>On Tue, 14 Mar 2006 16:37:17 +0200, "Marius Van Deventer - Umzimkulu"
><[hidden email]> wrote:
>
>  
>
>>Maybe the team should consider this for the OpenBSD 4.0 artwork.
>>
>>Maybe with a tagline like "The Admin who could not read" or "Annie get
>>your Glasses".
>>
>>OR, (in light of so many users who expect list members to hold their
>>hands) it could say something about the value of man pages.
>>
>>I'm sure any new user who sees that on his new CD jewel case will think
>>twice before posting silly questions.
>>    
>>
>
>Maybe I'm just a bit too jaded but...
>
>These days, you see computer security mentioned on the nightly news, yet
>there's never any mention of correctness or quality. The result has been
>obvious; people have flocked to OpenBSD in hopes of attaining this
>supposed "security" thing but they never realized there is a lot of work
>and learning required.
>
>The often used OpenBSD phrase "Secure By Default" actually encourages
>the lazy attitudes and lack of learning. Worse yet, "Secure By Default"
>is fairly misleading since systems are always secured by knowledge,
>effort and dedication.
>
>Though he dumbed down the details a lot, before Theo's post on this
>thread, how many people had any clue how dangerous X and/or video
>drivers (particularly closed source blob drivers) really are? More
>importantly, how many people would extend the effort to try solving the
>problem?
>
>If a slogan was used that is less buzzword compliant, less inviting and
>less misleading, the situation might improve or at least potential users
>would be forewarned about the study and effort required.
>
>Personally, I lean towards "Difficult By Default" but probably because
>it also applies to my personality. ;-)
>
>kind regards,
>jcr
>
>  
>
I think that man afterboot(8) should contain stuff that looks a lot like
Theo's E-mail. Something with a little bit of scare so as to get my
attention, but also something dumbed down to the point that I can read
it. Of course it's a developers' OS, but if it's going to remain secure
in the hands of someone like me stuff like Theo's E-mail will be very
helpful. Moreover Theo's E-mail enticed my desire to learn more about
the inherint problem associated with the "Evil in the video cards" (an
honest "thank you" goes out for that).

Just my $0.02

-JR

Reply | Threaded
Open this post in threaded view
|

Re: Reminder about the X Aperture

J.C. Roberts-2
In reply to this post by Darrin Chandler
On Tue, 14 Mar 2006 17:50:31 -0700, Darrin Chandler
<[hidden email]> wrote:

>>The often used OpenBSD phrase "Secure By Default" actually encourages
>>the lazy attitudes and lack of learning. Worse yet, "Secure By Default"
>>is fairly misleading since systems are always secured by knowledge,
>>effort and dedication.
>>  
>>
>I don't think "Secure by Default" is a bad thing. Neither perceptually
>nor in practice. I really like the ability to bring up an OpenBSD box on
>a public IP without much concern that it'll get hacked in 30 minutes.

It seems I failed to be clear. Having sane default settings is a good
thing. I very much enjoy and appreciate both the utility and the
bragging rights of "Secure By Default" as much (if not more) than most
OpenBSD users.

The sane default settings we enjoy have come from process of looking at
things critically so as to better understand all the implications.

The point I failed to be clear on, is I think the same process of
critical thinking and understanding implications should also be applied
to the rhetoric we use for promotion.

Go ask you mom or a nontechnical friend what she thinks when she hears
an operating system is "secure by default"? Ask her what it implies? Ask
her what she thinks it will require from her?

My mom, in her late 60's, hates computers, hates the web, hates email
and has no interest in learning about computers but none the less, she
uses OpenBSD daily for web access and email. Her replies to those
questions were quite enlightening.

kind regards,
JCR

Reply | Threaded
Open this post in threaded view
|

Re: Reminder about the X Aperture

Alexander Bochmann
In reply to this post by Theo de Raadt
...on Tue, Mar 14, 2006 at 05:41:44PM -0700, Theo de Raadt wrote:

 > > > Yes, they have DMA engines.  If the privilege seperate X server has a
 > > > bug, it can still wiggle the IO registers of the card to do DMA to
 > > > physical addresses, entirely bypassing system security.
 > > Wow. As if running a binary blob was not bad enough, video card  
 > > binary blobs are suddenly found to be all-powerful.
 > This issue is not about binary blobs for video cards.

Using GPU shader programs to read from main
memory was one of the ways mentioned as a
possible attack on the XBox 360 security system
in a presentation at 22C3 last year, though
limited by the system's memory encryption in
that case.

(Could well be contained in some binary blob,
but that's another issue.)

Alex.

Reply | Threaded
Open this post in threaded view
|

Re: Reminder about the X Aperture

Andrew Ng
In reply to this post by J.C. Roberts-2
The current "slogan" for 3.8 is "Free, Functional & Secure". My opinion
is that it presents the project goals well in 4 simple words. It is not
boastful, remember "Nothing is Impossible", or aims to create false
belief/concept. We have our fair share of those, just switch on your TV.

Theo and others did and are still doing a great job in sticking to the
project goals. Didn't know how the "Secure By Default" phrase came
about, I do agree that it can be misleading for your case. You could
refer your mother or nontechnical friends to the Project Goals page(not
too long, 2 pages on my system). Also, I believe Theo and others would
give it some consideration if you can come up with a better "slogan".

Regards

On Tue, 14 Mar 2006 18:40:13 -0800, "J.C. Roberts" <[hidden email]>
said:

> On Tue, 14 Mar 2006 17:50:31 -0700, Darrin Chandler
> <[hidden email]> wrote:
>
> >>The often used OpenBSD phrase "Secure By Default" actually encourages
> >>the lazy attitudes and lack of learning. Worse yet, "Secure By Default"
> >>is fairly misleading since systems are always secured by knowledge,
> >>effort and dedication.
> >>  
> >>
> >I don't think "Secure by Default" is a bad thing. Neither perceptually
> >nor in practice. I really like the ability to bring up an OpenBSD box on
> >a public IP without much concern that it'll get hacked in 30 minutes.
>
> It seems I failed to be clear. Having sane default settings is a good
> thing. I very much enjoy and appreciate both the utility and the
> bragging rights of "Secure By Default" as much (if not more) than most
> OpenBSD users.
>
> The sane default settings we enjoy have come from process of looking at
> things critically so as to better understand all the implications.
>
> The point I failed to be clear on, is I think the same process of
> critical thinking and understanding implications should also be applied
> to the rhetoric we use for promotion.
>
> Go ask you mom or a nontechnical friend what she thinks when she hears
> an operating system is "secure by default"? Ask her what it implies? Ask
> her what she thinks it will require from her?
>
> My mom, in her late 60's, hates computers, hates the web, hates email
> and has no interest in learning about computers but none the less, she
> uses OpenBSD daily for web access and email. Her replies to those
> questions were quite enlightening.
>
> kind regards,
> JCR
>
--
  Andrew Ng
  [hidden email]

--
http://www.fastmail.fm - A fast, anti-spam email service.

Reply | Threaded
Open this post in threaded view
|

Here we go - more nonesence out changein things (was: Reminder about the X Aperture)

Chris Silva-2
Andrew Ng wrote:

> The current "slogan" for 3.8 is "Free, Functional & Secure". My opinion
> is that it presents the project goals well in 4 simple words. It is not
> boastful, remember "Nothing is Impossible", or aims to create false
> belief/concept. We have our fair share of those, just switch on your TV.
>
> Theo and others did and are still doing a great job in sticking to the
> project goals. Didn't know how the "Secure By Default" phrase came
> about, I do agree that it can be misleading for your case. You could
> refer your mother or nontechnical friends to the Project Goals page(not
> too long, 2 pages on my system). Also, I believe Theo and others would
> give it some consideration if you can come up with a better "slogan".

Last I recall - Secure by Default was based on a default installation.
And If I recall, it's stated on the site.  If users can't take the time
to read what's here - they should not run something as complex as ANY Unix.

So, why is everyone out to change everything and anything about the BSD's?

First it was NetBSD and its logo, then FreeBSD went and did something
likewise, now we have this nimbrod suggesting to someone that he/she
ought to come up with a new slogan - and that project would do well to
consider it?!

It the project team feels things are great as is, leave it alone.
Besides, don't you have more to do with your life then to start some
crusade about nothing that needs to be changed?

Life calls - you should answer mate.

Regards,

Chris

Reply | Threaded
Open this post in threaded view
|

Re: Reminder about the X Aperture

Robert Jacobs-2
In reply to this post by Theo de Raadt
I think the slogan "Secure by default" is an excellent description of
OpenBSD.
It implies that it is secure out of the box, and can only be made less
secure
by the user. As soon as you deviate from the "default" you are obviously
losing security points. Just my 2.


Robert

Reply | Threaded
Open this post in threaded view
|

Re: Here we go - more nonesence out changein things (was: Reminder about the X Aperture)

Andrew Ng
In reply to this post by Chris Silva-2
Hi Chris,

cool it. I think you meant nimrod. I said "I believe Theo and others
would give it some consideration ...", I didn't said they must or have
to.

Regards

On Wed, 15 Mar 2006 08:11:49 -0600, "Chris" <[hidden email]> said:

> Andrew Ng wrote:
> > The current "slogan" for 3.8 is "Free, Functional & Secure". My opinion
> > is that it presents the project goals well in 4 simple words. It is not
> > boastful, remember "Nothing is Impossible", or aims to create false
> > belief/concept. We have our fair share of those, just switch on your TV.
> >
> > Theo and others did and are still doing a great job in sticking to the
> > project goals. Didn't know how the "Secure By Default" phrase came
> > about, I do agree that it can be misleading for your case. You could
> > refer your mother or nontechnical friends to the Project Goals page(not
> > too long, 2 pages on my system). Also, I believe Theo and others would
> > give it some consideration if you can come up with a better "slogan".
>
> Last I recall - Secure by Default was based on a default installation.
> And If I recall, it's stated on the site.  If users can't take the time
> to read what's here - they should not run something as complex as ANY
> Unix.
>
> So, why is everyone out to change everything and anything about the
> BSD's?
>
> First it was NetBSD and its logo, then FreeBSD went and did something
> likewise, now we have this nimbrod suggesting to someone that he/she
> ought to come up with a new slogan - and that project would do well to
> consider it?!
>
> It the project team feels things are great as is, leave it alone.
> Besides, don't you have more to do with your life then to start some
> crusade about nothing that needs to be changed?
>
> Life calls - you should answer mate.
>
> Regards,
>
> Chris
>
--
  Andrew Ng
  [hidden email]

--
http://www.fastmail.fm - mmm... Fastmail...

Reply | Threaded
Open this post in threaded view
|

Here we go - more nonesence out changein things (was: Reminder about the X Aperture)

Roger Neth Jr
On 3/15/06, Andrew Ng <[hidden email]> wrote:
> Hi Chris,
>
> cool it. I think you meant nimrod. I said "I believe Theo and others
> snip<

Can anyone guess who nimrod was in history? : )

rogern

John 3:16

Reply | Threaded
Open this post in threaded view
|

Re: Here we go - more nonesence out changein things (was: Reminder about the X Aperture)

Andrew Ng
http://dictionary.reference.com/search?q=nimrod

On Wed, 15 Mar 2006 07:59:26 -0800, "Roger Neth Jr" <[hidden email]>
said:

> On 3/15/06, Andrew Ng <[hidden email]> wrote:
> > Hi Chris,
> >
> > cool it. I think you meant nimrod. I said "I believe Theo and others
> > snip<
>
> Can anyone guess who nimrod was in history? : )
>
> rogern
>
> John 3:16
>
--
  Andrew Ng
  [hidden email]

--
http://www.fastmail.fm - The professional email service

12