softraid: adding volumes, CPU requirements, RAID5

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

softraid: adding volumes, CPU requirements, RAID5

Hugo Osvaldo Barrera
Hi,

I'm building myself an openbsd-based fileserver, which will initially
have three disks with softraid in RAID5 mode.

I've three questions regarding softraid:

1) I intend on using a single-core 1.8Ghz Atom processor I have lying
around. Would that limit my performance too much? I'll be using this
fileserver mostly for media (movies/series/music) and some ocassional
backups. Can anyone share what CPU they've used and their experience? (I'm
clarifying my intended usage for the fileserver since I think it's quite
relevant to say if the CPU is or isn't enough).

2) How do I add additional volumes to an already created softraid
volume? I intend on adding additional disks as necessary. Is it possible?

3) The man pages report RAID5 as experimental. I'm curious, why is
this so? Is it just not-very-thoroughly tested, or is there some
missing feature? I read on a 2010 presentation that rebuild was not
implemented yet, is this still so?

Thanks,

--
Hugo Osvaldo Barrera

[demime 1.01d removed an attachment of type application/pgp-signature]

Reply | Threaded
Open this post in threaded view
|

Re: softraid: adding volumes, CPU requirements, RAID5

Nick Holland
On 05/20/13 00:52, Hugo Osvaldo Barrera wrote:

> Hi,
>
> I'm building myself an openbsd-based fileserver, which will initially
> have three disks with softraid in RAID5 mode.
>
> I've three questions regarding softraid:
>
> 1) I intend on using a single-core 1.8Ghz Atom processor I have lying
> around. Would that limit my performance too much? I'll be using this
> fileserver mostly for media (movies/series/music) and some ocassional
> backups. Can anyone share what CPU they've used and their experience? (I'm
> clarifying my intended usage for the fileserver since I think it's quite
> relevant to say if the CPU is or isn't enough).

Wrong question, I think.  More than processor is memory (caching) and
disk interface (ahci rocks), network interface, etc.

> 2) How do I add additional volumes to an already created softraid
> volume? I intend on adding additional disks as necessary. Is it possible?

Not in the way you are likely thinking.
Besides, your Atom board probably has a rather finite amount of
expandability.

> 3) The man pages report RAID5 as experimental. I'm curious, why is
> this so? Is it just not-very-thoroughly tested, or is there some
> missing feature? I read on a 2010 presentation that rebuild was not
> implemented yet, is this still so?

That's really a question you will need to find out though
experimentation before you implement (i.e., you MUST practice this
recovery stuff before going into production), but yes, RAID5 rebuild is
still not there, so I would NOT recommend going this route.

However, a nice little RAID1 system to start, hopefully leaving you two
SATA ports for the next generation/upgrade disks.

Nick.

Reply | Threaded
Open this post in threaded view
|

Re: softraid: adding volumes, CPU requirements, RAID5

Hugo Osvaldo Barrera
On 2013-05-20 07:46, Nick Holland wrote:

> On 05/20/13 00:52, Hugo Osvaldo Barrera wrote:
> > Hi,
> >
> > I'm building myself an openbsd-based fileserver, which will initially
> > have three disks with softraid in RAID5 mode.
> >
> > I've three questions regarding softraid:
> >
> > 1) I intend on using a single-core 1.8Ghz Atom processor I have lying
> > around. Would that limit my performance too much? I'll be using this
> > fileserver mostly for media (movies/series/music) and some ocassional
> > backups. Can anyone share what CPU they've used and their experience?
(I'm
> > clarifying my intended usage for the fileserver since I think it's quite
> > relevant to say if the CPU is or isn't enough).
>
> Wrong question, I think.  More than processor is memory (caching) and
> disk interface (ahci rocks), network interface, etc.

Oh, great, that's good to know. I though processor power was a very
limiting factor in this. Memory and network won't be an issue in this
case.

>
> > 2) How do I add additional volumes to an already created softraid
> > volume? I intend on adding additional disks as necessary. Is it possible?
>
> Not in the way you are likely thinking.
> Besides, your Atom board probably has a rather finite amount of
> expandability.

Hmm. That makes everything far more complicated. :/
Actually, this motherboard I've lying around has four ports, and there
are some other mini-itx one with up to seven ports.

>
> > 3) The man pages report RAID5 as experimental. I'm curious, why is
> > this so? Is it just not-very-thoroughly tested, or is there some
> > missing feature? I read on a 2010 presentation that rebuild was not
> > implemented yet, is this still so?
>
> That's really a question you will need to find out though
> experimentation before you implement (i.e., you MUST practice this
> recovery stuff before going into production), but yes, RAID5 rebuild is
> still not there, so I would NOT recommend going this route.

Yes, indeed. It's way to dangerous and I don't have the storage to create
a dump and rebuild if a disk fails.

>
> However, a nice little RAID1 system to start, hopefully leaving you two
> SATA ports for the next generation/upgrade disks.

Regrettably, I've too much data to take this route. The costs are
prohibitive, and I'd need way too many disks.

>
> Nick.
>

Thanks,

--
Hugo Osvaldo Barrera

[demime 1.01d removed an attachment of type application/pgp-signature]

Reply | Threaded
Open this post in threaded view
|

Re: softraid: adding volumes, CPU requirements, RAID5

Jean-Francois Simon
In reply to this post by Nick Holland
Le 20/05/2013 13:46, Nick Holland a écrit :

> On 05/20/13 00:52, Hugo Osvaldo Barrera wrote:
>> Hi,
>>
>> I'm building myself an openbsd-based fileserver, which will initially
>> have three disks with softraid in RAID5 mode.
>>
>> I've three questions regarding softraid:
>>
>> 1) I intend on using a single-core 1.8Ghz Atom processor I have lying
>> around. Would that limit my performance too much? I'll be using this
>> fileserver mostly for media (movies/series/music) and some ocassional
>> backups. Can anyone share what CPU they've used and their experience? (I'm
>> clarifying my intended usage for the fileserver since I think it's quite
>> relevant to say if the CPU is or isn't enough).
> Wrong question, I think.  More than processor is memory (caching) and
> disk interface (ahci rocks), network interface, etc.
>
>> 2) How do I add additional volumes to an already created softraid
>> volume? I intend on adding additional disks as necessary. Is it possible?
> Not in the way you are likely thinking.
> Besides, your Atom board probably has a rather finite amount of
> expandability.
>
>> 3) The man pages report RAID5 as experimental. I'm curious, why is
>> this so? Is it just not-very-thoroughly tested, or is there some
>> missing feature? I read on a 2010 presentation that rebuild was not
>> implemented yet, is this still so?
> That's really a question you will need to find out though
> experimentation before you implement (i.e., you MUST practice this
> recovery stuff before going into production), but yes, RAID5 rebuild is
> still not there, so I would NOT recommend going this route.
>
> However, a nice little RAID1 system to start, hopefully leaving you two
> SATA ports for the next generation/upgrade disks.
>
> Nick.

"RAID5 rebuild is still not there" Can you please make it more clear what actual state of soft raid can and what it cannot do under RAID 5 ... I'm not so sure to get it, thank you.

J.-F.

Reply | Threaded
Open this post in threaded view
|

Re: softraid: adding volumes, CPU requirements, RAID5

Nick Holland
On 07/02/13 17:07, Jean-Francois Simon wrote:
> Le 20/05/2013 13:46, Nick Holland a écrit :
>> On 05/20/13 00:52, Hugo Osvaldo Barrera wrote:
...

>>> 3) The man pages report RAID5 as experimental. I'm curious, why
>>> is this so? Is it just not-very-thoroughly tested, or is there
>>> some missing feature? I read on a 2010 presentation that rebuild
>>> was not implemented yet, is this still so?
>> That's really a question you will need to find out though
>> experimentation before you implement (i.e., you MUST practice this
>> recovery stuff before going into production), but yes, RAID5
>> rebuild is still not there, so I would NOT recommend going this
>> route.
>>
>> However, a nice little RAID1 system to start, hopefully leaving you
>> two SATA ports for the next generation/upgrade disks.
>>
>> Nick.
>
> "RAID5 rebuild is still not there" Can you please make it more clear
> what actual state of soft raid can and what it cannot do under RAID 5
> ... I'm not so sure to get it, thank you.
>
> J.-F.
>

"RAID5 rebuild is still not there" -> there's no RAID5 rebuild.  I'm not
sure how to make it more clear...

Ok, let's try this...
Today, you take four 1TB disks, and make a 3TB RAID5 volume.  You can do
that.  Works great.

Now, a lot of people might call this "Job Done".  Not me.  The point of
RAID isn't to build complicated systems, but to have the system keep
your butt out of the fire when things go wrong.

Next month, one of those drive fail.  That's ok, RAID5 is designed to
keep your data usable with one drive down.  THAT is the point of RAID.

You pat yourself on the back and say, "I'm glad I am using RAID5".
You replace the failed drive and...
...
um... now what?
You have a three drive degraded RAID5 system with no remaining
redundancy...and a new drive that is currently unused.  You have no
ability to rebuild the function of the failed drive into the new
drive...because the RAID5 rebuild is not there.

Oh, poo.

Your options?  Well,
* you can build a NEW array on other disks (hope you have enough ports
to plug them into), copy the data from the old one to the new one
* you can hope your backup system is perfect, and rebuild the entire
array and reload from backup
* you can hope a second drive doesn't fail in your array... for the life
of the system.

Not much else I can think of.

If you want to play with softraid and raid5, hey, have a blast.  You
want to put critical data on it?  I'd not suggest that.  A job ago, I
had some relatively large chunks of data to hash through to find some
needles of data in and no disks handy that could do it in one
chunk...but I had some big disk array boxes, and a lot of smallish SCSI
disks I could stick in them (and the office space was really cold, so a
bit of heat under my desk was not unappreciated).  I think I did them as
softraid RAID0, but I could have done it as RAID5 with this system --
the data is there just for analysis, not storage.  RAID5 might give me a
few minutes to pull data off that I realized was important only after
the drive failed, but otherwise the loss of data on this array would not
have been catastrophic at all.

Now, anyone who drops important data on any kind of RAID system without
figuring out how to deal with disk (and controller) failures deserves
what they get.  So if I was a nice guy, I'd have said "Go try it out on
some spare hardware and unimportant data and answer your own question",
but being the evil bastard that I am, I'm denying you a very important
learning experience.

Nick.

Reply | Threaded
Open this post in threaded view
|

Re: softraid: adding volumes, CPU requirements, RAID5

Jean-Francois Simon
Le 03/07/2013 00:53, Nick Holland a écrit :

> On 07/02/13 17:07, Jean-Francois Simon wrote:
>> Le 20/05/2013 13:46, Nick Holland a écrit :
>>> On 05/20/13 00:52, Hugo Osvaldo Barrera wrote:
> ...
>>>> 3) The man pages report RAID5 as experimental. I'm curious, why
>>>> is this so? Is it just not-very-thoroughly tested, or is there
>>>> some missing feature? I read on a 2010 presentation that rebuild
>>>> was not implemented yet, is this still so?
>>> That's really a question you will need to find out though
>>> experimentation before you implement (i.e., you MUST practice this
>>> recovery stuff before going into production), but yes, RAID5
>>> rebuild is still not there, so I would NOT recommend going this
>>> route.
>>>
>>> However, a nice little RAID1 system to start, hopefully leaving you
>>> two SATA ports for the next generation/upgrade disks.
>>>
>>> Nick.
>> "RAID5 rebuild is still not there" Can you please make it more clear
>> what actual state of soft raid can and what it cannot do under RAID 5
>> ... I'm not so sure to get it, thank you.
>>
>> J.-F.
>>
> "RAID5 rebuild is still not there" -> there's no RAID5 rebuild.  I'm not
> sure how to make it more clear...
>
> Ok, let's try this...
> Today, you take four 1TB disks, and make a 3TB RAID5 volume.  You can do
> that.  Works great.
>
> Now, a lot of people might call this "Job Done".  Not me.  The point of
> RAID isn't to build complicated systems, but to have the system keep
> your butt out of the fire when things go wrong.
>
> Next month, one of those drive fail.  That's ok, RAID5 is designed to
> keep your data usable with one drive down.  THAT is the point of RAID.
>
> You pat yourself on the back and say, "I'm glad I am using RAID5".
> You replace the failed drive and...
> ...
> um... now what?
> You have a three drive degraded RAID5 system with no remaining
> redundancy...and a new drive that is currently unused.  You have no
> ability to rebuild the function of the failed drive into the new
> drive...because the RAID5 rebuild is not there.
>
> Oh, poo.
>
> Your options?  Well,
> * you can build a NEW array on other disks (hope you have enough ports
> to plug them into), copy the data from the old one to the new one
> * you can hope your backup system is perfect, and rebuild the entire
> array and reload from backup
> * you can hope a second drive doesn't fail in your array... for the life
> of the system.
>
> Not much else I can think of.
>
> If you want to play with softraid and raid5, hey, have a blast.  You
> want to put critical data on it?  I'd not suggest that.  A job ago, I
> had some relatively large chunks of data to hash through to find some
> needles of data in and no disks handy that could do it in one
> chunk...but I had some big disk array boxes, and a lot of smallish SCSI
> disks I could stick in them (and the office space was really cold, so a
> bit of heat under my desk was not unappreciated).  I think I did them as
> softraid RAID0, but I could have done it as RAID5 with this system --
> the data is there just for analysis, not storage.  RAID5 might give me a
> few minutes to pull data off that I realized was important only after
> the drive failed, but otherwise the loss of data on this array would not
> have been catastrophic at all.
>
> Now, anyone who drops important data on any kind of RAID system without
> figuring out how to deal with disk (and controller) failures deserves
> what they get.  So if I was a nice guy, I'd have said "Go try it out on
> some spare hardware and unimportant data and answer your own question",
> but being the evil bastard that I am, I'm denying you a very important
> learning experience.
>
> Nick.

Great, I did not find this information in the manual in fact ... except
considered experimental if that covers this fact.
Thanks again, not a bad guy explanation to me, for sure.

Regards JF

Reply | Threaded
Open this post in threaded view
|

Re: softraid: adding volumes, CPU requirements, RAID5

Hugo Osvaldo Barrera
In reply to this post by Nick Holland
On 2013-07-02 18:53, Nick Holland wrote:

> On 07/02/13 17:07, Jean-Francois Simon wrote:
> > Le 20/05/2013 13:46, Nick Holland a écrit :
> >> On 05/20/13 00:52, Hugo Osvaldo Barrera wrote:
> ...
> >>> 3) The man pages report RAID5 as experimental. I'm curious, why
> >>> is this so? Is it just not-very-thoroughly tested, or is there
> >>> some missing feature? I read on a 2010 presentation that rebuild
> >>> was not implemented yet, is this still so?
> >> That's really a question you will need to find out though
> >> experimentation before you implement (i.e., you MUST practice this
> >> recovery stuff before going into production), but yes, RAID5
> >> rebuild is still not there, so I would NOT recommend going this
> >> route.
> >>
> >> However, a nice little RAID1 system to start, hopefully leaving you
> >> two SATA ports for the next generation/upgrade disks.
> >>
> >> Nick.
> >
> > "RAID5 rebuild is still not there" Can you please make it more clear
> > what actual state of soft raid can and what it cannot do under RAID 5
> > ... I'm not so sure to get it, thank you.
> >
> > J.-F.
> >
>
> "RAID5 rebuild is still not there" -> there's no RAID5 rebuild.  I'm not
> sure how to make it more clear...
>
> Ok, let's try this...
> Today, you take four 1TB disks, and make a 3TB RAID5 volume.  You can do
> that.  Works great.
>
> Now, a lot of people might call this "Job Done".  Not me.  The point of
> RAID isn't to build complicated systems, but to have the system keep
> your butt out of the fire when things go wrong.
>
> Next month, one of those drive fail.  That's ok, RAID5 is designed to
> keep your data usable with one drive down.  THAT is the point of RAID.
>
> You pat yourself on the back and say, "I'm glad I am using RAID5".
> You replace the failed drive and...
> ...
> um... now what?
> You have a three drive degraded RAID5 system with no remaining
> redundancy...and a new drive that is currently unused.  You have no
> ability to rebuild the function of the failed drive into the new
> drive...because the RAID5 rebuild is not there.
>
> Oh, poo.
>
> Your options?  Well,
> * you can build a NEW array on other disks (hope you have enough ports
> to plug them into), copy the data from the old one to the new one
> * you can hope your backup system is perfect, and rebuild the entire
> array and reload from backup
> * you can hope a second drive doesn't fail in your array... for the life
> of the system.
>
> Not much else I can think of.
>
> If you want to play with softraid and raid5, hey, have a blast.  You
> want to put critical data on it?  I'd not suggest that.  A job ago, I
> had some relatively large chunks of data to hash through to find some
> needles of data in and no disks handy that could do it in one
> chunk...but I had some big disk array boxes, and a lot of smallish SCSI
> disks I could stick in them (and the office space was really cold, so a
> bit of heat under my desk was not unappreciated).  I think I did them as
> softraid RAID0, but I could have done it as RAID5 with this system --
> the data is there just for analysis, not storage.  RAID5 might give me a
> few minutes to pull data off that I realized was important only after
> the drive failed, but otherwise the loss of data on this array would not
> have been catastrophic at all.
>
> Now, anyone who drops important data on any kind of RAID system without
> figuring out how to deal with disk (and controller) failures deserves
> what they get.  So if I was a nice guy, I'd have said "Go try it out on
> some spare hardware and unimportant data and answer your own question",
> but being the evil bastard that I am, I'm denying you a very important
> learning experience.
>
> Nick.
>

Indeed! I wanted to make sure I'd know how to rebuild the RAID after it
failed, and that was my initial doubt.

You can be pretty much assured that I didn't use RAID5 in the end (I
don't have anywhere to copy all my stuff while I rebuild the array).

I'm wondering though; is it *so* hard to implement the rebuildage,
or is there simply no interest on behalf of the devs?

--
Hugo Osvaldo Barrera

[demime 1.01d removed an attachment of type application/pgp-signature]

Reply | Threaded
Open this post in threaded view
|

Re: softraid: adding volumes, CPU requirements, RAID5

Boris Goldberg
In reply to this post by Nick Holland
Hello guys,

Tuesday, July 2, 2013, 5:53:04 PM, Nick Holland wrote:

NH> "RAID5 rebuild is still not there" -> there's no RAID5 rebuild.  I'm not
NH> sure how to make it more clear...

NH> Ok, let's try this...
NH> Today, you take four 1TB disks, and make a 3TB RAID5 volume.  You can do
NH> that.  Works great.

NH> Now, a lot of people might call this "Job Done".  Not me.  The point of
NH> RAID isn't to build complicated systems, but to have the system keep
NH> your butt out of the fire when things go wrong.

NH> Next month, one of those drive fail.  That's ok, RAID5 is designed to
NH> keep your data usable with one drive down.  THAT is the point of RAID.

NH> You pat yourself on the back and say, "I'm glad I am using RAID5".
NH> You replace the failed drive and...
NH> ...
NH> um... now what?
NH> You have a three drive degraded RAID5 system with no remaining
NH> redundancy...and a new drive that is currently unused.  You have no
NH> ability to rebuild the function of the failed drive into the new
NH> drive...because the RAID5 rebuild is not there.

NH> Oh, poo.

NH> Your options?  Well,
NH> * you can build a NEW array on other disks (hope you have enough ports
NH> to plug them into), copy the data from the old one to the new one
NH> * you can hope your backup system is perfect, and rebuild the entire
NH> array and reload from backup
NH> * you can hope a second drive doesn't fail in your array... for the life
NH> of the system.

NH> Not much else I can think of.

  If the softraid is so raw yet, why the old good RAIDFrame was removed
starting the 5.2? It works just fine for me. Big volumes rebuilds take a
long while, but it's something working.

--
Best regards,
 Boris                            mailto:[hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: softraid: adding volumes, CPU requirements, RAID5

Nick Holland
On 07/04/13 09:46, Boris Goldberg wrote:
> Hello guys,
...
>   If the softraid is so raw yet, why the old good RAIDFrame was removed
> starting the 5.2? It works just fine for me. Big volumes rebuilds take a
> long while, but it's something working.

That's quite a leap from "RAID 5 is not ready for use" to "softraid is
so raw".  RAID5 is one discipline of several that isn't complete.  RAID0
is ready for use, RAID1 is ready for use, crypto is ready for use.

It is also quite a leap to call old RAIDframe "good".
It was horribly old, unmaintained code, which wasn't well loved by
developers when it was fresh and current.

Your assumptions are wrong.

Nick.

Reply | Threaded
Open this post in threaded view
|

Re: softraid: adding volumes, CPU requirements, RAID5

Boris Goldberg
Hello guys,

Thursday, July 4, 2013, 12:40:50 PM, Nick Holland wrote:

>>   If the softraid is so raw yet, why the old good RAIDFrame was removed
>> starting the 5.2? It works just fine for me. Big volumes rebuilds take a
>> long while, but it's something working.

NH> That's quite a leap from "RAID 5 is not ready for use" to "softraid is
NH> so raw".  RAID5 is one discipline of several that isn't complete.  RAID0
NH> is ready for use, RAID1 is ready for use, crypto is ready for use.

  I've tried to use the nicer word. "Not fully functional" and "raw" are
synonyms.

NH> It is also quite a leap to call old RAIDframe "good".
NH> It was horribly old, unmaintained code, which wasn't well loved by
NH> developers when it was fresh and current.

NH> Your assumptions are wrong.

  I am not assuming, I'm talking from experience. It works. I can install
to it (after a small tweak in the script). I boot from it (after a small
tweak in the code to pick up swap on raid). It continues to work if one
disk fails. It repairs (automatically if you replace the disk and boot -
doing much better job than md from Linux). In other words - it's fully
functional with some flaws. "Fully functional" is the key expression here.

  Is the RAIDFrame old? Yes, but old isn't necessary bad if it's working.
  Did it need a replacement? Yes if no one was willing to maintain it.
  Did you need to kill it *before* the replacement is ready? Definitely no.

  Could you, please, return the RAIDframe support until the softraid is
ready?

--
Best regards,
 Boris                            mailto:[hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: softraid: adding volumes, CPU requirements, RAID5

Tony Abernethy
It works.
Translation:
It has worked (mostly) for me. (A few times)

(Seems like Theo has a good quote about gcc)

Boris Goldberg wrote:
Hello guys,

Thursday, July 4, 2013, 12:40:50 PM, Nick Holland wrote:

>>   If the softraid is so raw yet, why the old good RAIDFrame was removed
>> starting the 5.2? It works just fine for me. Big volumes rebuilds take a
>> long while, but it's something working.

NH> That's quite a leap from "RAID 5 is not ready for use" to "softraid is
NH> so raw".  RAID5 is one discipline of several that isn't complete.  RAID0
NH> is ready for use, RAID1 is ready for use, crypto is ready for use.

  I've tried to use the nicer word. "Not fully functional" and "raw" are
synonyms.

NH> It is also quite a leap to call old RAIDframe "good".
NH> It was horribly old, unmaintained code, which wasn't well loved by
NH> developers when it was fresh and current.

NH> Your assumptions are wrong.

  I am not assuming, I'm talking from experience. It works. I can install
to it (after a small tweak in the script). I boot from it (after a small
tweak in the code to pick up swap on raid). It continues to work if one
disk fails. It repairs (automatically if you replace the disk and boot -
doing much better job than md from Linux). In other words - it's fully
functional with some flaws. "Fully functional" is the key expression here.

  Is the RAIDFrame old? Yes, but old isn't necessary bad if it's working.
  Did it need a replacement? Yes if no one was willing to maintain it.
  Did you need to kill it *before* the replacement is ready? Definitely no.

  Could you, please, return the RAIDframe support until the softraid is
ready?

--
Best regards,
 Boris                            mailto:[hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: softraid: adding volumes, CPU requirements, RAID5

Boris Goldberg
Hello Tony,

Friday, July 5, 2013, 10:09:37 AM, you wrote:

TA> It works.
TA> Translation:
TA> It has worked (mostly) for me. (A few times)

  Don't try to translate from the language you don't understand.
  It's in production on more than a few servers now, and has been for more
than ten years.

--
Best regards,
 Boris                            mailto:[hidden email]