Future of ccd(4) and raid(4)?

classic Classic list List threaded Threaded
16 messages Options
Reply | Threaded
Open this post in threaded view
|

Future of ccd(4) and raid(4)?

Matthew Dempsky-3
What should be done about ccd(4) and raid(4)?  They both seem
superseded in functionality by softraid(4), which also has much more
developer interest and active development.

Are there any users still using ccd(4) and/or raid(4) and unable to
upgrade to softraid(4)?  Will anyone be up a creek if ccd(4)/raid(4)
were removed?

Reply | Threaded
Open this post in threaded view
|

Re: Future of ccd(4) and raid(4)?

Kenneth R Westerback
On Thu, Jun 23, 2011 at 04:39:19PM -0700, Matthew Dempsky wrote:
> What should be done about ccd(4) and raid(4)?  They both seem
> superseded in functionality by softraid(4), which also has much more
> developer interest and active development.
>
> Are there any users still using ccd(4) and/or raid(4) and unable to
> upgrade to softraid(4)?  Will anyone be up a creek if ccd(4)/raid(4)
> were removed?
>

I use neither but know people claim to be using one or the other,
but mostly raid(4), a.k.a. raidframe. In particular until softraid
can reliably be booted from, I think raid(4) will be useful to have.

.... Ken

Reply | Threaded
Open this post in threaded view
|

Re: Future of ccd(4) and raid(4)?

Matthew Dempsky-3
On Thu, Jun 23, 2011 at 7:29 PM, Kenneth R Westerback
<[hidden email]> wrote:
> I use neither but know people claim to be using one or the other,
> but mostly raid(4), a.k.a. raidframe.

Then it sounds like the solution is to subtly break them so we can
smoke out these claimed users! ;)

> In particular until softraid
> can reliably be booted from, I think raid(4) will be useful to have.

I don't think you can boot from raid(4) either though?

Reply | Threaded
Open this post in threaded view
|

Re: Future of ccd(4) and raid(4)?

Matthew Dempsky-3
In reply to this post by Matthew Dempsky-3
[+misc@, for users not subscribed to tech@]

On Thu, Jun 23, 2011 at 4:39 PM, Matthew Dempsky <[hidden email]> wrote:
> What should be done about ccd(4) and raid(4)?  They both seem
> superseded in functionality by softraid(4), which also has much more
> developer interest and active development.
>
> Are there any users still using ccd(4) and/or raid(4) and unable to
> upgrade to softraid(4)?  Will anyone be up a creek if ccd(4)/raid(4)
> were removed?

Reply | Threaded
Open this post in threaded view
|

Re: Future of ccd(4) and raid(4)?

Kenneth R Westerback
In reply to this post by Matthew Dempsky-3
On Thu, Jun 23, 2011 at 07:44:52PM -0700, Matthew Dempsky wrote:

> On Thu, Jun 23, 2011 at 7:29 PM, Kenneth R Westerback
> <[hidden email]> wrote:
> > I use neither but know people claim to be using one or the other,
> > but mostly raid(4), a.k.a. raidframe.
>
> Then it sounds like the solution is to subtly break them so we can
> smoke out these claimed users! ;)
>
> > In particular until softraid
> > can reliably be booted from, I think raid(4) will be useful to have.
>
> I don't think you can boot from raid(4) either though?

I may be confusing 'boot from' and 'provide root'.

Again, never tried it myself that I can recall.  The important point
being that whatever it can do, softraid must do better. :-)

.... Ken

Reply | Threaded
Open this post in threaded view
|

Re: Future of ccd(4) and raid(4)?

Benny Lofgren
In reply to this post by Matthew Dempsky-3
On 2011-06-24 01.39, Matthew Dempsky wrote:
> What should be done about ccd(4) and raid(4)?  They both seem
> superseded in functionality by softraid(4), which also has much more
> developer interest and active development.

Never used ccd(4) so can't comment on that, but RAIDframe (raid(4)) has
a lot of functionality that is not yet implemented in softraid(4).

It has (for good reason given that softraid(4) is in the works) received
little developer attention and has a few bugs and other shortcomings.

I've tried in the past to address those I've run up against, but I know
there are probably more problems with it than is worth fixing (in
particular I've had problems with very large disks and raid sets) so I
have high hopes for softraid(4) in the future.

> Are there any users still using ccd(4) and/or raid(4) and unable to
> upgrade to softraid(4)?  Will anyone be up a creek if ccd(4)/raid(4)
> were removed?

I for one will be up the worst of creeks if raid(4) was removed, that
would force me to stay on 4.9 until softraid(4) have evolved enough
(which I have no doubt will happen eventually), so please please don't
remove raid(4) just yet. :-)

My wish list for softraid(4) to enable me to say goodbye to RAIDframe
is something like this (not exhaustive and in no particular order):

- More complete RAID support overall, including
  - ability to tune stripe sizes
  - parity initialization / rebuilding, preferrably with background
    mode
  - Hot spare support
  - Better handling of stripe (disk) failures
  - Better handling of recovery from failed stripes (ability to hot
    plug a replacement disk and rebuild on the spot for example)
  - Full stripe writes for perfomance
  - Usable status reporting
  - Stripe on stripe (on stripe ...) support to be able to build
    RAID 0+1 and RAID50 sets, as well as crypto on raid (this may
    work now, haven't tried lately)
  - RAID6 support (way way back in priority though)

- Bootable/rootable raid sets (I know this is close now)

- More consistent sd<n> unit allocation (perhaps this is achievable
  with DUID, I haven't had time to explore that yet)

- Probably other small features as well, that I'll probably think
  of the moment I've sent this mail off...


Regards,
/Benny

--
internetlabbet.se     / work:   +46 8 551 124 80      / "Words must
Benny LC6fgren        /  mobile: +46 70 718 11 90     /   be weighed,
                    /   fax:    +46 8 551 124 89    /    not counted."
                   /    email:  benny -at- internetlabbet.se

Reply | Threaded
Open this post in threaded view
|

Re: Future of ccd(4) and raid(4)?

Joel Sing-3
In reply to this post by Matthew Dempsky-3
On Friday 24 June 2011, Matthew Dempsky wrote:

> On Thu, Jun 23, 2011 at 7:29 PM, Kenneth R Westerback
>
> <[hidden email]> wrote:
> > I use neither but know people claim to be using one or the other,
> > but mostly raid(4), a.k.a. raidframe.
>
> Then it sounds like the solution is to subtly break them so we can
> smoke out these claimed users! ;)
>
> > In particular until softraid
> > can reliably be booted from, I think raid(4) will be useful to have.
>
> I don't think you can boot from raid(4) either though?

grep -i raid /usr/src/sys/arch/sparc64/stand/bootblk/*

No idea if it actually works or if you can on other architectures...
--

    "Reason is not automatic. Those who deny it cannot be conquered by it.
     Do not count on them. Leave them alone." -- Ayn Rand

Reply | Threaded
Open this post in threaded view
|

Re: Future of ccd(4) and raid(4)?

Christian Weisgerber
In reply to this post by Matthew Dempsky-3
Matthew Dempsky <[hidden email]> wrote:

> What should be done about ccd(4) and raid(4)?  They both seem
> superseded in functionality by softraid(4), which also has much more
> developer interest and active development.

Is softraid ready at all?  I thought it was experimental, under
construction, incomplete, don't-use-unless-you-want-to-contribute
code.

--
Christian "naddy" Weisgerber                          [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Future of ccd(4) and raid(4)?

Thordur Bjornsson-2
On Fri, Jun 24, 2011 at 03:38:48PM +0000, Christian Weisgerber wrote:
> Matthew Dempsky <[hidden email]> wrote:
>
> > What should be done about ccd(4) and raid(4)?  They both seem
> > superseded in functionality by softraid(4), which also has much more
> > developer interest and active development.
>
> Is softraid ready at all?  I thought it was experimental, under
> construction, incomplete, don't-use-unless-you-want-to-contribute
> code.
I'm pretty sure it left that state some time ago, in all fairness
I'd sooner trust softraid for my data then ccd/raidframe.

softraid needs some of the bells and whistles raidframe has as
Benny already pointed out, but I think ccd(4) ought to go the
way of the Dodo.

Reply | Threaded
Open this post in threaded view
|

Re: Future of ccd(4) and raid(4)?

Joel Sing-3
In reply to this post by Christian Weisgerber
On Saturday 25 June 2011, Christian Weisgerber wrote:
> Matthew Dempsky <[hidden email]> wrote:
> > What should be done about ccd(4) and raid(4)?  They both seem
> > superseded in functionality by softraid(4), which also has much more
> > developer interest and active development.
>
> Is softraid ready at all?  I thought it was experimental, under
> construction, incomplete, don't-use-unless-you-want-to-contribute
> code.

The RAID 0, RAID 1 and crypto disciplines have been stable for the better part
of the last two years (RAID 1 got rebuild and hotspare support about 2 years
ago). The remaining disciplines (RAID 4, RAID 5, RAID 6, AOE) are
certainly "experimental" and incomplete.
--

    "Reason is not automatic. Those who deny it cannot be conquered by it.
     Do not count on them. Leave them alone." -- Ayn Rand

Reply | Threaded
Open this post in threaded view
|

Re: Future of ccd(4) and raid(4)?

Joel Sing-3
In reply to this post by Benny Lofgren
On Friday 24 June 2011, Benny Lofgren wrote:

> On 2011-06-24 01.39, Matthew Dempsky wrote:
> > What should be done about ccd(4) and raid(4)?  They both seem
> > superseded in functionality by softraid(4), which also has much more
> > developer interest and active development.
>
> Never used ccd(4) so can't comment on that, but RAIDframe (raid(4)) has
> a lot of functionality that is not yet implemented in softraid(4).
>
> It has (for good reason given that softraid(4) is in the works) received
> little developer attention and has a few bugs and other shortcomings.
>
> I've tried in the past to address those I've run up against, but I know
> there are probably more problems with it than is worth fixing (in
> particular I've had problems with very large disks and raid sets) so I
> have high hopes for softraid(4) in the future.
>
> > Are there any users still using ccd(4) and/or raid(4) and unable to
> > upgrade to softraid(4)?  Will anyone be up a creek if ccd(4)/raid(4)
> > were removed?
>
> I for one will be up the worst of creeks if raid(4) was removed, that
> would force me to stay on 4.9 until softraid(4) have evolved enough
> (which I have no doubt will happen eventually), so please please don't
> remove raid(4) just yet. :-)
>
> My wish list for softraid(4) to enable me to say goodbye to RAIDframe
> is something like this (not exhaustive and in no particular order):
>
> - More complete RAID support overall, including
>   - ability to tune stripe sizes

Easily doable - not sure about the benefit since MAXPHYS should be close to
optimal.

>   - parity initialization / rebuilding, preferrably with background
>     mode

The RAID 4/5/6 disciplines are still lacking this (scrubbing), along with
other things.

>   - Hot spare support

We've had that for almost 2 years.

>   - Better handling of stripe (disk) failures

Not sure what you're wanting here.

>   - Better handling of recovery from failed stripes (ability to hot
>     plug a replacement disk and rebuild on the spot for example)

We've had that for almost 2 years as well.

>   - Full stripe writes for perfomance

Meaning?

>   - Usable status reporting

Are you talking about error messages, or bioctl(8) output?

>   - Stripe on stripe (on stripe ...) support to be able to build
>     RAID 0+1 and RAID50 sets, as well as crypto on raid (this may
>     work now, haven't tried lately)

This works, although is not officially supported at this stage.

>   - RAID6 support (way way back in priority though)
>
> - Bootable/rootable raid sets (I know this is close now)
>
> - More consistent sd<n> unit allocation (perhaps this is achievable
>   with DUID, I haven't had time to explore that yet)

sd(4) unit allocation will always be inconsistent and unpredicatable - DUIDs
will let you avoid this entirely.

> - Probably other small features as well, that I'll probably think
>   of the moment I've sent this mail off...
>
>
> Regards,
> /Benny
--

    "Reason is not automatic. Those who deny it cannot be conquered by it.
     Do not count on them. Leave them alone." -- Ayn Rand

Reply | Threaded
Open this post in threaded view
|

Re: sd<n> allocation and umass(4)

Christopher Zimmermann
On 06/24/11 18:46, Joel Sing wrote:
> On Friday 24 June 2011, Benny Lofgren wrote:
>> - More consistent sd<n> unit allocation (perhaps this is achievable
>>   with DUID, I haven't had time to explore that yet)
>
> sd(4) unit allocation will always be inconsistent and unpredicatable
> - DUIDs will let you avoid this entirely.


By the way, is there a way to mount umass(4) devices without looking at
dmesg for the number of the sd<n> device?

Reply | Threaded
Open this post in threaded view
|

Re: sd<n> allocation and umass(4)

Alexander Polakov-2
* Christopher Zimmermann <[hidden email]> [110624 21:24]:

> On 06/24/11 18:46, Joel Sing wrote:
> > On Friday 24 June 2011, Benny Lofgren wrote:
> >> - More consistent sd<n> unit allocation (perhaps this is achievable
> >>   with DUID, I haven't had time to explore that yet)
> >
> > sd(4) unit allocation will always be inconsistent and unpredicatable
> > - DUIDs will let you avoid this entirely.
>
>
> By the way, is there a way to mount umass(4) devices without looking at
> dmesg for the number of the sd<n> device?
>

hotplugd(8)

--
Alexander Polakov | plhk.ru

Reply | Threaded
Open this post in threaded view
|

Re: sd<n> allocation and umass(4)

Christopher Zimmermann
On 06/24/11 19:40, Alexander Polakov wrote:

> * Christopher Zimmermann <[hidden email]> [110624 21:24]:
>> On 06/24/11 18:46, Joel Sing wrote:
>>> On Friday 24 June 2011, Benny Lofgren wrote:
>>>> - More consistent sd<n> unit allocation (perhaps this is achievable
>>>>   with DUID, I haven't had time to explore that yet)
>>>
>>> sd(4) unit allocation will always be inconsistent and unpredicatable
>>> - DUIDs will let you avoid this entirely.
>>
>>
>> By the way, is there a way to mount umass(4) devices without looking at
>> dmesg for the number of the sd<n> device?
>>
>
> hotplugd(8)

That's not what I thought about, but even better - hotplugd the BSD way.
Just great :) Thanks!

Reply | Threaded
Open this post in threaded view
|

Re: sd<n> allocation and umass(4)

Matthias Kilian
On Fri, Jun 24, 2011 at 08:34:15PM +0200, Christopher Zimmermann wrote:
> >> By the way, is there a way to mount umass(4) devices without looking at
> >> dmesg for the number of the sd<n> device?
> >
> > hotplugd(8)
>
> That's not what I thought about, but even better - hotplugd the BSD way.

If you just need the disknames without looking at the dmesg output,
just use sysctl hw.disknames ;-)

Ciao,
        Kili

Reply | Threaded
Open this post in threaded view
|

Re: Future of ccd(4) and raid(4)?

Benny Lofgren
In reply to this post by Joel Sing-3
On 2011-06-24 18.46, Joel Sing wrote:

> On Friday 24 June 2011, Benny Lofgren wrote:
>> On 2011-06-24 01.39, Matthew Dempsky wrote:
>>> What should be done about ccd(4) and raid(4)?  They both seem
>>> superseded in functionality by softraid(4), which also has much more
>>> developer interest and active development.
>>
>> Never used ccd(4) so can't comment on that, but RAIDframe (raid(4)) has
>> a lot of functionality that is not yet implemented in softraid(4).
>>
>> It has (for good reason given that softraid(4) is in the works) received
>> little developer attention and has a few bugs and other shortcomings.
>>
>> I've tried in the past to address those I've run up against, but I know
>> there are probably more problems with it than is worth fixing (in
>> particular I've had problems with very large disks and raid sets) so I
>> have high hopes for softraid(4) in the future.
>>
>>> Are there any users still using ccd(4) and/or raid(4) and unable to
>>> upgrade to softraid(4)?  Will anyone be up a creek if ccd(4)/raid(4)
>>> were removed?
>>
>> I for one will be up the worst of creeks if raid(4) was removed, that
>> would force me to stay on 4.9 until softraid(4) have evolved enough
>> (which I have no doubt will happen eventually), so please please don't
>> remove raid(4) just yet. :-)
>>
>> My wish list for softraid(4) to enable me to say goodbye to RAIDframe
>> is something like this (not exhaustive and in no particular order):
>>
>> - More complete RAID support overall, including
>>   - ability to tune stripe sizes
>
> Easily doable - not sure about the benefit since MAXPHYS should be close to
> optimal.

I'm not sure either, but concerning RAIDframe different use cases
could definitely be made to perform better or worse depending on how
you tuned the stripes, with regard to overall speed, memory (stripe
buffer) usage and so on. Perhaps it isn't an issue with softraid.

>>   - parity initialization / rebuilding, preferrably with background
>>     mode
> The RAID 4/5/6 disciplines are still lacking this (scrubbing), along with
> other things.

That's the main reason I still use RAIDframe, to be able to make very
large RAID5 volumes. For mirroring I use softraid, even without the
ability to boot from the mirror. When that comes online I'll be even
happier. :-)

>>   - Hot spare support
> We've had that for almost 2 years.

I must admit that I haven't read the source, but I see no indication
in the man pages of hot spare support for either of the disciplines?

>>   - Better handling of stripe (disk) failures
> Not sure what you're wanting here.

Maybe it's depending on the underlying hardware and/or their drivers,
but whenever I have a disk failure on a system it often deadlocks or
misbehaves in some other way. I'm on vacation and can't specify this
in more detail, but the next time it happens I'll trace it down more
thorough (unfortunately these are production systems, so most often
they need to be back up again asap, so there's little opportunity to
debug).

Also, when a hot spare is defined for a raid set, it would be great
if it would be possible to automatically kick off a rebuild to that
device when an array is degraded. (Perhaps it is, but I can't find
it in the man pages.)

>>   - Better handling of recovery from failed stripes (ability to hot
>>     plug a replacement disk and rebuild on the spot for example)
> We've had that for almost 2 years as well.

Great! Is that the -R option to bioctl you're referring to?

>>   - Full stripe writes for perfomance
> Meaning?

In a RAID5 environment, a "full stripe write" is an optimization to
avoid the read data + read parity + update data + update parity cycle
normally necessary for each data block write, when there is enough
data buffered for writing to span an entire stripe.

In that case, given a stripe width of n, only n+1 writes of n data
blocks are necessary, compared to 2n reads + 2n writes in the
partial stripe write case.

(The other kind of RAID5 write optimization that makes sense is of
course to use a dedicated RAID controller with battery backed up
buffer cache, which can achieve a similar effect on a block by block
basis, only updating the parity block when it feels like it.)

>>   - Usable status reporting
> Are you talking about error messages, or bioctl(8) output?

Both, I think. :-)  I can elaborate more on that when I'm back in town
and behind the wheel of my systems again.

>>   - Stripe on stripe (on stripe ...) support to be able to build
>>     RAID 0+1 and RAID50 sets, as well as crypto on raid (this may
>>     work now, haven't tried lately)
> This works, although is not officially supported at this stage.

Cool!

>> - More consistent sd<n> unit allocation (perhaps this is achievable
>>   with DUID, I haven't had time to explore that yet)
> sd(4) unit allocation will always be inconsistent and unpredicatable - DUIDs
> will let you avoid this entirely.

Great. I'll look into that as well.


Regards,
/Benny

--
internetlabbet.se     / work:   +46 8 551 124 80      / "Words must
Benny LC6fgren        /  mobile: +46 70 718 11 90     /   be weighed,
                    /   fax:    +46 8 551 124 89    /    not counted."
                   /    email:  benny -at- internetlabbet.se