WAPBL?

classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

WAPBL?

Martijn Rijkeboer
Hi,

Just out of curiosity, what has happend with WAPBL? There were some patches
floating around on tech@ in the last months of 2015, but then it became
quiet. I'm not complaining just curious.

Kind regards,


Martijn Rijkeboer

Reply | Threaded
Open this post in threaded view
|

Re: WAPBL?

Walter Souza
Hi,

I'm not working on it for a while. Sadly I am with no time, but trying
to escape to return. :(

2016-03-26 16:27 GMT-03:00 Martijn Rijkeboer <[hidden email]>:

> Hi,
>
> Just out of curiosity, what has happend with WAPBL? There were some patches
> floating around on tech@ in the last months of 2015, but then it became
> quiet. I'm not complaining just curious.
>
> Kind regards,
>
>
> Martijn Rijkeboer

Reply | Threaded
Open this post in threaded view
|

Re: WAPBL?

Predrag Punosevac-2
In reply to this post by Martijn Rijkeboer
Walter Neto wrote:

>
> Hi,
>
> I'm not working on it for a while. Sadly I am with no time, but trying
> to escape to return. :(
>

This is most regrettable. I was following your work on porting WAPBL and
the correspondence on tech@openbsd with great interest. Do you think
that a help from OpenBSD foundation could enable you to resume the work
on porting WAPBL?


Predrag


> 2016-03-26 16:27 GMT-03:00 Martijn Rijkeboer <[hidden email]>:
> > Hi,
> >
> > Just out of curiosity, what has happend with WAPBL? There were some
> patches
> > floating around on tech@ in the last months of 2015, but then it
> became
> > quiet. I'm not complaining just curious.
> >
> > Kind regards,
> >
> >
> > Martijn Rijkeboer

Reply | Threaded
Open this post in threaded view
|

Re: WAPBL?

Martijn Rijkeboer
In reply to this post by Walter Souza
Hi,

> I'm not working on it for a while. Sadly I am with no time, but trying
> to escape to return. :(

Thanks for the update.

Kind regards,


Martijn Rijkeboer

Reply | Threaded
Open this post in threaded view
|

Re: WAPBL?

Walter Souza
In reply to this post by Predrag Punosevac-2
Hi Predrag,

2016-03-28 22:42 GMT-03:00 Predrag Punosevac <[hidden email]>:

> Walter Neto wrote:
>
>>
>> Hi,
>>
>> I'm not working on it for a while. Sadly I am with no time, but trying
>> to escape to return. :(
>>
>
> This is most regrettable. I was following your work on porting WAPBL and
> the correspondence on tech@openbsd with great interest. Do you think
> that a help from OpenBSD foundation could enable you to resume the work
> on porting WAPBL?
>

It would be perfect, but I need to finish some work commitments first.

>
> Predrag
>
>
>> 2016-03-26 16:27 GMT-03:00 Martijn Rijkeboer <[hidden email]>:
>> > Hi,
>> >
>> > Just out of curiosity, what has happend with WAPBL? There were some
>> patches
>> > floating around on tech@ in the last months of 2015, but then it
>> became
>> > quiet. I'm not complaining just curious.
>> >
>> > Kind regards,
>> >
>> >
>> > Martijn Rijkeboer

Reply | Threaded
Open this post in threaded view
|

Re: WAPBL?

Bob Beck-2
In reply to this post by Martijn Rijkeboer
I have more up to date versions of these patches around here.

The problem with them is that fundamentally, the WAPBL implementation
as it is assumes that it may infinitely steal
buffers from the buffer cache and hold onto them indefinitely - and it
assumes it can always get buffers from it. While the patch as it sits
may "work" in the "happy case" on many people's machines, as it sits
today it is dangerous and can lock up your machine and corrupt things
in low memory situations.

Basically in order to progres WAPBL (renamed "FFS Journalling" here)
needs to have a mechanism added to allow
it be told "no it can't have a buffer" and let it deal with it
correctly.  The first part is done, the latter part is complex.


On Sat, Mar 26, 2016 at 1:27 PM, Martijn Rijkeboer <[hidden email]> wrote:

> Hi,
>
> Just out of curiosity, what has happend with WAPBL? There were some patches
> floating around on tech@ in the last months of 2015, but then it became
> quiet. I'm not complaining just curious.
>
> Kind regards,
>
>
> Martijn Rijkeboer

Reply | Threaded
Open this post in threaded view
|

Re: WAPBL?

Amit Kulkarni-5
I see the writes are not being done to disk in case of a simple cvs update,
and the machine locks up for a solid couple of minutes afterwards also.
This happens in a dual CPU config with plenty of free memory, even with
stefan, mpi and kettenis recent diffs. For a curious kernel reader, where
could the bug(s) be? in amap, uvm/buffer cache, rthreads???

Thanks in advance


On Fri, Apr 1, 2016 at 9:06 AM, Bob Beck <[hidden email]> wrote:

> I have more up to date versions of these patches around here.
>
> The problem with them is that fundamentally, the WAPBL implementation
> as it is assumes that it may infinitely steal
> buffers from the buffer cache and hold onto them indefinitely - and it
> assumes it can always get buffers from it. While the patch as it sits
> may "work" in the "happy case" on many people's machines, as it sits
> today it is dangerous and can lock up your machine and corrupt things
> in low memory situations.
>
> Basically in order to progres WAPBL (renamed "FFS Journalling" here)
> needs to have a mechanism added to allow
> it be told "no it can't have a buffer" and let it deal with it
> correctly.  The first part is done, the latter part is complex.
>
>
> On Sat, Mar 26, 2016 at 1:27 PM, Martijn Rijkeboer <[hidden email]>
> wrote:
> > Hi,
> >
> > Just out of curiosity, what has happend with WAPBL? There were some
> patches
> > floating around on tech@ in the last months of 2015, but then it became
> > quiet. I'm not complaining just curious.
> >
> > Kind regards,
> >
> >
> > Martijn Rijkeboer

Reply | Threaded
Open this post in threaded view
|

Re: WAPBL?

Bob Beck-2
I would hazard a guess that if you are running a random diff, the
problem is with the diff you are running - not those other things.

On Fri, Apr 1, 2016 at 9:30 AM, Amit Kulkarni <[hidden email]> wrote:

> I see the writes are not being done to disk in case of a simple cvs update,
> and the machine locks up for a solid couple of minutes afterwards also. This
> happens in a dual CPU config with plenty of free memory, even with stefan,
> mpi and kettenis recent diffs. For a curious kernel reader, where could the
> bug(s) be? in amap, uvm/buffer cache, rthreads???
>
> Thanks in advance
>
>
> On Fri, Apr 1, 2016 at 9:06 AM, Bob Beck <[hidden email]> wrote:
>>
>> I have more up to date versions of these patches around here.
>>
>> The problem with them is that fundamentally, the WAPBL implementation
>> as it is assumes that it may infinitely steal
>> buffers from the buffer cache and hold onto them indefinitely - and it
>> assumes it can always get buffers from it. While the patch as it sits
>> may "work" in the "happy case" on many people's machines, as it sits
>> today it is dangerous and can lock up your machine and corrupt things
>> in low memory situations.
>>
>> Basically in order to progres WAPBL (renamed "FFS Journalling" here)
>> needs to have a mechanism added to allow
>> it be told "no it can't have a buffer" and let it deal with it
>> correctly.  The first part is done, the latter part is complex.
>>
>>
>> On Sat, Mar 26, 2016 at 1:27 PM, Martijn Rijkeboer <[hidden email]>
>> wrote:
>> > Hi,
>> >
>> > Just out of curiosity, what has happend with WAPBL? There were some
>> > patches
>> > floating around on tech@ in the last months of 2015, but then it became
>> > quiet. I'm not complaining just curious.
>> >
>> > Kind regards,
>> >
>> >
>> > Martijn Rijkeboer

Reply | Threaded
Open this post in threaded view
|

Re: WAPBL?

Amit Kulkarni-5
Nope, my cvs tree is clean. i only applied those diffs since they are small.

On Fri, Apr 1, 2016 at 10:56 AM, Bob Beck <[hidden email]> wrote:

> I would hazard a guess that if you are running a random diff, the
> problem is with the diff you are running - not those other things.
>
> On Fri, Apr 1, 2016 at 9:30 AM, Amit Kulkarni <[hidden email]> wrote:
> > I see the writes are not being done to disk in case of a simple cvs
> update,
> > and the machine locks up for a solid couple of minutes afterwards also.
> This
> > happens in a dual CPU config with plenty of free memory, even with
> stefan,
> > mpi and kettenis recent diffs. For a curious kernel reader, where could
> the
> > bug(s) be? in amap, uvm/buffer cache, rthreads???
> >
> > Thanks in advance
> >
> >
> > On Fri, Apr 1, 2016 at 9:06 AM, Bob Beck <[hidden email]> wrote:
> >>
> >> I have more up to date versions of these patches around here.
> >>
> >> The problem with them is that fundamentally, the WAPBL implementation
> >> as it is assumes that it may infinitely steal
> >> buffers from the buffer cache and hold onto them indefinitely - and it
> >> assumes it can always get buffers from it. While the patch as it sits
> >> may "work" in the "happy case" on many people's machines, as it sits
> >> today it is dangerous and can lock up your machine and corrupt things
> >> in low memory situations.
> >>
> >> Basically in order to progres WAPBL (renamed "FFS Journalling" here)
> >> needs to have a mechanism added to allow
> >> it be told "no it can't have a buffer" and let it deal with it
> >> correctly.  The first part is done, the latter part is complex.
> >>
> >>
> >> On Sat, Mar 26, 2016 at 1:27 PM, Martijn Rijkeboer <[hidden email]>
> >> wrote:
> >> > Hi,
> >> >
> >> > Just out of curiosity, what has happend with WAPBL? There were some
> >> > patches
> >> > floating around on tech@ in the last months of 2015, but then it
> became
> >> > quiet. I'm not complaining just curious.
> >> >
> >> > Kind regards,
> >> >
> >> >
> >> > Martijn Rijkeboer

Reply | Threaded
Open this post in threaded view
|

Re: WAPBL?

Karel Gardas
In reply to this post by Bob Beck-2
On Fri, Apr 1, 2016 at 4:06 PM, Bob Beck <[hidden email]> wrote:
> I have more up to date versions of these patches around here.
>
> The problem with them is that fundamentally, the WAPBL implementation
> as it is assumes that it may infinitely steal
> buffers from the buffer cache and hold onto them indefinitely - and it
> assumes it can always get buffers from it. While the patch as it sits
> may "work" in the "happy case" on many people's machines, as it sits
> today it is dangerous and can lock up your machine and corrupt things
> in low memory situations.

so basically the situation is like with the current softdep which is
also dangerous in slow-write-drive low-memory situation and yet it's
in tree.

> Basically in order to progres WAPBL (renamed "FFS Journalling" here)
> needs to have a mechanism added to allow
> it be told "no it can't have a buffer" and let it deal with it
> correctly.  The first part is done, the latter part is complex.

the question is if it's better to hold those patches in your tree or
push them to the tree so others may try their luck with them. Well you
are the judge here, is WAPBL already on softdep quality level? Thanks!

Reply | Threaded
Open this post in threaded view
|

Re: WAPBL?

Martijn Rijkeboer
In reply to this post by Bob Beck-2
> I have more up to date versions of these patches around here.
>
> The problem with them is that fundamentally, the WAPBL implementation
> as it is assumes that it may infinitely steal
> buffers from the buffer cache and hold onto them indefinitely - and it
> assumes it can always get buffers from it. While the patch as it sits
> may "work" in the "happy case" on many people's machines, as it sits
> today it is dangerous and can lock up your machine and corrupt things
> in low memory situations.
>
> Basically in order to progres WAPBL (renamed "FFS Journalling" here)
> needs to have a mechanism added to allow
> it be told "no it can't have a buffer" and let it deal with it
> correctly.  The first part is done, the latter part is complex.

Thanks for the update. I completely understand that you are holding
back these patches. This attitude to correctness is one of the pillars
that make OpenBSD great.

Kind regards,


Martijn Rijkeboer

Reply | Threaded
Open this post in threaded view
|

Re: WAPBL?

tinkr
In reply to this post by Karel Gardas
On 2016-04-02 17:22, Karel Gardas wrote:
..
> so basically the situation is like with the current softdep which is
> also dangerous in slow-write-drive low-memory situation and yet it's
> in tree.

Is "softdep" dangerous? :-O

I thought it was a benevolent filesystem optimization, is it malevolent
or can I guarantee that my system always will have integrity, what's the
recipe or should I just switch to "sync"?

Thanks for highlighting.

Reply | Threaded
Open this post in threaded view
|

Re: WAPBL?

Mike Burns
On 2016-04-04 14.58.33 +0700, Tinker wrote:
> Is "softdep" dangerous? :-O

This thread explains more:
https://marc.info/?l=openbsd-misc&m=142164001816142&w=2