For a FFS on an SSD, which of "-o" nil, "sync" &/ "softdep" is more data-safe and fast?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

For a FFS on an SSD, which of "-o" nil, "sync" &/ "softdep" is more data-safe and fast?

Tinker
Hi!

If I understand mount(8) (http://man.openbsd.org/mount) right, FFS
mounts have a metadata I/O mode and a data I/O mode. By default,
metadata is accessed synchronously and data is accessed
asynchronously.

"-o sync" will force both to synchronous mode, and "-o softdep" would
change the metadata I/O mode to the alternative softdep access mode.


What is the optimal sync &/ softdep configuration for minimizing risk
of data loss on unexpected reboot, and maximizing overall filesystem
I/O speed?

By "data loss" I mean in particular filesystem structure issues that
could cause fsck(8) hickups and stability issues (e.g. the "/var/log"
directory disappearing altogether - I've seen that happen), secondarily
that file data content updates not would have stuck on the disk.

Either nothing (no "-o" setting), or "-o sync,softdep", or "-o sync",
would all look like they'd make sense.

(Can you even do "-o sync,softdep", would that serve any purpose?)

Will appreciate to learn your thoughts on this one a lot,

Thanks,
Tinker

Reply | Threaded
Open this post in threaded view
|

Re: For a FFS on an SSD, which of "-o" nil, "sync" &/ "softdep" is more data-safe and fast?

Tom Smyth
Also use  noatime  mount option so whe reading files you are not updating
access time

On 8 Feb 2018 7:36 PM, "Tinker" <[hidden email]> wrote:

> Hi!
>
> If I understand mount(8) (http://man.openbsd.org/mount) right, FFS
> mounts have a metadata I/O mode and a data I/O mode. By default,
> metadata is accessed synchronously and data is accessed
> asynchronously.
>
> "-o sync" will force both to synchronous mode, and "-o softdep" would
> change the metadata I/O mode to the alternative softdep access mode.
>
>
> What is the optimal sync &/ softdep configuration for minimizing risk
> of data loss on unexpected reboot, and maximizing overall filesystem
> I/O speed?
>
> By "data loss" I mean in particular filesystem structure issues that
> could cause fsck(8) hickups and stability issues (e.g. the "/var/log"
> directory disappearing altogether - I've seen that happen), secondarily
> that file data content updates not would have stuck on the disk.
>
> Either nothing (no "-o" setting), or "-o sync,softdep", or "-o sync",
> would all look like they'd make sense.
>
> (Can you even do "-o sync,softdep", would that serve any purpose?)
>
> Will appreciate to learn your thoughts on this one a lot,
>
> Thanks,
> Tinker
>
>
Reply | Threaded
Open this post in threaded view
|

Re: For a FFS on an SSD, which of "-o" nil, "sync" &/ "softdep" is more data-safe and fast?

Tom Smyth
Also use  noatime  mount option so whe reading files you are not updating
access time

Ie there would be writes to disk everytime u access a file if noatime is
not set....



On 8 Feb 2018 7:36 PM, "Tinker" <[hidden email]> wrote:

> Hi!
>
> If I understand mount(8) (http://man.openbsd.org/mount) right, FFS
> mounts have a metadata I/O mode and a data I/O mode. By default,
> metadata is accessed synchronously and data is accessed
> asynchronously.
>
> "-o sync" will force both to synchronous mode, and "-o softdep" would
> change the metadata I/O mode to the alternative softdep access mode.
>
>
> What is the optimal sync &/ softdep configuration for minimizing risk
> of data loss on unexpected reboot, and maximizing overall filesystem
> I/O speed?
>
> By "data loss" I mean in particular filesystem structure issues that
> could cause fsck(8) hickups and stability issues (e.g. the "/var/log"
> directory disappearing altogether - I've seen that happen), secondarily
> that file data content updates not would have stuck on the disk.
>
> Either nothing (no "-o" setting), or "-o sync,softdep", or "-o sync",
> would all look like they'd make sense.
>
> (Can you even do "-o sync,softdep", would that serve any purpose?)
>
> Will appreciate to learn your thoughts on this one a lot,
>
> Thanks,
> Tinker
>
>
Reply | Threaded
Open this post in threaded view
|

Re: For a FFS on an SSD, which of "-o" nil, "sync" &/ "softdep" is more data-safe and fast?

Martin Brandenburg
In reply to this post by Tinker
> From [hidden email] Thu Feb  8 23:37:59 2018
> From: Tom Smyth <[hidden email]>
> Date: Thu, 8 Feb 2018 19:40:23 +0000
> Subject: Re: For a FFS on an SSD, which of "-o" nil, "sync" &/ "softdep" is
>  more data-safe and fast?
> To: Tinker <[hidden email]>
>
> Also use  noatime  mount option so whe reading files you are not updating
> access time

But then don't complain when your favorite software package either
doesn't work or does unexpected things.

Martin

Reply | Threaded
Open this post in threaded view
|

Re: For a FFS on an SSD, which of "-o" nil, "sync" &/ "softdep" is more data-safe and fast?

Rupert Gallagher
The only problem I've encountered is rsync unable to preserve the original time of files: copied files have the time of the copy.

Sent from ProtonMail Mobile

On Sat, Feb 10, 2018 at 01:50, <[hidden email]> wrote:

>> From [hidden email] Thu Feb 8 23:37:59 2018 > From: Tom Smyth  > Date: Thu, 8 Feb 2018 19:40:23 +0000 > Subject: Re: For a FFS on an SSD, which of "-o" nil, "sync" &/ "softdep" is > more data-safe and fast? > To: Tinker  > > Also use noatime mount option so whe reading files you are not updating > access time But then don't complain when your favorite software package either doesn't work or does unexpected things. Martin @protonmail.ch> @wirelessconnect.eu>
Reply | Threaded
Open this post in threaded view
|

Re: For a FFS on an SSD, which of "-o" nil, "sync" &/ "softdep" is more data-safe and fast?

Tom Smyth
In reply to this post by Martin Brandenburg
Hi Martin... can you give a specific case where you have experienced
negative impacts from thevmount options i suggested...
It would be good to know...

Thanks Martin

Tom Smyth

On 10 Feb 2018 12:50 AM, <[hidden email]> wrote:

> > From [hidden email] Thu Feb  8 23:37:59 2018
> > From: Tom Smyth <[hidden email]>
> > Date: Thu, 8 Feb 2018 19:40:23 +0000
> > Subject: Re: For a FFS on an SSD, which of "-o" nil, "sync" &/ "softdep"
> is
> >  more data-safe and fast?
> > To: Tinker <[hidden email]>
> >
> > Also use  noatime  mount option so whe reading files you are not updating
> > access time
>
> But then don't complain when your favorite software package either
> doesn't work or does unexpected things.
>
> Martin
>
Reply | Threaded
Open this post in threaded view
|

Re: For a FFS on an SSD, which of "-o" nil, "sync" &/ "softdep" is more data-safe and fast?

Martin Brandenburg
In reply to this post by Tinker
> From [hidden email] Sat Feb 10 11:28:46 2018
> From: Tom Smyth <[hidden email]>
> Date: Sat, 10 Feb 2018 09:10:30 +0000
> Subject: Re: For a FFS on an SSD, which of "-o" nil, "sync" &/ "softdep" is
>  more data-safe and fast?
> To: [hidden email]
>
> Hi Martin... can you give a specific case where you have experienced
> negative impacts from thevmount options i suggested...
> It would be good to know...

Well I don't run noatime, so I can't give specific examples.

The point I was making is that noatime is a tradeoff.  You give up
adherence to POSIX and historical Unix behavior in exchange for
performance and less disk wear.

Now I will admit that programs that rely on atime updates are few and
far between.  It was always a bit niche.  Nowadays, most software is
written on Linux, and most Linux systems use relatime.

It might be an acceptable tradeoff.  To use the examples from the
manpage, if you know you're not going run any programs that need it and
that you'll never leave your laptop up for seven days or if you know
your news server doesn't need it, go ahead and disable it.

Just don't forget to consider it as a potential problem if you run into
trouble.

Here are some lines from /etc/daily.  Without atime updates, it could
delete files.  (Some of these might get more frequent mtime or ctime
updates which would bump the atime under noatime.)

The second set might be uncommented on a big multiuser system to clear
out /tmp since users don't.  They might be a little irritated when the
file they've been reading for the last day gets deleted.

next_part "Removing scratch and junk files:"
if [ -d /tmp -a ! -L /tmp ]; then
        cd /tmp && {
        find -x . \
            \( -path './ssh-*' -o -path ./.X11-unix -o -path ./.ICE-unix \
                -o -path './tmux-*' \) \
            -prune -o -type f -atime +7 -execdir rm -f -- {} \; 2>/dev/null
        find -x . -type d -mtime +1 ! -path ./vi.recover ! -path ./.X11-unix \
            ! -path ./.ICE-unix ! -name . \
            -execdir rmdir -- {} \; >/dev/null 2>&1; }
fi

# Additional junk directory cleanup would go like this:
#if [ -d /scratch -a ! -L /scratch ]; then
# cd /scratch && {
# find . ! -name . -atime +1 -execdir rm -f -- {} \;
# find . ! -name . -type d -mtime +1 -execdir rmdir -- {} \; \
#    >/dev/null 2>&1; }
#fi

Martin

Reply | Threaded
Open this post in threaded view
|

Re: For a FFS on an SSD, which of "-o" nil, "sync" &/ "softdep" is more data-safe and fast?

Ax0n
I use ls -lu and find -atime quite frequently. I've also been using OpenBSD
as a desktop (with all the insane disk activity of a browser cache, temp
files, git, etc) while doing absolutely nothing special using SATA SSD
drives. I'm running Softraid crypto on one, running without softraid on the
other. No snazzy mount options outside the defaults. It's been as reliable
as any spinning rust I've used. You're keeping good backups regardless,
right? Because that protects you from many more likely failure modes than
wear-level-exhaustion on SSD.

On Sat, Feb 10, 2018 at 10:38 AM, <[hidden email]> wrote:

> > From [hidden email] Sat Feb 10 11:28:46 2018
> > From: Tom Smyth <[hidden email]>
> > Date: Sat, 10 Feb 2018 09:10:30 +0000
> > Subject: Re: For a FFS on an SSD, which of "-o" nil, "sync" &/ "softdep"
> is
> >  more data-safe and fast?
> > To: [hidden email]
> >
> > Hi Martin... can you give a specific case where you have experienced
> > negative impacts from thevmount options i suggested...
> > It would be good to know...
>
> Well I don't run noatime, so I can't give specific examples.
>
> The point I was making is that noatime is a tradeoff.  You give up
> adherence to POSIX and historical Unix behavior in exchange for
> performance and less disk wear.
>
> Now I will admit that programs that rely on atime updates are few and
> far between.  It was always a bit niche.  Nowadays, most software is
> written on Linux, and most Linux systems use relatime.
>
> It might be an acceptable tradeoff.  To use the examples from the
> manpage, if you know you're not going run any programs that need it and
> that you'll never leave your laptop up for seven days or if you know
> your news server doesn't need it, go ahead and disable it.
>
> Just don't forget to consider it as a potential problem if you run into
> trouble.
>
> Here are some lines from /etc/daily.  Without atime updates, it could
> delete files.  (Some of these might get more frequent mtime or ctime
> updates which would bump the atime under noatime.)
>
> The second set might be uncommented on a big multiuser system to clear
> out /tmp since users don't.  They might be a little irritated when the
> file they've been reading for the last day gets deleted.
>
> next_part "Removing scratch and junk files:"
> if [ -d /tmp -a ! -L /tmp ]; then
>         cd /tmp && {
>         find -x . \
>             \( -path './ssh-*' -o -path ./.X11-unix -o -path ./.ICE-unix \
>                 -o -path './tmux-*' \) \
>             -prune -o -type f -atime +7 -execdir rm -f -- {} \; 2>/dev/null
>         find -x . -type d -mtime +1 ! -path ./vi.recover ! -path
> ./.X11-unix \
>             ! -path ./.ICE-unix ! -name . \
>             -execdir rmdir -- {} \; >/dev/null 2>&1; }
> fi
>
> # Additional junk directory cleanup would go like this:
> #if [ -d /scratch -a ! -L /scratch ]; then
> #       cd /scratch && {
> #       find . ! -name . -atime +1 -execdir rm -f -- {} \;
> #       find . ! -name . -type d -mtime +1 -execdir rmdir -- {} \; \
> #           >/dev/null 2>&1; }
> #fi
>
> Martin
>
>
Reply | Threaded
Open this post in threaded view
|

Re: For a FFS on an SSD, which of "-o" nil, "sync" &/ "softdep" is more data-safe and fast?

Martin Schröder
In reply to this post by Rupert Gallagher
2018-02-10 7:28 GMT+01:00 Rupert Gallagher <[hidden email]>:
> The only problem I've encountered is rsync unable to preserve the original time of files: copied files have the time of the copy.

man rsync

-t, --times                 preserve modification times

You want
-a, --archive               archive mode; equals -rlptgoD (no -H,-A,-X)

Best
    Martin

Reply | Threaded
Open this post in threaded view
|

Re: For a FFS on an SSD, which of "-o" nil, "sync" &/ "softdep" is more data-safe and fast?

Tinker
My original question was which mounting options are optimal for FFS
filesystems on SSD:s, for data-safety and relative IO speed.

It seems noone in this thread had any data-safety issues ever and so
there was no feedback beyond "use the defaults".

I guess "noatime" as a measure to lower write load should be of some
utility for data-safety.


Part of my question with this thread was if the "softdep" algorithm
used could be expected to be more data-safe for filesystem metadata
than the normal "sync" algorithm, for instance by appending rather
than overwriting.


Tinker

Reply | Threaded
Open this post in threaded view
|

Re: For a FFS on an SSD, which of "-o" nil, "sync" &/ "softdep" is more data-safe and fast?

Marcus MERIGHI
[hidden email] (Tinker), 2018.02.11 (Sun) 06:06 (CET):

> My original question was which mounting options are optimal for FFS
> filesystems on SSD:s, for data-safety and relative IO speed.
>
> It seems noone in this thread had any data-safety issues ever and so
> there was no feedback beyond "use the defaults".
>
> I guess "noatime" as a measure to lower write load should be of some
> utility for data-safety.
>
>
> Part of my question with this thread was if the "softdep" algorithm
> used could be expected to be more data-safe for filesystem metadata
> than the normal "sync" algorithm, for instance by appending rather
> than overwriting.

what I found yesterday after reading your questions and not replying...

ratchov@
https://marc.info/?l=openbsd-misc&m=142166078120639
nick@
https://marc.info/?l=openbsd-misc&m=142170287802566
schwarze@
https://marc.info/?l=openbsd-misc&m=142204644507870
tedu@
https://marc.info/?l=openbsd-misc&m=142250784228719
https://marc.info/?l=openbsd-misc&m=142294185000751
https://marc.info/?l=openbsd-tech&m=143770961724850
clint pachl
https://marc.info/?l=openbsd-misc&m=142294090200592
sthen@
https://marc.info/?l=openbsd-tech&m=143774281500609

Marcus

Reply | Threaded
Open this post in threaded view
|

Re: For a FFS on an SSD, which of "-o" nil, "sync" &/ "softdep" is more data-safe and fast?

Tom Smyth
In reply to this post by Tinker
Hi Tinker,

If you don't expect to run into unexpected shutdowns (power .. you will
be ok with defaults ) ie if you believe the system will always
have reliable power, (and you will have access to the console to help
fsck on reboot, in the unlikely event of loss of power to the running system)

the other thing I would suggest to you (forgot to mention)
is modify the default offset of the first partition from 64 sectors
to 2048 sectors (1 MByte)
and size the rest of the partitions in sectors so that the number you
enter is divisible evenly by 2048, this will (hopefully)  mean that allocation
units on the filesystem are aligned with the underlying SSD disk
which are usually split in to 1MB blocks

I Hope this helps

Thanks
Tom Smyth

On 11 February 2018 at 05:06, Tinker <[hidden email]> wrote:

> My original question was which mounting options are optimal for FFS
> filesystems on SSD:s, for data-safety and relative IO speed.
>
> It seems noone in this thread had any data-safety issues ever and so
> there was no feedback beyond "use the defaults".
>
> I guess "noatime" as a measure to lower write load should be of some
> utility for data-safety.
>
>
> Part of my question with this thread was if the "softdep" algorithm
> used could be expected to be more data-safe for filesystem metadata
> than the normal "sync" algorithm, for instance by appending rather
> than overwriting.
>
>
> Tinker
>



--
Kindest regards,
Tom Smyth

Mobile: +353 87 6193172
The information contained in this E-mail is intended only for the
confidential use of the named recipient. If the reader of this message
is not the intended recipient or the person responsible for
delivering it to the recipient, you are hereby notified that you have
received this communication in error and that any review,
dissemination or copying of this communication is strictly prohibited.
If you have received this in error, please notify the sender
immediately by telephone at the number above and erase the message
You are requested to carry out your own virus check before
opening any attachment.

Reply | Threaded
Open this post in threaded view
|

Re: For a FFS on an SSD, which of "-o" nil, "sync" &/ "softdep" is more data-safe and fast?

Lampshade
In reply to this post by Tinker
> Hi!
>
> If I understand mount(8) (http://man.openbsd.org/mount) right, FFS
> mounts have a metadata I/O mode and a data I/O mode. By default,
> metadata is accessed synchronously and data is accessed
> asynchronously.
>
> "-o sync" will force both to synchronous mode, and "-o softdep" would
> change the metadata I/O mode to the alternative softdep access mode.

No. softdep and async are different concepts.
Default: metadata, data accordingly: sync, async
not that there is async - not softdep

You can do the oposite by options: nosync, noasync
It would mean asynchronous mode for metadata and synchronous for data - which is stupid (slow and dangerous), so don't.

I think default are quite good for SSD. Maybe add noatime for some partitions.