file timestamps wrap around

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

file timestamps wrap around

Bruno Haible
>Synopsis: file timestamps wrap around
>Category: <PR category (one line)>
>Environment:
        System      : OpenBSD 6.0
        Details     : OpenBSD 6.0 (GENERIC) #1917: Tue Jul 26 12:48:33 MDT 2016
                         [hidden email]:/usr/src/sys/arch/i386/compile/GENERIC

        Architecture: OpenBSD.i386
        Machine     : i386
>Description:

Use of 'touch', the kernel's file system, and the 'ls' program produces
unintended time stamps. Time stamps that were meant to be in the past
come out as time stamps in the future, and vice versa.

>How-To-Repeat:

$ export TZ=UTC0

$ touch -t 190101010000 in
$ ls -l in
-rw-r--r--  1 bruno  bruno  0 Feb  6  2037 in
$ ls -lT in
-rw-r--r--  1 bruno  bruno  0 Feb  6  06:28:16 2037 in

$ touch -t 203801190314.08 in2
$ ls -l in2
-rw-r--r--  1 bruno  bruno  0 Dec 13  1901 in2
$ ls -lT in2
-rw-r--r--  1 bruno  bruno  0 Dec 13 20:45:52 1901 in2

$ touch -t 210602070628.15 in3
$ ls -l in3
-rw-r--r--  1 bruno  bruno  0 Dec 31  1969 in3
$ ls -lT in3
-rw-r--r--  1 bruno  bruno  0 Dec 31 23:59:59 in3

Credit for these instructions goes to Paul Eggert.
https://lists.gnu.org/archive/html/bug-gzip/2017-11/msg00020.html

>Fix:

The 'touch' program should properly validate its input, and reject
time stamps that it or the kernel's file system cannot handle.
See https://cwe.mitre.org/data/definitions/20.html


dmesg:
OpenBSD 6.0 (GENERIC) #1917: Tue Jul 26 12:48:33 MDT 2016
    [hidden email]:/usr/src/sys/arch/i386/compile/GENERIC
cpu0: Intel(R) Core(TM) m3-6Y30 CPU @ 0.90GHz ("GenuineIntel" 686-class) 1.52 GHz
cpu0: FPU,V86,DE,PSE,TSC,MSR,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,SSE3,PCLMUL,MWAIT,SSSE3,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AES,XSAVE,AVX,RDRAND,LAHF,ABM,3DNOWP,ITSC,RDSEED,CLFLUSHOPT
real mem  = 1073168384 (1023MB)
avail mem = 1039958016 (991MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: date 06/23/99, BIOS32 rev. 0 @ 0xfda00, SMBIOS rev. 2.5 @ 0xe1000 (10 entries)
bios0: vendor innotek GmbH version "VirtualBox" date 12/01/2006
bios0: innotek GmbH VirtualBox
acpi0 at bios0: rev 2
acpi0: sleep states S0 S5
acpi0: tables DSDT FACP SSDT
acpi0: wakeup devices
acpitimer0 at acpi0: 3579545 Hz, 32 bits
acpiprt0 at acpi0: bus 0 (PCI0)
"PNP0303" at acpi0 not configured
"PNP0F03" at acpi0 not configured
acpibat0 at acpi0: BAT0 model "1" serial 0 type VBOX oem "innotek"
acpiac0 at acpi0: AC unit online
acpivideo0 at acpi0: GFX0
bios0: ROM list: 0xc0000/0x8000 0xe2000/0xd400
cpu0 at mainbus0: (uniprocessor)
mtrr: CPU supports MTRRs but not enabled by BIOS
cpu0: mwait min=64, max=64
pci0 at mainbus0 bus 0: configuration mode 1 (bios)
pchb0 at pci0 dev 0 function 0 "Intel 82441FX" rev 0x02
pcib0 at pci0 dev 1 function 0 "Intel 82371SB ISA" rev 0x00
pciide0 at pci0 dev 1 function 1 "Intel 82371AB IDE" rev 0x01: DMA, channel 0 configured to compatibility, channel 1 configured to compatibility
wd0 at pciide0 channel 0 drive 0: <VBOX HARDDISK>
wd0: 128-sector PIO, LBA, 5120MB, 10485760 sectors
wd0(pciide0:0:0): using PIO mode 4, Ultra-DMA mode 2
atapiscsi0 at pciide0 channel 1 drive 0
scsibus1 at atapiscsi0: 2 targets
cd0 at scsibus1 targ 0 lun 0: <VBOX, CD-ROM, 1.0> ATAPI 5/cdrom removable
cd0(pciide0:1:0): using PIO mode 4, Ultra-DMA mode 2
vga1 at pci0 dev 2 function 0 "InnoTek VirtualBox Graphics Adapter" rev 0x00
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
em0 at pci0 dev 3 function 0 "Intel 82540EM" rev 0x02: irq 10, address 08:00:27:1a:87:0f
"InnoTek VirtualBox Guest Service" rev 0x00 at pci0 dev 4 function 0 not configured
auich0 at pci0 dev 5 function 0 "Intel 82801AA AC97" rev 0x01: irq 11, ICH AC97
ac97: codec id 0x83847600 (SigmaTel STAC9700)
audio0 at auich0
ohci0 at pci0 dev 6 function 0 "Apple Intrepid USB" rev 0x00: irq 11, version 1.0
piixpm0 at pci0 dev 7 function 0 "Intel 82371AB Power" rev 0x08: SMBus disabled
isa0 at pcib0
isadma0 at isa0
pckbc0 at isa0 port 0x60/5 irq 1 irq 12
pckbd0 at pckbc0 (kbd slot)
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pms0 at pckbc0 (aux slot)
wsmouse0 at pms0 mux 0
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
npx0 at isa0 port 0xf0/16: reported by CPUID; using exception 16
usb0 at ohci0: USB revision 1.0
uhub0 at usb0 "Apple OHCI root hub" rev 1.00/1.00 addr 1
vscsi0 at root
scsibus2 at vscsi0: 256 targets
softraid0 at root
scsibus3 at softraid0: 256 targets
root on wd0a (c1ea2b9ce0536372.a) swap on wd0b dump on wd0b

usbdevs:
Controller /dev/usb0:
addr 1: full speed, self powered, config 1, OHCI root hub(0x0000), Apple(0x106b), rev 1.00
 port 1 powered
 port 2 powered
 port 3 powered
 port 4 powered
 port 5 powered
 port 6 powered
 port 7 powered
 port 8 powered
 port 9 powered
 port 10 powered
 port 11 powered
 port 12 powered

Reply | Threaded
Open this post in threaded view
|

Re: file timestamps wrap around

Otto Moerbeek
On Sun, Nov 12, 2017 at 06:29:38PM +0100, Bruno Haible wrote:

> >Synopsis: file timestamps wrap around
> >Category: <PR category (one line)>
> >Environment:
> System      : OpenBSD 6.0
> Details     : OpenBSD 6.0 (GENERIC) #1917: Tue Jul 26 12:48:33 MDT 2016
> [hidden email]:/usr/src/sys/arch/i386/compile/GENERIC
>
> Architecture: OpenBSD.i386
> Machine     : i386
> >Description:
>
> Use of 'touch', the kernel's file system, and the 'ls' program produces
> unintended time stamps. Time stamps that were meant to be in the past
> come out as time stamps in the future, and vice versa.
>
> >How-To-Repeat:
>
> $ export TZ=UTC0
>
> $ touch -t 190101010000 in
> $ ls -l in
> -rw-r--r--  1 bruno  bruno  0 Feb  6  2037 in
> $ ls -lT in
> -rw-r--r--  1 bruno  bruno  0 Feb  6  06:28:16 2037 in
>
> $ touch -t 203801190314.08 in2
> $ ls -l in2
> -rw-r--r--  1 bruno  bruno  0 Dec 13  1901 in2
> $ ls -lT in2
> -rw-r--r--  1 bruno  bruno  0 Dec 13 20:45:52 1901 in2
>
> $ touch -t 210602070628.15 in3
> $ ls -l in3
> -rw-r--r--  1 bruno  bruno  0 Dec 31  1969 in3
> $ ls -lT in3
> -rw-r--r--  1 bruno  bruno  0 Dec 31 23:59:59 in3
>
> Credit for these instructions goes to Paul Eggert.
> https://lists.gnu.org/archive/html/bug-gzip/2017-11/msg00020.html
>
> >Fix:
>
> The 'touch' program should properly validate its input, and reject
> time stamps that it or the kernel's file system cannot handle.
> See https://cwe.mitre.org/data/definitions/20.html

OpenBSD uses ffs1 by default. I suppose that is waht you call the
"kernel filesystem".  If you would have ran you test cases using an
ffs2 filesystem it would have worked as expected:

-rw-r--r--   1 otto  wheel    0 Jan  1  1901 in
-rw-r--r--   1 otto  wheel    0 Jan 19  2038 in2
-rw-r--r--   1 otto  wheel    0 Feb  7  2106 in3

As for the general case, for timestamps before 1970, we *are* allowed
to have implementation defined behaviour. Posix says:

"If the resulting time precedes the Epoch, the behavior is
implementation-defined. If the time cannot be represented as the
file's timestamp, touch shall exit immediately with an error status."

For timestamps from 2038 the big problem is that implementing this is
a challange, since the kernel (let alone touch(1)) does not know the
filesystem's timestamp limitations in all cases. Things especially
become hairy in the case of network file systems.

        -Otto


 

Reply | Threaded
Open this post in threaded view
|

Re: file timestamps wrap around

Mark Kettenis
> Date: Sun, 12 Nov 2017 19:54:24 +0100
> From: Otto Moerbeek <[hidden email]>
>
> On Sun, Nov 12, 2017 at 06:29:38PM +0100, Bruno Haible wrote:
>
> > >Synopsis: file timestamps wrap around
> > >Category: <PR category (one line)>
> > >Environment:
> > System      : OpenBSD 6.0
> > Details     : OpenBSD 6.0 (GENERIC) #1917: Tue Jul 26 12:48:33 MDT 2016
> > [hidden email]:/usr/src/sys/arch/i386/compile/GENERIC
> >
> > Architecture: OpenBSD.i386
> > Machine     : i386
> > >Description:
> >
> > Use of 'touch', the kernel's file system, and the 'ls' program produces
> > unintended time stamps. Time stamps that were meant to be in the past
> > come out as time stamps in the future, and vice versa.
> >
> > >How-To-Repeat:
> >
> > $ export TZ=UTC0
> >
> > $ touch -t 190101010000 in
> > $ ls -l in
> > -rw-r--r--  1 bruno  bruno  0 Feb  6  2037 in
> > $ ls -lT in
> > -rw-r--r--  1 bruno  bruno  0 Feb  6  06:28:16 2037 in
> >
> > $ touch -t 203801190314.08 in2
> > $ ls -l in2
> > -rw-r--r--  1 bruno  bruno  0 Dec 13  1901 in2
> > $ ls -lT in2
> > -rw-r--r--  1 bruno  bruno  0 Dec 13 20:45:52 1901 in2
> >
> > $ touch -t 210602070628.15 in3
> > $ ls -l in3
> > -rw-r--r--  1 bruno  bruno  0 Dec 31  1969 in3
> > $ ls -lT in3
> > -rw-r--r--  1 bruno  bruno  0 Dec 31 23:59:59 in3
> >
> > Credit for these instructions goes to Paul Eggert.
> > https://lists.gnu.org/archive/html/bug-gzip/2017-11/msg00020.html
> >
> > >Fix:
> >
> > The 'touch' program should properly validate its input, and reject
> > time stamps that it or the kernel's file system cannot handle.
> > See https://cwe.mitre.org/data/definitions/20.html
>
> OpenBSD uses ffs1 by default. I suppose that is waht you call the
> "kernel filesystem".  If you would have ran you test cases using an
> ffs2 filesystem it would have worked as expected:
>
> -rw-r--r--   1 otto  wheel    0 Jan  1  1901 in
> -rw-r--r--   1 otto  wheel    0 Jan 19  2038 in2
> -rw-r--r--   1 otto  wheel    0 Feb  7  2106 in3
>
> As for the general case, for timestamps before 1970, we *are* allowed
> to have implementation defined behaviour. Posix says:
>
> "If the resulting time precedes the Epoch, the behavior is
> implementation-defined. If the time cannot be represented as the
> file's timestamp, touch shall exit immediately with an error status."
>
> For timestamps from 2038 the big problem is that implementing this is
> a challange, since the kernel (let alone touch(1)) does not know the
> filesystem's timestamp limitations in all cases. Things especially
> become hairy in the case of network file systems.

It certainly doesn't make sense to have touch(1) do the checks.

Note that POSIX says that futimens(2) shall faill if:

[EINVAL]
    A new file timestamp would be a value whose tv_sec component is
    not a value supported by the file system.

which is something we don't implement for the filesystems where it
matters.  Also note that NFSv3 still uses 32 bits for storing the
timestamp so there wil;l be issues regardless of the underlying
filesystem.

Reply | Threaded
Open this post in threaded view
|

Re: file timestamps wrap around

Otto Moerbeek
On Sun, Nov 12, 2017 at 08:12:26PM +0100, Mark Kettenis wrote:

> > Date: Sun, 12 Nov 2017 19:54:24 +0100
> > From: Otto Moerbeek <[hidden email]>
> >
> > On Sun, Nov 12, 2017 at 06:29:38PM +0100, Bruno Haible wrote:
> >
> > > >Synopsis: file timestamps wrap around
> > > >Category: <PR category (one line)>
> > > >Environment:
> > > System      : OpenBSD 6.0
> > > Details     : OpenBSD 6.0 (GENERIC) #1917: Tue Jul 26 12:48:33 MDT 2016
> > > [hidden email]:/usr/src/sys/arch/i386/compile/GENERIC
> > >
> > > Architecture: OpenBSD.i386
> > > Machine     : i386
> > > >Description:
> > >
> > > Use of 'touch', the kernel's file system, and the 'ls' program produces
> > > unintended time stamps. Time stamps that were meant to be in the past
> > > come out as time stamps in the future, and vice versa.
> > >
> > > >How-To-Repeat:
> > >
> > > $ export TZ=UTC0
> > >
> > > $ touch -t 190101010000 in
> > > $ ls -l in
> > > -rw-r--r--  1 bruno  bruno  0 Feb  6  2037 in
> > > $ ls -lT in
> > > -rw-r--r--  1 bruno  bruno  0 Feb  6  06:28:16 2037 in
> > >
> > > $ touch -t 203801190314.08 in2
> > > $ ls -l in2
> > > -rw-r--r--  1 bruno  bruno  0 Dec 13  1901 in2
> > > $ ls -lT in2
> > > -rw-r--r--  1 bruno  bruno  0 Dec 13 20:45:52 1901 in2
> > >
> > > $ touch -t 210602070628.15 in3
> > > $ ls -l in3
> > > -rw-r--r--  1 bruno  bruno  0 Dec 31  1969 in3
> > > $ ls -lT in3
> > > -rw-r--r--  1 bruno  bruno  0 Dec 31 23:59:59 in3
> > >
> > > Credit for these instructions goes to Paul Eggert.
> > > https://lists.gnu.org/archive/html/bug-gzip/2017-11/msg00020.html
> > >
> > > >Fix:
> > >
> > > The 'touch' program should properly validate its input, and reject
> > > time stamps that it or the kernel's file system cannot handle.
> > > See https://cwe.mitre.org/data/definitions/20.html
> >
> > OpenBSD uses ffs1 by default. I suppose that is waht you call the
> > "kernel filesystem".  If you would have ran you test cases using an
> > ffs2 filesystem it would have worked as expected:
> >
> > -rw-r--r--   1 otto  wheel    0 Jan  1  1901 in
> > -rw-r--r--   1 otto  wheel    0 Jan 19  2038 in2
> > -rw-r--r--   1 otto  wheel    0 Feb  7  2106 in3
> >
> > As for the general case, for timestamps before 1970, we *are* allowed
> > to have implementation defined behaviour. Posix says:
> >
> > "If the resulting time precedes the Epoch, the behavior is
> > implementation-defined. If the time cannot be represented as the
> > file's timestamp, touch shall exit immediately with an error status."
> >
> > For timestamps from 2038 the big problem is that implementing this is
> > a challange, since the kernel (let alone touch(1)) does not know the
> > filesystem's timestamp limitations in all cases. Things especially
> > become hairy in the case of network file systems.
>
> It certainly doesn't make sense to have touch(1) do the checks.
>
> Note that POSIX says that futimens(2) shall faill if:
>
> [EINVAL]
>     A new file timestamp would be a value whose tv_sec component is
>     not a value supported by the file system.
>
> which is something we don't implement for the filesystems where it
> matters.  Also note that NFSv3 still uses 32 bits for storing the
> timestamp so there wil;l be issues regardless of the underlying
> filesystem.

One thing to consider: touch first creates a file using
open(...O_CREAT...)  and then sets it's timestamp using futimens(2).
That means that we can only determine something went wrong after the
fact and if there's would be an error we end up with a file with the
wrong filestamp.

        -Otto

Reply | Threaded
Open this post in threaded view
|

Re: file timestamps wrap around

Otto Moerbeek
On Sun, Nov 12, 2017 at 08:19:42PM +0100, Otto Moerbeek wrote:

> On Sun, Nov 12, 2017 at 08:12:26PM +0100, Mark Kettenis wrote:
>
> > > Date: Sun, 12 Nov 2017 19:54:24 +0100
> > > From: Otto Moerbeek <[hidden email]>
> > >
> > > On Sun, Nov 12, 2017 at 06:29:38PM +0100, Bruno Haible wrote:
> > >
> > > > >Synopsis: file timestamps wrap around
> > > > >Category: <PR category (one line)>
> > > > >Environment:
> > > > System      : OpenBSD 6.0
> > > > Details     : OpenBSD 6.0 (GENERIC) #1917: Tue Jul 26 12:48:33 MDT 2016
> > > > [hidden email]:/usr/src/sys/arch/i386/compile/GENERIC
> > > >
> > > > Architecture: OpenBSD.i386
> > > > Machine     : i386
> > > > >Description:
> > > >
> > > > Use of 'touch', the kernel's file system, and the 'ls' program produces
> > > > unintended time stamps. Time stamps that were meant to be in the past
> > > > come out as time stamps in the future, and vice versa.
> > > >
> > > > >How-To-Repeat:
> > > >
> > > > $ export TZ=UTC0
> > > >
> > > > $ touch -t 190101010000 in
> > > > $ ls -l in
> > > > -rw-r--r--  1 bruno  bruno  0 Feb  6  2037 in
> > > > $ ls -lT in
> > > > -rw-r--r--  1 bruno  bruno  0 Feb  6  06:28:16 2037 in
> > > >
> > > > $ touch -t 203801190314.08 in2
> > > > $ ls -l in2
> > > > -rw-r--r--  1 bruno  bruno  0 Dec 13  1901 in2
> > > > $ ls -lT in2
> > > > -rw-r--r--  1 bruno  bruno  0 Dec 13 20:45:52 1901 in2
> > > >
> > > > $ touch -t 210602070628.15 in3
> > > > $ ls -l in3
> > > > -rw-r--r--  1 bruno  bruno  0 Dec 31  1969 in3
> > > > $ ls -lT in3
> > > > -rw-r--r--  1 bruno  bruno  0 Dec 31 23:59:59 in3
> > > >
> > > > Credit for these instructions goes to Paul Eggert.
> > > > https://lists.gnu.org/archive/html/bug-gzip/2017-11/msg00020.html
> > > >
> > > > >Fix:
> > > >
> > > > The 'touch' program should properly validate its input, and reject
> > > > time stamps that it or the kernel's file system cannot handle.
> > > > See https://cwe.mitre.org/data/definitions/20.html
> > >
> > > OpenBSD uses ffs1 by default. I suppose that is waht you call the
> > > "kernel filesystem".  If you would have ran you test cases using an
> > > ffs2 filesystem it would have worked as expected:
> > >
> > > -rw-r--r--   1 otto  wheel    0 Jan  1  1901 in
> > > -rw-r--r--   1 otto  wheel    0 Jan 19  2038 in2
> > > -rw-r--r--   1 otto  wheel    0 Feb  7  2106 in3
> > >
> > > As for the general case, for timestamps before 1970, we *are* allowed
> > > to have implementation defined behaviour. Posix says:
> > >
> > > "If the resulting time precedes the Epoch, the behavior is
> > > implementation-defined. If the time cannot be represented as the
> > > file's timestamp, touch shall exit immediately with an error status."
> > >
> > > For timestamps from 2038 the big problem is that implementing this is
> > > a challange, since the kernel (let alone touch(1)) does not know the
> > > filesystem's timestamp limitations in all cases. Things especially
> > > become hairy in the case of network file systems.
> >
> > It certainly doesn't make sense to have touch(1) do the checks.
> >
> > Note that POSIX says that futimens(2) shall faill if:
> >
> > [EINVAL]
> >     A new file timestamp would be a value whose tv_sec component is
> >     not a value supported by the file system.
> >
> > which is something we don't implement for the filesystems where it
> > matters.  Also note that NFSv3 still uses 32 bits for storing the
> > timestamp so there wil;l be issues regardless of the underlying
> > filesystem.
>
> One thing to consider: touch first creates a file using
> open(...O_CREAT...)  and then sets it's timestamp using futimens(2).
> That means that we can only determine something went wrong after the
> fact and if there's would be an error we end up with a file with the
> wrong filestamp.
>
> -Otto

to be more explicit: a conforming implementation of touch(1) should
have knowledge of the filesystem, since Posix claims that touch(1)
should do the validation taking into account the filesystem
limitations. If we do not want to do that and rely on futimens(2) to
detect the error cases, we have cases where we end up with wrong
timestamp because of the two-stelp way of doing things. We could
delete delete the file with the wrong filestamp on error, but that is
a race condition I would not like to have.

        -Otto

Reply | Threaded
Open this post in threaded view
|

Re: file timestamps wrap around

Philip Guenther
In reply to this post by Otto Moerbeek
On Sun, 12 Nov 2017, Otto Moerbeek wrote:
> On Sun, Nov 12, 2017 at 08:12:26PM +0100, Mark Kettenis wrote:
...

> > It certainly doesn't make sense to have touch(1) do the checks.
> >
> > Note that POSIX says that futimens(2) shall faill if:
> >
> > [EINVAL]
> >     A new file timestamp would be a value whose tv_sec component is
> >     not a value supported by the file system.
> >
> > which is something we don't implement for the filesystems where it
> > matters.  Also note that NFSv3 still uses 32 bits for storing the
> > timestamp so there wil;l be issues regardless of the underlying
> > filesystem.

Yeah, for NFS the best we can do is reflect the protocol limitation; NFS
export of FFSv1 will still see the misbehavior, but that can't be fixed
without an NFS protocol addition.


> One thing to consider: touch first creates a file using
> open(...O_CREAT...)  and then sets it's timestamp using futimens(2).
> That means that we can only determine something went wrong after the
> fact and if there's would be an error we end up with a file with the
> wrong filestamp.

The "Right" way to solve that would be to add a new pathconf(2) item(s)
that would provide info on the timestamp range, or maybe direct max+min
queries.  The API design would be a bit tricky as pathconf(2) returns
long, not long long, so either some sort of scaling (log scale by
returning the # of bits supported?) or an alternate entry point to return
the full value would be necessary.  Returning log-scale, say,
_PC_FILETIMEBITS, would be rather annoying to actually *use*, but adding
pathconfll() just to fix this situation in touch(1) feels like squashing a
bug with a pile-driver.


Hmm, here's an idea: add a new AT_UTIME_TEST flag for use with utimensat()
that, when used, makes utimensat() do no actual filesystem change but
rather simply succeed if and only if the supplied timestamps are within
the range of the filesystem of the named path, ignoring permission checks.  
touch(1) could then, when the named file doesn't already exist, use
utimensat(AT_UTIME_TEST) on the directory holding the target.


Philip