Re: Something hosing my msdos/FAT32 file system

classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: Something hosing my msdos/FAT32 file system

Alexander Hall
A long time ago, Pedro Martelletto wrote:

> Alexander, can you please try to build a test-case that rules out NFS,
> if at all possible?

I ran into this just this week while moving stuff like crazy. Succeeded
to boil stuff down to a very limited set of operations that fscks things
up. I run tests on a (disposable :) vnd device but I got the same
results on an ordinary partition (wd0n).

Upgraded to snapshot from Nov 25 to make sure this wasn't fixed since I
noticed some changes but no better luck there.

Maybe you, or anyone else, can do somthing of it. "Fails like a charm"
every time for me.

/Alexander

----
$ dd if=/dev/zero of=msdos_fs bs=1024 count=1024
1024+0 records in
1024+0 records out
1048576 bytes transferred in 0.014 secs (70148247 bytes/sec)
$ sudo vnconfig vnd0 msdos_fs
$ sudo newfs_msdos /dev/rvnd0c
/dev/rvnd0c: 2008 sectors in 251 FAT12 clusters (4096 bytes/cluster)
bps=512 spc=8 res=1 nft=2 rde=512 sec=2048 mid=0xf0 spf=1 spt=63 hds=1 hid=0
$ sudo mount_msdos -m 777 -l /dev/vnd0c /mnt/test/
$ cd /mnt/test
$ mkdir a aa ab
$ find .
.
./a
./aa
./ab
$ mv aa ab a
$ find .
.
./a
./a/aa
$ ll a
total 16
drwxrwxrwx  1 root  wheel  4096 Nov 26 00:52 aa/
drwxrwxrwx  1 root  wheel  4096 Nov 26 00:52 ab/
$ find -L .
.
./a
./a/aa
./a/ab

Reply | Threaded
Open this post in threaded view
|

Re: Something hosing my msdos/FAT32 file system

Pedro Martelletto
On Sat, Nov 26, 2005 at 12:57:06AM +0100, Alexander Hall wrote:

> I ran into this just this week while moving stuff like crazy. Succeeded
> to boil stuff down to a very limited set of operations that fscks things
> up. I run tests on a (disposable :) vnd device but I got the same
> results on an ordinary partition (wd0n).
>
> Upgraded to snapshot from Nov 25 to make sure this wasn't fixed since I
> noticed some changes but no better luck there.
>
> Maybe you, or anyone else, can do somthing of it. "Fails like a charm"
> every time for me.
>
> /Alexander

Thanks for the report, I will investigate on it.

-p.

Reply | Threaded
Open this post in threaded view
|

Re: Something hosing my msdos/FAT32 file system

Pedro Martelletto
In reply to this post by Alexander Hall
On Sat, Nov 26, 2005 at 12:57:06AM +0100, Alexander Hall wrote:
> $ dd if=/dev/zero of=msdos_fs bs=1024 count=1024
> $ sudo vnconfig vnd0 msdos_fs
> $ sudo newfs_msdos /dev/rvnd0c
> $ sudo mount_msdos -m 777 -l /dev/vnd0c /mnt/test/
> $ cd /mnt/test
> $ mkdir a aa ab

> $ find .
> .
> ./a
> ./aa
> ./ab

Interesting. If you don't do this preliminary 'find', then all is fine.

> $ mv aa ab a
> $ find .
> .
> ./a
> ./a/aa
> $ ll a
> total 16
> drwxrwxrwx  1 root  wheel  4096 Nov 26 00:52 aa/
> drwxrwxrwx  1 root  wheel  4096 Nov 26 00:52 ab/
> $ find -L .
> .
> ./a
> ./a/aa
> ./a/ab

And if you try 'rm -rf a' now, you're likely to get:

rm: fts_read: No such file or directory

-p.

Reply | Threaded
Open this post in threaded view
|

Re: Something hosing my msdos/FAT32 file system

Alexander Hall
Pedro Martelletto wrote:

> On Sat, Nov 26, 2005 at 12:57:06AM +0100, Alexander Hall wrote:
>
>>$ dd if=/dev/zero of=msdos_fs bs=1024 count=1024
>>$ sudo vnconfig vnd0 msdos_fs
>>$ sudo newfs_msdos /dev/rvnd0c
>>$ sudo mount_msdos -m 777 -l /dev/vnd0c /mnt/test/
>>$ cd /mnt/test
>>$ mkdir a aa ab
>
>>$ find .
>>.
>>./a
>>./aa
>>./ab
>
> Interesting. If you don't do this preliminary 'find', then all is fine.

Yes I noticed that, too. :)

>>$ mv aa ab a

If I do ``mv aa a; mv ab a'', all is fine, too. AFAICT, this should
require the exact same syscalls to perform as ``mv aa ab a''; one for
each file to move. It seems that if things happen too fast things go bad.

Thinking of it, since the first "find" is required to get bad results, I
get the feeling that some caches/vnodes/... are populated with corrupted
values, (but I don't know much about this, so I maybe just better stop
guessing). What I mean is that if so, the error must not be within "rename".

>>$ find .
>>.
>>./a
>>./a/aa
>>$ ll a
>>total 16
>>drwxrwxrwx  1 root  wheel  4096 Nov 26 00:52 aa/
>>drwxrwxrwx  1 root  wheel  4096 Nov 26 00:52 ab/
>>$ find -L .
>>.
>>./a
>>./a/aa
>>./a/ab

This one is funny. Don't ask me how come I tested it. Symlinks on
msdosfs? And what are they linking to? :)

> And if you try 'rm -rf a' now, you're likely to get:
>
> rm: fts_read: No such file or directory

Actually, after testing copying aa and ab separately, I cannot reproduce
the previous errors again. Maybe a reboot will "help".

FWIW, I think that unmounting and mounting the fs again restored the
order (or so it seemed).

Please let me know if there is anything else I can do to help you.

/Alexander

Reply | Threaded
Open this post in threaded view
|

Re: Something hosing my msdos/FAT32 file system

Pedro Martelletto
On Sat, Nov 26, 2005 at 03:05:30AM +0100, Alexander Hall wrote:
> Actually, after testing copying aa and ab separately, I cannot reproduce
> the previous errors again. Maybe a reboot will "help".
>
> FWIW, I think that unmounting and mounting the fs again restored the
> order (or so it seemed).
>
> Please let me know if there is anything else I can do to help you.
>
> /Alexander

On a side note, I'm fairly convinced it has someting to do with the
FTS_PHYSICAL option being passed to fts_open(3).

That's why 'find -L' works.

-p.

Reply | Threaded
Open this post in threaded view
|

Re: Something hosing my msdos/FAT32 file system

Ted Unangst-2
In reply to this post by Alexander Hall
On 11/25/05, Alexander Hall <[hidden email]> wrote:

> $ mkdir a aa ab
> $ find .
> .
> ./a
> ./aa
> ./ab
> $ mv aa ab a
> $ find .
> .
> ./a
> ./a/aa
> $ ll a
> total 16
> drwxrwxrwx  1 root  wheel  4096 Nov 26 00:52 aa/
> drwxrwxrwx  1 root  wheel  4096 Nov 26 00:52 ab/
> $ find -L .
> .
> ./a
> ./a/aa
> ./a/ab

there may be a missing cache_purge in msdosfs_rename.

Reply | Threaded
Open this post in threaded view
|

Re: Something hosing my msdos/FAT32 file system

Pedro Martelletto
On Sat, Nov 26, 2005 at 01:45:47AM -0800, Ted Unangst wrote:
> there may be a missing cache_purge in msdosfs_rename.

I've looked at the code and yes, you're correct. There was a missing
cache_purge() in msdosfs_rename(), and that fixed the issue for me.

Alexander, can you confirm the problem is gone?

-p.

Index: msdosfs_vnops.c
===================================================================
RCS file: /cvs/src/sys/msdosfs/msdosfs_vnops.c,v
retrieving revision 1.55
diff -u -r1.55 msdosfs_vnops.c
--- msdosfs_vnops.c 19 Nov 2005 02:18:01 -0000 1.55
+++ msdosfs_vnops.c 26 Nov 2005 14:27:32 -0000
@@ -1151,6 +1151,9 @@
  VOP_UNLOCK(fdvp, 0, p);
  goto bad;
  }
+
+ cache_purge(fvp);
+
  if (!doingdirectory) {
  error = pcbmap(dp, de_cluster(pmp, to_diroffset), 0,
        &ip->de_dirclust, 0);

Reply | Threaded
Open this post in threaded view
|

Re: Something hosing my msdos/FAT32 file system

Alexander Hall
Pedro Martelletto wrote:
> On Sat, Nov 26, 2005 at 01:45:47AM -0800, Ted Unangst wrote:
>
>>there may be a missing cache_purge in msdosfs_rename.
>
>
> I've looked at the code and yes, you're correct. There was a missing
> cache_purge() in msdosfs_rename(), and that fixed the issue for me.
>
> Alexander, can you confirm the problem is gone?

Yeah, seems to work just fine now! At least the previously failing
mv+find stuff does not fail anymore (AFAICT). Thanks Pedro and Ted! :)

I was wondering if this could cause a lot of other errors, like
corrupted files and such. I am thinking about if I dare to mount and use
msdos filesystems r/w again... But then again, if noone dares to do it,
we'll never know if it works as it is supposed to, so maybe it's better
to just backup regularly and go for it! :-)

/Alexander

Reply | Threaded
Open this post in threaded view
|

Re: Something hosing my msdos/FAT32 file system

Ted Unangst-2
On 11/27/05, Alexander Hall <[hidden email]> wrote:
> I was wondering if this could cause a lot of other errors, like
> corrupted files and such. I am thinking about if I dare to mount and use
> msdos filesystems r/w again... But then again, if noone dares to do it,
> we'll never know if it works as it is supposed to, so maybe it's better
> to just backup regularly and go for it! :-)

it shouldn't.  you might get files landing in the wrong directory (i'm
not really sure this is possible either, but it's probably the worst
that could happen), but there shouldn't be any real disk corruption.

Reply | Threaded
Open this post in threaded view
|

Re: Something hosing my msdos/FAT32 file system

frantisek holop
hmm, on Sun, Nov 27, 2005 at 12:24:14AM -0800, Ted Unangst said that

> On 11/27/05, Alexander Hall <[hidden email]> wrote:
> > I was wondering if this could cause a lot of other errors, like
> > corrupted files and such. I am thinking about if I dare to mount and use
> > msdos filesystems r/w again... But then again, if noone dares to do it,
> > we'll never know if it works as it is supposed to, so maybe it's better
> > to just backup regularly and go for it! :-)
>
> it shouldn't.  you might get files landing in the wrong directory (i'm
> not really sure this is possible either, but it's probably the worst
> that could happen), but there shouldn't be any real disk corruption.


i had files not showing up on the fat partition right after moving
them over from ffs.  windows chkdsk found them as lost clusters
so apart losing their names, they were ok ;-)

-f
--
god gives the nuts, but he does not crack them.

Reply | Threaded
Open this post in threaded view
|

Re: Something hosing my msdos/FAT32 file system

Ted Unangst-2
On 11/27/05, frantisek holop <[hidden email]> wrote:
> > it shouldn't.  you might get files landing in the wrong directory (i'm
> > not really sure this is possible either, but it's probably the worst
> > that could happen), but there shouldn't be any real disk corruption.
>
>
> i had files not showing up on the fat partition right after moving
> them over from ffs.  windows chkdsk found them as lost clusters
> so apart losing their names, they were ok ;-)

that would be a different problem.  when did it happen?

Reply | Threaded
Open this post in threaded view
|

Re: Something hosing my msdos/FAT32 file system

Tan Dang
On 11/28/05, Ted Unangst <[hidden email]> wrote:

> On 11/27/05, frantisek holop <[hidden email]> wrote:
> > > it shouldn't.  you might get files landing in the wrong directory (i'm
> > > not really sure this is possible either, but it's probably the worst
> > > that could happen), but there shouldn't be any real disk corruption.
> >
> >
> > i had files not showing up on the fat partition right after moving
> > them over from ffs.  windows chkdsk found them as lost clusters
> > so apart losing their names, they were ok ;-)
>
> that would be a different problem.  when did it happen?
>
>


I have had this problem of files not showing up on the fat partition
after moving the files over from ffs also.  I dual boot OpenBSD 3.8
and Windows XP on my laptop.  Both os's share a fat partition.  For my
particular case, I put Windows XP into hibernation mode and booted to
OpenBSD.  I copied some ogg files from the ffs partition to the fat
partition.  I was able to play the ogg files from the fat partition
with mplayer fine.  On a boot to Windows XP, I was unable to see the
directories that held my ogg files.  I rebooted back to OpenBSD and
the files were now not seen from OpenBSD.

When I shutdown Windows cleanly and move the files to the fat
partition from OpenBSD, then Windows seems to see the files fines.

Reply | Threaded
Open this post in threaded view
|

Re: Something hosing my msdos/FAT32 file system

Josh Grosse
On Mon, Nov 28, 2005 at 09:47:25PM -0600, Tan Dang wrote:

> ....I dual boot OpenBSD 3.8
> and Windows XP on my laptop.  Both os's share a fat partition.  For my
> particular case, I put Windows XP into hibernation mode and booted to
> OpenBSD.  I copied some ogg files from the ffs partition to the fat
> partition.  I was able to play the ogg files from the fat partition
> with mplayer fine.  On a boot to Windows XP, I was unable to see the
> directories that held my ogg files.  I rebooted back to OpenBSD and
> the files were now not seen from OpenBSD.
>
> When I shutdown Windows cleanly and move the files to the fat
> partition from OpenBSD, then Windows seems to see the files fines.
>
This is operator error.  Each operating system expects exclusive write access
to a logical disk device -- in this case, the FAT partition.  What you did
was to have a mounted partition on Windows, that you wrote upon with OpenBSD,
and Windows was never aware of.  Any changes you made to the magnetic media
was without the knowledge of Windows.  Windows, like any operating system,
keeps structure information in memory, and expects exclusive access.

Reply | Threaded
Open this post in threaded view
|

Re: Something hosing my msdos/FAT32 file system

Ted Unangst-2
In reply to this post by Tan Dang
On 11/28/05, Tan Dang <[hidden email]> wrote:
> I have had this problem of files not showing up on the fat partition
> after moving the files over from ffs also.  I dual boot OpenBSD 3.8
> and Windows XP on my laptop.  Both os's share a fat partition.  For my
> particular case, I put Windows XP into hibernation mode and booted to
> OpenBSD.  I copied some ogg files from the ffs partition to the fat
> partition.  I was able to play the ogg files from the fat partition
> with mplayer fine.  On a boot to Windows XP, I was unable to see the
> directories that held my ogg files.  I rebooted back to OpenBSD and
> the files were now not seen from OpenBSD.

i would never under any circumstances ever do this.

Reply | Threaded
Open this post in threaded view
|

Re: Something hosing my msdos/FAT32 file system

Damien Miller
In reply to this post by Tan Dang
On Mon, 28 Nov 2005, Tan Dang wrote:

> I have had this problem of files not showing up on the fat partition
> after moving the files over from ffs also.  I dual boot OpenBSD 3.8
> and Windows XP on my laptop.  Both os's share a fat partition.  For my
> particular case, I put Windows XP into hibernation mode and booted to
> OpenBSD.  I copied some ogg files from the ffs partition to the fat
> partition.  I was able to play the ogg files from the fat partition
> with mplayer fine.  On a boot to Windows XP, I was unable to see the
> directories that held my ogg files.  I rebooted back to OpenBSD and
> the files were now not seen from OpenBSD.
>
> When I shutdown Windows cleanly and move the files to the fat
> partition from OpenBSD, then Windows seems to see the files fines.

You are asking for trouble if you are doing this - it is likely that
Windows is caching filesystem metadata and not refreshing it when it
resumes from suspend.

Don't fiddle with an (effectively) mounted device.

-d