Poor NFS performance

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Poor NFS performance

Sebastian Neuper
Hi again.

Long time ago, I asked about [1] hardware recommendations to build
a home file / video server, which streams HD videos for Avid.

Quite late, but thanks for your responses.

I bought a raid-controller  and an em-network card and recycled a
really old pc, installed OpenBSD 5.3 Current on a 2TB HDD ( My first
one and Nick knows that disk [2] ) and made a Softraid RAID 1 with
2x2.5' 1TB Drives. Then I upgraded Windows 7 to Ultimate to get the
build-in NFS Client. Started nfsd, statd, lockd etc.

Good news: It works. But only in SD. On HD it hangs every 10 sec
for a few. And I don't think, this is so, because I play
Kate Ryan LoveLife music video to test it. ;)

When I look at `top`, the CPU stays below 10% most of the time. The
network is 1Gbit/s both sides and switch. So I think I could get a
better performance than 20MB/s while coping files to and around
4MB/s when coping from that server.

But where is the bottleneck?
 
I uploaded the output of these commands:

dmesg
netstat -m
nfsstat
nfsstat -c
ps ax
systat ifstat
systat nfsserver
systat vm
top
vmstat
and a screenshot of Windows netdrive options

on http://www.sqrt2.de/log .

If you need more, just mail.

Thanks,

Sebastian.

[1] http://marc.info/?l=openbsd-misc&m=135646615716842&w=2
[2] http://marc.info/?l=openbsd-misc&m=105889683722295&w=2

--
Sebastian Neuper <[hidden email]>

Reply | Threaded
Open this post in threaded view
|

Re: Poor NFS performance

Sebastian Neuper
> When I look at `top`, the CPU stays below 10% most of the time. The
> network is 1Gbit/s both sides and switch. So I think I could get a
> better performance than 20MB/s while coping files to and around
> 4MB/s when coping from that server.
>
> But where is the bottleneck?
>  
> I uploaded the output of these commands: [...]

For the archives, find the output of the commands also here:

### dmesg ###                      

OpenBSD 5.3-current (GENERIC) #149: Tue May  7 12:44:38 MDT 2013
    [hidden email]:/usr/src/sys/arch/i386/compile/GENERIC
cpu0: Intel(R) Pentium(R) 4 CPU 2.66GHz ("GenuineIntel" 686-class) 2.68
GHz cpu0:
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,CNXT-ID,PERF
real mem  = 1073213440 (1023MB) avail mem = 1044254720 (995MB)
mainbus0 at root
bios0 at mainbus0: AT/286+ BIOS, date 09/11/02, BIOS32 rev. 0 @
0xfb3d0, SMBIOS rev. 2.3 @ 0xf0800 (32 entries) bios0: vendor Phoenix
Technologies, LTD version "6.00 PG" date 09/11/2002 bios0: MEDIONPC
MS-6701 acpi0 at bios0: rev 0
acpi0: sleep states S0 S3 S4 S5
acpi0: tables DSDT FACP APIC
acpi0: wakeup devices FUTS(S4) USB0(S5) USB1(S5) USB2(S5) USB3(S5) MAC0
(S5) AMR0(S4) UAR1(S5) PS2M(S5) PS2K(S4) PCI0(S5) acpitimer0 at acpi0:
3579545 Hz, 24 bits acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: apic clock running at 133MHz
ioapic0 at mainbus0: apid 2 pa 0xfec00000, version 14, 24 pins
ioapic0: misconfigured as apic 0, remapped to apid 2
acpiprt0 at acpi0: bus 0 (PCI0)
acpicpu0 at acpi0
acpitz0 at acpi0: critical temperature is 100 degC
acpibtn0 at acpi0: PWRB
acpibtn1 at acpi0: FUTS
bios0: ROM list: 0xc0000/0xf800 0xd0000/0x8000!
pci0 at mainbus0 bus 0: configuration mode 1 (bios)
pchb0 at pci0 dev 0 function 0 "SiS 648 PCI" rev 0x02
sisagp0 at pchb0
agp0 at sisagp0: aperture at 0xe0000000, size 0x8000000
ppb0 at pci0 dev 1 function 0 "SiS 86C201 AGP" rev 0x00
pci1 at ppb0 bus 1
vga1 at pci1 dev 0 function 0 "NVIDIA GeForce FX 5900 Ultra" rev 0xa1
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
pcib0 at pci0 dev 2 function 0 "SiS 85C503 System" rev 0x04
"SiS 7007 FireWire" rev 0x00 at pci0 dev 2 function 3 not configured
pciide0 at pci0 dev 2 function 5 "SiS 5513 EIDE" rev 0x00: 648: DMA,
channel 0 configured to compatibility, channel 1 configured to
compatibility wd0 at pciide0 channel 0 drive 0: <ST52520A> wd0:
16-sector PIO, LBA, 2446MB, 5009760 sectors wd0(pciide0:0:0): using PIO
mode 4, DMA mode 2 pciide0: channel 1 disabled (no drives)
auich0 at pci0 dev 2 function 7 "SiS 7012 AC97" rev 0xa0: apic 2 int
18, SiS7012 AC97 ac97: codec id 0x414c4720 (Avance Logic ALC650)
ac97: codec features 20 bit DAC, 18 bit ADC, Realtek 3D
audio0 at auich0
ohci0 at pci0 dev 3 function 0 "SiS 5597/5598 USB" rev 0x0f: apic 2 int
20, version 1.0, legacy support ohci1 at pci0 dev 3 function 1 "SiS
5597/5598 USB" rev 0x0f: apic 2 int 21, version 1.0, legacy support
ohci2 at pci0 dev 3 function 2 "SiS 5597/5598 USB" rev 0x0f: apic 2 int
22, version 1.0, legacy support ehci0 at pci0 dev 3 function 3 "SiS
7002 USB" rev 0x00: apic 2 int 23 usb0 at ehci0: USB revision 2.0 uhub0
at usb0 "SiS EHCI root hub" rev 2.00/1.00 addr 1 sis0 at pci0 dev 4
function 0 "SiS 900 10/100BaseTX" rev 0x91: apic 2 int 19, address
00:10:dc:7e:88:23 rlphy0 at sis0 phy 1: RTL8201L 10/100 PHY, rev. 1 em0
at pci0 dev 6 function 0 "Intel PRO/1000GT (82541GI)" rev 0x05: apic 2
int 17, address 90:e2:ba:30:ba:a0 pciide1 at pci0 dev 7 function 0 "CMD
Technology SiI3114 SATA" rev 0x02: DMA pciide1: using apic 2 int 18 for
native-PCI interrupt pciide1: port 2: device present, speed: 1.5Gb/s
wd1 at pciide1 channel 2 drive 0: <SAMSUNG HN-M101MBB> wd1: 16-sector
PIO, LBA48, 953869MB, 1953525168 sectors wd1(pciide1:2:0): using BIOS
timings, Ultra-DMA mode 6 pciide1: port 3: device present, speed:
1.5Gb/s wd2 at pciide1 channel 3 drive 0: <SAMSUNG HN-M101MBB>
wd2: 16-sector PIO, LBA48, 953869MB, 1953525168 sectors
wd2(pciide1:3:0): using BIOS timings, Ultra-DMA mode 6
isa0 at pcib0
isadma0 at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
pckbc0 at isa0 port 0x60/5
pckbd0 at pckbc0 (kbd slot)
pckbc0: using irq 1 for kbd slot
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pms0 at pckbc0 (aux slot #0)
pckbc0: using irq 12 for aux slot #0
wsmouse0 at pms0 mux 0
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
lpt0 at isa0 port 0x378/4 irq 7
wbsio0 at isa0 port 0x2e/2: W83697HF rev 0x16
lm1 at wbsio0 port 0x290/8: W83697HF
npx0 at isa0 port 0xf0/16: reported by CPUID; using exception 16
fdc0 at isa0 port 0x3f0/6 irq 6 drq 2
usb1 at ohci0: USB revision 1.0
uhub1 at usb1 "SiS OHCI root hub" rev 1.00/1.00 addr 1
usb2 at ohci1: USB revision 1.0
uhub2 at usb2 "SiS OHCI root hub" rev 1.00/1.00 addr 1
usb3 at ohci2: USB revision 1.0
uhub3 at usb3 "SiS OHCI root hub" rev 1.00/1.00 addr 1
mtrr: Pentium Pro MTRR support
vscsi0 at root
scsibus0 at vscsi0: 256 targets
softraid0 at root
scsibus1 at softraid0: 256 targets
softraid0: trying to bring up sd0 degraded
sd0 at scsibus1 targ 1 lun 0: <OPENBSD, SR RAID 1, 005> SCSI2 0/direct
fixed sd0: 953869MB, 512 bytes/sector, 1953524576 sectors
softraid0: roaming device  -> wd2a
root on wd0a (58c4113a2922eb55.a) swap on wd0b dump on wd0b


### netstat -m ###

69 mbufs in use:
        39 mbufs allocated to data
        12 mbufs allocated to packet headers
        18 mbufs allocated to socket names and addresses
20/46/6144 mbuf 2048 byte clusters in use (current/peak/max)
0/8/6144 mbuf 4096 byte clusters in use (current/peak/max)
0/8/6144 mbuf 8192 byte clusters in use (current/peak/max)
0/8/6144 mbuf 9216 byte clusters in use (current/peak/max)
0/8/6144 mbuf 12288 byte clusters in use (current/peak/max)
0/8/6144 mbuf 16384 byte clusters in use (current/peak/max)
0/8/6144 mbuf 65536 byte clusters in use (current/peak/max)
312 Kbytes allocated to network (16% in use)
0 requests for memory denied
0 requests for memory delayed
0 calls to protocol drain routines


### nfsstat ###

Client Info:
  Getattr   Setattr    Lookup  Readlink      Read     Write
Create    Remove 0         0         0         0         0
0         0         0 Rename      Link   Symlink     Mkdir     Rmdir
Readdir  RdirPlus    Access 0         0         0         0
0         0         0         0 Mknod    Fsstat    Fsinfo  PathConf
Commit 0         1         1         0         0
Rpc Info:
 TimedOut   Invalid X Replies   Retries  Requests   FrcSync
        0         0         0         0         2         0
Cache Info:
Attr Hits    Misses Lkup Hits    Misses BioR Hits    Misses BioW
Hits    Misses 0         0         0         0         0
0         0         0 BioRLHits    Misses BioD Hits    Misses DirE
Hits    Misses 0         0         0         0         0         0

Server Info:
  Getattr   Setattr    Lookup  Readlink      Read     Write
Create    Remove 254        86        89         0    859487
53286        17         0 Rename      Link   Symlink     Mkdir
Rmdir   Readdir  RdirPlus    Access 0         0         0
4         0        38         4         0 Mknod    Fsstat    Fsinfo
PathConf    Commit 0      2059         2         1       863
Server Ret-Failed
               50
Server Faults
            0
Server Cache Stats:
   Inprog      Idem  Non-idem    Misses
        0         0         0         2
Server Write Gathering:
 WriteOps  WriteRPC   Opsaved
    53286     53286         0


### nfsstat -c ###

Client Info:
  Getattr   Setattr    Lookup  Readlink      Read     Write
Create    Remove 0         0         0         0         0
0         0         0 Rename      Link   Symlink     Mkdir     Rmdir
Readdir  RdirPlus    Access 0         0         0         0
0         0         0         0 Mknod    Fsstat    Fsinfo  PathConf
Commit 0         1         1         0         0
Rpc Info:
 TimedOut   Invalid X Replies   Retries  Requests   FrcSync
        0         0         0         0         2         0
Cache Info:
Attr Hits    Misses Lkup Hits    Misses BioR Hits    Misses BioW
Hits    Misses 0         0         0         0         0
0         0         0 BioRLHits    Misses BioD Hits    Misses DirE
Hits    Misses 0         0         0         0         0         0


### ps ax ###  

PID TT  STAT       TIME COMMAND
    1 ??  Is      0:01.00 /sbin/init
28471 ??  Is      0:00.00 syslogd: [priv] (syslogd)
18983 ??  S       0:00.03 /usr/sbin/syslogd -a /var/www/dev/log
-a /var/empty/dev/log 15552 ??  Is      0:00.00 pflogd: [priv] (pflogd)
28657 ??  Is      0:00.00 /usr/sbin/portmap
27053 ??  S       0:00.23 pflogd: [running] -s 160 -i pflog0
-f /var/log/pflog (pflogd) 20579 ??  Is      0:00.02 /sbin/mountd
31362 ??  Is      0:00.00 nfsd: master (nfsd)
12977 ??  I       0:00.00 nfsd: server (nfsd)
 3955 ??  I       1:35.45 nfsd: server (nfsd)
24186 ??  I       0:00.00 nfsd: server (nfsd)
 5109 ??  I       0:00.00 nfsd: server (nfsd)
31214 ??  Is      0:00.03 /usr/sbin/rpc.lockd
 7408 ??  Is      0:00.00 /usr/sbin/rpc.statd
22977 ??  Is      0:00.00 /usr/sbin/sshd
22089 ??  S       0:00.65 smtpd: delivery (smtpd)
18955 ??  Ss      0:01.10 smtpd: [priv] (smtpd)
30282 ??  S       0:02.00 smtpd: control (smtpd)
 8489 ??  S       0:00.78 smtpd: lookup (smtpd)
 4442 ??  S       0:00.66 smtpd: filter (smtpd)
 3216 ??  S       0:00.78 smtpd: transfer (smtpd)
  119 ??  S       0:00.84 smtpd: queue (smtpd)
11483 ??  S       0:00.80 smtpd: smtp (smtpd)
29320 ??  S       0:00.51 smtpd: scheduler (smtpd)
27111 ??  Is      0:00.00 nginx: master process /usr/sbin/nginx
25831 ??  I       0:00.00 nginx: worker process (nginx)
25532 ??  I<s     0:00.00 /usr/bin/sndiod
 4257 ??  Is      0:00.00 /usr/sbin/apmd
 1294 ??  Is      0:00.02 /usr/sbin/cron
15268 ??  Is      0:00.00 ssh-agent -s
19723 ??  S       0:05.77 /usr/X11R6/bin/X :0
-auth /home/phancy/.serverauth.29344 (Xorg) 5474 ??  I       0:00.00 X:
[priv] (Xorg) 6557 ??  Is      0:00.05 /usr/X11R6/bin/xterm
 6305 ??  Ss      0:04.58 /usr/X11R6/bin/xterm
10497 p0  Is+     0:00.01 ksh
11054 p2  Is+     0:00.01 ksh
 1042 p3  Is      0:00.01 ksh
17691 p3  S       0:00.10 ksh
32227 p3  R+      0:00.00 ps -ax
21353 C0  Is      0:00.02 -ksh (ksh)
29344 C0  I+      0:00.01 /bin/sh /usr/X11R6/bin/startx
 9451 C0  I+      0:00.01 xinit /home/phancy/.xinitrc
-- /usr/X11R6/bin/X :0 -auth /home/phancy/.serverauth.29344 30372 C0
I       0:00.00 sh /home/phancy/.xinitrc 3026 C0  I
0:00.21 /usr/X11R6/bin/xterm 19221 C0  I
0:00.06 /usr/X11R6/bin/cwm 1931 C0  I
0:00.03 /usr/X11R6/bin/xconsole 16685 C0  I
0:00.06 /usr/X11R6/bin/xclock 5277 C1  Is+
0:00.00 /usr/libexec/getty std.9600 ttyC1 23136 C2  Is+
0:00.00 /usr/libexec/getty std.9600 ttyC2 13609 C3  Is+
0:00.00 /usr/libexec/getty std.9600 ttyC3 19566 C5  Is+
0:00.00 /usr/libexec/getty std.9600 ttyC5

### systat ifstat ###

    4 users    Load 0.45 0.43 0.29                     Thu May  9 16:20:56 2013

IFACE     STATE DESC             IPKTS IBYTES  IERRS  OPKTS OBYTES  OERRS  COLLS
sis0      dn                         0      0      0      0      0      0      0
em0       up:U                      17   1354      0     29  41193      0      0
enc0      dn:U                       0      0      0      0      0      0      0
lo0       up                         0      0      0      0      0      0      0
pflog0    up                         0      0      0      0      0      0      0
Totals                              17   1354      0     29  41193      0      0


### systat nfsserver ###

    4 users    Load 0.51 0.44 0.29                     Thu May  9 16:19:46 2013

  RPC COUNTS                   CACHE STATS               WRITES                
     Getattr            254         Inprog             0 WriteOps          53286
     Setattr             86           Idem             0 WriteRPC          53286
      Lookup             89       Non-idem             0 Opsaved               0
    Readlink              0         Misses             2                        
        Read         892742                                                    
       Write          53286                                                    
      Create             17                                                    
      Remove              0                                                    
      Rename              0                                                    
        Link              0                                                    
     Symlink              0                                                    
       Mkdir              4                                                    
       Rmdir              0                                                    
     Readdir             38                                                    
    RdirPlus              4                                                    
      Access              0                                                    
       Mknod              0                                                    
      Fsstat           2059                                                    
      Fsinfo              2                                                    
    PathConf              1                                                    
      Commit            863                                                    
                                                                               
  Ret-Failed             50         Faults             0                        
                     


### systat vm ###

    4 users    Load 0.47 0.44 0.29                     Thu May  9
16:20:42 2013

### top ###

load averages:  0.76,  0.31,  0.21    gaia.sqrt2.de 16:14:59
52 processes: 51 idle, 1 on processor
CPU states:  0.1% user,  0.0% nice,  0.7% system,  1.0% interrupt, 98.2% idle
Memory: Real: 38M/252M act/tot Free: 745M Cache: 190M Swap: 0K/121M

  PID USERNAME PRI NICE  SIZE   RES STATE     WAIT      TIME    CPU COMMAND
 3955 root       2    0  216K  128K sleep     nfsd      1:08  5.62% nfsd
19723 phancy     2    0   11M   16M sleep     select    0:05  0.00% Xorg
 6305 phancy     2    0 2444K 5196K sleep     select    0:04  0.00% xterm
30282 _smtpd     2    0 1596K 2084K sleep     kqread    0:02  0.00% smtpd
18955 root       2    0 1548K 2136K sleep     kqread    0:01  0.00% smtpd
    1 root      10    0  512K  352K idle      wait      0:01  0.00% init
  119 _smtpd     2    0 1488K 1996K sleep     kqread    0:01  0.00% smtpd
11483 _smtpd     2    0 1332K 1864K sleep     kqread    0:01  0.00% smtpd
 8489 _smtpd     2    0 1404K 1856K sleep     kqread    0:01  0.00% smtpd
19465 phancy     2    0  880K 1768K sleep     poll      0:01  0.00% top
 3216 _smtpd     2    0 1256K 1664K sleep     kqread    0:01  0.00% smtpd
 4442 _smtpd     2    0 1192K 1640K sleep     kqread    0:01  0.00% smtpd
22089 _smtpd     2    0 1256K 1672K sleep     kqread    0:01  0.00% smtpd
29320 _smtpd     2    0 1120K 1592K sleep     kqread    0:00  0.00% smtpd
27053 _pflogd    4    0  732K  328K sleep     bpf       0:00  0.00% pflogd
 3026 phancy     2    0 1648K 3676K sleep     select    0:00  0.00% xterm
16685 phancy     2    0  888K 4248K idle      poll      0:00  0.00% xclock
19221 phancy     2    0  884K 3812K idle      select    0:00  0.00% cwm

### vmstat ###

 procs    memory       page                    disks    traps          cpu
 r b w    avm     fre  flt  re  pi  po  fr  sr wd0 wd1  int   sys   cs us sy id
 0 0 0  38444  771772   13   0   0   0   0   0   1  33  381   215  250  0  2 98

--
Sebastian Neuper <[hidden email]>

Reply | Threaded
Open this post in threaded view
|

Re: Poor NFS performance

Brett Mahar-2
On 05/10/13 14:04, Sebastian Neuper wrote:>> When I look at `top`, the
CPU stays below 10% most of the time. The
 >> network is 1Gbit/s both sides and switch. So I think I could get a
 >> better performance than 20MB/s while coping files to and around
 >> 4MB/s when coping from that server.
 >>
 >> But where is the bottleneck?

Not sure if it will help in your case but in the past when I used NFS,
setting vfs.nfs.iothreads=20 in /etc/sysctl.conf (the default is 4)
resulted in more stable performance.

Brett.

Reply | Threaded
Open this post in threaded view
|

Re: Poor NFS performance

Sebastian Neuper
On Fri, 10 May 2013 19:22:08 +1000
Brett Mahar <[hidden email]> wrote:

> On 05/10/13 14:04, Sebastian Neuper wrote:>> When I look at `top`, the
> CPU stays below 10% most of the time. The
>  >> network is 1Gbit/s both sides and switch. So I think I could get a
>  >> better performance than 20MB/s while coping files to and around
>  >> 4MB/s when coping from that server.
>  >>
>  >> But where is the bottleneck?
>
> Not sure if it will help in your case but in the past when I used NFS,
> setting vfs.nfs.iothreads=20 in /etc/sysctl.conf (the default is 4)
> resulted in more stable performance.
>
> Brett.
>

Hi Brett. Thanks for the hint, but there is no real change in
performance. Sebastian.

--
Sebastian Neuper <[hidden email]>

Reply | Threaded
Open this post in threaded view
|

Re: Poor NFS performance

Jan Stary
In reply to this post by Brett Mahar-2
On May 10 19:22:08, [hidden email] wrote:

> On 05/10/13 14:04, Sebastian Neuper wrote:>> When I look at `top`,
> the CPU stays below 10% most of the time. The
> >> network is 1Gbit/s both sides and switch. So I think I could get a
> >> better performance than 20MB/s while coping files to and around
> >> 4MB/s when coping from that server.
> >>
> >> But where is the bottleneck?
>
> Not sure if it will help in your case but in the past when I used
> NFS, setting vfs.nfs.iothreads=20 in /etc/sysctl.conf (the default
> is 4) resulted in more stable performance.

The same has made my 5.2-current panic a few months ago.

Reply | Threaded
Open this post in threaded view
|

Re: Poor NFS performance

Jan Stary
In reply to this post by Sebastian Neuper
On May 09 16:03:30, [hidden email] wrote:

> Hi again.
>
> Long time ago, I asked about [1] hardware recommendations to build
> a home file / video server, which streams HD videos for Avid.
>
> Quite late, but thanks for your responses.
>
> I bought a raid-controller  and an em-network card and recycled a
> really old pc, installed OpenBSD 5.3 Current on a 2TB HDD ( My first
> one and Nick knows that disk [2] ) and made a Softraid RAID 1 with
> 2x2.5' 1TB Drives. Then I upgraded Windows 7 to Ultimate to get the
> build-in NFS Client. Started nfsd, statd, lockd etc.
>
> Good news: It works. But only in SD. On HD it hangs every 10 sec
> for a few. And I don't think, this is so, because I play
> Kate Ryan LoveLife music video to test it. ;)
>
> When I look at `top`, the CPU stays below 10% most of the time. The
> network is 1Gbit/s both sides and switch. So I think I could get a
> better performance than 20MB/s while coping files to and around
> 4MB/s when coping from that server.
>
> But where is the bottleneck?
>  
> I uploaded the output of these commands:
>
> dmesg
> netstat -m
> nfsstat
> nfsstat -c
> ps ax
> systat ifstat
> systat nfsserver
> systat vm
> top
> vmstat
> and a screenshot of Windows netdrive options
>
> on http://www.sqrt2.de/log .
>
> If you need more, just mail.

How is the filesystem mounted on the OpenBSD NFS server,
how is it exported, and how is it mounted on the Win client?

Reply | Threaded
Open this post in threaded view
|

Re: Poor NFS performance

David Coppa
On Mon, Jul 8, 2013 at 9:00 AM, Jan Stary <[hidden email]> wrote:

> On May 09 16:03:30, [hidden email] wrote:
>> Hi again.
>>
>> Long time ago, I asked about [1] hardware recommendations to build
>> a home file / video server, which streams HD videos for Avid.
>>
>> Quite late, but thanks for your responses.
>>
>> I bought a raid-controller  and an em-network card and recycled a
>> really old pc, installed OpenBSD 5.3 Current on a 2TB HDD ( My first
>> one and Nick knows that disk [2] ) and made a Softraid RAID 1 with
>> 2x2.5' 1TB Drives. Then I upgraded Windows 7 to Ultimate to get the
>> build-in NFS Client. Started nfsd, statd, lockd etc.
>>
>> Good news: It works. But only in SD. On HD it hangs every 10 sec
>> for a few. And I don't think, this is so, because I play
>> Kate Ryan LoveLife music video to test it. ;)
>>
>> When I look at `top`, the CPU stays below 10% most of the time. The
>> network is 1Gbit/s both sides and switch. So I think I could get a
>> better performance than 20MB/s while coping files to and around
>> 4MB/s when coping from that server.
>>
>> But where is the bottleneck?

Can you try the attached diff?

[demime 1.01d removed an attachment of type application/x-gzip which had a name of nfs-performance.patch.gz]

Reply | Threaded
Open this post in threaded view
|

Re: Poor NFS performance

Sebastian Neuper
Hi David, hi Jan. Thanks for your response.

On Mon, 8 Jul 2013 09:00:52 +0200
Jan Stary <[hidden email]> wrote:

> How is the filesystem mounted on the OpenBSD NFS server,
> how is it exported, and how is it mounted on the Win client?

the output of mount:

/dev/wd0a on / type ffs (local)
/dev/wd0e on /home type ffs (NFS exported, local, nodev, nosuid)
/dev/wd0d on /usr type ffs (local, nodev)
/dev/sd0a on /usr/src type ffs (NFS exported, local, nodev, softdep)
localhost:/home/phancy/www on /var/www/htdocs type nfs (nodev, nosuid,
v3, udp, soft, intr, timeo=100)

my exports file:

# $OpenBSD: exports,v 1.2 2002/05/31 08:15:44 pjanzen Exp $
#
# NFS exports Database
# See exports(5) for more information.  Be very careful:  misconfiguration
# of this file can result in your filesystems being readable by the world.

/home/phancy/www -ro localhost
/usr/src/media -network 192.168.2 -mask=255.255.255.0

mounted on win simply with Ultimate Build-in NFS Client,
here the command:

mount \\192.168.2.108\usr\src\media H:

All other free NFS clients on Windows crashes, or give me wrong
numbers, or can't mount the share.


On Mon, 8 Jul 2013 09:56:07 +0200
David Coppa <[hidden email]> wrote:

> On Mon, Jul 8, 2013 at 9:00 AM, Jan Stary <[hidden email]> wrote:
> > On May 09 16:03:30, [hidden email] wrote:
> >> But where is the bottleneck?
>
> Can you try the attached diff?
>

Tried it (dmesg below), but it didn't improve the performance.
On websites, no one here recommends, I found iperf, which give me
following results:

openbsd nfs server as iperf client to windows
$ iperf -c 192.168.2.106
------------------------------------------------------------
Client connecting to 192.168.2.106, TCP port 5001
TCP window size: 17.0 KByte (default)
------------------------------------------------------------
[  3] local 192.168.2.108 port 11633 connected with 192.168.2.106 port
5001 [ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec   566 MBytes   475 Mbits/sec

openbsd nfs server as iperf server, windows as client:
$ iperf -s
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 16.0 KByte (default)
------------------------------------------------------------
[  4] local 192.168.2.108 port 5001 connected with 192.168.2.106 port
49449 [ ID] Interval       Transfer     Bandwidth
[  4]  0.0-10.0 sec   516 MBytes   432 Mbits/sec

If I boot both devices to openbsd, iperf shows:

$ iperf -c 192.168.2.106
------------------------------------------------------------
Client connecting to 192.168.2.106, TCP port 5001
TCP window size: 31.1 KByte (default)
------------------------------------------------------------
[  3] local 192.168.2.108 port 22735 connected with 192.168.2.106 port
5001 [ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec   559 MBytes   469 Mbits/sec

$ iperf -s
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 16.0 KByte (default)
------------------------------------------------------------
[  4] local 192.168.2.108 port 5001 connected with 192.168.2.106 port
28918 [ ID] Interval       Transfer     Bandwidth
[  4]  0.0-10.0 sec   807 MBytes   675 Mbits/sec

So the "OpenBSD network" seems to be faster, but with 469Mbit/sec too
slow as an HD Videoserver. Perhaps the PCI bus, perhaps the PCI NIC
itself, or some bridge? I'm not well versed in it, but it is the old
hardware for sure. Is it worth digging?

Best.

Sebastian.



pic 2 int 17 for native-PCI interrupt
pciide1: port 2: device$present, speed: 1.5Gb/s
wd1 at pciide1 channel 2 drive 0: <SAMSUNG HN-M101MBB>
wd1: 16-sector PIO, LBA48, 953869MB, 1953525168 secuors
wd1(pciide5:2:0): using BIOS timings, Ultra\M--DMA mode 6
pciide1: port 3: device present, speed: 1.5Gb/s
wd2 at pciide1 channel 3 drive 0: <SAMSUNG HN-M101MBB>
wd2: 16-sector PIO, LBA48, 953869MB, 1953525168 sectors
wd2(pciide1:3:0): usi\M-ng BIOS timings, Ultra-DMA mode 6
em0 at pci0 dev 7 function 0 "Intel 82541GI" rev 0x05: apic 2 int 18,
address 90:e2:ba:30:ba:a0 isa0 at pcib0
isadma0 at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
pckbc0 at isa0 port 0x60/5
pckbd0 at pckbc0 (kbd slot)
pckbc0: using izq 1 for kbd slot
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pms0 at pckbc0 (aux slot)
pckbc0: using irq 12 for aux slot\M^Jwsmouse0 at pms0 mux 0
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
lpt0 at isa0 port 0x378/4 irq 7
wbsio0 at isa0 port 0x2e/2: W83697HF!rev 0x16
lm1 at wbsio0 port 0x290/8: W8\M-3697HF
npx0 at isa0 port 0xf0/16: reported by CPUID; using exception 16
fdc0 at isa0 port 0x3f0/6 irq 6 drq 2
usb1 at ohci0: USB revision 1.0
uhub1 at usb1 "SiS OHCI"root hub" rev 1.00/1.00 addr 1
usb2 at ohci1: USB revision 1.0
uhub2 at usb2 \M-"SiS OHCI root hub" rev 1.00/1.00 addr 1
usb3 at ohci2: USB revision 1.0
uhub3 at usb3 "SiS OHCI root hub" rev 1.00/1.00 addr 1
mtrr: Pentium Pro MTRR support
vscsi0 at root
scsibus0 at vscsi0: 256 targets
softraid0 at root
scsibus1 at softraid0: 256 targets
sd0 at scsibus1 targ 1 lun 0: <OPENBSD, SR RAID 1, 005> SCSI2 0/direct
fixed sd0: 953869MB, 512 bytes/sector, 1953524576 sectors
root on wd0a (83291e85336d8f4d.a) swap on wd0b dump on wd0b
OpenBSD 5.3-current (GENERIC) #10: Thu Jun 27 16:20:01 MDT 2013
    [hidden email]:/usr/src/sys/arch/i386/compile/GENERIC
cpu0: Intel(R) Pentium(R) 4 CPU 2.66GHz ("GenuineIntel" 686-class) 2.68
GHz cpu0:
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,CNXT-ID,PERF
real mem  = 1073205248 (1023MB) avail mem = 1044221952 (995MB)
mainbus0 at root
bios0 at mainbus0: AT/286+ BIOS, date 09/11/02, BIOS32 rev. 0 @
0xfb3d0, SMBIOS rev. 2.3 @ 0xf0800 (32 entries) bios0: vendor Phoenix
Technologies, LTD version "6.00 PG" date 09/11/2002 bios0: MEDIONPC
MS-6701 acpi0 at bios0: rev 0
acpi0: sleep states S0 S3 S4 S5
acpi0: tables DSDT FACP APIC
acpi0: wakeup devices FUTS(S4) USB0(S5) USB1(S5) USB2(S5) USB3(S5) MAC0
(S5) AMR0(S4) UAR1(S5) PS2M(S5) PS2K(S4) PCI0(S5) acpitimer0 at acpi0:
3579545 Hz, 24 bits acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: apic clock running at 133MHz
ioapic0 at mainbus0: apid 2 pa 0xfec00000, version 14, 24 pins
ioapic0: misconfigured as apic 0, remapped to apid 2
acpiprt0 at acpi0: bus 0 (PCI0)
acpicpu0 at acpi0
acpitz0 at acpi0: critical temperature is 100 degC
acpibtn0 at acpi0: PWRB
acpibtn1 at acpi0: FUTS
bios0: ROM list: 0xc0000/0xf800 0xd0000/0x8000!
pci0 at mainbus0 bus 0: configuration mode 1 (bios)
pchb0 at pci0 dev 0 function 0 "SiS 648 PCI" rev 0x02
sisagp0 at pchb0
agp0 at sisagp0: aperture at 0xe0000000, size 0x8000000
ppb0 at pci0 dev 1 function 0 "SiS 86C201 AGP" rev 0x00
pci1 at ppb0 bus 1
vga1 at pci1 dev 0 function 0 "NVIDIA GeForce FX 5900 Ultra" rev 0xa1
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
pcib0 at pci0 dev 2 function 0 "SiS 85C503 System" rev 0x04
"SiS 7007 FireWire" rev 0x00 at pci0 dev 2 function 3 not configured
pciide0 at pci0 dev 2 function 5 "SiS 5513 EIDE" rev 0x00: 648: DMA,
channel 0 configured to compatibility, channel 1 configured to
compatibility wd0 at pciide0 channel 0 drive 0: <ST52520A> wd0:
16-sector PIO, LBA, 2446MB, 5009760 sectors wd0(pciide0:0:0): using PIO
mode 4, DMA mode 2 pciide0: channel 1 disabled (no drives)
auich0 at pci0 dev 2 function 7 "SiS 7012 AC97" rev 0xa0: apic 2 int
18, SiS7012 AC97 ac97: codec id 0x414c4720 (Avance Logic ALC650)
ac97: codec features 20 bit DAC, 18 bit ADC, Realtek 3D
audio0 at auich0
ohci0 at pci0 dev 3 function 0 "SiS 5597/5598 USB" rev 0x0f: apic 2 int
20, version 1.0, legacy support ohci1 at pci0 dev 3 function 1 "SiS
5597/5598 USB" rev 0x0f: apic 2 int 21, version 1.0, legacy support
ohci2 at pci0 dev 3 function 2 "SiS 5597/5598 USB" rev 0x0f: apic 2 int
22, version 1.0, legacy support ehci0 at pci0 dev 3 function 3 "SiS
7002 USB" rev 0x00: apic 2 int 23 usb0 at ehci0: USB revision 2.0 uhub0
at usb0 "SiS EHCI root hub" rev 2.00/1.00 addr 1 sis0 at pci0 dev 4
function 0 "SiS 900 10/100BaseTX" rev 0x91: apic 2 int 19, address
00:10:dc:7e:88:23 rlphy0 at sis0 phy 1: RTL8201L 10/100 PHY, rev. 1
pciide1 at pci0 dev 6 function 0 "CMD Technology SiI3114 SATA" rev
0x02: DMA pciide1: using apic 2 int 17 for native-PCI interrupt
pciide1: port 2: device present, speed: 1.5Gb/s wd1 at pciide1 channel
2 drive 0: <SAMSUNG HN-M101MBB> wd1: 16-sector PIO, LBA48, 953869MB,
1953525168 sectors wd1(pciide1:2:0): using BIOS timings, Ultra-DMA mode
6 pciide1: port 3: device present, speed: 1.5Gb/s
wd2 at pciide1 channel 3 drive 0: <SAMSUNG HN-M101MBB>
wd2: 16-sector PIO, LBA48, 953869MB, 1953525168 sectors
wd2(pciide1:3:0): using BIOS timings, Ultra-DMA mode 6
em0 at pci0 dev 7 function 0 "Intel 82541GI" rev 0x05: apic 2 int 18,
address 90:e2:ba:30:ba:a0 isa0 at pcib0
isadma0 at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
pckbc0 at isa0 port 0x60/5
pckbd0 at pckbc0 (kbd slot)
pckbc0: using irq 1 for kbd slot
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pms0 at pckbc0 (aux slot)
pckbc0: using irq 12 for aux slot
wsmouse0 at pms0 mux 0
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
lpt0 at isa0 port 0x378/4 irq 7
wbsio0 at isa0 port 0x2e/2: W83697HF rev 0x16
lm1 at wbsio0 port 0x290/8: W83697HF
npx0 at isa0 port 0xf0/16: reported by CPUID; using exception 16
fdc0 at isa0 port 0x3f0/6 irq 6 drq 2
usb1 at ohci0: USB revision 1.0
uhub1 at usb1 "SiS OHCI root hub" rev 1.00/1.00 addr 1
usb2 at ohci1: USB revision 1.0
uhub2 at usb2 "SiS OHCI root hub" rev 1.00/1.00 addr 1
usb3 at ohci2: USB revision 1.0
uhub3 at usb3 "SiS OHCI root hub" rev 1.00/1.00 addr 1
mtrr: Pentium Pro MTRR support
vscsi0 at root
scsibus0 at vscsi0: 256 targets
softraid0 at root
scsibus1 at softraid0: 256 targets
softraid0: sd0 was not shutdown properly
sd0 at scsibus1 targ 1 lun 0: <OPENBSD, SR RAID 1, 005> SCSI2 0/direct
fixed sd0: 953869MB, 512 bytes/sector, 1953524576 sectors
root on wd0a (83291e85336d8f4d.a) swap on wd0b dump on wd0b
WARNING: / was not properly unmounted
syncing disks...
OpenBSD 5.4-beta (GENERIC) #0: Mon Jul  8 19:48:38 CEST 2013
    [hidden email]:/usr/src/sys/arch/i386/compile/GENERIC
cpu0: Intel(R) Pentium(R) 4 CPU 2.66GHz ("GenuineIntel" 686-class) 2.68
GHz cpu0:
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,CNXT-ID,PERF
real mem  = 1073205248 (1023MB) avail mem = 1044221952 (995MB)
mainbus0 at root
bios0 at mainbus0: AT/286+ BIOS, date 09/11/02, BIOS32 rev. 0 @
0xfb3d0, SMBIOS rev. 2.3 @ 0xf0800 (32 entries) bios0: vendor Phoenix
Technologies, LTD version "6.00 PG" date 09/11/2002 bios0: MEDIONPC
MS-6701 acpi0 at bios0: rev 0
acpi0: sleep states S0 S3 S4 S5
acpi0: tables DSDT FACP APIC
acpi0: wakeup devices FUTS(S4) USB0(S5) USB1(S5) USB2(S5) USB3(S5) MAC0
(S5) AMR0(S4) UAR1(S5) PS2M(S5) PS2K(S4) PCI0(S5) acpitimer0 at acpi0:
3579545 Hz, 24 bits acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: apic clock running at 133MHz
ioapic0 at mainbus0: apid 2 pa 0xfec00000, version 14, 24 pins
ioapic0: misconfigured as apic 0, remapped to apid 2
acpiprt0 at acpi0: bus 0 (PCI0)
acpicpu0 at acpi0
acpitz0 at acpi0: critical temperature is 100 degC
acpibtn0 at acpi0: PWRB
acpibtn1 at acpi0: FUTS
bios0: ROM list: 0xc0000/0xf800 0xd0000/0x8000!
pci0 at mainbus0 bus 0: configuration mode 1 (bios)
pchb0 at pci0 dev 0 function 0 "SiS 648 PCI" rev 0x02
sisagp0 at pchb0
agp0 at sisagp0: aperture at 0xe0000000, size 0x8000000
ppb0 at pci0 dev 1 function 0 "SiS 86C201 AGP" rev 0x00
pci1 at ppb0 bus 1
vga1 at pci1 dev 0 function 0 "NVIDIA GeForce FX 5900 Ultra" rev 0xa1
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
pcib0 at pci0 dev 2 function 0 "SiS 85C503 System" rev 0x04
"SiS 7007 FireWire" rev 0x00 at pci0 dev 2 function 3 not configured
pciide0 at pci0 dev 2 function 5 "SiS 5513 EIDE" rev 0x00: 648: DMA,
channel 0 configured to compatibility, channel 1 configured to
compatibility wd0 at pciide0 channel 0 drive 0: <ST52520A> wd0:
16-sector PIO, LBA, 2446MB, 5009760 sectors wd0(pciide0:0:0): using PIO
mode 4, DMA mode 2 pciide0: channel 1 disabled (no drives)
auich0 at pci0 dev 2 function 7 "SiS 7012 AC97" rev 0xa0: apic 2 int
18, SiS7012 AC97 ac97: codec id 0x414c4720 (Avance Logic ALC650)
ac97: codec features 20 bit DAC, 18 bit ADC, Realtek 3D
audio0 at auich0
ohci0 at pci0 dev 3 function 0 "SiS 5597/5598 USB" rev 0x0f: apic 2 int
20, version 1.0, legacy support ohci1 at pci0 dev 3 function 1 "SiS
5597/5598 USB" rev 0x0f: apic 2 int 21, version 1.0, legacy support
ohci2 at pci0 dev 3 function 2 "SiS 5597/5598 USB" rev 0x0f: apic 2 int
22, version 1.0, legacy support ehci0 at pci0 dev 3 function 3 "SiS
7002 USB" rev 0x00: apic 2 int 23 usb0 at ehci0: USB revision 2.0 uhub0
at usb0 "SiS EHCI root hub" rev 2.00/1.00 addr 1 sis0 at pci0 dev 4
function 0 "SiS 900 10/100BaseTX" rev 0x91: apic 2 int 19, address
00:10:dc:7e:88:23 rlphy0 at sis0 phy 1: RTL8201L 10/100 PHY, rev. 1
pciide1 at pci0 dev 6 function 0 "CMD Technology SiI3114 SATA" rev
0x02: DMA pciide1: using apic 2 int 17 for native-PCI interrupt
pciide1: port 2: device present, speed: 1.5Gb/s wd1 at pciide1 channel
2 drive 0: <SAMSUNG HN-M101MBB> wd1: 16-sector PIO, LBA48, 953869MB,
1953525168 sectors wd1(pciide1:2:0): using BIOS timings, Ultra-DMA mode
6 pciide1: port 3: device present, speed: 1.5Gb/s
wd2 at pciide1 channel 3 drive 0: <SAMSUNG HN-M101MBB>
wd2: 16-sector PIO, LBA48, 953869MB, 1953525168 sectors
wd2(pciide1:3:0): using BIOS timings, Ultra-DMA mode 6
em0 at pci0 dev 7 function 0 "Intel 82541GI" rev 0x05: apic 2 int 18,
address 90:e2:ba:30:ba:a0 isa0 at pcib0
isadma0 at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
pckbc0 at isa0 port 0x60/5
pckbd0 at pckbc0 (kbd slot)
pckbc0: using irq 1 for kbd slot
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pms0 at pckbc0 (aux slot)
pckbc0: using irq 12 for aux slot
wsmouse0 at pis0 mux 0
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
lpt0 at isa0 port 0x378/4 irq 7
wbsio0 at isa0 port 0x2e/2: W83697HF rev 0x16
lm1 at wbsio0 port 0x290/8: W83697HF
npx0 at isa0 port 0xf0/16: reported by CPUID; using exception 16
fdc0 at isa0 port 0x3f0/6 irq 6 drq 2
usb1 at ohci0: USB revision 1.0
uhub1 at usb1 "SiS OHCI root hub" rev 1.00/1.00 addr 1
usb2 at ohci1: USB revision 1.0
uhub2 at usb2 "SiS OHCI root hub" rev 1.00/1.00 addr 1
usb3 at ohci2: USB revision 1.0
uhub3 at usb3 "SiS OHCI root hub" rev 1.00/1.00 addr 1
mtrr: Pentium Pro MTRR support
vscsi0 at root
scsibus0 at vscsi0: 256 targets
softraid0 at root
scsibus1 at softraid0: 256 targets
sd0 at scsibus1 targ 1 lun 0: <OPENBSD, SR RAID 1, 005> SCSI2 0/direct
fixed sd0: 953869MB, 512 bytes/sector, 1953524576 sectors
root on wd0a (83291e85336d8f4d.a) swap on wd0b dump on wd0b
syncing disks... upt
p
OpenBSD 5.4-beta (GENERIC) #0: Mon Jul  8 19:48:38 CEST 2013
    [hidden email]:/usr/src/sys/arch/i386/compile/GENERIC
cpu0: Intel(R) Pentium(R) 4 CPU 2.66GHz ("GenuineIntel" 686-class) 2.68
GHz cpu0:
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,CNXT-ID,PERF
real mem  = 1073205248 (1023MB) avail mem = 1044221952 (995MB)
mainbus0 at root
bios0 at mainbus0: AT/286+ BIOS, date 09/11/02, BIOS32 rev. 0 @
0xfb3d0, SMBIOS rev. 2.3 @ 0xf0800 (32 entries) bios0: vendor Phoenix
Technologies, LTD version "6.00 PG" date 09/11/2002 bios0: MEDIONPC
MS-6701 acpi0 at bios0: rev 0
acpi0: sleep states S0 S3 S4 S5
acpi0: tables DSDT FACP APIC
acpi0: wakeup devices FUTS(S4) USB0(S5) USB1(S5) USB2(S5) USB3(S5) MAC0
(S5) AMR0(S4) UAR1(S5) PS2M(S5) PS2K(S4) PCI0(S5) acpitimer0 at acpi0:
3579545 Hz, 24 bits acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: apic clock running at 133MHz
ioapic0 at mainbus0: apid 2 pa 0xfec00000, version 14, 24 pins
ioapic0: misconfigured as apic 0, remapped to apid 2
acpiprt0 at acpi0: bus 0 (PCI0)
acpicpu0 at acpi0
acpitz0 at acpi0: critical temperature is 100 degC
acpibtn0 at acpi0: PWRB
acpibtn1 at acpi0: FUTS
bios0: ROM list: 0xc0000/0xf800 0xd0000/0x8000!
pci0 at mainbus0 bus 0: configuration mode 1 (bios)
pchb0 at pci0 dev 0 function 0 "SiS 648 PCI" rev 0x02
sisagp0 at pchb0
agp0 at sisagp0: aperture at 0xe0000000, size 0x8000000
ppb0 at pci0 dev 1 function 0 "SiS 86C201 AGP" rev 0x00
pci1 at ppb0 bus 1
vga1 at pci1 dev 0 function 0 "NVIDIA GeForce FX 5900 Ultra" rev 0xa1
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
pcib0 at pci0 dev 2 function 0 "SiS 85C503 System" rev 0x04
"SiS 7007 FireWire" rev 0x00 at pci0 dev 2 function 3 not configured
pciide0 at pci0 dev 2 function 5 "SiS 5513 EIDE" rev 0x00: 648: DMA,
channel 0 configured to compatibility, channel 1 configured to
compatibility wd0 at pciide0 channel 0 drive 0: <ST52520A> wd0:
16-sector PIO, LBA, 2446MB, 5009760 sectors wd0(pciide0:0:0): using PIO
mode 4, DMA mode 2 pciide0: channel 1 disabled (no drives)
auich0 at pci0 dev 2 function 7 "SiS 7012 AC97" rev 0xa0: apic 2 int
18, SiS7012 AC97 ac97: codec id 0x414c4720 (Avance Logic ALC650)
ac97: codec features 20 bit DAC, 18 bit ADC, Realtek 3D
audio0 at auich0
ohci0 at pci0 dev 3 function 0 "SiS 5597/5598 USB" rev 0x0f: apic 2 int
20, version 1.0, legacy support ohci1 at pci0 dev 3 function 1 "SiS
5597/5598 USB" rev 0x0f: apic 2 int 21, version 1.0, legacy support
ohci2 at pci0 dev 3 function 2 "SiS 5597/5598 USB" rev 0x0f: apic 2 int
22, version 1.0, legacy support ehci0 at pci0 dev 3 function 3 "SiS
7002 USB" rev 0x00: apic 2 int 23 usb0 at ehci0: USB revision 2.0 uhub0
at usb0 "SiS EHCI root hub" rev 2.00/1.00 addr 1 sis0 at pci0 dev 4
function 0 "SiS 900 10/100BaseTX" rev 0x91: apic 2 int 19, address
00:10:dc:7e:88:23 rlphy0 at sis0 phy 1: RTL8201L 10/100 PHY, rev. 1
pciide1 at pci0 dev 6 function 0 "CMD Technology SiI3114 SATA" rev
0x02: DMA pciide1: using apic 2 int 17 for native-PCI interrupt
pciide1: port 2: device present, speed: 1.5Gb/s wd1 at pciide1 channel
2 drive 0: <SAMSUNG HN-M101MBB> wd1: 16-sector PIO, LBA48, 953869MB,
1953525168 sectors wd1(pciide1:2:0): using BIOS timings, Ultra-DMA mode
6 pciide1: port 3: device present, speed: 1.5Gb/s
wd2 at pciide1 channel 3 drive 0: <SAMSUNG HN-M101MBB>
wd2: 16-sector PIO, LBA48, 953869MB, 1953525168 sectors
wd2(pciide1:3:0): using BIOS timings, Ultra-DMA mode 6
em0 at pci0 dev 7 function 0 "Intel 82541GI" rev 0x05: apic 2 int 18,
address 90:e2:ba:30:ba:a0 isa0 at pcib0
isadma0 at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
pckbc0 at isa0 port 0x60/5
pckbd0 at pckbc0 (kbd slot)
pckbc0: using irq 1 for kbd slot
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pms0 at pckbc0 (aux slot)
pckbc0: using irq 12 for aux slot
wsmouse0 at pms0 mux 0
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
lpt0 at isa0 port 0x378/4 irq 7
wbsio0 at isa0 port 0x2e/2: W83697HF rev 0x16
lm1 at wbsio0 port 0x290/8: W83697HF
npx0 at isa0 port 0xf0/16: reported by CPUID; using exception 16
fdc0 at isa0 port 0x3f0/6 irq 6 drq 2
usb1 at ohci0: USB revision 1.0
uhub1 at usb1 "SiS OHCI root hub" rev 1.00/1.00 addr 1
usb2 at ohci1: USB revision 1.0
uhub2 at usb2 "SiS OHCI root hub" rev 1.00/1.00 addr 1
usb3 at ohci2: USB revision 1.0
uhub3 at usb3 "SiS OHCI root hub" rev 1.00/1.00 addr 1
mtrr: Pentium Pro MTRR support
vscsi0 at root
scsibus0 at vscsi0: 256 targets
softraid0 at root
scsibus1 at softraid0: 256 targets
sd0 at scsibus1 targ 1 lun 0: <OPENBSD, SR RAID 1, 005> SCSI2 0/direct
fixed sd0: 953869MB, 512 bytes/sector, 1953524576 sectors
root on wd0a (83291e85336d8f4d.a) swap on wd0b dump on wd0b

--
Sebastian Neuper <[hidden email]>

Reply | Threaded
Open this post in threaded view
|

Re: Poor NFS performance

Jan Stary
On Jul 08 22:07:53, [hidden email] wrote:

> Hi David, hi Jan. Thanks for your response.
>
> On Mon, 8 Jul 2013 09:00:52 +0200
> Jan Stary <[hidden email]> wrote:
>
> > How is the filesystem mounted on the OpenBSD NFS server,
> > how is it exported, and how is it mounted on the Win client?
>
> the output of mount:
>
> /dev/wd0a on / type ffs (local)
> /dev/wd0e on /home type ffs (NFS exported, local, nodev, nosuid)
> /dev/wd0d on /usr type ffs (local, nodev)
> /dev/sd0a on /usr/src type ffs (NFS exported, local, nodev, softdep)
> localhost:/home/phancy/www on /var/www/htdocs type nfs (nodev, nosuid,
> v3, udp, soft, intr, timeo=100)
>
> my exports file:
>
> # $OpenBSD: exports,v 1.2 2002/05/31 08:15:44 pjanzen Exp $
> #
> # NFS exports Database
> # See exports(5) for more information.  Be very careful:  misconfiguration
> # of this file can result in your filesystems being readable by the world.
>
> /home/phancy/www -ro localhost
> /usr/src/media -network 192.168.2 -mask=255.255.255.0
>
> mounted on win simply with Ultimate Build-in NFS Client,
> here the command:
>
> mount \\192.168.2.108\usr\src\media H:

Details matter. Is this NFS2/NFS3? Is this a tcp/udp mount?
mount_nfs (as we know it on BSD) offers quite a few options
that can make a difference. Does the Win client have any
options?

In particular, being a video server, I suppose the Win client
does long sequential reads. I remember -r speeding it up.

> openbsd nfs server as iperf client to windows
> $ iperf -c 192.168.2.106
> ------------------------------------------------------------
> Client connecting to 192.168.2.106, TCP port 5001
> TCP window size: 17.0 KByte (default)
> ------------------------------------------------------------
> [  3] local 192.168.2.108 port 11633 connected with 192.168.2.106 port
> 5001 [ ID] Interval       Transfer     Bandwidth
> [  3]  0.0-10.0 sec   566 MBytes   475 Mbits/sec
>
> openbsd nfs server as iperf server, windows as client:
> $ iperf -s
> ------------------------------------------------------------
> Server listening on TCP port 5001

Is your NFS mount a TCP mount?

> TCP window size: 16.0 KByte (default)
> ------------------------------------------------------------
> [  4] local 192.168.2.108 port 5001 connected with 192.168.2.106 port
> 49449 [ ID] Interval       Transfer     Bandwidth
> [  4]  0.0-10.0 sec   516 MBytes   432 Mbits/sec
>
> If I boot both devices to openbsd, iperf shows:
>
> $ iperf -c 192.168.2.106
> ------------------------------------------------------------
> Client connecting to 192.168.2.106, TCP port 5001
> TCP window size: 31.1 KByte (default)
> ------------------------------------------------------------
> [  3] local 192.168.2.108 port 22735 connected with 192.168.2.106 port
> 5001 [ ID] Interval       Transfer     Bandwidth
> [  3]  0.0-10.0 sec   559 MBytes   469 Mbits/sec
> $ iperf -s
> ------------------------------------------------------------
> Server listening on TCP port 5001
> TCP window size: 16.0 KByte (default)
> ------------------------------------------------------------
> [  4] local 192.168.2.108 port 5001 connected with 192.168.2.106 port
> 28918 [ ID] Interval       Transfer     Bandwidth
> [  4]  0.0-10.0 sec   807 MBytes   675 Mbits/sec
> So the "OpenBSD network" seems to be faster, but with 469Mbit/sec too
> slow as an HD Videoserver.

The 469 is what iperf does, not how fast your NFS client can read.

> Perhaps the PCI bus, perhaps the PCI NIC itself, or some bridge?
> I'm not well versed in it, but it is the old
> hardware for sure. Is it worth digging?

It would be my wild guess that the NFS is your bottleneck
(rather than the hardware). Try various (NFS) mount options.

        Jan


BTW,

(1) why are you poluting a standardized directory like /usr/src
with something like /usr/src/media, serving videos?

(2) why are you "exporting" /home/phancy/www to /var/www/htdocs,
on your 2G wd0 disk?

Reply | Threaded
Open this post in threaded view
|

Re: Poor NFS performance

Sebastian Neuper
Hi Jan.

On Tue, 9 Jul 2013 10:22:48 +0200
Jan Stary <[hidden email]> wrote:

> > mount \\192.168.2.108\usr\src\media H:
>
> Details matter. Is this NFS2/NFS3? Is this a tcp/udp mount?
> mount_nfs (as we know it on BSD) offers quite a few options
> that can make a difference. Does the Win client have any
> options?
>
> In particular, being a video server, I suppose the Win client
> does long sequential reads. I remember -r speeding it up.

The options I know of, are on Microsoft Technet [1], no -r option.
Only read and write buffer size, time out, retry, soft or hard
mount and some file access and encoding options.
The netdrive properties of H: as mounted above give me:
read and write buffer size 32768 bytes (max), time out 0.8, retry 1,
soft mount and NFS3.

I tried with many different read and write sizes but no real
improvements.

> Is your NFS mount a TCP mount?

Client was set to UDP, but I also tested it with TCP and TCP+UDP.
TCP seems to be slower for Avid, I got more hangs there.

> It would be my wild guess that the NFS is your bottleneck
> (rather than the hardware). Try various (NFS) mount options.

nfsd(8) on openbsd only has 4 options. In sysctl you only can set
vfs.nfs.iothreads to max 20. And I don't know which other sysctl
variables also affects nfsd. On windows side there is the read/write
buffer which defaults to its max. And TCP/UDP, where UDP seems better.

> Jan
>
>
> BTW,
>
> (1) why are you poluting a standardized directory like /usr/src
> with something like /usr/src/media, serving videos?

I had to recompile the kernel, and my 2G system drive is slow and
loud. So I changed the mount point of my softraid from /media to
/usr/src and made a media folder for my video files. Simply a
workaround for silent and proper compiling and testing your patch.

> (2) why are you "exporting" /home/phancy/www to /var/www/htdocs,
> on your 2G wd0 disk?

Want to stay on current, but also everytime a clean install.
So all my data lives in my home directory (backup simpler)
also my webpage, which is exported with nfs to the 'right' htdocs
directory. With a simple backup script, an install.site, rc.firsttime
and siteXX.tgz, staying on current feels like just a long reboot.
 
Thanks for your help!

Best,
Sebastian.

[1] http://technet.microsoft.com/en-us/library/cc754350.aspx


--
Sebastian Neuper <[hidden email]>