Disk performance/benchmarking

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Disk performance/benchmarking

Gabriel George POPA
            Hello all,

   Could someone tell me which is the most used disk-benchmarking
solution for OpenBSD 3.8?
I'm running a small production server and I don't want to disrupt (too
much) its activity. I would like
of course to perform non-destructive tests. I think there's somethink
wrong with my disk/os performance
but I don't know where to start. I think a benchmark utility will be
good (SMART neveals nothing).
   He he he. I think all people are concerned about OpenBSD performance
these days...

                                                                                                             
Yours in BSDness,
                                                                                                                       
Gabriel George POPA

Reply | Threaded
Open this post in threaded view
|

Re: Disk performance/benchmarking

Sean Cody
If it is doing anything but the benchmark the results will be useless  
but you probably already know that.

Assuming the machine is completely idle with no services currently  
running (other than the one used to log in) then just try a bunch of  
test cases for the usage pattern you expect to see (or want to use)  
and interpret the numbers accordingly.  Reducing the variables that  
interact with what you are metering is the first step.

If you want a 'generic' (ie. not real usage) benchmark then try the  
iogen port.

But with any benchmarks don't put a lot of trust in the numbers  
especially if you only run one test set.
Run a bunch and pull out that old Stats 1 text and take the results  
with a big huge grain of salt and realize you are not benchmarking  
OpenBSD you are benching your hardware and configuration.

http://www.openbsd.org/cgi-bin/cvsweb/ports/sysutils/iogen/pkg/DESCR?
rev=1.2&content-type=text/x-cvsweb-markup
"iogen is an i/o generator.  It forks child processes that each run a  
mix of
reads and writes.  The idea is to generate heavily fragmented files  
to make the
hardware suffer as much as possible.  This tool has been used to test
filesystems, drivers, firmware and hardware devices.  It is by no  
means meant
as a performance measuring tool since it tries to recreate the worst  
case
scenario i/o."

You may already have considered all of the above but included in case  
others on the list have not.

On 27-Jun-06, at 7:57 AM, Gabriel George POPA wrote:

>
>   Could someone tell me which is the most used disk-benchmarking  
> solution for OpenBSD 3.8?
> I'm running a small production server and I don't want to disrupt  
> (too much) its activity. I would like
> of course to perform non-destructive tests. I think there's  
> somethink wrong with my disk/os performance
> but I don't know where to start. I think a benchmark utility will  
> be good (SMART neveals nothing).
>   He he he. I think all people are concerned about OpenBSD  
> performance these days...

--
Sean

Reply | Threaded
Open this post in threaded view
|

Re: Disk performance/benchmarking

Gabriel George POPA
I was mainly wanting to see a rough estimation of disk throughput
(MB/sec). And now I am interested to see
if packets get lost over my wide LAN here (I think a switch is
deffective, but I don't know what). What should I do?


Sean Cody wrote:

> If it is doing anything but the benchmark the results will be useless  
> but you probably already know that.
>
> Assuming the machine is completely idle with no services currently  
> running (other than the one used to log in) then just try a bunch of  
> test cases for the usage pattern you expect to see (or want to use)  
> and interpret the numbers accordingly.  Reducing the variables that  
> interact with what you are metering is the first step.
>
> If you want a 'generic' (ie. not real usage) benchmark then try the  
> iogen port.
>
> But with any benchmarks don't put a lot of trust in the numbers  
> especially if you only run one test set.
> Run a bunch and pull out that old Stats 1 text and take the results  
> with a big huge grain of salt and realize you are not benchmarking  
> OpenBSD you are benching your hardware and configuration.
>
> http://www.openbsd.org/cgi-bin/cvsweb/ports/sysutils/iogen/pkg/DESCR?
> rev=1.2&content-type=text/x-cvsweb-markup
> "iogen is an i/o generator.  It forks child processes that each run a  
> mix of
> reads and writes.  The idea is to generate heavily fragmented files  
> to make the
> hardware suffer as much as possible.  This tool has been used to test
> filesystems, drivers, firmware and hardware devices.  It is by no  
> means meant
> as a performance measuring tool since it tries to recreate the worst  
> case
> scenario i/o."
>

> You may already have considered all of the above but included in case  
> others on the list have not.
>
> On 27-Jun-06, at 7:57 AM, Gabriel George POPA wrote:
>
>>
>>   Could someone tell me which is the most used disk-benchmarking  
>> solution for OpenBSD 3.8?
>> I'm running a small production server and I don't want to disrupt  
>> (too much) its activity. I would like
>> of course to perform non-destructive tests. I think there's  
>> somethink wrong with my disk/os performance
>> but I don't know where to start. I think a benchmark utility will  be
>> good (SMART neveals nothing).
>>   He he he. I think all people are concerned about OpenBSD  
>> performance these days...

Reply | Threaded
Open this post in threaded view
|

Re: Disk performance/benchmarking

Sean Cody
dd(1) and iogen should give you said rough estimation.

As for transmission issues.
First take a look on either side for network 'errors'..
$ netstat -I hme0
Name    Mtu   Network     Address              Ipkts Ierrs    Opkts  
Oerrs Colls
hme0    1500  <Link>      08:00:20:c2:5f:f0  1498294     0    
742115    10 99837

Ierrs and Oerrs should be really low.
Colls are collissions.

If files get corrupted over the wire make a test file and hash [md5
(1)] it before transfer on test machine and after it has been  
transfered hash it on the target machine.
That will determine if the file corruption happened during transit.

For packet loss... ping the target machine then generate a lot of  
traffic to it (tcpblast, dd+netcat, etc.).
Then stop ping and the stats at the end will give you the relative  
packet loss (to ping).
Change target machine's switch port and try again.
Change testing machine's switch port and try again.
Try a different switch.

On 27-Jun-06, at 8:53 AM, Gabriel George POPA wrote:

> I was mainly wanting to see a rough estimation of disk throughput  
> (MB/sec). And now I am interested to see
> if packets get lost over my wide LAN here (I think a switch is  
> deffective, but I don't know what). What should I do?
>

--
Sean

Reply | Threaded
Open this post in threaded view
|

Re: Disk performance/benchmarking

Gabriel George POPA
            Thank you very much. That's what I was looking for.
Unfortunately at this time I have no
phisical access to some switches here (they are locked in two offices
and I don't have the key)...



Sean Cody wrote:

> dd(1) and iogen should give you said rough estimation.
>
> As for transmission issues.
> First take a look on either side for network 'errors'..
> $ netstat -I hme0
> Name    Mtu   Network     Address              Ipkts Ierrs    Opkts  
> Oerrs Colls
> hme0    1500  <Link>      08:00:20:c2:5f:f0  1498294     0    
> 742115    10 99837
>
> Ierrs and Oerrs should be really low.
> Colls are collissions.
>
> If files get corrupted over the wire make a test file and hash [md5
> (1)] it before transfer on test machine and after it has been  
> transfered hash it on the target machine.
> That will determine if the file corruption happened during transit.
>
> For packet loss... ping the target machine then generate a lot of  
> traffic to it (tcpblast, dd+netcat, etc.).
> Then stop ping and the stats at the end will give you the relative  
> packet loss (to ping).
> Change target machine's switch port and try again.
> Change testing machine's switch port and try again.
> Try a different switch.
>
> On 27-Jun-06, at 8:53 AM, Gabriel George POPA wrote:
>
>> I was mainly wanting to see a rough estimation of disk throughput  
>> (MB/sec). And now I am interested to see
>> if packets get lost over my wide LAN here (I think a switch is  
>> deffective, but I don't know what). What should I do?

Reply | Threaded
Open this post in threaded view
|

Re: Disk performance/benchmarking

no@spam@mgedv.net
In reply to this post by Gabriel George POPA
> I was mainly wanting to see a rough estimation of disk throughput
> (MB/sec).

try this in a state, where the machine is more or less idle
(you'd be able to setup a cronjob for this):

you will .5GB space for that ;-)

for i in 1 2 3 4 5
do
        h_file="/a/clean/dir/on/your/disk/test_$i.data";
        time dd if=/dev/zero of=$h_file bs=1m count=100;
        time dd if=$h_file of=/dev/null bs=1m;
done;

don't forget to remove the files
dd will give you an avg bytes/sec on transfer, but
don't forget that these are with hw/os caching,
interferences with other active processes, and so on...

it's what you wanted: just rough estimation of how
fast your system will do basic disk io from userland.

this doesn't show how fast it could be, but will show
faster values than normal processing would deliver.
you could change the blocksize/count to increase/decrease
the work the system has to do for writing a specific amount
of data. the more the system has to work, the lesser the
blocksize and the lesser the throughput will be.
so "how fast is my disk" is always very depending on what
kind of action you do with it. transferring/working on
millions of small files will be much slower than working
on a few large files.