Sun, May 07, 2006 at 05:57:43PM +0200, Tonnerre LMBARD wrote:
Salut,

It seems that disk I/ is massively slow under NetBSD 3.0 on amd64. I am
running it on a DualCore 175 machine with 2G RAM (Sun Fire X2100),
and when I e.g. dd if=/dev/zero of=/some/ffsv2/path bs=16k count=16k, the
average write speed is around 5 MB/s on a 7.2krpm SATA disk.

Is this a tuning problem or some known bug or do we have to track the
problem down? (It doesn't behave much better on 3.99.18 with the new
BUFQ stuff enabled either) It doesn't appear to be a problem of processing
power, since the processing power used is usually around 1.5% of one CPU.

What is your disk system ? Maybe your SATA controller isn't properly
supported.
Also, you may have better speed using 64k blocks with dd

Mon, May 08, 2006 at 06:22:43AM +0200, Tonnerre LMBARD wrote:
Salut,

Mon, May 08, 2006 at 12:15:12AM +0200, Manuel Bouyer wrote:
What is your disk system ? Maybe your SATA controller isn't properly
supported.

It's NVidia NForce, using the viaide driver.

Also, you may have better speed using 64k blocks with dd

Shouldn't the scheduler accumulate 4 operations with some kind of
"Nagle's Algorithm for I/ schedulers"?

It should, and it does on my system at last (in my case, bs=16k was even
a little faster )

Can you try the attached program ? It will benchmark the IDE bus's bandwidth:
/tst /dev/rwd0d 10000

it should run for a few seconds to give usefull results; if it doesn't
use something larger than 10000

Tue, May 09, 2006 at 07:31:35AM +0200, Tonnerre LMBARD wrote:
Salut,

Mon, May 08, 2006 at 01:23:22PM +0200, Manuel Bouyer wrote:
Can you try the attached program ? It will benchmark the IDE bus's bandwidth:
./tst /dev/rwd0d 10000

it should run for a few seconds to give usefull results; if it doesn't
use something larger than 10000

I added the SYNC parameter to your open() and it sanitized the output
quite a bit:

As the tool is using the raw device it shouldn't matter. What problem did
you notice ?

vic# ./tst /dev/rwd0d 10000
5409749 us, 115.532162 MB/s
vic#

Bonnie gives the following output:

Input--
-Per Char- -Rewrite-- -Per Char-
MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec %CPU
256 4802 3.5 5217 1.8 6430 1.8 105399 75.6 627604 100.0 5970.5 31.6

The worst thing is that sometimes disk I/ operations hang for quite a
while during heavy disk I/ (especially when the I/ takes place from two
or more programs, so it seems that the I/ bandwidth isn't really
distributed equally). I might bring up more test cases on that tonight.

It looks to me that either the VM can't stand much parallel I/ or that
there is a speed issue in the file system layer.

I don't have such issues with others disk subsystems:
tango# /tmp/tst /dev/rwd0d 10000
5175742 us, 120.755633 MB/s
tango# bonnie -c 256
Input--
-Per Char- -Rewrite-- -Per Char-
MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec %CPU
256 43852 28.9 42751 8.2 28485 6.5 60864 42.9 1057795 100.0 34763.0 71.0
The same with -s 1024 (the box has 512Mb RAM):
Input--
-Per Char- -Rewrite-- -Per Char-
MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec %CPU
1024 46547 32.0 46896 8.8 20691 4.3 44786 32.4 47385 5.7 146.6 0.5

So the problem is probably with either the disk or the controller.
It seems that the controller can read from disk at full speed, so
it's probably setup properly, at last for communications with the disk.
You don't have the disk's write cache turned off, do you ?

BTW, you could also try to benchmark raw write speed: boot single user and
use:
dd if=/dev/zero of=/dev/rwd0b bs=64k (assuming your swap partition is wd0b)

Tue, May 09, 2006 at 11:36:51PM +0200, Tonnerre LMBARD wrote:
Salut,

Tue, May 09, 2006 at 12:06:42PM +0200, Manuel Bouyer wrote:
I added the SYNC parameter to your open() and it sanitized the output
quite a bit:

As the tool is using the raw device it shouldn't matter. What problem did
you notice ?

The read speed was around 4000MB/s, which was unreasonable.

Hum, then I guess you used the block device instead of the raw device

BTW, you could also try to benchmark raw write speed: boot single user and
use:
dd if=/dev/zero of=/dev/rwd0b bs=64k (assuming your swap partition is wd0b)

There appears to be a significant problem in write speed:

vic# dd if=/dev/zero of=/dev/rwd0b bs=64k count=1536
100663296 bytes transferred in 41.410 secs (2430893 bytes/sec)

That's way to slow. So it looks like the issue is really somewhere
between the disk and the controller.
I would blame the controller, or more probably the way we use it (but without
doc it's hard to fix this :/