From: Alexei_Roudnev (Alexei_Roudnev_at_exigengroup.com)
Date: Wed Apr 11 2007 - 19:58:53 CEST
Message-ID: <19ef01c77c63$110400b0$6f31a8c0@sjc.exigengroup.com> From: "Alexei_Roudnev" <Alexei_Roudnev@exigengroup.com> Date: Wed, 11 Apr 2007 10:58:53 -0700 Subject: Re: Antw: Re: [suse-sles-e] Re: catastrophic disk-performance on SLES10 with SATA drives
Did you try the same with ReiserFS?
I found interestring difference in IO in my lab. Have UL4.4 == RHEL4.4, and
SLES10 SP1 beta, both with iSCSI on 2 lun's on Netapp,
and with LVM stripped over 2 LUN's. Have ext3 on UL4.4 and ReiserFS on
SLES10.
Run iometer tests emulation oracle load (direct_io option is on, block size
is 64K - too slow with 8K).
All results are almost the same (difference is 5%) but random writes are 3
times faster on UL4.4 . I counted it on possible bad block aligment on
SLES10 + LVM + reiserFS /it is possible that it have nothing with SLES but
only with disk partitioning and FS parameters/.
So, is it possible that you have the same problem - just wrong FS block
aligment relating to the physical disk / raid block(s)?
Anyway, it (what you see) should not be in the normal conditions - defaults
are quite good for the most RAID systems. Something wrong with you disk
subsystem or disk drivers (for example, they dont send interruption when IO
is completed, so if system works in sync mode then it waits al the time, but
when you switch it into write back mode it sends requests in parallel and
works well on the status polling instead of interruption... or block
aligment is wrong.. or something else...)
----- Original Message -----
From: "Markus Roeffen" <markus.roeffen@iws.uni-stuttgart.de>
To: "Alexei_Roudnev" <Alexei_Roudnev@exigengroup.com>;
<suse-sles-e@suse.com>; "matilda matilda" <matilda@grandel.de>
Sent: Wednesday, April 11, 2007 12:22 AM
Subject: Re: Antw: Re: [suse-sles-e] Re: catastrophic disk-performance on
SLES10 with SATA drives
Hello Alexei,
hello Andreas,
hello mailinglist,
I've got good news from the incredible p-machine.
The following options have increased the IO performance dramatically:
1. scheduler parameter "nr_requests" set to 1024
2. Format the partition with:
mkfs.ext3 /dev/sda1 -O dir_index
tune2fs -o journal_data_writeback
(You can view these settings with: tune2fs -l /dev/sda1 )
3. Mount it with noatime and data=writeback:
mount -t ext3 /dev/sda1 /home -o noatime -o data=writeback
At least rsync with the home directories from our other server has _not_
increased the load and the io wait.
And now some benchmark results with the old (SLES-default) settings and
the new settings.
old settings:
bonnie -s 200
Bonnie: Warning: You have 1004MB RAM, but you test with only 200MB datasize!
Bonnie: This might yield unrealistically good results,
Bonnie: for reading and seeking and writing.
Bonnie 1.4: File './Bonnie.17650', size: 209715200, volumes: 1
Writing with putc()... done: 31108 kB/s 57.3 %CPU
Rewriting... done: 36260 kB/s 9.2 %CPU
Writing intelligently... done: 44453 kB/s 9.5 %CPU
Reading with getc()... done: 62697 kB/s 99.9 %CPU
Reading intelligently... done: 1421620 kB/s 100.0 %CPU
Seeker 1...Seeker 2...Seeker 3...start 'em...done...done...done...
---Sequential Output (nosync)--- ---Sequential Input-- --Rnd
Seek-
-Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --04k
(03)-
Machine MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec
%CPU
server 1* 200 31108 57.3 44453 9.5 36260 9.2 62697 99.91421620 100
117875.9 94.3
bonnie -s 2000
Bonnie 1.4: File './Bonnie.17671', size: 2097152000, volumes: 1
Writing with putc()... done: 23158 kB/s 43.0 %CPU
Rewriting... done: 26146 kB/s 7.2 %CPU
Writing intelligently... done: 35760 kB/s 9.4 %CPU
Reading with getc()... done: 54578 kB/s 89.3 %CPU
Reading intelligently... done: 95385 kB/s 9.5 %CPU
Seeker 1...Seeker 2...Seeker 3...start 'em...done...done...done...
---Sequential Output (nosync)--- ---Sequential Input-- --Rnd
Seek-
-Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --04k
(03)-
Machine MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec
%CPU
server 1*2000 23158 43.0 35760 9.4 26146 7.2 54578 89.3 95385 9.5 661.4
1.2
new settings:
bonnie -s 200
Bonnie: Warning: You have 1004MB RAM, but you test with only 200MB datasize!
Bonnie: This might yield unrealistically good results,
Bonnie: for reading and seeking and writing.
Bonnie 1.4: File './Bonnie.16120', size: 209715200, volumes: 1
Writing with putc()... done: 55636 kB/s 99.0 %CPU
Rewriting... done: 183626 kB/s 37.3 %CPU
Writing intelligently... done: 101891 kB/s 16.7 %CPU
Reading with getc()... done: 62315 kB/s 99.6 %CPU
Reading intelligently... done: 1468458 kB/s 100.4 %CPU
Seeker 1...Seeker 2...Seeker 3...start 'em...done...done...done...
---Sequential Output (nosync)--- ---Sequential Input-- --Rnd
Seek-
-Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --04k
(03)-
Machine MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec
%CPU
server 1* 200 55636 99.0101891 16.7 183626 37.3 62315 99.61468458 100
108831.7 98.0
bonnie -s 2000
Bonnie 1.4: File './Bonnie.16133', size: 2097152000, volumes: 1
Writing with putc()... done: 47224 kB/s 83.6 %CPU
Rewriting... done: 43536 kB/s 9.8 %CPU
Writing intelligently... done: 99917 kB/s 18.9 %CPU
Reading with getc()... done: 49738 kB/s 81.2 %CPU
Reading intelligently... done: 95502 kB/s 8.8 %CPU
Seeker 1...Seeker 2...Seeker 3...start 'em...done...done...done...
---Sequential Output (nosync)--- ---Sequential Input-- --Rnd
Seek-
-Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --04k
(03)-
Machine MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec
%CPU
server 1*2000 47224 83.6 99917 18.9 43536 9.8 49738 81.2 95502 8.8 647.1
1.3
Please note that there was only one user on the server. We haven't
tested our performance under normal circumstances (nfs/afs and more
users) yet.
If the settings also improve your performance or if you have other hints
for me, please let me know.
Best regards
Markus
Alexei_Roudnev wrote:
> Interesting.
>
> We got the same, dramatic, IO problem with oracle on tested SLES10 and
RHEL5
> (both used new kernel and open-iscsi), and
> it's possible that the problem is the same. Stil investigation what is
> wrong.
>
>
>
>
-- ------------------------------- Markus Roeffen Institut für Wasserbau IWS Universität Stuttgart Pfaffenwaldring 61 70569 Stuttgart Tel: +49(0)711/685-67010 Fax: +49(0)711/685-67020 Mail: markus.roeffen@iws.uni-stuttgart.de --------------------------------------------------------------------- To unsubscribe, e-mail: suse-sles-e-unsubscribe@suse.com For additional commands, e-mail: suse-sles-e-help@suse.com --------------------------------------------------------------------- To unsubscribe, e-mail: suse-sles-e-unsubscribe@suse.com For additional commands, e-mail: suse-sles-e-help@suse.com
This archive was generated by hypermail 2.1.7 : Wed Apr 11 2007 - 22:05:36 CEST