Re: Re: [suse-sles-e] Very high MD latencies with SLES10 SP1.

From: Alexei_Roudnev (Alexei_Roudnev_at_exigengroup.com)
Date: Mon Dec 10 2007 - 20:16:54 CET

Next message: Alexei_Roudnev: "Re: [suse-sles-e] Very high MD latencies with SLES10 SP1."
Previous message: Matthias Eble: "[suse-sles-e] SLES10 Mounting cifs share from Netapp FAS-3020 fails (permission denied)"
In reply to: matilda matilda: "Re: Re: [suse-sles-e] Very high MD latencies with SLES10 SP1."
Next in thread: Norbert Paschedag: "Re: [suse-sles-e] Very high MD latencies with SLES10 SP1."
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ] [ attachment ]

Message-ID: <0e4e01c83b61$39c8b320$7031a8c0@exigengroup.com>
From: "Alexei_Roudnev" <Alexei_Roudnev@exigengroup.com>
Date: Mon, 10 Dec 2007 11:16:54 -0800
Subject: Re: Re: [suse-sles-e] Very high MD latencies with SLES10 SP1.

I have base-lines for NetApp iSCSI disks and NetApp NFS (and for some
PERC/SCSI and FC/SATA RAID-s), but these numbers depends of a huge ## of
factors (such as NetApp model, are lun's 'reallocated' or are not, and so
on).

I used 'iozone' to mimic Oracle load (iozone have 'direct io' mode).

I can send my performance table here into the list if you want.

----- Original Message -----
From: "matilda matilda" <matilda@grandel.de>
To: "Manfred Hollstein" <manfred@die-hollsteins.de>; <suse-sles-e@suse.com>
Sent: Monday, December 10, 2007 3:10 AM
Subject: Re: Re: [suse-sles-e] Very high MD latencies with SLES10 SP1.

Hi all,

a little bit offtopic but not completely.
Sometime we also had the problem that we thought the
"disk performance" is not satisfying. But: How to check.
As soon as you get this feeling you have to ask:
What would have been "normal"? What can you expect?

Are there interested people out there who are interested to provide
disk performance data in a way that this "base lines" could be compared?

If you follow this list you get the feeling that there are not too much
different hardware combinations out there. In this thread someone compared
his current performance to an old one to get the feeling that something must
be wrong.

Would it not be better to compare base lines of comparable hardware?
(E.g. we have HP DL380 G3/G4 with SmartArray 5i/6i and 10000 SCSI-Disks,
SLES9, SLES10)

Comments appreciated.

Best regards
Andreas Mock

>>> Manfred Hollstein <manfred@die-hollsteins.de> 10.12.2007 12:01 >>>
Hi there,

I forwarded your message to Neil Brown, who's the MD maintainer at SUSE
amongst other things. Here's his reply:

(I'm not on this list, but this message was forwarded to me...)

On Wednesday December 5, noe@physik.unizh.ch wrote:
> Hi,
>
> we're having a bit of a performance problem with our MD RAID1 setup after
> having switched to SLES10 SP1 (from SLES9 SP3). For random-I/O in
> particular,
> we're seeing maximum read and write latencies up to _several seconds_
> (with averages
> around 30ms) where SLES9 showed maxima around 500ms with avg around 20ms.
>
> Typical outputs (randomio-1.3, using O_DIRECT):
>
> sles9sp3:
> total | read: latency (ms) | write: latency (ms)
> iops | iops min avg max sdev | iops min avg max
> sdev
> --------+-----------------------------------+----------------------------------
> 636.8 | 476.7 0.2 20.6 501.6 39.1 | 160.1 0.4 1.0 25.0
> 1.4
>
> sles10sp1:
> total | read: latency (ms) | write: latency (ms)
> iops | iops min avg max sdev | iops min avg max
> sdev
> --------+-----------------------------------+----------------------------------
> 350.1 | 262.5 0.2 29.9 7114.5 284.6 | 87.6 0.5 22.0 6511.6
> 253.6

This looks bad. I've tried on may machine and I cannot reproduce it,
so presumably drive subsystem as a significant effect (I've got some
fairly boring SATA drives on a cheap SATA card).

I presume you are running 'randomio' on a file that you created in the
XFS partition - is that correct?

Could you try failing one half of the mirror so that all the requests go
just to the one drive?
Also, would it be possible to make a single drive into a RAID0. That
might help show if the problem is specific to raid1, or if it affect
md more generally.

NeilBrown

---------------------------------------------------------------------
To unsubscribe, e-mail: suse-sles-e-unsubscribe@suse.com
For additional commands, e-mail: suse-sles-e-help@suse.com

Next message: Alexei_Roudnev: "Re: [suse-sles-e] Very high MD latencies with SLES10 SP1."
Previous message: Matthias Eble: "[suse-sles-e] SLES10 Mounting cifs share from Netapp FAS-3020 fails (permission denied)"
In reply to: matilda matilda: "Re: Re: [suse-sles-e] Very high MD latencies with SLES10 SP1."
Next in thread: Norbert Paschedag: "Re: [suse-sles-e] Very high MD latencies with SLES10 SP1."
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ] [ attachment ]

This archive was generated by hypermail 2.1.7 : Mon Dec 10 2007 - 09:17:29 CET