From: Alexei_Roudnev (Alexei_Roudnev_at_exigengroup.com)
Date: Tue Oct 10 2006 - 20:39:22 CEST
Message-ID: <0f3301c6ec9b$679fa6c0$6f31a8c0@sjc.exigengroup.com> From: "Alexei_Roudnev" <Alexei_Roudnev@exigengroup.com> Date: Tue, 10 Oct 2006 11:39:22 -0700 Subject: Re: [suse-sles-e] Re: [suse-oracle] Raw Devices
Be very (extremely) careful with lvmn2 and RAC. lvm2 is not cluster aware,
so
if you want to change volumes, you must:
- shutdown all instances but one
- change volume
- restart instances which you shutdown
- check that they see the change.
I (again) advise for ASM and against both, RAW and OCFSv2 (esp. against
raw). Exception - if you can
just assign 1 LUN per Oracle object on SAN system (easy on NetAPp, for
example) then you can proceed with raw.
If your DBA had a problem resizing ASM, he will definitely will have a
nightmare resizing RAC database, located on LVM and RAW.
OCFSv2 is a possible choice, but don't forget that
- it have not online resize
- it have not (yet) offline resize
- if you allocate it on lvm, see point above
- it adds instability because of a very primitive heartbeat and fencing
(except fi you run it al on SLES10 with evms, which I am not sure if it is
tested well yet).
So. If it is not a lab, your only good choices will be:
- raw, 1 partition per LUN, all changes are doing on SAN and you NEVER
RESIZE (only add new LUNs);
OR
- ASM on RAW, you never resize LUN's, but just add new LUN;s into ASM
groups.
(It is my personal opinion; I dont insist, but I am just too lazy to wake up
at 3 am because of DBA have a problems and broke disk system , making
inconsistant raw resizing, or because OCFSv2 panicked in 10-th time...)
----- Original Message -----
From: "C'est Pierre" <cestpierre@gmail.com>
To: "Alexei_Roudnev" <Alexei_Roudnev@exigengroup.com>
Cc: "Fabrizio Magni" <fabrizio.magni@gmail.com>; <suse-oracle@suse.com>;
<suse-sles-e@suse.com>
Sent: Tuesday, October 10, 2006 3:09 AM
Subject: Re: [suse-sles-e] Re: [suse-oracle] Raw Devices
> Hello all!
>
> Sorry for the late reply, but I had a few other tasks to solve in between.
> We're going to try the lvm + raw approach. Any issues?
>
> In case of failure, we'll resort to OCFSv2. I've been to Oracle's OCFS
> page and they say we require SLES 9 SP3's kernel 2.6.5-7.257. However,
> my SP3 iso, downloaded from Novell's website, has these files:
>
>
./x86_64/update/SUSE-CORE/9/rpm/x86_64/kernel-source-2.6.5-7.244.x86_64.rpm
> ./x86_64/update/SUSE-CORE/9/rpm/x86_64/kernel-syms-2.6.5-7.244.x86_64.rpm
>
./x86_64/update/SUSE-CORE/9/rpm/x86_64/kernel-default-2.6.5-7.244.x86_64.rpm
> ./x86_64/update/SUSE-CORE/9/rpm/x86_64/kernel-smp-2.6.5-7.244.x86_64.rpm
>
> Any ideas?
>
> Thanks,
> Pierre
>
>
> On 10/9/06, Alexei_Roudnev <Alexei_Roudnev@exigengroup.com> wrote:
> > I think, that your DBA tried to do a bad thing - change disk size on
SAN
> > system and then resize ASM device., It don';t work because Linux don't
see
> > new size if disk is mounted by any way.
> >
> > You should not
> > - use ASMLib, it is a piece of garbage.
> > - don't try to resize disk when ASM device is full.
> >
> > You should, instead, ADD new disk into the group, when previous one is
> > filled in (and system will redistribute ASM files), OR add new
> > ASM disk group and assign new datafile into the tablespace.
> >
> > OCFSv2 is a controvercial thing - it is fine when it work, buit it adds
an
> > element of instability. I;d better use NFS for backups, but we are
runnimng
> > OCFSv2 in the lab for low-performance file systems (archive logs and
> > backups) and it works fine (if we have not infrastructure failures).
> >
> >
> >
> > ----- Original Message -----
> > From: "C'est Pierre" <cestpierre@gmail.com>
> > To: "Fabrizio Magni" <fabrizio.magni@gmail.com>
> > Cc: <suse-oracle@suse.com>; <suse-sles-e@suse.com>
> > Sent: Monday, October 09, 2006 8:41 AM
> > Subject: [suse-sles-e] Re: [suse-oracle] Raw Devices
> >
> >
> > > Hi Fabrizio,
> > >
> > > We skipped the option of using ASMLib on this cluster. Our dba said it
> > > had a limitation (or bug) when a tablespace (or was it raw device?)
> > > filled up, they couldn't extend the size anymore. This is a migration
> > > database, so they'll need lots more space than we already have in the
> > > long run.
> > >
> > > We're using RAC and raw devices only, from what he said. An
> > > alternative in mind is OCFS.
> > >
> > > Any thoughts?
> > >
> > >
> > > Thanks,
> > > Pierre
> > >
> > >
> > > On 10/9/06, Fabrizio Magni <fabrizio.magni@gmail.com> wrote:
> > > > Hi Pierre,
> > > > just personal curiosity: what's the advantage to have 47 partition
on
> > the
> > > > same disk?
> > > > If you are still using the ASM then you can simply use a single
> > partition
> > > > (or device).
> > > > Tablespaces are already divided inside that device optimizing the
space
> > > > usage.
> > > >
> > > > Regards
> > > > Fabrizio
> > > >
> > > >
> > > > On 10/9/06, C'est Pierre <cestpierre@gmail.com> wrote:
> > > > >
> > > > > Hello all,
> > > > >
> > > > > First, I am sys-administrator, not a DBA, so excuse me if I am no
> > > > > precise with some tech. aspects of Oracle.
> > > > >
> > > > > I am in the process of installing a new RAC cluster. Our previous
one,
> > > > > went smoothly, but we used ASMLib, which we aren't using now.
Problem:
> > > > > I created 47 partitions on a scsi disk presented to the server
thru a
> > > > > fibre channel card. These are sda1 thru sda47. However, only the
15
> > > > > first (except one, which is the extended one - sda4) are usable.
> > > > >
> > > > > I wrote the /etc/raw, mapping sda's to raw's just as the dba
sugested
> > > > > for organizational purposes (they aren't sequential, e.g raw51
maps
> > > > > to sda28).
> > > > >
> > > > > I then made this line of bash to create all device nods that
weren't
> > > > > there (and even those that were...just in case):
> > > > >
> > > > > for i in `seq -f %g 1 47`; do echo mknod sda$i b 8 $i ; done
> > > > >
> > > > > Problem: I can't access past sda16, I get this error:
> > > > >
> > > > > dd: opening `/dev/sda16': No such device or address
> > > > >
> > > > > when I look at dmesg and /proc/partitions, I only get to see the
first
> > > > > 15 partitions there (or 16, if you count with 'sda')
> > > > >
> > > > > sda: sda1 sda2 sda3 sda4 < sda5 sda6 sda7 sda8 sda9 sda10 sda11
sda12
> > > > > sda13 sda14 sda15 >
> > > > >
> > > > > I've googled for this problem and it seems there's a limitation on
> > > > > scsi disks of 16 partitions per device. Is this true? if so, what
is
> > > > > the solution for this problem? how can I make 47 raw partitions?
> > > > >
> > > > > I can still ask our storage administrator to divide this disk into
> > > > > several disks and then we will group partitions 16 partitions on
each,
> > > > > but this isn't a good solution. Another alternative solution which
our
> > > > > dba presented, was adding ocfs 2 support and make it dance with
the
> > > > > devil by the pale moonlight ;-)
> > > > >
> > > > > Let me know your thoughts and especially solutions!!
> > > > >
> > > > > Also, your thoughts on ocfs or not as a side note would be
apreciated.
> > > > >
> > > > > Thank you
> > > > >
> > > > > Pierre
> > > > >
> > > > > P.S: I'm not french but I love french bread! ;)
> > > > >
> > > > > --
> > > > > To unsubscribe, email: suse-oracle-unsubscribe@suse.com
> > > > > For additional commands, email: suse-oracle-help@suse.com
> > > > > Please see http://www.suse.com/oracle/ before posting
> > > > >
> > > > >
> > > >
> > > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: suse-sles-e-unsubscribe@suse.com
> > > For additional commands, e-mail: suse-sles-e-help@suse.com
> > >
> > >
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: suse-sles-e-unsubscribe@suse.com
> For additional commands, e-mail: suse-sles-e-help@suse.com
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: suse-sles-e-unsubscribe@suse.com
For additional commands, e-mail: suse-sles-e-help@suse.com
This archive was generated by hypermail 2.1.7 : Tue Oct 10 2006 - 20:36:33 CEST