[sles-beta] Creating XEN VM files to use with HA cluster

Darren Thompson darrent at akurit.com.au
Tue Jun 3 20:55:43 MDT 2014 

Team

This is embarrassing...

Looks like I missed out on a required ordering constraint...

Please add
"order clusterfs-then-VM-sles12 inf: c-clusterfs VM-sles12"
to your configurations or "bad things" could happen to the VM on restart or
by puting the hosting node into "standby"

I found out the "bad things" retrospectively and now need to reinstall my
VM due to bad btrfs corruption. :-(

This is how it "should have" been configured ;-)

node 1084777673: node1 \
        attributes standby=off maintenance=off
node 1084777674: node2 \
        attributes standby=off maintenance=off
primitive VM-sles12 VirtualDomain \
        params config="/srv/clusterfs/XEN01/configs/sles12.xml"
autoset_utilization_cpu=true autoset_utilization_hv_memory=true
migration_transport=ssh \
        op monitor interval=10 timeout=30 \
        op start interval=0 timeout=90 \
        op stop interval=0 timeout=90 \
        op migrate_from interval=0 timeout=180 \
        op migrate_to interval=0 timeout=180 \
        meta target-role=Stopped allow-migrate=true \
        utilization cpu=1 hv_memory=1025
primitive clusterfs Filesystem \
        params
device="/dev/disk/by-id/dm-uuid-mpath-36001405bebeea89675546079c48419ec"
directory="/srv/clusterfs/XEN01" fstype=ocfs2 \
        op start interval=0 timeout=60 \
        op stop interval=0 timeout=60 \
        op monitor interval=20 timeout=40
primitive dlm ocf:pacemaker:controld \
        op start timeout=90 interval=0 \
        op stop timeout=100 interval=0 \
        op monitor interval=60 timeout=60
primitive fencing-sbd stonith:external/sbd \
        op monitor timeout=20 interval=3600 \
        op start timeout=20 interval=0 \
        op stop timeout=15 interval=0 \
        meta target-role=Started
clone base-clone dlm \
        meta interleave=true target-role=Started
clone c-clusterfs clusterfs \
        meta interleave=true target-role=Started
colocation clusterfs-with-base inf: c-clusterfs base-clone
order clusterfs-then-VM-sles12 inf: c-clusterfs VM-sles12
order base-then-clusterfs inf: base-clone c-clusterfs
property cib-bootstrap-options: \
        dc-version=1.1.11-11.13-3ca8c3b \
        cluster-infrastructure=corosync \
        cluster-name=cluster \
        stonith-enabled=true \
        stonith-timeout=120s \
        no-quorum-policy=ignore \
        last-lrm-refresh=1401848656
rsc_defaults rsc-options: \
        resource-stickiness=1

Regards
Darren



On 4 June 2014 11:21, Darren Thompson <darrent at akurit.com.au> wrote:

> Team
>
> Here are some sample files from my "test lab".
>
> They should give some "working" examples of a XEN VM with "live migration"
> for a HA cluster of XEN hosts.
>
> It's a zip file (let me know if it needs to be packaged differently to get
> past any antivirus scanners etc) sles12-cluster_example.zip
>
> File contents:
> 1. crm-config.txt: (export of cluster config using 'crm config show >
> crm-config.txt')
> You should be able to import modifed subsections of this straight in using
> the 'crm' shell environment.
> 2. working-cib-cleaned.xml: (cleaned up export using cibadmin -Q >
> working-cib.xml, with nodes,status sections removed and conflicting infor
> removed for crm header).
> I can import this using
>  cibadmin -E --force
>  cibadmin -C -f ./working-cib-cleaned.xml
> I'm old fashioned and still favour the xml format ;-)
> 3. sles12.xml: the "exported" vm configuration file used for the HA/VM
> resource.
> => This is an example, you will need to create your own working files ;-)
> 4. xl.conf: from /etc/xen, this is my configuration which allows live
> migrations using EITHER 'tcp" of "ssh" as 'migration transport'
> Note, I have created shared SSH keys so that my cluster nodes can ssh to
> each other without passwords.
>
> Notes on My HA Cluster configuration
> I use a OCFS2 "shared storage" configuration to hold the VM configuration
> files and also the "boot" disk images for the VM.
>
> The OCFS2 cluster resources were created using "hawk" and the new OCFS2
> cluster filesytem template.
>
> The actual filesystem was initialised with this command:
> 'mkfs.ocfs2 -b 4k -C 1M -L XEN01 --fs-feature-level=max-features
> --cluster-stack=pcmk --cluster-name=cluster
> /dev/disk/by-id/dm-uuid-mpath-36001405bebeea89675546079c48419ec -F'
>
> I use SBD for STONITH (there are probably better choices in production)
>
> I left most of the file paths as "defaults" but have created symbolic
> links into the OCFS2 shared filesystem for:
> node1:/var/lib/libvirt # ls -la images
> lrwxrwxrwx 1 root root 28 Jun  2 15:41 images ->
> /srv/clusterfs/XEN01/images/
>
> Please note, this configuration "works" but bypasses a lot of
> authentication/hardening options so is almost certainly not recommended "in
> production".
>
> I hope they are useful to you/others.
>
> Regards
> Darren
>
>
> On 3 June 2014 00:53, <fpoeta at pandminc.com> wrote:
>
>> Thanks Darren I would like to see them
>>
>>
>> Francis Poeta
>> President
>> P & M Computers, Inc.
>> An IBM Advanced Business Partner
>> 201-943-0353(v)
>> 201-943-0227(f)
>>
>>
>>  *Darren Thompson <darrent at akurit.com.au <darrent at akurit.com.au>>*
>> Sent by: sles-beta-bounces at lists.suse.com
>>
>> 06/01/2014 11:37 PM
>>   To
>> SUSE Linux Enterprise High Availability Authorized Beta Program <
>> sleha-beta at lists.suse.com>, "sles-beta at lists.suse.com" <
>> sles-beta at lists.suse.com>
>> cc
>>   Subject
>> Re: [sles-beta] Creating XEN VM files to use with HA cluster
>>
>>
>>
>>
>> Team
>>
>> A good day (I have XEN live migration working in a SLES12 HA cluster).
>>
>> Nothing tels it like a raw log output:
>> 2014-06-02T13:29:22.065250+10:00 node2 crmd[1824]:   notice:
>> te_rsc_command: Initiating action 32: migrate_to VM-sles12_migrate_to_0 on
>> node2 (local)
>> 2014-06-02T13:29:22.774946+10:00 node2 VirtualDomain(VM-sles12)[10318]:
>> INFO: sles12: Starting live migration to node1 (using remote hypervisor URI
>> xen+tcp://node1/ ).
>> 2014-06-02T13:29:47.058146+10:00 node2 VirtualDomain(VM-sles12)[10318]:
>> INFO: sles12: live migration to node1 succeeded.
>> 2014-06-02T13:29:47.071165+10:00 node2 lrmd[1821]:   notice:
>> operation_finished: VM-sles12_migrate_to_0:10318:stderr [ 2014-06-02
>> 03:29:47.035+0000: 10360: info : libvirt version: 1.2.4 ]
>> 2014-06-02T13:29:47.072310+10:00 node2 lrmd[1821]:   notice:
>> operation_finished: VM-sles12_migrate_to_0:10318:stderr [ 2014-06-02
>> 03:29:47.035+0000: 10360: warning : virDomainMigrateVersion3Full:4993 :
>> Guest sles12 probably left in &#x27;paused&#x27; state on source ]
>> 2014-06-02T13:29:47.073332+10:00 node2 crmd[1824]:   notice:
>> process_lrm_event: LRM operation VM-sles12_migrate_to_0 (call=82, rc=0,
>> cib-update=267, confirmed=true) ok
>> 2014-06-02T13:29:47.082142+10:00 node2 crmd[1824]:   notice:
>> te_rsc_command: Initiating action 33: migrate_from VM-sles12_migrate_from_0
>> on node1
>> 2014-06-02T13:29:47.975935+10:00 node2 crmd[1824]:   notice:
>> te_rsc_command: Initiating action 29: stop VM-sles12_stop_0 on node2 (local)
>> 2014-06-02T13:29:48.679317+10:00 node2 VirtualDomain(VM-sles12)[10524]:
>> INFO: Domain sles12 already stopped.
>> 2014-06-02T13:29:48.694079+10:00 node2 crmd[1824]:   notice:
>> process_lrm_event: LRM operation VM-sles12_stop_0 (call=83, rc=0,
>> cib-update=268, confirmed=true) ok
>> 2014-06-02T13:29:49.269012+10:00 node1 VirtualDomain(VM-sles12)[10813]:
>> INFO: sles12: live migration from node2 succeeded.
>> 2014-06-02T13:29:49.281838+10:00 node1 crmd[1621]:   notice:
>> process_lrm_event: LRM operation VM-sles12_migrate_from_0 (call=77, rc=0,
>> cib-update=85, confirmed=true) ok
>> 2014-06-02T13:31:05.180010+10:00 node2 crmd[1824]:   notice:
>> te_rsc_command: Initiating action 32: migrate_to VM-sles12_migrate_to_0 on
>> node1
>> 2014-06-02T13:31:06.941294+10:00 node1 VirtualDomain(VM-sles12)[13060]:
>> INFO: sles12: Starting live migration to node2 (using remote hypervisor URI
>> xen+tcp://node2/ ).
>> 2014-06-02T13:31:25.198502+10:00 node2 crmd[1824]:   notice:
>> te_rsc_command: Initiating action 33: migrate_from VM-sles12_migrate_from_0
>> on node2 (local)
>> 2014-06-02T13:31:25.745563+10:00 node2 VirtualDomain(VM-sles12)[11467]:
>> INFO: sles12: live migration from node1 succeeded.
>> 2014-06-02T13:31:25.760436+10:00 node2 crmd[1824]:   notice:
>> process_lrm_event: LRM operation VM-sles12_migrate_from_0 (call=84, rc=0,
>> cib-update=271, confirmed=true) ok
>> 2014-06-02T13:31:25.771805+10:00 node2 crmd[1824]:   notice:
>> te_rsc_command: Initiating action 29: stop VM-sles12_stop_0 on node1
>> 2014-06-02T13:31:26.552432+10:00 node1 VirtualDomain(VM-sles12)[13060]:
>> INFO: sles12: live migration to node2 succeeded.
>> 2014-06-02T13:31:26.570635+10:00 node1 crmd[1621]:   notice:
>> process_lrm_event: LRM operation VM-sles12_migrate_to_0 (call=80, rc=0,
>> cib-update=87, confirmed=true) ok
>> 2014-06-02T13:31:27.415782+10:00 node1 VirtualDomain(VM-sles12)[13231]:
>> INFO: Domain sles12 already stopped.
>> 2014-06-02T13:31:27.429105+10:00 node1 crmd[1621]:   notice:
>> process_lrm_event: LRM operation VM-sles12_stop_0 (call=81, rc=0,
>> cib-update=88,
>> If anyone is intersted in doing the same, post a mail to these lists and
>> I will publish my working config exampes.
>> Darren
>>
>>
>>
>>
>>
>> On 2 June 2014 12:08, Darren Thompson <*darrent at akurit.com.au*
>> <darrent at akurit.com.au>> wrote:
>> Team
>>
>> OK I now have managed to create a cluster XEN VM (using a OCFS2 shared
>> file-system and the notes from my previous correspondence).
>>
>> It looks like the 'live migration/relocation' function is now managed by
>> 'lib-virt' and not xend (there is no xend-config.sxp file for example).
>>
>> Is there any notes on the configuration steps required for 'live
>> migration/relocation' now, the YAST2 'relocation server' module does not
>> seem to do anything and it's setting do not "stick".
>>
>> Darren
>>
>>
>>
>> On 2 June 2014 10:54, Darren Thompson <*darrent at akurit.com.au*
>> <darrent at akurit.com.au>> wrote:
>> Team
>>
>> Found it, old dog needs to learn some new tricks ;-)
>>
>> "# virsh dumpxml *GuestID* > *guest.xml"*
>>
>> From:
>> *https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Virtualization_Administration_Guide/form-Virtualization-Managing_guests_with_virsh-Creating_a_virtual_machine_XML_dump_configuration_file.html*
>> <https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Virtualization_Administration_Guide/form-Virtualization-Managing_guests_with_virsh-Creating_a_virtual_machine_XML_dump_configuration_file.html>
>> (Is there updated SLES12 documentation on this yet)
>>
>> Also, the original Xen OCF cluster resource cannot be used as it's hard
>> coded to "xm" which is depreciated.
>>
>> There is a new 'VirtualDomain' OCF file which is used for clustering
>> 'lib-virt' compatible VM's (which seems to support Xen, KVM etc).
>>
>> Darren
>>
>>
>> On 31 May 2014 15:58, Darren Thompson <*darrent at akurit.com.au*
>> <darrent at akurit.com.au>> wrote:
>> Team
>>
>> I must be missing something obvious, but how do you get the VM config
>> files for HA
>>
>> In SLES11+ the VM create tool created the configuration file or you could
>> run "xm list {vmname} -l <= That;s from memory and my not be correct syntax.
>>
>> I have created a VM using the new virt-manager but it does not seem to
>> have created a config file.
>>
>> The xl list command does not seem to "find" the VM when it's running and
>> does not list it at all when stopped (I can only start it from virt-manager
>> itself).
>>
>> Any hints/tips appreciated as I'm sure that I'm missing something that
>> should be obvious.
>>
>> Darren
>>
>>
>> --
>>
>> Darren Thompson
>>
>> Professional Services Engineer / Consultant
>>
>>  [image: cid:image001.jpg at 01CB7C0C.6C6A2AE0]
>>
>> Level 3, 60 City Road
>>
>> Southgate, VIC 3006
>>
>> Mb: 0400 640 414
>>
>> Mail: darrent*@akurit.com.au* <steve at akurit.com.au>
>> Web: *www.akurit.com.au* <http://www.akurit.com.au/>
>>
>>
>>
>>
>> --
>>
>> Darren Thompson
>>
>> Professional Services Engineer / Consultant
>>
>>  [image: cid:image001.jpg at 01CB7C0C.6C6A2AE0]
>>
>> Level 3, 60 City Road
>>
>> Southgate, VIC 3006
>>
>> Mb: 0400 640 414
>>
>> Mail: darrent*@akurit.com.au* <steve at akurit.com.au>
>> Web: *www.akurit.com.au* <http://www.akurit.com.au/>
>>
>>
>>
>>
>> --
>>
>> Darren Thompson
>>
>> Professional Services Engineer / Consultant
>>
>>  [image: cid:image001.jpg at 01CB7C0C.6C6A2AE0]
>>
>> Level 3, 60 City Road
>>
>> Southgate, VIC 3006
>>
>> Mb: 0400 640 414
>>
>> Mail: darrent*@akurit.com.au* <steve at akurit.com.au>
>> Web: *www.akurit.com.au* <http://www.akurit.com.au/>
>>
>>
>>
>>
>> --
>>
>> Darren Thompson
>>
>> Professional Services Engineer / Consultant
>>
>>  [image: cid:image001.jpg at 01CB7C0C.6C6A2AE0]
>>
>> Level 3, 60 City Road
>>
>> Southgate, VIC 3006
>>
>> Mb: 0400 640 414
>>
>> Mail: darrent*@akurit.com.au* <steve at akurit.com.au>
>> Web: *www.akurit.com.au* <http://www.akurit.com.au/>
>> _______________________________________________
>> sles-beta mailing list
>> sles-beta at lists.suse.com
>> http://lists.suse.com/mailman/listinfo/sles-beta
>>
>>
>
>
> --
>
> Darren Thompson
>
> Professional Services Engineer / Consultant
>
>  *[image: cid:image001.jpg at 01CB7C0C.6C6A2AE0]*
>
> Level 3, 60 City Road
>
> Southgate, VIC 3006
>
> Mb: 0400 640 414
>
> Mail: darrent at akurit.com.au <steve at akurit.com.au>
> Web: www.akurit.com.au
>



-- 

Darren Thompson

Professional Services Engineer / Consultant

 *[image: cid:image001.jpg at 01CB7C0C.6C6A2AE0]*

Level 3, 60 City Road

Southgate, VIC 3006

Mb: 0400 640 414

Mail: darrent at akurit.com.au <steve at akurit.com.au>
Web: www.akurit.com.au
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.suse.com/mailman/private/sles-beta/attachments/20140604/14609eb2/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 3692 bytes
Desc: not available
URL: <http://lists.suse.com/mailman/private/sles-beta/attachments/20140604/14609eb2/attachment.jpeg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 3692 bytes
Desc: not available
URL: <http://lists.suse.com/mailman/private/sles-beta/attachments/20140604/14609eb2/attachment-0001.jpeg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 3692 bytes
Desc: not available
URL: <http://lists.suse.com/mailman/private/sles-beta/attachments/20140604/14609eb2/attachment-0002.jpeg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 3692 bytes
Desc: not available
URL: <http://lists.suse.com/mailman/private/sles-beta/attachments/20140604/14609eb2/attachment-0003.jpeg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image003.jpg
Type: image/jpeg
Size: 3692 bytes
Desc: not available
URL: <http://lists.suse.com/mailman/private/sles-beta/attachments/20140604/14609eb2/attachment.jpg>

More information about the sles-beta mailing list