[sles-beta] Creating XEN VM files to use with HA cluster

Tue Jun 3 19:21:11 MDT 2014

Team

Here are some sample files from my "test lab".

They should give some "working" examples of a XEN VM with "live migration"
for a HA cluster of XEN hosts.

It's a zip file (let me know if it needs to be packaged differently to get
past any antivirus scanners etc) sles12-cluster_example.zip

File contents:
1. crm-config.txt: (export of cluster config using 'crm config show >
crm-config.txt')
You should be able to import modifed subsections of this straight in using
the 'crm' shell environment.
2. working-cib-cleaned.xml: (cleaned up export using cibadmin -Q >
working-cib.xml, with nodes,status sections removed and conflicting infor
removed for crm header).
I can import this using
 cibadmin -E --force
 cibadmin -C -f ./working-cib-cleaned.xml
I'm old fashioned and still favour the xml format ;-)
3. sles12.xml: the "exported" vm configuration file used for the HA/VM
resource.
=> This is an example, you will need to create your own working files ;-)
4. xl.conf: from /etc/xen, this is my configuration which allows live
migrations using EITHER 'tcp" of "ssh" as 'migration transport'
Note, I have created shared SSH keys so that my cluster nodes can ssh to
each other without passwords.

Notes on My HA Cluster configuration
I use a OCFS2 "shared storage" configuration to hold the VM configuration
files and also the "boot" disk images for the VM.

The OCFS2 cluster resources were created using "hawk" and the new OCFS2
cluster filesytem template.

The actual filesystem was initialised with this command:
'mkfs.ocfs2 -b 4k -C 1M -L XEN01 --fs-feature-level=max-features
--cluster-stack=pcmk --cluster-name=cluster
/dev/disk/by-id/dm-uuid-mpath-36001405bebeea89675546079c48419ec -F'

I use SBD for STONITH (there are probably better choices in production)

I left most of the file paths as "defaults" but have created symbolic links
into the OCFS2 shared filesystem for:
node1:/var/lib/libvirt # ls -la images
lrwxrwxrwx 1 root root 28 Jun  2 15:41 images ->
/srv/clusterfs/XEN01/images/

Please note, this configuration "works" but bypasses a lot of
authentication/hardening options so is almost certainly not recommended "in
production".

I hope they are useful to you/others.

Regards
Darren

On 3 June 2014 00:53, <fpoeta at pandminc.com> wrote:

> Thanks Darren I would like to see them
>
>
> Francis Poeta
> President
> P & M Computers, Inc.
> An IBM Advanced Business Partner
> 201-943-0353(v)
> 201-943-0227(f)
>
>
>  *Darren Thompson <darrent at akurit.com.au <darrent at akurit.com.au>>*
> Sent by: sles-beta-bounces at lists.suse.com
>
> 06/01/2014 11:37 PM
>   To
> SUSE Linux Enterprise High Availability Authorized Beta Program <
> sleha-beta at lists.suse.com>, "sles-beta at lists.suse.com" <
> sles-beta at lists.suse.com>
> cc
>   Subject
> Re: [sles-beta] Creating XEN VM files to use with HA cluster
>
>
>
>
> Team
>
> A good day (I have XEN live migration working in a SLES12 HA cluster).
>
> Nothing tels it like a raw log output:
> 2014-06-02T13:29:22.065250+10:00 node2 crmd[1824]:   notice:
> te_rsc_command: Initiating action 32: migrate_to VM-sles12_migrate_to_0 on
> node2 (local)
> 2014-06-02T13:29:22.774946+10:00 node2 VirtualDomain(VM-sles12)[10318]:
> INFO: sles12: Starting live migration to node1 (using remote hypervisor URI
> xen+tcp://node1/ ).
> 2014-06-02T13:29:47.058146+10:00 node2 VirtualDomain(VM-sles12)[10318]:
> INFO: sles12: live migration to node1 succeeded.
> 2014-06-02T13:29:47.071165+10:00 node2 lrmd[1821]:   notice:
> operation_finished: VM-sles12_migrate_to_0:10318:stderr [ 2014-06-02
> 03:29:47.035+0000: 10360: info : libvirt version: 1.2.4 ]
> 2014-06-02T13:29:47.072310+10:00 node2 lrmd[1821]:   notice:
> operation_finished: VM-sles12_migrate_to_0:10318:stderr [ 2014-06-02
> 03:29:47.035+0000: 10360: warning : virDomainMigrateVersion3Full:4993 :
> Guest sles12 probably left in &#x27;paused&#x27; state on source ]
> 2014-06-02T13:29:47.073332+10:00 node2 crmd[1824]:   notice:
> process_lrm_event: LRM operation VM-sles12_migrate_to_0 (call=82, rc=0,
> cib-update=267, confirmed=true) ok
> 2014-06-02T13:29:47.082142+10:00 node2 crmd[1824]:   notice:
> te_rsc_command: Initiating action 33: migrate_from VM-sles12_migrate_from_0
> on node1
> 2014-06-02T13:29:47.975935+10:00 node2 crmd[1824]:   notice:
> te_rsc_command: Initiating action 29: stop VM-sles12_stop_0 on node2 (local)
> 2014-06-02T13:29:48.679317+10:00 node2 VirtualDomain(VM-sles12)[10524]:
> INFO: Domain sles12 already stopped.
> 2014-06-02T13:29:48.694079+10:00 node2 crmd[1824]:   notice:
> process_lrm_event: LRM operation VM-sles12_stop_0 (call=83, rc=0,
> cib-update=268, confirmed=true) ok
> 2014-06-02T13:29:49.269012+10:00 node1 VirtualDomain(VM-sles12)[10813]:
> INFO: sles12: live migration from node2 succeeded.
> 2014-06-02T13:29:49.281838+10:00 node1 crmd[1621]:   notice:
> process_lrm_event: LRM operation VM-sles12_migrate_from_0 (call=77, rc=0,
> cib-update=85, confirmed=true) ok
> 2014-06-02T13:31:05.180010+10:00 node2 crmd[1824]:   notice:
> te_rsc_command: Initiating action 32: migrate_to VM-sles12_migrate_to_0 on
> node1
> 2014-06-02T13:31:06.941294+10:00 node1 VirtualDomain(VM-sles12)[13060]:
> INFO: sles12: Starting live migration to node2 (using remote hypervisor URI
> xen+tcp://node2/ ).
> 2014-06-02T13:31:25.198502+10:00 node2 crmd[1824]:   notice:
> te_rsc_command: Initiating action 33: migrate_from VM-sles12_migrate_from_0
> on node2 (local)
> 2014-06-02T13:31:25.745563+10:00 node2 VirtualDomain(VM-sles12)[11467]:
> INFO: sles12: live migration from node1 succeeded.
> 2014-06-02T13:31:25.760436+10:00 node2 crmd[1824]:   notice:
> process_lrm_event: LRM operation VM-sles12_migrate_from_0 (call=84, rc=0,
> cib-update=271, confirmed=true) ok
> 2014-06-02T13:31:25.771805+10:00 node2 crmd[1824]:   notice:
> te_rsc_command: Initiating action 29: stop VM-sles12_stop_0 on node1
> 2014-06-02T13:31:26.552432+10:00 node1 VirtualDomain(VM-sles12)[13060]:
> INFO: sles12: live migration to node2 succeeded.
> 2014-06-02T13:31:26.570635+10:00 node1 crmd[1621]:   notice:
> process_lrm_event: LRM operation VM-sles12_migrate_to_0 (call=80, rc=0,
> cib-update=87, confirmed=true) ok
> 2014-06-02T13:31:27.415782+10:00 node1 VirtualDomain(VM-sles12)[13231]:
> INFO: Domain sles12 already stopped.
> 2014-06-02T13:31:27.429105+10:00 node1 crmd[1621]:   notice:
> process_lrm_event: LRM operation VM-sles12_stop_0 (call=81, rc=0,
> cib-update=88,
> If anyone is intersted in doing the same, post a mail to these lists and I
> will publish my working config exampes.
> Darren
>
>
>
>
>
> On 2 June 2014 12:08, Darren Thompson <*darrent at akurit.com.au*
> <darrent at akurit.com.au>> wrote:
> Team
>
> OK I now have managed to create a cluster XEN VM (using a OCFS2 shared
> file-system and the notes from my previous correspondence).
>
> It looks like the 'live migration/relocation' function is now managed by
> 'lib-virt' and not xend (there is no xend-config.sxp file for example).
>
> Is there any notes on the configuration steps required for 'live
> migration/relocation' now, the YAST2 'relocation server' module does not
> seem to do anything and it's setting do not "stick".
>
> Darren
>
>
>
> On 2 June 2014 10:54, Darren Thompson <*darrent at akurit.com.au*
> <darrent at akurit.com.au>> wrote:
> Team
>
> Found it, old dog needs to learn some new tricks ;-)
>
> "# virsh dumpxml *GuestID* > *guest.xml"*
>
> From:
> *https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Virtualization_Administration_Guide/form-Virtualization-Managing_guests_with_virsh-Creating_a_virtual_machine_XML_dump_configuration_file.html*
> <https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Virtualization_Administration_Guide/form-Virtualization-Managing_guests_with_virsh-Creating_a_virtual_machine_XML_dump_configuration_file.html>
> (Is there updated SLES12 documentation on this yet)
>
> Also, the original Xen OCF cluster resource cannot be used as it's hard
> coded to "xm" which is depreciated.
>
> There is a new 'VirtualDomain' OCF file which is used for clustering
> 'lib-virt' compatible VM's (which seems to support Xen, KVM etc).
>
> Darren
>
>
> On 31 May 2014 15:58, Darren Thompson <*darrent at akurit.com.au*
> <darrent at akurit.com.au>> wrote:
> Team
>
> I must be missing something obvious, but how do you get the VM config
> files for HA
>
> In SLES11+ the VM create tool created the configuration file or you could
> run "xm list {vmname} -l <= That;s from memory and my not be correct syntax.
>
> I have created a VM using the new virt-manager but it does not seem to
> have created a config file.
>
> The xl list command does not seem to "find" the VM when it's running and
> does not list it at all when stopped (I can only start it from virt-manager
> itself).
>
> Any hints/tips appreciated as I'm sure that I'm missing something that
> should be obvious.
>
> Darren
>
>
> --
>
> Darren Thompson
>
> Professional Services Engineer / Consultant
>
>  [image: cid:image001.jpg at 01CB7C0C.6C6A2AE0]
>
> Level 3, 60 City Road
>
> Southgate, VIC 3006
>
> Mb: 0400 640 414
>
> Mail: darrent*@akurit.com.au* <steve at akurit.com.au>
> Web: *www.akurit.com.au* <http://www.akurit.com.au/>
>
>
>
>
> --
>
> Darren Thompson
>
> Professional Services Engineer / Consultant
>
>  [image: cid:image001.jpg at 01CB7C0C.6C6A2AE0]
>
> Level 3, 60 City Road
>
> Southgate, VIC 3006
>
> Mb: 0400 640 414
>
> Mail: darrent*@akurit.com.au* <steve at akurit.com.au>
> Web: *www.akurit.com.au* <http://www.akurit.com.au/>
>
>
>
>
> --
>
> Darren Thompson
>
> Professional Services Engineer / Consultant
>
>  [image: cid:image001.jpg at 01CB7C0C.6C6A2AE0]
>
> Level 3, 60 City Road
>
> Southgate, VIC 3006
>
> Mb: 0400 640 414
>
> Mail: darrent*@akurit.com.au* <steve at akurit.com.au>
> Web: *www.akurit.com.au* <http://www.akurit.com.au/>
>
>
>
>
> --
>
> Darren Thompson
>
> Professional Services Engineer / Consultant
>
>  [image: cid:image001.jpg at 01CB7C0C.6C6A2AE0]
>
> Level 3, 60 City Road
>
> Southgate, VIC 3006
>
> Mb: 0400 640 414
>
> Mail: darrent*@akurit.com.au* <steve at akurit.com.au>
> Web: *www.akurit.com.au* <http://www.akurit.com.au/>
> _______________________________________________
> sles-beta mailing list
> sles-beta at lists.suse.com
> http://lists.suse.com/mailman/listinfo/sles-beta
>
>

-- 

Darren Thompson

Professional Services Engineer / Consultant

 *[image: cid:image001.jpg at 01CB7C0C.6C6A2AE0]*

Level 3, 60 City Road

Southgate, VIC 3006

Mb: 0400 640 414

Mail: darrent at akurit.com.au <steve at akurit.com.au>
Web: www.akurit.com.au
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.suse.com/mailman/private/sles-beta/attachments/20140604/64f3bfcc/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 3692 bytes
Desc: not available
URL: <http://lists.suse.com/mailman/private/sles-beta/attachments/20140604/64f3bfcc/attachment.jpeg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 3692 bytes
Desc: not available
URL: <http://lists.suse.com/mailman/private/sles-beta/attachments/20140604/64f3bfcc/attachment-0001.jpeg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 3692 bytes
Desc: not available
URL: <http://lists.suse.com/mailman/private/sles-beta/attachments/20140604/64f3bfcc/attachment-0002.jpeg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image003.jpg
Type: image/jpeg
Size: 3692 bytes
Desc: not available
URL: <http://lists.suse.com/mailman/private/sles-beta/attachments/20140604/64f3bfcc/attachment.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 3692 bytes
Desc: not available
URL: <http://lists.suse.com/mailman/private/sles-beta/attachments/20140604/64f3bfcc/attachment-0003.jpeg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: sles12-cluster_example.zip
Type: application/zip
Size: 4204 bytes
Desc: not available
URL: <http://lists.suse.com/mailman/private/sles-beta/attachments/20140604/64f3bfcc/attachment.zip>