[Deepsea-users] [ses-users] Trying out migrations
Stephan Müller
smueller at suse.com
Thu Jul 6 03:40:58 MDT 2017
Hi I tried to migrate from filestore to bluestore, according to this
mail, but it was without success. After that I tried step 1 from this
mail http://mailman.suse.de/mlarch/SuSE/ses-users/2017/ses-users.2017.0
5/msg00201.html and successfully put my cluster into a error state it
hasn't recovered for over half an hour (no change at all in output of
ceph -s).
I will describe where the problems started in your mail.
Am Donnerstag, den 29.06.2017, 10:41 -0400 schrieb Eric Jackson:
> Hello all,
> For everyone wanting to try out the migration functionality with
> DeepSea
> 0.7.15 ( and soon node by node with 0.7.16), here is a primitive
> workflow. I
> would suggest using the new proposal runner to create multiple
> configurations.
>
> 1) Start with a fresh cluster
My cluster wasn't completely fresh as I have used it for 1 or 2 weeks,
but to only to create pools and RBDs with no files on them.
> 2) Run "salt '*' osd.report"
> The result should agree that your current configuration is
> active
They were.
> 3) Create a new hardware profile
> e.g. salt-run proposal.populate nvme-ssd=True ratio=3
> name=bluestore+waldb
> I used this for servers with 6 SSDs and 2 NVMe devices.
>
I did it returned {} when I ran it the first time.
Later I have run it again and got a config as the return but also this
warning message:
"[WARNING ] not overwriting existing proposal stargazer-sle-
1.oa.suse.de"
> 4) Edit your policy.cfg, comment out the existing profile and add the
> new
> profile. I commented out these lines:
>
> profile-2Intel745GB-6INTEL372GB-1/cluster/*.sls
> profile-2Disk2GB-1/stack/default/ceph/minions/data*.yml
>
> and added
>
> profile-bluestore+waldb/cluster/blueshark[4-8]*.sls
> profile-bluestore+waldb/stack/default/ceph/minions/blueshark[4-
> 8]*.yml
>
I did the same.
> 5) Refresh the pillar (e.g. salt-run state.orch ceph.stage.2)
>
I got the following error:
"profiles_populated : ['There are no files under the profiles
directory. Probably an issue with the discovery stage.']
[ERROR ] No highstate or sls specified, no execution made"
Than I started running step 3 again, with the warning seen under step 3
but continued with step 5 getting the same error.
Than I started to copy the missing files from the old comment out
profile. That worked but didn't change anything (If I had really worked
with deepSea before I would have known that)
> 6) Run "salt '*' osd.report"
> Depending on the configuration changes between the existing and
> new
> profile, expect to see some list of devices
>
> 7) Run the migration
> salt-run state.orch ceph.migrate.osds
> Unfortunately, no progress is returned. Running 'ceph osd tree'
> will give
> an indication of which server and OSD is currently reconfiguring.
To run this you first have to run:
salt-run disengage.safety
After I have run this after I die step 1 from the other mentioned mail
because nothing had changed, my cluster ended up in a unrecoverable
error state (at least for me).
>
> 8) Run "salt '*' osd.report" when complete. The expectation is that
> everything is converted. If any devices did not succeed, check the
> /var/log/salt/minion for commands related to that device. All
> commands (e.g.
> sgdisk, prepare, activate, etc.) are logged.
>
> With our working wip-osd branch, Stage 3 will be able to correct
> broken OSDs
> under some conditions. I expect this to be available with 0.7.16.
>
> ----
> Another comment about the current strategy: Stage 3 will only add an
> OSD and
> the migrate will only reconfigure an OSD. I can see the argument
> that Stage 3
> should just make the cluster the way the admin wants it.
>
> Any thoughts on this either way would be appreciated.
>
My final question is why did Step 3 not succeed leaving me clueless
whats wrong :/
--
Stephan Müller
SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton,
HRB 21284 (AG Nürnberg)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 819 bytes
Desc: This is a digitally signed message part
URL: <http://lists.suse.com/pipermail/deepsea-users/attachments/20170706/c8a550a8/attachment.sig>
More information about the Deepsea-users
mailing list