[Deepsea-users] filestore to bluestore migration

[Thomas Sundell]

Hi all,
we recently upgraded our SES4 cluster to SES5 successfully, there were
a couple of very minor issues on the way that we managed to solve,
good work and great documentation!

Now we are at the step to migrate from filestore to bluestore backend.
We followed the documentation (
https://www.suse.com/documentation/suse-enterprise-storage-5/book_storage_deployment/data/ceph_upgrade_4to5.html
) and ran "salt-run state.orch ceph.migrate.policy", this created a
new policy.cfg and policy.cfg-original and also a directory named
migrated-profile-1Intel238GB-8ST4000VN0001-1SF3726GB-1.

Then we ran "salt-run state.orch ceph.migrate.osds" which ran
successfully but watching "ceph osd tree" the weight of the osds
stayed the same, so my guess was that the migration did not start.

So I did a diff on policy.cfg and policy.cfg-original and noticed the
content was the same, I edited the file and changed the profile to
point to the migrated folder:
migrated-profile-1Intel238GB-8ST4000VN0001-1SF3726GB-1/cluster/*.sls
migrated-profile-1Intel238GB-8ST4000VN0001-1SF3726GB-1/stack/default/ceph/minions/*.yml

Then I reran "salt-run state.orch ceph.migrate.osds" which completed
successfully again but still the osd weight didn't change.

After this I noticed that another directory also had been created
during the upgrade named profile-default. Comparing the content of the
yml files in that directory vs the migrated directory I noticed some
difference:

This is from the migrated profile directory:
ceph:
  storage:
    osds:
      /dev/disk/by-id/ata-ST4000VN0001-1SF178_Z4F0PS2P:
        format: bluestore
...

This is from the profile-default directory:
ceph:
  storage:
    osds:
      /dev/disk/by-id/ata-ST4000VN0001-1SF178_Z4F0PS49:
        db: /dev/disk/by-id/nvme-20000000001000000e4d25c0bf42f4c00
        db_size: 500m
        format: bluestore
        wal: /dev/disk/by-id/nvme-20000000001000000e4d25c0bf42f4c00
        wal_size: 500m
...

So now I'm thinking that maybe policy.cfg should point to
profile-default/../, maybe that change would get the migration to
bluestore rolling? I wanted to ask here before I try, anyone have a
clue if I'm onto something or am I completely lost :)

First time poster and new to this list and SES.

Thank's for listening.

Thomas