[Deepsea-users] Single configuration approach for DeepSea

[Ricardo Dias]

Hi,

I've been developing an alternative approach to the current "policy.cfg" to
configure the deployment of Ceph with DeepSea. The implementation resides in my
own fork of DeepSea in branch wip-single-config:

https://github.com/rjfd/DeepSea/tree/wip-single-config

The main idea is to use a single configuration file (single source of truth) in
the pillar, written in yaml, that can specify several deployment properties and
customize specific options for the several Ceph components, such as OSDs, MONs,
etc...

This configuration file can be created before any stage 0 of DeepSea because it
does not rely on any information generated during stages 0..5.
An example of this configuration file can be seen in:

https://github.com/rjfd/DeepSea/blob/wip-single-config/srv/pillar/ceph/config.sls

The structure in the above file is a proposal and can be (and should be) subject
to changes.
The include/exclude sets that appear in several section of the config file
contain minion names and support gobling. These sets only take effect from stage
1 onwards. Stage 0 is still not supported.

Inside the "osds" section, we can specify options that affect the discovery
algorithm of stage 1, and we can also specify OSD properties, such as
"journal_size" or the use of "dmcrypt".

You can specify the OSD properties in three different scopes:
 - global: it applies to all OSDs in the cluster
 - minion: it applies to all OSDs in a minion
 - drive: it applies only to the OSD using that drive

The properties specified in the inner scope will override the properties from
the outer scope.

Regarding the way we can identify the drive/device we can use the device
name (with support of globs), or use any other way that can identify
each drive, such as vendor, or model. The current implementation only supports
the device name.

iscsi, rgw, mds, cephfs sections are not supported yet, since this is just a
prototype of an idea to check if it's worth to pursue this approach.

Another section that isn't still supported is the specification of the
public/cluster networks.

To use this single config file approach, just checkout the code from my fork,
run "make install" to install DeepSea. Create the config file based on the
example that is already in the repository (notice that "make install" will not
copy the config file sample) in /srv/pillar/ceph/config.sls.
Then run stages 0..3 as usual. When running stage 1, it will output the
information of OSDs, MONs, networks that will be deployed in the following
stages. If you find that the configuration does not fit your needs, you can make
changes to the config file and run stage 1 again.

This implementation only supports running stages up to stage 3, therefore it
only supports deploying a Ceph cluster. It doesn't allow you to deploy iSCSI
gateways, nor CephFS for instance.

Please note that this is just an idea that I'm exploring, and it is not even
planned to be included in DeepSea yet. Moreover, the implementation was only
tested in a virtual environment, and may contain a lot of bugs.

I'm looking for people to experiment with it and make suggestions on how to
improve it.

Thanks,
Ricardo Dias