[Deepsea-users] stage 1: network discovery

Ricardo Dias rdias at suse.com
Mon Dec 12 08:50:15 MST 2016

On 2016-12-12 10:28:10, Eric Jackson wrote:
> On Monday, December 12, 2016 03:12:50 PM Ricardo Dias wrote:
> > Hi,
> > 
> > I've been running DeepSea (DS) in a virtual environment and I'm wondering
> > what are the network requirements of DeepSea to successfully deploy a
> > Ceph cluster.
> > 
> > In my current environment, every node has a single network interface
> > configured with a netmask of
> Are you using virtualbox where every node has the same address?   I cannot 
> think of any production network where all Ceph nodes would use host routes.  

I'm using a cluster created in OVH cloud provider. Every node has a
different IP address, but have a netmask of /32. The IP addresses are
public and the access rules are controlled by the security group of
the instances.

> > 
> > I've looked into how the discovery code in populate.py is identifying
> > the network "topology" of the cluster and I have a set of
> > questions/doubts:
> > 
> > - Does DS expects to exist two networks (cluster and public) in the
> >   cluster, or it can cope with the existence of only one network and
> >   use it has both the "cluster" and "public" networks?
> >   -- Just found out in the code that if only one network exists it is used
> >      as both the "cluster" and "public" networks
> Right.  The simplest network is normally a single one used for management, 
> public and cluster traffic.  I personally use three separate networks since I 
> believe many production users would be similar.  I am aware some have multiple 
> management networks.
> > 
> > - DS is identifying a network by using the netmask of the network
> >   interface. In my case, the netmask is and DS is unable
> >   to configure MONs public address due to the way the IP addresses are
> >   verified to belong to the "chosen" public address.
> >   Maybe we should rethink the way we are currently discovering the
> >   existent networks of the cluster.
> I never considered a netmask of /32.  I would like to understand your 
> topology.  If this is easy enough to detect (i.e. no network seems to exist 
> and all are /32), what is the correct answer?A

I think we can assume that all networks (in this case there is a network
for each node) are public. And by reading the "Network Configuration"
section [1] in Ceph documentation, we can specify several public
networks. This means we just need to extend a bit the code to deal with
a list of public networks instead of only one.

The rule for network detection should be:
Given a set of networks NTS:
 1) if each network in NTS only contains one node, then we consider all
    networks public
 2) otherwise we fallback to the current logic, which is:
    pick the network with more nodes as the public network, and the
    second with more nodes as the cluster network.

> > 
> > - Why is DS storing the "public_network" and "cluster_network" in
> >   "pillar/ceph/stack/default/ceph/cluster.yml"? What are their purposes?
> >   Do we really need them?
> Take a look at /srv/salt/ceph/configuration/files/ceph.conf.j2.  The public and 
> cluster networks are specified there.

As I said in the comment above, we can add a list of public networks in
the "public_network" config option.

[1] http://docs.ceph.com/docs/jewel/rados/configuration/network-config-ref/

More information about the Deepsea-users mailing list