[Deepsea-users] Pull-requests and automated testing

Ricardo Dias rdias at suse.com
Fri Dec 2 10:04:24 MST 2016


On 2016-12-02 17:54:54, Jan Fajerski wrote:
> On Fri, Dec 02, 2016 at 04:32:30PM +0000, Ricardo Dias wrote:
> > On 2016-12-02 17:11:50, Jan Fajerski wrote:
> > > On Fri, Dec 02, 2016 at 12:00:52PM +0000, Ricardo Dias wrote:
> > > > Of course all of this assumes that we have tests to run, and we should
> > > > start planning how unit tests should be developed for DS, based on Salt
> > > > unit tests. But for the start, I think we should have some way to pick
> > > > the changes from the submitted PR, spawn some virtual machines and
> > > > deploy a Ceph cluster, and respective services with DS+changes, and
> > > > check that everything is working correctly.
> > > I'm not sure if unit tests will be sufficient. salt is hard to test and to
> > > my knowledge we can only test runners with unit tests. The real question is
> > > how we will test the rest.
> > 
> > As a first step, I will be happy to just deploy a Ceph cluster using the PR
> > branch and check for HEALTH_OK.
> > Also, we can check if each stage has succeeded.
> Yeah same here. I'm also looking to somewhat automate this process with vagrant.
> > 
> > > >
> > > > A side effect of having status checks activated for the PRs is that we
> > > > protect the master branch from "force" pushes and from deletion, which
> > > > is also nice.
> > > >
> > > > Now what do we need to actually put this plan to work?
> > > > Here is a checklist (maybe incomplete):
> > > > * public jenkins instance
> > > > * computing resources for testing DS
> > > > * Scripts to glue everything with github
> > > Being on my way back from the QA/Testing/Automation meeting in Prag, I would
> > > strongly suggest we look into integrating DeepSea testing into Teuthology.
> > > This has various advantages and shouldn't be too hard. Ceph deploy is tested
> > > in Teuthology too...so why not DS.
> > > Imho the tests to run (besides unit tests for the runners) would simply
> > > exercise DeepSeas functionality.
> > 
> > Yes, testing DS in teuthology is an important goal, which I think it's
> > still a bit far away from being reached. There are two ways of testing
> > DS within teuthology in my opinion: a) swap ceph-ansible for DS to
> > deploy the cluster as configured in each test suite; b) test DS as a
> > single suite test. Either approaches will require significant work,
> > specially a).
> We probably want to have both really. From what I understood from Nathans
> explanations re. Teuthology, it doesn't actually use Teuthology for cluster
> deployment. There is a ceph-cm-ansible component that takes care of
> provisioning the nodes, but does not actually deploy it. Is ceph-ansible
> also used?

Sorry, I was referring to ceph-cm-ansible and yes, you're right,
ceph-cm-ansible just provisions. The deployment per se is done by
teuthology itself if I'm not mistaken.

> > 
> > But I think using teuthology to test DeepSea in the context of a GitHub
> > PR's status check is not an adequate solution. We need something to be
> > run quickly, no more than 30 minutes in my opinion.
> Would this be impossible with Teuthology? Can't we just run our DS specific
> test suite? If that gets so extensive, that it takes so long we can use
> Teuthologies approach of partitioning suites and run a partition at a time.

Teuthology is a heavyweight and resource hungry testing framework, not sure
if we can run a simple DeepSea suite within a small time-frame, but
without really run it I cannot have 100% sure.

> > 
> > > >
> > > > _______________________________________________
> > > > Deepsea-users mailing list
> > > > Deepsea-users at lists.suse.com
> > > > http://lists.suse.com/mailman/listinfo/deepsea-users
> > > 
> > > --
> > > Jan Fajerski
> > > Engineer Enterprise Storage
> > > SUSE Linux GmbH
> > > jfajerski at suse.com
> > > _______________________________________________
> > > Deepsea-users mailing list
> > > Deepsea-users at lists.suse.com
> > > http://lists.suse.com/mailman/listinfo/deepsea-users
> > _______________________________________________
> > Deepsea-users mailing list
> > Deepsea-users at lists.suse.com
> > http://lists.suse.com/mailman/listinfo/deepsea-users
> _______________________________________________
> Deepsea-users mailing list
> Deepsea-users at lists.suse.com
> http://lists.suse.com/mailman/listinfo/deepsea-users


More information about the Deepsea-users mailing list