[Deepsea-users] Pull-requests and automated testing

Ricardo Dias rdias at suse.com
Fri Dec 2 09:32:30 MST 2016


On 2016-12-02 17:11:50, Jan Fajerski wrote:
> On Fri, Dec 02, 2016 at 12:00:52PM +0000, Ricardo Dias wrote:
> > Of course all of this assumes that we have tests to run, and we should
> > start planning how unit tests should be developed for DS, based on Salt
> > unit tests. But for the start, I think we should have some way to pick
> > the changes from the submitted PR, spawn some virtual machines and
> > deploy a Ceph cluster, and respective services with DS+changes, and
> > check that everything is working correctly.
> I'm not sure if unit tests will be sufficient. salt is hard to test and to
> my knowledge we can only test runners with unit tests. The real question is
> how we will test the rest.

As a first step, I will be happy to just deploy a Ceph cluster using the PR
branch and check for HEALTH_OK.
Also, we can check if each stage has succeeded.

> > 
> > A side effect of having status checks activated for the PRs is that we
> > protect the master branch from "force" pushes and from deletion, which
> > is also nice.
> > 
> > Now what do we need to actually put this plan to work?
> > Here is a checklist (maybe incomplete):
> > * public jenkins instance
> > * computing resources for testing DS
> > * Scripts to glue everything with github
> Being on my way back from the QA/Testing/Automation meeting in Prag, I would
> strongly suggest we look into integrating DeepSea testing into Teuthology.
> This has various advantages and shouldn't be too hard. Ceph deploy is tested
> in Teuthology too...so why not DS.
> Imho the tests to run (besides unit tests for the runners) would simply
> exercise DeepSeas functionality.

Yes, testing DS in teuthology is an important goal, which I think it's
still a bit far away from being reached. There are two ways of testing
DS within teuthology in my opinion: a) swap ceph-ansible for DS to
deploy the cluster as configured in each test suite; b) test DS as a
single suite test. Either approaches will require significant work,
specially a).

But I think using teuthology to test DeepSea in the context of a GitHub
PR's status check is not an adequate solution. We need something to be
run quickly, no more than 30 minutes in my opinion.

> > 
> > _______________________________________________
> > Deepsea-users mailing list
> > Deepsea-users at lists.suse.com
> > http://lists.suse.com/mailman/listinfo/deepsea-users
> 
> -- 
> Jan Fajerski
> Engineer Enterprise Storage
> SUSE Linux GmbH
> jfajerski at suse.com
> _______________________________________________
> Deepsea-users mailing list
> Deepsea-users at lists.suse.com
> http://lists.suse.com/mailman/listinfo/deepsea-users


More information about the Deepsea-users mailing list