[Deepsea-users] Teuthology is for functional testing

[Nathan Cutler]

In the upstream Ceph testing model, you've got two types of tests:

1. make check tests (unit tests) - consume source code
2. teuthology tests (functional tests) - consume packages

The make check tests all run on a single machine - if the environment is 
set up correctly you just type "make check" and patiently await the 
result. (There is also run-make-check.sh which attempts to automate even 
setting up the environment.)

These "make check" tests are what is run (via Jenkins in OVH) on 
upstream Ceph PRs.

Teuthology is a heavy-duty tool for functional testing of clusters, and 
it's designed to run entire suites of tests. For example, 250 tests 
running in parallel with each test using 1-5 VMs. For each of these 250 
tests, a set of VMs is provisioned, the test is run, results gathered, 
and then the VMs are destroyed.

Needless to say, even in a high-powered teuthology lab environment it 
can take hours for a suite run to complete, and failures due to 
"infrastructure noise" are inevitable since there are so many moving 
parts. Analyzing the failures requires special skills and is generally 
done by developers.

So, we should think about how much testing we want to do for DeepSea 
itself and whether it would make sense to implement such a distinction 
between unit tests (which run on a single machine) and functional tests 
(which run on clusters).

-- 
Nathan Cutler
Software Engineer Distributed Storage
SUSE LINUX, s.r.o.
Tel.: +420 284 084 037