[Deepsea-users] Choosing unittesting frameworks

[Abhishek L]

Nathan Cutler writes:

> I asked Loic, here's his reply:
>
> (01:55:08 PM) loicd: pytest is *wwwaaaaaaay* better
> (02:01:38 PM) smithfarm: can I quote you? ;-)
> (02:01:47 PM) loicd: yes
> (02:02:18 PM) loicd: seriously, there is nothing but pytest nowadays

yay!
>
> On 01/31/2017 01:12 PM, Joshua Schmid wrote:
>> Hey list,
>>
>> first of all I'd like to get the terminology straight.
>>
>> We use 'tox' serves us as a virtualenv manager and allows us:
>>
>> * checking your package installs correctly with different Python
>> versions and interpreters
>> * running your tests in each of the environments, configuring your test
>> tool of choice
>> * acting as a frontend to Continuous Integration servers, greatly
>> reducing boilerplate and merging CI and shell-based testing.
>>
>> tox internally executes 'pytest' which acts currently (for us) as a
>> testcollector that scans directories for 'test_*' files or methods
>> starting with 'test_'.
>>
>> pytest could be replaced with nosetests but after reading some articles
>> pytest seem to have more functionality and a bigger user-base.
>>
>> pytest does not solely act act as a testcollector and runner, but is
>> also a full-fledged unittest framework.
>>
>> as we are moving towards more unittesting, we should agree _now_ on one
>> framework. I for example picked the native python 'unittest' library
>> because it's widespread and has a rather complete documentation.
>>
>> Abhi otoh went for 'pytest'. After some reading I see some benefits in
>> also switching to it.
>>
>> benefits suchs as:
>>
>> * writing setup_functions for specific blocks of tests rather than for
>> an entire test class.
>> * less boilerplate assert vs self.asserTrue(e.g)

This was pretty much my motivation for using pytest style tests as well,
using just assert, and minimal boilerplate, no need to write the main
function to just call unittest.main etc. Also their fixtures _look_ neat
compared to the unittest.

>>
>>
>> we also will use mocking quite extensively. We have only one options
>> here: Mock, which is a standalone lib in py2.7 but was adopted in the
>> native unittest lib in py3.x. (There is also a thin wrapper for pytest
>> called pytest-mocker but this is out of scope here I guess.)

Yeah I also expect we'll have to use mock heavily, and mock seems good
enough and well documented. There is a module called salttesting which
salt upstream uses[1], if we find at some stage that mocking is not
sufficient for a few salt calls (not sure, but maybe __pillar__,
__salt__ calls have some magic around them) we may consider adding this
dependency. But again as I understand this wouldn't modify any existing
tests in the framework we choose.

[1]: https://docs.saltstack.com/en/latest/topics/tutorials/writing_tests.html
>>
>> I don't have a strong opinion because I basically just started reading..
>> If anyone has a profound python background, please speak up now :)
>>
>> I currently lean towards pytest + mock.
>>
>> Thoughts?
>>


--
Abhishek Lekshmanan
SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)