From 679206166470dede587068c9b358d3d0b6afcb55 Mon Sep 17 00:00:00 2001 From: Lars Wirzenius Date: Sun, 11 Nov 2007 05:09:00 +0200 Subject: Wrote a README. --- README | 139 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 139 insertions(+) create mode 100644 README (limited to 'README') diff --git a/README b/README new file mode 100644 index 0000000..533bff4 --- /dev/null +++ b/README @@ -0,0 +1,139 @@ +CoverageTestRunner +================== + +The Python standard library unittest aids in implementing and running +unit tests. The coverage.py module (not in the standard library) measures +which parts of a program are executed. A proper unit test suite will +execute all parts of a program. CoverageTestRunner uses coverage.py to +run test suites implemented with unittest, and fails the tests if they +do not test the entire program. + +CoverageTestRunner makes some assumptions, the most important of which is +that the test suite is written so that each module corresponds to a test +module, and that test module should execute everything in the module. + +Ideally, we would test each class and its corresponding test suite, but +unfortunately coverage.py only allows module granularity. It also only +measures at the statement level. This is, however, typically better +than can be achieved without any measurement tools. + + +Example +======= + +Assume a Python module in foo.py, and its test suite in foo_tests.py. +Then running the test suite using CoverageTestRunner can be very simple: + + python -m CoverageTestRunner + +This will invoke the module's main program, which scans the current +directory (and its subdirectories) for test modules with corresponding +code modules, based on filenames, and then executes them pairwise. + +If you have a more complicated setup, you can pair the code and test +modules manually, using a wrapper script such as follows: + + from CoverageTestRunner import CoverageTestRunner + + r = CoverageTestRunner() + r.add_pair("code/foo.py", "tests/foo.py") + r.run() + + +Output +====== + +CoverageTestRunner writes its output to the standard output, and for +a successful run it looks like this: + + Running test 2/2: testTrue (foo_tests.FooTests) + + OK + Time: 0.0 s + +An unsuccessful run might look like this (shortened for space reasons): + + Running test 226/226: test (configTests.WriteDefaultConfigTests) + + FAILED + + ERROR: testEmptyFile (ioTests.FileContentsTests) + Traceback (most recent call last): + File "unittests/ioTests.py", line 228, in testEmptyFile + id = obnam.io.create_file_contents_object(self.context, filename) + File "/home/liw/Braawi/obnam/new-storage/obnam/io.py", line 240, in create_file_contents_object + stdout=subprocess.PIPE) + File "subprocess.py", line 593, in __init__ + errread, errwrite) + File "subprocess.py", line 1135, in _execute_child + raise child_exception + OSError: [Errno 2] No such file or directory + + Statements missed by per-module tests: + Module Missed statements + /home/liw/Braawi/obnam/new-storage/obnam/cfgfile.py 70, 303 + /home/liw/Braawi/obnam/new-storage/obnam/cmp.py 155 + /home/liw/Braawi/obnam/new-storage/obnam/config.py 234-237 + /home/liw/Braawi/obnam/new-storage/obnam/filelist.py 88 + /home/liw/Braawi/obnam/new-storage/obnam/format.py 74, 97 + + 1 failures, 3 errors + Time: 5.4 s + +In this case, after the line with "FAILED" comes a list of all the +failed tests, with some indication of why they failed, complete with +Python stack traces. Additionally, if some parts of the code were not +executed during the tests, a list of files and statement numbers is +included in the output. + + +Your tests are too slow +======================= + +CoverageTestRunner keeps track of how long each test takes. If the total +run time of the tests is more than ten seconds, even if all tests pass, +it then prints out a list of the slowest tests. This is useful for projects +whose test suites should run quickly so that developers don't mind running +them all the time. + +This is obviously a bad thing for projects with large test suites. If it +bothers you, please ask me, and I'll provide a way to disable it, or to +set the time limit. + + +Contacts +======== + +Python's unittest is part of the standard library, and was (originally?) +written by Steve Purcell. coverage.py was originally written by Gareth +Rees and is now maintained by Net Batchelder. Its home page is at + + http://www.nedbatchelder.com/code/modules/coverage.html + +CoverageTestRunner was written by Lars Wirzenius (liw@iki.fi), and is +licensed under the GNU General Public License version 2 or at your option +any later version. The home page is at + + http://braawi.org/coveragetestrunner.html + +I welcome any feedback you may have. + + +License +======= + +Copyright (C) 2007 Lars Wirzenius + +This program is free software; you can redistribute it and/or modify +it under the terms of the GNU General Public License as published by +the Free Software Foundation; either version 2 of the License, or +(at your option) any later version. + +This program is distributed in the hope that it will be useful, +but WITHOUT ANY WARRANTY; without even the implied warranty of +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +GNU General Public License for more details. + +You should have received a copy of the GNU General Public License along +with this program; if not, write to the Free Software Foundation, Inc., +51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. -- cgit v1.2.1