summaryrefslogtreecommitdiff
path: root/README
diff options
context:
space:
mode:
authorLars Wirzenius <liw@iki.fi>2007-11-11 05:09:00 +0200
committerLars Wirzenius <liw@iki.fi>2007-11-11 05:09:00 +0200
commit679206166470dede587068c9b358d3d0b6afcb55 (patch)
treef0305a865f8b17daca01cc40f95613ae99bcf984 /README
parent80e2285c030eb53f0f39603c83fa064f52a4ad66 (diff)
downloadcoverage-test-runner-679206166470dede587068c9b358d3d0b6afcb55.tar.gz
Wrote a README.
Diffstat (limited to 'README')
-rw-r--r--README139
1 files changed, 139 insertions, 0 deletions
diff --git a/README b/README
new file mode 100644
index 0000000..533bff4
--- /dev/null
+++ b/README
@@ -0,0 +1,139 @@
+CoverageTestRunner
+==================
+
+The Python standard library unittest aids in implementing and running
+unit tests. The coverage.py module (not in the standard library) measures
+which parts of a program are executed. A proper unit test suite will
+execute all parts of a program. CoverageTestRunner uses coverage.py to
+run test suites implemented with unittest, and fails the tests if they
+do not test the entire program.
+
+CoverageTestRunner makes some assumptions, the most important of which is
+that the test suite is written so that each module corresponds to a test
+module, and that test module should execute everything in the module.
+
+Ideally, we would test each class and its corresponding test suite, but
+unfortunately coverage.py only allows module granularity. It also only
+measures at the statement level. This is, however, typically better
+than can be achieved without any measurement tools.
+
+
+Example
+=======
+
+Assume a Python module in foo.py, and its test suite in foo_tests.py.
+Then running the test suite using CoverageTestRunner can be very simple:
+
+ python -m CoverageTestRunner
+
+This will invoke the module's main program, which scans the current
+directory (and its subdirectories) for test modules with corresponding
+code modules, based on filenames, and then executes them pairwise.
+
+If you have a more complicated setup, you can pair the code and test
+modules manually, using a wrapper script such as follows:
+
+ from CoverageTestRunner import CoverageTestRunner
+
+ r = CoverageTestRunner()
+ r.add_pair("code/foo.py", "tests/foo.py")
+ r.run()
+
+
+Output
+======
+
+CoverageTestRunner writes its output to the standard output, and for
+a successful run it looks like this:
+
+ Running test 2/2: testTrue (foo_tests.FooTests)
+
+ OK
+ Time: 0.0 s
+
+An unsuccessful run might look like this (shortened for space reasons):
+
+ Running test 226/226: test (configTests.WriteDefaultConfigTests)
+
+ FAILED
+
+ ERROR: testEmptyFile (ioTests.FileContentsTests)
+ Traceback (most recent call last):
+ File "unittests/ioTests.py", line 228, in testEmptyFile
+ id = obnam.io.create_file_contents_object(self.context, filename)
+ File "/home/liw/Braawi/obnam/new-storage/obnam/io.py", line 240, in create_file_contents_object
+ stdout=subprocess.PIPE)
+ File "subprocess.py", line 593, in __init__
+ errread, errwrite)
+ File "subprocess.py", line 1135, in _execute_child
+ raise child_exception
+ OSError: [Errno 2] No such file or directory
+
+ Statements missed by per-module tests:
+ Module Missed statements
+ /home/liw/Braawi/obnam/new-storage/obnam/cfgfile.py 70, 303
+ /home/liw/Braawi/obnam/new-storage/obnam/cmp.py 155
+ /home/liw/Braawi/obnam/new-storage/obnam/config.py 234-237
+ /home/liw/Braawi/obnam/new-storage/obnam/filelist.py 88
+ /home/liw/Braawi/obnam/new-storage/obnam/format.py 74, 97
+
+ 1 failures, 3 errors
+ Time: 5.4 s
+
+In this case, after the line with "FAILED" comes a list of all the
+failed tests, with some indication of why they failed, complete with
+Python stack traces. Additionally, if some parts of the code were not
+executed during the tests, a list of files and statement numbers is
+included in the output.
+
+
+Your tests are too slow
+=======================
+
+CoverageTestRunner keeps track of how long each test takes. If the total
+run time of the tests is more than ten seconds, even if all tests pass,
+it then prints out a list of the slowest tests. This is useful for projects
+whose test suites should run quickly so that developers don't mind running
+them all the time.
+
+This is obviously a bad thing for projects with large test suites. If it
+bothers you, please ask me, and I'll provide a way to disable it, or to
+set the time limit.
+
+
+Contacts
+========
+
+Python's unittest is part of the standard library, and was (originally?)
+written by Steve Purcell. coverage.py was originally written by Gareth
+Rees and is now maintained by Net Batchelder. Its home page is at
+
+ http://www.nedbatchelder.com/code/modules/coverage.html
+
+CoverageTestRunner was written by Lars Wirzenius (liw@iki.fi), and is
+licensed under the GNU General Public License version 2 or at your option
+any later version. The home page is at
+
+ http://braawi.org/coveragetestrunner.html
+
+I welcome any feedback you may have.
+
+
+License
+=======
+
+Copyright (C) 2007 Lars Wirzenius <liw@iki.fi>
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; either version 2 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License along
+with this program; if not, write to the Free Software Foundation, Inc.,
+51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.