Twisted中的单元测试 (Unit Tests in Twisted)

TableOfContents

• 每个单元测试用来测试一个功能点。单元测试完全自动进行，执行迅速。对于整个系统的单元测试被整合到一个测试集合中，整个测试集可以批量执行。单元测试的结果是十分简单：通过或是不通过。所有这些意味着你可以方便的在任何时候测试你系统，并且能够迅速看到测试结果是通过还是没通过。 Each unit test tests one bit of functionality in the software. Unit tests are entirely automated and complete quickly. Unit tests for the entire system are gathered into one test suite, and may all be run in a single batch. The result of a unit test is simple: either it passes, or it doesn't. All this means you can test the entire system at any time without inconvenience, and quickly see what passes and what fails.

Twisted中的单元测试哲学（Unit Tests in the Twisted Philosophy）

• Twisted开发小组坚持实践极限编程的思想，而使用单元测试的正是极限编程的基础。单元测试是一个不断增长你信心的工具。假设你修改了一个算法，你想知道你是否做错了什么东西吗？请运行单元测试。如果某个测试失败，你能够很清楚的知道哪儿出了问题，因为每个测试只覆盖了很少量的代码，而你清楚的知道你对代码做过怎样的修改。如果所有的测试都通过了，那么没问题，你可以继续了，你再也不需要担心你的修改可能会无意中影响了其它人的程序了。 The Twisted development team adheres to the practice of Extreme Programming (XP), and the usage of unit tests is a cornerstone XP practice. Unit tests are a tool to give you increased confidence. You changed an algorithm -- did you break something? Run the unit tests. If a test fails, you know where to look, because each test covers only a small amount of code, and you know it has something to do with the changes you just made. If all the tests pass, you're good to go, and you don't need to second-guess yourself or worry that you just accidently broke someone else's program.

什么是应该测试的，什么是不应该测试的（What to Test, What Not to Test）

• 你不必为每个方法都编写测试程序，只需为那些可能会发生变化的方法编写测试程序就够了。
  • －－Kent Beck，极限编程解析，58页。
  You don't have to write a test for every single method you write, only production methods that could possibly break.
  • -- Kent Beck, Extreme Programming Explained, p. 58.

运行测试程序（Running the Tests）

• == 怎样运行（How） ==

  •     $ Twisted/admin/runtests

    在你的Emacs初始化文件中加上下面的代码，这会极大的方便你在Emacs中执行单元测试。 You'll find that having something like this in your emacs init files is quite handy:

        (defun runtests () (interactive)
          (compile "python /somepath/Twisted/admin/runtests"))
        
        (global-set-key [(alt t)] 'runtests)

  == 什么时候（When） ==
  • 永远要在提交代码前确定你的代码已经通过了所有的测试。如果其他人开始开发，却发现刚刚checkout到的代码无法通过测试，他们可能会非常生气并把你揪出来。 Always always always be sure all the tests pass before committing any code. If someone else checks out code at the start of a development session and finds failing tests, they will not be happy and may decide to hunt you down. 由于开发队伍的成员分布在世界各地，那个能够帮助你调好代码的人可能并不你的隔壁。你可能想把你的工作进展与整个网络上的同伴分享，但是你又必须保证CVS的主版本树永远保持一个正确的状态。此时，你需要使用分支技术，当你解决了你的问题，并通过了所有的测试之后，你就可以把你的修改合并到主版本中了。 Since this is a geographically dispersed team, the person who can help you get your code working probably isn't in the room with you. You may want to share your work in progress over the network, but you want to leave the main CVS tree in good working order. So use a branch, and merge your changes back in only after your problem is solved and all the unit tests pass again.

加入一个测试程序（Adding a Test）

• 在没有向Twisted中加入相应的测试程序之前，请不要向Twisted中加入新的模块。否则，你可能会在毫不知情的情况下破坏了你的模块。当你知道出了问题以后（可能是马上就知道，也可能是在一个release版本之后），由于已经做过太多的修改，你根本无法确定到底是那里出了问题。 Please don't add new modules to Twisted without adding tests for them too. Otherwise we could change something which breaks your module and not find out until later, making it hard to know exactly what the change that broke it was, or until after a release, and nobody wants broken code in a release.

  所有的测试程序都放在Twisted/twisted/test/目录下，测试程序的名称由“test_”前缀加上对应模块或是包的名称组成。在下面的链接这小节中可以找到很多关于使用PyUnit框架编写单元测试的文档。

  Tests go in Twisted/twisted/test/, and are named test_foo.py, where foo is the name of the module or package being tested. Extensive documentation on using the PyUnit framework for writing unit tests can be found in the links section below.

  一个与标准PyUnit文档像背离的事实：为了确保测试结果的任何变化都是由代码或是测试环境所造成的，而与测试的过程无关，Twisted自己移植了一个兼容的测试框架。这就意味着，无论何时想要导入单元测试模块，你都应该使用 from twisted.trial import unittest 来代替标准的 import unittest 。

  One deviation from the standard PyUnit documentation: To ensure that any variations in test results are due to variations in the code or environment and not the test process itself, Twisted ships with its own, compatible, testing framework. That just means that when you import the unittest module, you will from twisted.trial import unittest instead of the standard import unittest. 只要遵循模块文件的命名与放置习惯，runtests就能聪明的找到你所加入的新的测试程序。 As long as you have followed the module naming and placement conventions, runtests will be smart enough to pick up any new tests you write.

忽略测试，TODO项目（Skipping tests, TODO items）

• Twisted单元测试框架－－Trial，对PyUnit做了一些小的扩展，用来鼓励开发者添加新的测试程序。测试程序在测试一些可选功能时常见的情况是：可能需要依赖于某些特定的外部库文件是否可用，可能只能工作在特定的操作系统上。最重要的问题是，没有人把这些限制当作一个bug。 Trial, the Twisted unit test framework, has some extensions which are designed to encourage developers to add new tests. One common situation is that a test exercises some optional functionality: maybe it depends upon certain external libraries being available, maybe it only works on certain operating systems. The important common factor is that nobody considers these limitations to be a bug.

  为了使测试尽可能的简便，在特定的环境下某些测试可能会被忽略。单个的测试用例可以抛出SkipTest异常来指出这些测试应该被忽略，于是这个测试程序中其余部分就不再被执行。在测试结果summary部分，除了会统计通过或是不通过的测试用例的个数以外，还会对测试中被忽略部分做专门的统计。抛出SkipTest异常这种方式也可以用在条件语句中，用来判断是否满足程序执行的必要条件。

  To make it easy to test as much as possible, some tests may be skipped in certain situations. Individual test cases can raise the SkipTest exception to indicate that they should be skipped, and the remainder of the test is not run. In the summary (the very last thing printed, at the bottom of the test output) the test is counted as a skip instead of a success or fail. This should be used inside a conditional which looks for the necessary prerequisites:

    def testSSHClient(self):
        if not ssh_path:
            raise unittest.SkipTest, "cannot find ssh, nothing to test"
        foo() # do actual test after the SkipTest

  你也可以在一个方法上设定skip属性，这个属性描述了测试为什么被忽略。使用这种机制可以十分方便的临时取消一个测试用例，当然也根据情况手动修改一个类的属性。 You can also set the .skip attribute on the method, with a string to indicate why the test is being skipped. This is convenient for temporarily turning off a test case, but it can also be set conditionally (by manipulating the class attributes after they've been defined):

    def testThing(self):
        dotest()
    testThing.skip = "disabled locally"
    
    class MyTestCase(unittest.TestCase):
        def testOne(self):
            ...
        def testThing(self):
            dotest()
  
    if not haveThing:
        MyTestCase.testThing.im_func.skip = "cannot test without Thing"
        # but testOne() will still run

  最后，你也可以通过设置对应类的skip属性来一次性取消整个测试用例。如果你根据代码的功能依赖关系定义测试程序，那么这种方式可以很方便的取消某种现在还不支持的功能的所需要的单元测试。

  Finally, you can turn off an entire TestCase at once by setting the .skip attribute on the class. If you organize your tests by the functionality they depend upon, this is a convenient way to disable just the tests which cannot be run.

    class SSLTestCase(unittest.TestCase):
       ...
    class TCPTestCase(unittest.TestCase):
       ...
    
    if not haveSSL:
        SSLTestCase.skip = "cannot test without SSL support"
        # but TCPTestCase will still run

  == todo属性与测试新功能（.todo and Testing New Functionality） ==
  • 产生于极限编程开发过程中的两个好的实践方法有时候看上去好像相互矛盾： Two good practices which arise from the XP development process are sometimes at odds with each other:
    • 单元测试是一个好东西。好的开发者一发现单元测试失败就应该心怀恐惧。他们应该把所有的更改都去掉，直到代码能够顺利通过测试。
    • Unit tests are a good thing. Good developers recoil in horror when they see a failing unit test. They should drop everything until the test has been fixed.
    • 好的开发者应该先写单元测试。测试代码完成以后，他们才开始编写具体的实现代码，并使之能够通过单元测试。然后当前的开发就可以告一段落了。
    • Good developers write the unit tests first. Once tests are done, they write implementation code until the unit tests pass. Then they stop.
    这两个目标有的时候是彼此矛盾的。在编写任何实现代码之前，就编写单元测试，这注定一定会失败的。我们总想让开发者快点儿提交代码，以提高代码的可靠性并改善研究同一问题的多个开发人员之间的协作程度。当一个开发人员为代码添加新特性的时候，其它（其工作不涉及到新特性）的开发者本应当不必关注新代码所引入的错误。而不是在一个模块还没有完成，且未开始其单元测试的时候，就高喊“狼来了”，同时不让我们头脑中业已形成的由“狼来了”所导致的测试失败恐惧综合症有丝毫的淡化。如果真这么干，其结果要么是教会模块作者放弃该模块，或者是逼迫其在实现所有的功能后再提交他们的单元测试代码，要么是教会其它的开发者忽略测试程序所检查到的失败信息。这两种方式都不好。 These two goals will sometimes conflict. The unit tests that are written first, before any implementation has been done, are certain to fail. We want developers to commit their code frequently, for reliability and to improve coordination between multiple people working on the same problem together. While the code is being written, other developers (those not involved in the new feature) should not have to pay attention to failures in the new code. We should not dilute our well-indoctrinated Failing Test Horror Syndrome by crying wolf when an incomplete module has not yet started passing its unit tests. To do so would either teach the module author to put off writing or committing their unit tests until after all the functionality is working, or it would teach the other developers to ignore failing test cases. Both are bad things. 为了解决这个问题，我们使用了todo属性。当一个开发者第一次开始为某个尚未实现的新功能编写单元测试的时候，他们可以将该测试方法的todo属性设置为“期望失败”。该方法的运行仍然会产生失败，但是其产生的失败并不会被作为普通的失败加以记录，而是将这种“被期望的失败”放到一个单独的“期望失败”的统计类别中。开发者应该学会将“期望失败”与“实际失败”这两者区别对待，“期望失败”的优先级要比“实际失败”的优先级低一些，应该先解决那些导致“实际失败”的问题。
  • todo is intended to solve this problem. When a developer first starts writing the unit tests for functionality that has not yet been implemented, they can set the .todo attribute on the test methods that are expected to fail. These methods will still be run, but their failure will not be counted the same as normal failures: they will go into an expected failures category. Developers should learn to treat this category as a second-priority queue, behind actual test failures. 当开发者实现了某个特性之后，对应的测试程序终于能够正常运行了。但是由于已经存在的todo属性将这些测试都被标记为“期望失败”，能够成功通过的测试程序反而会把成功信息放到一个“不被期望的成功”的类别中。此时，开发者就应该将测试程序中的todo属性去掉了。从这一点开始，我们才开始正常的测试，而任何一点儿失败都将会引发整个开发团队的立即反映。 As the developer implements the feature, the tests will eventually start passing. This is surprising: after all those tests are marked as being expected to fail. The .todo tests which nevertheless pass are put into a unexpected success category. The developer should remove the .todo tag from these tests. At that point, they become normal tests, and their failure is once again cause for immediate action by the entire development team. 整个测试的生命周期可以描述为：
    1. 建立测试，将其todo属性标记为“期望失败”，此时的测试失败是一种“被期望失败”。
    2. 代码实现某个特定的功能，测试通过。此时的通过是一种“不被期望的通过”。
    3. 去掉todo标签。测试通过，此时的通过是正常通过。
    4. 修改代码，导致测试失败。开发者发现这种失败，并有所行动。
    5. 修复代码，使得测试再次成功，代码暂时稳定。
    The life cycle of a test is thus:
    1. Test is created, marked .todo. Test fails: expected failure.
    2. Code is written, test starts to pass. unexpected success.
    3. .todo tag is removed. Test passes. success.
    4. Code is broken, test starts to fail. failure. Developers spring into action.
    5. Code is fixed, test passes once more. success.
    任何长时间保持todo标签没有去掉的测试都应该被仔细检查。因为这可能反映出某个功能长期没有人维护，或是当前的代码已经无法通过测试，需要尽快修复。 Any test which remains marked with .todo for too long should be examined. Either it represents functionality which nobody is working on, or the test is broken in some fashion and needs to be fixed.

如何关联测试用例与被测试的代码（Associating Test Cases With Source Files）

• 请在需要测试的代码中添加一个test-case-name的标签，并确认你的新测试程序的测试范围已经覆盖了这个待测试文件的功能点。添加了test-case-name标签的待测试文件的头部应该像这样： Please add a test-case-name tag to the source file that is covered by your new test. This is a comment at the beginning of the file which looks like one of the following:

    # -*- test-case-name: twisted.test.test_defer -*-

  或是这样： or

    #!/usr/bin/python
    # -*- test-case-name: twisted.test.test_defer -*-

  Emacs能够理解这个格式，并标记一个文件变量。这么做的目的是为了让Emacs能够在文件的头一两行就能够获取test-case-name的值，而不是放在文件的尾部的Emacs文件变量列表中。如果你还需要定义其它的Emacs文件变量，你既可以使用前面介绍的方法，在文件的头两行中用分号分隔多个变量的定义，和可以采用Emacs支持的放在文件结尾的文件变量列表方式。（译者注：可以参考Emacs手册中 AD.2.5 Local Variables in Files 这个小节的说明，google上search一下吧，^_） This format is understood by emacs to mark File Variables. The intention is to accept test-case-name anywhere emacs would on the first or second line of the file (but not in the File Variables: block that emacs accepts at the end of the file). If you need to define other emacs file variables, you can either put them in the File Variables: block or use a semicolon-separated list of variable definitions:

    # -*- test-case-name: twisted.test.test_defer; fill-column: 75; -*-

  如果一份源码需要使用多个测试用例来测试，多个测试用例的名称使用逗号隔开。如下所示（注意，可以通过trial --testmodule检测所有的测试用例，但不是所有的工具都支持这种用法。）： If the code is exercised by multiple test cases, those may be marked by using a comma-separated list of tests, as follows: (NOTE: not all tools can handle this yet.. trial --testmodule does, though)

    # -*- test-case-name: twisted.test.test_defer,twisted.test.test_tcp -*-

  trial --testmodule twisted/dir/myfile.py会使用test-case-name标签来检测为了完整测试mylife.py这个文件都需要运行那些测试用例。许多工具（包括twisted-dev.el的F9命令）都使用这种方式来正确的运行所有的测试用例。 The test-case-name tag will allow trial --testmodule twisted/dir/myfile.py to determine which test cases need to be run to exercise the code in myfile.py. Several tools (as well as twisted-dev.el's F9 command) use this to automatically run the right tests.