botbotbot 's blog

Python Testing

11 Jul 2014 Tags: python ·

There are difference between ‘building it right’ and ‘building the right thing’. -Python Testing Cookbook by Greg L. Turnquist

Good Test Habits

  • A few test is better than nothing
  • Coverage isn’t everything
    • make working software not a 100% coverage
  • Capturing a bug in an automated test
    1. Write Tests for the bug
    2. Add it to your test suite
    3. Fix the bug
    4. Veriry test suite passed
  • Testing Concept
    1. AAA - Arrange Act Assert
    2. BDD - Give When Then

Tips:

  • Run All unittest in module:

    python -m unittest discover <test_module>
  • Run nosetests with python3.
    • Install nose into python3 package with pip3 then you can run nosetests with python3 via nosetests-3.x ex. nosetests-3.4 if your python3 is version 3.4
    # ubuntu install python3-pip first
# $ sudo apt-get install python3-ip
$ pip3 install nose
$ nosetests-3.x

  • Problems using nose in a virtualenv

    you can’t run nose in system-packages to using packages in virtualenv (sandbox)

  • Run PDB with nosetest:

    from nose.tools import set_trace; set_trace()

Nose Multiprocess

You should make unittest run faster with mocking better than using multi processes.
There are many problem with nose plugin ex. rednose, coverage, xunit.

In worst case more processeses can make your tests slower depends on your tests.

$ nosetests --processes=NUMBER_OF_PROCESSORS --process-timeout <sec>

Plugin Problem:

  1. Rednose - disable rednose by remove –rednose option and NOSE_REDNOSE in enviorment variable
  2. coverage - fix by nose-cov
  3. xunit - fix by nose_xunitmp

Test Coverage

Stmts - statements
Miss - statements that not executed
Cover - (Stmts - Miss)/Stmts * 100

  • Coverage html
    • Generate an Pretty HTML report
    • powerful way to visually inspect the code and see the lines that missing
    nosetests --with-coverage --cover-html

  • Coverage erase

    coverage erase
coverage -e
nosetests --with-coverage --cover-erase

  • Coverage with filter Third-party Libraries out

    nosetests --with-coverage --cover-package=<package,package2>
nosetests --with-coverage --cover-package=$(ls | grep '.py$' | sed 's/[.]py$//' | xargs | sed 's/[\ ]/,/g')

Mock

  • mocking pattern

    1.record/replay pattern
    2.action/assert pattern

  • mock tests behavior
    • test the method was called or not called by assert_called_with(), etc.
  • stubs tests state
    • recording method class and its result

Behavior Driven Development

  • BDD - Given, When, and Then

  • BDD Stroy
    -Feature
    – Scenario
    — steps

  • TDD vs BDD
    • setUp == Given
    • call unit == Then
    • assert == When
  • Lettuce is a Cucumber-like BDD tool built for Python.

Robot Framework - Acceptance Testing

  • Acceptance testing is block box testing
  • Customers are usually more interested in what the software does, not how it does it.
  • Robot Framework uses keywords to define tests, test steps, variables, and other testing components and need to maps Robot Framework keywords to our Python code
  • Robot Framework uses data-driven test suite

Jekin(Java) - Continuous Integration

  • nosetests –with-xunit
  • tests upon commit(trigger),when scheduled

Tools:

pip install nose rednose coverage
echo "export NOSE_REDNOSE=1" >> .zshrc #enable rednose

pip install watchdog
  • Tox - multiple environments testing
pip install tox

Testing Python

  1. Python Testing Start Here
  2. Testing in Python: using nose & mocks
  3. Mock - Mocking and Testing Library
  4. An Introduction to Mocking in Python
  5. INTRODUCING BDD
  6. Mock Like a Boss
  7. Beginning Test-Driven Development in Python
  8. Behavior-Driven Development in Python
  9. Testing with mock
  10. Software Testing Course
  11. Patching Mocks in Python
  12. Python Nose: Speed up the runner
  13. Beginning Test-Driven Development in Python
  14. python-test-double
  15. Mocking in Python - How to Bypass Expensive and External Code Within Your Tests

Latest Posts