py.test also can generate "errors"

Here is e.g. corresponding summary:

    ===== 12 failed, 14 passed, 4 xfailed, 12 error in 19.58 seconds =====
5 jobs for master
in 0 seconds, using 0 compute credits, and was queued for 0 seconds