How tests will be processed by Autotester
-----------------------------------------

Tests will be processed by calling the specified command in the
``<command>`` section of the test description. In case the command
contains variables, they will be replaced first.

- If the test configuration is invalid (invalid syntax, unknown variables etc.), the result will be **ATC_FAIL**.
- If the command fails (expected return code does not match the actual return code),
  the result will be **EXECUTION_FAIL**.
  The expected return code defaults to 0 but can be specified using an attribute of the ``<call>``
  tag in the test description: ``<call expected_return_code="7">``.
- If a timeout for the program call is specified (using the optional ``<timeout>`` tag) and the specified duration
  is exceeded by the running process, the process will be killed and the result will be **EXECUTION_TIMEOUT**.

In case none of the above error events occurred, the validations of the
test will be processed. Similar to the processing of tests, the each
validation will be started as described in its ``<call>`` section. The
result of the test will be **VALIDATION_FAIL** in case any of the
validation fails, **VALIDATION_TIMEOUT** in case one or more validations
exceeded their limit, or **PASS** in case all validations were
successful. When there are different validation errors for one test, the
most severe failure determines the result status of the test case.
**VALIDATION_FAIL** will be considered more severe than
**VALIDATION_TIMEOUT**.

Autotesters return code
~~~~~~~~~~~~~~~~~~~~~~~

The return code of Autotester will be the number of testcases that have
failed, independently for the reason they failed, or how many of their
validations have failed.