How tests are created/verified









Tests are typically created in four steps as described below:
  1. Development of Test Assertions
    Test Assertions comprise a textual description of a specific detail of the MHP specification that shall be tested in a single test, the reference to the specification and further information as e.g. relevance or testability. Test Assertions are derived by both, a detailed analysis the MHP specification ("paper work"), and by analysis of interoperability problems.
  2. Development of Test Strategies
    Test Strategies are developed from Test Assertions and give a detailed guidance to a programmer how to implement a test.
  3. Implementation of Tests
    This comprises the actual programming of a test.
  4. Verification of Tests
    As any software development result, newly implemented tests usually contain errors or non-allowed dependencies of an underlying implementation (e.g. on manufacturer-dependent and irrelevant details of an MHP). In order to eliminate these problems, the tests need to be run and evaluated on a series of different MHPs. The results need to be jointly evaluated and further debugging work  needs need to be split up amongst partners. Since the number of tests dealt with is in the area of several thousands, this requires underlying structures such as common databases for tests (code and results) and an automated environment to run large sequences of tests without manual interaction.

After successful verification of a test code, it is available for further packaging with according documentation, e.g. the related assertions and strategies developed.

Steps one and four are typically R&D centric and require most efforts and most detailed knowledge of MHP.

Example for a Test Assertion:
The following text shows an assertion for the MHP API "org.davic.mpeg.sections.SimpleSectionFilter":

When multiple calls are made to getSection() with no intermediate calls to startFiltering(), and a Section has been received that matches the filter characteristics that have been specified, then the same Section will always be returned.

Example for a Test Strategy:
The following description gives detailed information to an implementer (programmer) how to translate the above assertion into Java code:

  • Create a SectionFilterGroup.
  • Obtain a TransportStream object for the TS that contains the application.
  • Attach the section filter group to the transport stream. Create a SimpleSectionFilter.
  • Create a new object to listen to filter events, and register it with the SimpleSectionFilter.
  • Start the filtering.
  • Wait for a SectionFilterEvent to be generated.
  • Check that the SectionFilterEvent is a SectionAvailableEvent.
  • Call getSection() twice, to retrieve the filtered Section.
  • Verify that the two calls to getSection() returned the same object.
  • Use the != operator to test for inequality.