Project Activities and Working Methods

The project identifies, specifies, creates and verifies conformance tests for MHPs and makes these validated tests available through the standardisation process of DVB and ETSI as a significant additional part of the official MHP Test Suite. These steps require the  following main activities:
  1. to identify areas where such tests are required and where interoperability can be improved, and to create and prioritise a series of test specifications (so-called  assertions)
  2. to implement and validate these tests
  3. to debug, correct and validate already existing, but incorrect tests
  4. to provide them to the standardisation body and to support the DVB official review and maintenance process
  5. to develop and propose business models to enable a longer-term support for continuous improvement and maintenance of the MHP Conformance Testing regime beyond the lifetime of this IST project
Activity (1) is driven from two directions:
  • A "specification centric approach" which analyses the actual MHP specification in every detail and extracts test conditions from every single detail given.
  • An "experience driven approach" which analyses practical experience and interoperability problems with real certified products and applications. This approach is new within the MHP conformance testing activities and can be established by MHP-CONFIDENCE since certified MHPs and regular applications are available on the market and the project partners have direct access to this experience through their own development departments, own experience and investigations, and feedback collected from the MHP user community. During an MHP Interoperability Workshop organised by IRT, the project has carried out an investigation on feedback from users.

This activity includes establishing of a communication to other contributors to the official MHP Test Suite in order to jointly further improve Conformance Testing at the transition between certain areas of by the MHP specification.

Once the test specifications have been developed, Activity (2) includes the selection of developers for sub-contracting of programming work in order to save resources of project partners to debug and scrutinise the tests on a series of MHP devices.

Activity (3) allows to include pre-existing work provided by the partners of the project.

Activity (4) covers the dissemination of the project's results to DVB and ETSI as well as the support required to accept and maintain a Test Suite for public use. This activity is not limited to only MHP and is open to derived and related standards, such e.g. in ARIB and OCAP. In cases where the MHP specification is found to be detailed, it also includes feedback to the group in charge of this specification. The project's results are submitted in a packaged format as required by DVB.

Activity (5) includes careful analysis of commercial and legal requirements of current or future MHP Test Suites, as well as the developing framework for Java Test Suites in general.


Working Methods
The project started with a set of existing test assets which were partly validated tests, implemented but not validated tests, or test assertions. Some parts of these assets were already contained in the official DVB MHP 1.0.2.b Test Suite valid at the time of the start of the project. These assets were further developed by the following activities:

Based on a coverage analysis of the current Test Suite 1.0.2.b and on the analysis of current interoperability problems experienced in the market with practical applications, the project identified areas where further test development should focus.

The project prioritised the results of that analysis, and defined clear areas which would benefit from additional conformance testing. From this work, a series of technical descriptions were created which served as a basis for implementation of new tests.

After implementation, tests were validated in a close and joint co-operation of the project partners as a major work item of the project. A joint approach is absolutely required for conformance tests, since these tests need to be of very high quality, need to stick exactly o the underlying specification, and must not test extra implementation specific details beyond the specification. The project established working methods and develop technical infrastucture to support processing of several thousands of tests in an effective way.