Validation of Control Software and Benchmarking

In the analysis of numerical methods and their implementation as numerical software it is extremely important to be able to test the correctness of the implementation as well as the performance of the method. This validation is one of the major steps in the construction of a software library, in particular if this library is used in practical applications.

In order to carry out such a test it is important to have tools that yield an evaluation of the performance of the method as well as the implementation with respect to correctness, accuracy and speed. Similar tools are needed to be able to compare different numerical methods, to test their robustness, and also to analyse the behaviour of the methods in extreme situation, i.e. on problems where the limit of the possible accuracy is reached.

In many application areas therefore benchmark collections have been created that can partially serve for this purpose. Such collections are heavily used. In our opinion, in order to have a fair evaluation and a comparison of methods and software, there should be a standardized set of examples, which are freely available and on which newly developed methods and their implementation can be tested. It is one of the goals of WGS to create such testing and validation environments for the area of numerical methods in control, in particular it is planned to accompany the SLICOT library with benchmark collections for each of the major problem areas. In order to make such collections useful it is important that the problems cover a wide range of problems and also problems are included that are difficult to solve in finite arithmetic. Such problems in particular drive the methods and their implementation to a limit. These are ideal test cases, since errors and failures usually occur only in extreme cases and these are often not covered by standard software validation procedures.

The SLICOT library currently contains such benchmark collections for:
- continuous-time and discrete-time standard and generalized Lyapunov matrix equations (see reports 2 and 3)
- standard and generalized continuous-time and discrete-time systems models (see reports 4 and 5)
- continuous-time and discrete-time algebraic Riccati matrix equations (see reports 6 and 7)
- identification (see report 8)
- model reduction of (high order) linear time invariant dynamical systems containing some useful "real world" examples reflecting current problems in applications can be found here (see also report 9)

Everybody is invited to submit such benchmark examples for the SLICOT benchmark collection. For submissions please contact Prof. Volker Mehrmann.