Talks about MBT on TAV33
TAV33 was the 33rd conference organized by the group “Test, Analysis and Verification (TAV)” of the German Gesellschaft für Informatik (like IEEE). It was held in Bad Honnef on the 3rd and 4th of May and was co-located with the group “Software-Reengineering” and “Programming languages and Computational Concepts”. So visitors were able to find many similarities between these groups, e.g. model-based approaches and the use of static analysis.
Two talks about MBT were presented: One from Prof. Dr. Andreas Zeller, Universität des Saarlandes, about Expermintal Program Analysis, and one from TAV’s spokesman Dr. Andrej Pietschker, Giesecke & Devrient, about MBT for real-time embedded software. I thank both speakers for their interesting presentations.
Andreas Zeller’s keynote about his Experimental Program Analysis (see slides) was quite innovative: It integrates aspects from static as well as dynamic analysis by iteratively cycling through test execution and evaluation on the one hand and through enriching specifications and extending test suites on the other. The twist is that specifications are not created manually and then tested for conformance to the source code; instead, the specification is generated automatically from the code, and then a human checks whether it conforms to his expectations.
Andreas Zeller’s resulting tool SpecMate has obtained several grants. Its application has found bugs in Facebook and the Princess on the Pea Phenomenon for Experimental Program Analysis, which states that it is more efficient to test a core library through its GUI than by testing it directly, since the GUI gives a sensible test prioritization.
I found the talk very interesting and hope to hear more about Andreas Zeller’s work, especially what kind of faults SpecMate is able to detect and how easily a human validates the automatically generated specifications.
Andrej Pietschker talked about some very successful model-based testing for real-time embedded software of a banknote processing system at Giesecke & Devrient (see slides): Since they already had a test bed and sufficiently detailed specifications, incorporating MBT only required two days.
The specifications were hierarchical, parallel state machines (like Statemate, but without history) and sufficiently formal and detailed to automatically generate test cases in TTCN-3 from them. For this, Conformiq Designer has been used, since it was able to carry over test cases already present and offered a good process to handle test cases that had become invalid. It also checked for deadlocks in the specification and considered statement and transition coverage of the specification during test generation. The resulting tests and test process were also effective: tests are understandable, reproducible, executable on real hardware, and can be generated simultaneously to software development – just faster 🙂
It would be great to see this application of MBT in action and more detail, e.g. in comparison to other tools and how the aspect of real-time is being handled.
All in all, TAV33 was a great success. Once again MBT proved to be one of the future technologies in the software testing area.