Report from the 2nd Model Based Testing User Conference
The second Model-Based Testing User Conference was held in Estonia`s capital Tallinn this year. Like last year`s conference, the focus was to convene active users of MBT rather than academic researchers.
The first keynote was held by Jan Plasberg from Skype. Although not directly covering MBT, this talk nevertheless was enlightening because it illustrated the massive scale of Skype`s service and the testing challenges associated with that.
Robert Binder held the second keynote. He argued that current MBT testing techniques that aim for pure functional testing are not sufficient to find possible faults that may arise in today`s software architectures. These typically are increasingly interdependent, heterogeneous and strongly distributed. Binder therefore proposes “multi-dimensional model-based testing” where functional MBT is interleaved with performance and robustness testing.
The third keynote, by Jeremy Dick, highlighted the importance to realize the (implicit) assumptions one makes when designing test cases and how this also applies to model-based testing.
In the paper sessions, many experience reports from applying MBT in various domains were presented. Some general observations could be made:
- MBT (still) is not a testing technique to get started with easily. Most presenters stated, however, that they could achieve improvements either in testing efficiency and/or effectiveness once the test process was established.
- Failure stories of MBT, as called for in the previous blog post, were absent. Maybe that would be an interesting point to explicitly add to the call for contributions for next year`s conference?
- Judging from the contributions, the major areas of application for MBT currently seem to be the telecommunications sector and the testing of embedded devices.
- The tool market is dominated by few vendors whose tools have made tremendous advantages in their capabilities recently. But there are still issues that could be improved, for example, the ability to handle large models without choking the test generator algorithms was mentioned.
As this event was organized by the European standardization organization ETSI, there were also reports of ETSI`s activities concerning MBT. Prof. Jens Grabowski talked about case studies that were executed to rate the applicability of MBT for validation testing of standard implementations at ETSI and to outline a tool independent MBT methodology.
Andreas Ulrich from Siemens talked about the efforts towards the ETSI Test Description Language (TDL). The purpose of TDL is to serve as a tool independent way to represent test descriptions (test cases). In comparison to the UML Testing Profile, the advantages of the TDL were amongst others stated as the ability to define various syntaxes in contrast of the graphical syntax of the UTP and the extendibility as well as reducibility to accommodate the exact user needs. It will be interesting to see whether the TDL will be used as standard output format for MBT tools.
All presentation slides can be downloaded from the conference web site.