Interview by Stephan Weißleder at ICST, Berlin, March 2011
Prof. Dr. Alexander Pretschner has worked at the Technical University in Munich, the ETH Zurich, the Fraunhofer-Institute IESE, and the Technical University in Kaiserslautern. He is currently professor at the Karlsruhe Institute of Technology. His research interests are focused on security of distributed data and systems, system engineering, usage control, privacy, model-based testing, automatic test case generation, and model-based development. He is the author of more than 50 publications and a well-known expert in his fields of research.
Model-Based Testing Community (MTC): How did you come to MBT?
Alexander Pretschner (AP): MBT offers possibilities to investigate practical and theoretical questions. It is a discipline that represents a link between models and the real world, a link between mathematics and software engineering.
MTC: What is your research focus in MBT? Or where do you apply MBT techniques in the industry?
AP: I currently put the focus on several aspects: The first one is random testing as a comparative means to evaluate the effort and benefit of other testing techniques. The second are domain-specific testing approaches like security-based testing in which – like in most other domains – it does not seem enough to just apply structural coverage criteria to measure test quality. Instead, properties seemt to be far more relevant. Last, I work on bridging the different abstraction layers in MBT: How can traces of a model automatically be compared to those of an SUT? This also includes runtime verification.
MTC: What does MBT stand for to you?
AP: It was the topic of my PhD thesis. And the PhD thesis often determines the rest of your life 😉
MTC: Did you expect MBT to gain so much attention when you first saw it back in (whenever)?
MTC: What do you expect to be the future challenges for MBT?
AP: The future challenges in MBT will be model quality, investigations on the cost effectiveness, parallel, non-deterministic behavior, process integration, and model management tools. First, there is the fundamental problem of “garbage in – garbage out”, which means that a low model quality can easily ruin all other efforts in MBT. Second, there is no broad knowledge about the cost efficiency of MBT. Maybe, this is a fundamental problem of all software engineering disciplines. Since cost effectiveness is said to be one of the main advantages of MBT, however, this point has to be investigated. Third, parallel and non-deterministic systems are hard to test for several reasons like controllability, observability, and adequate properties to be tested. Fourth, I am not sure a development process that does not use explicit models will be very likely to absorb MBT approaches. Finally, I don’t think we have Eclipse-like IDEs with debugging and concurrent editing facilities for models – but I am convinced that this is absolutely necessary to work productively.
MTC: Where is MBT well applicable / where not?
AP: MBT is well applicable in domains like hardware, protocols, etc. There are, however, many data-intensive systems like databases for which the efficient and effective applicability of MBT technology remains to be shown.
MTC: The importance of certification is growing in the testing domain, like the ISTQB certifications schemes. How do you value the usefulness / importance of such certifications for industry / academia? Do we need an MBT certification scheme in the future?
AP: In my opinion, it is not necessary, yet, to transfer detailed expert testing knowledge to industry. I think that the usefulness of such certification schemes is limited.
MTC: Do you think current MBT tools already have industrial strength?
AP: There are several industrial MBT tools. There are, however, several missing aspects in the existing tool chains that make me doubt the “industrial strength” of such tools. As one example that I already mentioned, model quality has a huge impact on the overall tool quality.
MTC: Do you think we have enough MBT companies (consulting + tool vendor) or should the market be more diversified?
MTC: Is MBT a mere hype, or will it become (/ is it already) a new main approach for software testing?
AP: I don’t doubt that there are tremendous advantages in using MBT. As said before, however, there is no empirical evidence on these advantages, yet. Thus, I would say that MBT is currently a hype.
MTC: With the maturing of formal verification techniques like model checking – do you see the importance of testing decrease in the future? Do you hope so?
MTC: Do you think MBT should go hand in hand with MDD/MDA or be totally separated from it?
AP: In my opinion, a separation of MBT and MDD/MDA is unlikely. There are thoughts about using MBT as a vehicle to bring modeling techniques to industry. I think, however, that the application of MBT alone is too expensive.
MTC: Testing became quite popular also in academia in the recent years. There is a lot of scientific output in form of papers / conferences / workshops. But looking at the quality and peer reviews of the publications it seems like the average quality is quite disappointing compared to other disciplines. Does testing attract less qualified people since testing appears at first sight less complicated and demanding than for instance formal verification?
AP: I don’t agree that testing papers are worse than papers from other areas.
MTC: How do you plan to pursue MBT at the Karlsruhe Institute of Technology?
AP: In my job as a professor at the KIT, I teach software engineering to the students. This also includes MBT with all its possible advantages, disadvantages, pitfalls, and – of course – the empirical evaluation of this technique. Especially the last point has to be discussed in a fair manner. Research-wise, we are currently working on property-based testing and on bridging levels of abstraction.
MTC: What are your personal goals with regard to model-based testing?
AP: One of my personal goals in MBT is to focus on important properties of systems rather than on purely syntactic coverage. In my opinion, the focus of future MBT research has to be on property-based quality metrics, fault models, process integration, model quality, and cost effectiveness of MBT.