In the last weeks I have attended many industrial conferences where we conducted many talks on MBT. My impression is that MBT still didn’t reach into people’s hearts.
At iqnite conference in Düsseldorf last week, visitors feedback has shown that only 2 % of attending companies are familiar with or have tried MBT. I learned more about the skepticism of attendees in the open discussion session. In this session we have discussed on the ROI of software testing techniques. More than 20 experienced testers or test managers had difficulties on formulating the ROI of any testing technique. Two main arguments were about reducing the costs for bug-fixes in late stages of the development process and about increasing the test coverage. However, the attendees were not convinced that we can find more bugs using MBT. On the other hand, increasing the test coverage was not a sufficient justification for investing in MBT. Also in my workshop on requirements-based testing on the last day of the conference, the attendees reported that model-based approaches are still not put into practice. They mostly work with textual requirements specifications which are used for testing purposes.
A few weeks ago on the German Testing Night, Helmut Götz gave a talk on ROI of MBT in Siemens. He explained that they could save more than 200 T€ in a project just by considering the errors detected in the requirements documents. He argumented that fixing those errors would cause this amount of money if they had been detected after delivery. In this calculation, he also considered the time effort saved for the test case design.
In summary, there is a need for clear arguments which show the “countable” value of MBT in order to motivate testers and test managers for trying and gaining experience with MBT. Many experience reports show that there is a clear benefit in MBT, especially in validating the quality of requirements.
Links for conferences:
Links for literature:
- Return On Investment @ MBT (System Level) – An Industrial MBT Case Study & its Return On Investment
- Integrate model-based testing to find quality problems early in development