Expensive bet lost because of missing model-based testing solution
This contribution comes from the Systems Engineering community which deals with developing (and thus also testing) heterogeneous systems that consist e.g. of mechanical hardware, electronics, microwave transmitters and a bit of software. When talking of model-based testing we thus talk about the model-based testing of such a system.
We would like to tell about a silly bet we made on April 2014: While Jesko doubted that model-based testing was able to fulfill his vision as of today, Tim was sure that it was. So we made the following bet:
Jesko to Tim: “I bet you a bottle of wine that you won’t be able to show me a solution meeting the requirements below by February 11th, 2016.”
The solution consists of a method and a tool supporting it. The requirements are as follows:
- REQ01: The method shall be identified as a method of model-based testing by at least two peer-reviewed publications or a book.
- REQ02: The method shall be so easy to learn that at least one systems engineers with the certification „Certified Systems Engineer GfSE ®“ can learn it within one week to an extent that he or she can explain it correctly afterwards (Success criterion: „explain it correctly“ means that at least one of the authors of the publications from REQ01 confirms that the explanation is correct).
- REQ03: A student at a “Fachhochschule” (Austrian / German / Swiss university of applied technology) who studies an engineering discipline in 2nd term without prior knowledge in Systems Engineering shall be able to learn the test executor role within 4 hours to an extent that enables him or her to do test execution with the tool according to predefined test procedures.
(Clarification: The 4 hours are the time needed for looking into the method and the tool. Possible additional trainings in using measurement equipment etc. are not included.
Definition: “without prior knowledge in Systems Engineering“ means: if asked: “do you know what systems engineering is“, then the student will reply “No.”)
- REQ04: The tool shall run on a Windows computer.
- REQ05: The student according to REQ03 shall be able to complete entering a piece of test data in the tool only 5 minutes after the login into Windows, once he or she has had the earlier-mentioned 4 hour introduction.
(Success criterion: The reference piece of test data is the result of a voltage measurement: “3.9 Volts“)
- REQ06: There shall be less than or equal to 10000 EUR initial software costs for the software license needed by the student according to REQ03. NOTE: The cost has to be computed based on nominal price information without any discount.
- REQ07: There shall be less than or equal to 2000 EUR yearly cost for software maintenance on the computer used by the student according to REQ03. NOTE: The cost has to be computed based on nominal price information without any discount.’
On February 11th 2016, Jesko and Tim assessed the outcome during a workshop at Hannover, Germany. By the time of the meeting they had identified three tools that might have helped Tim winning the bet, but no one had been able to show that they meet the above requirements. The three tools are as follows:
- ParTeG: http://parteg.sourceforge.net/
- MateLo: http://www.all4tec.net/Matelo/model-based-testing.html
- Conformiq: http://www.conformiq.com/
Tim’s resulting inevitable fate is shown in the picture below.
We like to encourage all who think they can present a solution that meets REQ01 to REQ07 above to contact us and explain the solution.