Model-Based Testing Community | Connecting the MBT world…

Apr/13

16

MBT needs Requirements

Hi all,

we are (again) thinking about the essence of model-based testing. One of the major aspects is the quality gain that can be reached by using MBT. This gain strongly depends on the quality of the test cases. And since the test cases are generated automatically and there are few ways to insert the quality into this automatic process, the quality of the test cases strongly depends on the quality of the test generation input – the requirements.

For safety-critical systems, there are further advances of well-defined requirements. For instance, one important point is traceability from tests to requirements: for each test case, one has to identify the requirements that are covered by it. There are several tools that allow to create and maintain this traceability during test generation like, e.g., by Conformiq or sepp.med.

Since requirements can often be complex and not easy to be represented in models  that are used for test generation by simple links to model elements, this approach is not always feasible. Different approaches to test generation make this approach even harder. There are, for instance, approaches like ConSequence in which sequences are interwoven with a state machine for test generation. If users want to use their test generator of choice, this approach is infeasible. Another approach that we recently published is based on decoupling test generation from requirements tracing by matching the generated test cases automatically to the requirements. This approach also requires a more fine-grained definition of requirements. More details can be found here.

And even though formulating requirements more precisely has been preached for years already, it has not been adopted in practice. We believe there are multiple causes:
* Requirements would have to be formulated more formally, which requires appropriate knowledge and tools.
* The gap between the domain language and the formal requirements language becomes larger, making the translation more difficult.
* A requirements-first approach (similar to test-first) would support precise requirements and leveraging them, but processes would need to change, which is often difficult to accomplish.

We would thus like to start a discussion about the necessity of (formal) requirements for the success of model-based testing and new approaches of creating and maintaining traceability during test generation and are very interested to learn about your opinion.

Cheers
Stephan

No tags

4 comments

  • Andres Kull · April 16, 2013 at 11:01 am

    No. You can’t force people to define requirements formally in sake of testing. MBT should adapter to the way people are used to do things and not vice versa.

    Reply

    • Author comment by David Farago · April 17, 2013 at 5:15 pm

      You have a good point there, Andres: MBT must be user friendly!

      Do you think a requirements-first approach is out of the question? After all, test first has become very popular for modern companies, in spite of having to strongly change processes and the way of thinking.

      Do you think a requirements-first approach would improve the quality of the requirements? I believe most reasons from test first (e.g. communication, finding faults early, setting the scope) apply here, too.

      Reply

    • Stephan Weißleder · April 22, 2013 at 7:56 pm

      Hi Andres,

      I think that this issue is not easy to solve. Of course, one wants the new appraoches (like MBT) to adapt to the existing tool chains and processes and not the other way round.
      In this case, however, the formalization of requirements is totally independent from MBT. It is an independent topic that is probably even older than MBT…
      I just used the combination of both to show the advantages of having (semi-)formal requirements – in this case for MBT. Many engineering task would benefit from this approach …

      Reply

  • Bruno Legeard · April 18, 2013 at 5:40 pm

    Hi!

    The difference between Requirements-firts, and Test-First is that tests are runnable artefacts. This means that there are always updated as soon as they must to be executed. This not the case of the requirements, so these artefacts are not updated (and so quicly out-of-date).

    The solution is obviously to design executable requirements. If we do so, functional tests then
    will not be useful anymore. Executable reqs = Tests. It’s partly what we try to do with our MBT models.

    Three formal artifacts (i.e. Formal Reqs, Model for test generation, application code) is not lean. Two artifacts are sufficient for redundancy.

    Cheers. Bruno.

    Reply

Leave a Reply

<<

>>

Theme Design by devolux.nh2.me
© by MBT Community 2011