Review of the JAMAICA 2014 workshop
The workshop on Joining AcadeMiA and Industry Contributions to Test Automation and Model-based Testing (JAMAICA) intends to foster the discussion on the current practices for automating and model-based testing activities among academia and industry (such as automotive, avionics, and factory automation). This should show the community what impact these activities may have on the overall cost/quality trade-off and on the development cycle. The second workshop, JAMAICA 2014, was co-located at ISSTA 2014 in California end of July.
The program started with an interesting keynote by Dr. Rajesh Subramanyan, Director Engineering at Siemens Healthcare Diagnostics, covering topics of Risk-Based Testing.
The submitted works covered a wide variety of topics within two sessions: Testing Techniques and Web and Distributed Applications. The presentations were distributed half by half between academia and industry contributions, which also benefited the discussion on all covered topics. The main topics have been: test automation, risk-based testing, user-interface (UI) testing, security testing, performance testing, and domain-specific testing, e.g., in the automotive and mobile applications industry. MBT was touched by several talks, but there has been no explicit presentation on MBT. The keynote and submitted works were very in line with the original intent of JAMAICA of discussing about the gap and the bridge between theoretically sound but hard-to-apply solutions and practical needs of industry.
The program finished with a lively debate as an open fishbowl discussion where we had three seats to be taken by discussion participants and one hot seat to be taken by any of the audience members. A central question has been why tools from academia are not in use within industry projects. Are they too outdated, are they too complex to handle, is there a lack of knowledge from the engineers or do they not scale to real world programs? One reason may be time, which is the industry’s ultimate budget; if you run out of it, engineers tend to use whatever method they already know, so there is often no time to learn and use new tools. Nevertheless, industry often requires idiot proven tools, which no PhD will design. There has been a common sense that more testing courses should be integrated during university education and testing should be seen as a career option. One participant stated that they compensate their testers better than their developers. The third discussion round has been dedicated to coverage metrics, analyzing if coverage is enough. Requirement specifications coverage is a good opportunity for a contract, but technically error models are more interesting. “Bugs the developer cares about will be fixed first”.
Due to the contributions and discussions, we plan on organizing JAMAICA again next year (at ISSTA 2015).
Harald Altinger, David Faragó, Roberto Pietrantuono