Now Online: How is Exploratory Testing Used? A State-of-the-Practice Survey – ESEM 2014

by Dietmar Pfahl, Huishi Yin, Mika V. Mäntylä, Jürgen Münch

 

Abstract: 

Context: Exploratory Testing has experienced a rise in popularity in the industry with the emergence of agile development practices, yet it remains unclear, in which domains and how it is used in practice.
Objective: To study how software engineers understand and apply the principles of exploratory testing, as well as the specific advantages and difficulties they experience. Method: We conducted an online survey in the period June to August 2013 among Estonian and Finnish software developers and testers.
Results: Our main findings are that the majority of testers, developers, and test managers using ET, (1) apply ET to usability-critical, performance-critical, security-critical and safety-critical software to a high degree; (2) use ET very flexibly in all types of test levels, activities, and phases; (3) perceive ET as an approach that supports creativity during testing and that is effective and efficient; and (4) find that ET is not easy to use and has little tool support.
Conclusion: The high degree of application of ET in critical domains is particularly interesting and indicates a need for future research to obtain a better understanding of the effects of ET in these domains. In addition, our findings suggest that more support to ET users should be given (guidance and tools).

 

Reference: 

  • [PDF] [DOI] Dietmar Pfahl, Huishi Yin, Mika V. Mäntylä, Jürgen Münch. How is Exploratory Testing Used? A State-of-the-Practice Survey. In Proceedings of the 8th ACM-IEEE International Symposium on Software Engineering and Measurement (ESEM 2014), Torino, Italy, September 2014.
    [Bibtex] [doi] [pdf]
    @inproceedings{ESEM2014,
    author = {Dietmar Pfahl, Huishi Yin, Mika V. Mäntylä, Jürgen Münch}, 
    title = {How is Exploratory Testing Used? A State-of-the-Practice Survey}, 
    booktitle = {Proceedings of the 8th ACM-IEEE International Symposium on Software Engineering and Measurement (ESEM 2014)},
    year = {2014},
    month = {September},
    doi={10.1145/2652524.2652531},
    address = {Torino, Italy}

 

Related Articles:

  • [PDF] [DOI] Simo Mäkinen, Jürgen Münch. Effects of Test-Driven Development: A Comparative Analysis of Empirical Studies. In Software Quality. Model-Based Approaches for Advanced Software and Systems Engineering. Proceedings of the 6th International Conference Software Quality Days (SWQD 2014), Vienna, Austria, volume 166 of Lecture Notes in Business Information Processing, pages 155-169. Springer Verlag, 2014.
    [Bibtex] [doi] [url] [pdf]
    @inproceedings{SWQD2014,
      author    = {Simo Mäkinen, Jürgen Münch},
      title     = {Effects of Test-Driven Development: A Comparative Analysis of Empirical Studies},
      booktitle = {Software Quality. Model-Based Approaches for Advanced Software and Systems Engineering. Proceedings of the 6th International Conference Software Quality Days (SWQD 2014), Vienna, Austria},
      publisher = {Springer Verlag},
      series = {Lecture Notes in Business Information Processing},
    volume = {166},
    pages = {155-169},
    year = {2014},
    doi = {10.1007/978-3-319-03602-1_10},
    url = {http://link.springer.com/chapter/10.1007/978-3-319-03602-1_10}
    }
  • [PDF] [DOI] Frank Elberzhager, Jürgen Münch, Vi Tran Ngoc Nha. A systematic mapping study on the combination of static and dynamic quality assurance techniques. Information and Software Technology, 54(1):1 – 15, 2012.
    [Bibtex] [doi] [url] [pdf]
    @article{Elberzhager20121,
    title = {A systematic mapping study on the combination of static and dynamic quality assurance techniques},
    journal = {Information and Software Technology},
    volume =  {54},
    number = {1},
    pages = {1 - 15},
    year = {2012},
    issn = {0950-5849},
    doi = {10.1016/j.infsof.2011.06.003},
    url = {http://www.sciencedirect.com/science/article/pii/S0950584911001388},
    author = {Frank Elberzhager, Jürgen Münch, Vi Tran Ngoc Nha},
    keywords = {Combination},
    keywords = {Dynamic quality assurance},
    keywords = {Inspection},
    keywords = {Static quality assurance},
    keywords = {Systematic mapping study},
    keywords = {Testing}
    }
  • [PDF] [DOI] Frank Elberzhager, Alla Rosbach, Jürgen Münch, Robert Eschbach. Reducing test effort: A systematic mapping study on existing approaches. Information and Software Technology, 54(10):1092 – 1106, 2012.
    [Bibtex] [doi] [url] [pdf]
    @article{Elberzhager20121092,
    title = {Reducing test effort: A systematic mapping study on existing approaches},
    journal = {Information and Software Technology},
    volume = {54},
    number = {10},
    pages = {1092 - 1106},
    year = {2012},
    issn = {0950-5849},
    doi = {10.1016/j.infsof.2012.04.007},
    url = {http://www.sciencedirect.com/science/article/pii/S0950584912000894},
    author = {Frank Elberzhager, Alla Rosbach, Jürgen Münch,  Robert Eschbach},
    keywords = {Efficiency improvement},
    keywords = {Mapping study},
    keywords = {Quality assurance},
    keywords = {Software testing},
    keywords = {Test effort reduction}
    }