Now Online: “Building Blocks for Continuous Experimentation” (RCoSE 2014)

by Fabian Fagerholm, Alejandro Sanchez Guinea, Hanna Mäenpää, Jürgen Münch

 

Abstract: Context: Development of software-intensive products and services increasingly occurs by continuously deploying product or service increments, such as new features and enhancements, to customers. Product and service developers need to continuously find out what customers want by direct customer feedback and observation of usage behaviour, rather than indirectly through up-front business analyses. Objective: This paper examines the preconditions for setting up an experimentation system for continuous customer experiments. It describes the building blocks required for such a system. Method: A model for continuous experimentation is analytically derived from prior work. The proposed model is validated against a case study examining a startup company. Results: Building blocks for a continuous experimentation system and infrastructure are presented. Conclusion: A suitable experimentation system requires at least the ability to release minimum viable products or features with suitable instrumentation, design and manage experiment plans, link experiment results with a product roadmap, and manage a flexible business strategy. The main challenges are proper and rapid design of experiments, advanced instrumentation of software to collect, analyse, and store relevant data, and the integration of experiment results in both the product development cycle and the software development process.

Keywords: Continuous Experimentation, Product Development, Architecture, Agile Software Development, Lean Software Development, Lean Startup

Reference:

  • [PDF] Fabian Fagerholm, Alejandro Sanchez Guinea, Hanna Mäenpää, Jürgen Münch. Building Blocks for Continuous Experimentation. In Proceedings of the 1st International Workshop on Rapid Continuous Software Engineering (RCoSE 2014), Hyderabad, India, pages 26-35, June 2014.
    [Bibtex] [pdf]
    @inproceedings{RCOSE2014,
      author    = {Fabian Fagerholm, Alejandro Sanchez Guinea, Hanna Mäenpää, Jürgen Münch},
      title     = {Building Blocks for Continuous Experimentation},
      booktitle = {Proceedings of the 1st International Workshop on Rapid Continuous Software Engineering (RCoSE 2014), Hyderabad, India},
      month = {June},
      pages = {26-35},
      year      = {2014}
    }

Related References: 

  • [PDF] [DOI] Jürgen Münch, Fabian Fagerholm, Patrik Johnson, Janne Pirttilahti, Juha Torkkel, Janne Järvinen. Creating Minimum Viable Products in Industry-Academia Collaborations. In Proceedings of the Lean Enterprise Software and Systems Conference (LESS 2013, Galway, Ireland, December 1-4), volume 167 of LNBIP, pages 137-151. Springer Berlin Heidelberg, 2013.
    [Bibtex] [doi] [url] [pdf]
    @inproceedings{LESS2013a,
      author    = {Jürgen Münch, Fabian Fagerholm, Patrik Johnson, Janne Pirttilahti, Juha Torkkel, Janne Järvinen},
      title     = {Creating Minimum Viable Products in Industry-Academia Collaborations},
      booktitle = {Proceedings of the Lean Enterprise Software and Systems Conference (LESS 2013, Galway, Ireland, December 1-4)},
      publisher = {Springer Berlin Heidelberg},
    series = {LNBIP},
    volume = {167},
    pages = {137-151},
      year      = {2013},
    series = {LNBIP},
    doi = {10.1007/978-3-642-44930-7_9},
    URL = {http://link.springer.com/chapter/10.1007/978-3-642-44930-7_9},
      year  = {2013}
    }
  • [PDF] [DOI] Victor R. Basili, Michael Lindvall, Myrna Regardie, Carolyn Seaman, Jens Heidrich, Jürgen Münch, Dieter Rombach, Adam Trendowicz. Linking Software Development and Business Strategy Through Measurement. Computer, 43(4):57-65, 2010.
    [Bibtex] [doi] [url] [pdf]
    @ARTICLE{5445168, 
    author={Victor R. Basili, Michael Lindvall, Myrna Regardie, Carolyn Seaman, Jens Heidrich, Jürgen Münch, Dieter Rombach, Adam Trendowicz}, 
    journal={Computer}, 
    title={Linking Software Development and Business Strategy Through Measurement}, 
    year={2010}, 
    volume={43}, 
    number={4}, 
    pages={57-65}, 
    keywords={organisational aspects;software engineering;GQM+Strategies approach;business strategy;enterprise wide support;measurement program;software development;Joining processes;Programming;Software measurement;GQM;Goal/question/metric paradigm;Metrics/Measurement;Software engineering}, 
    doi={10.1109/MC.2010.108}, 
    URL = {http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=5445168},
    ISSN={0018-9162},}
  • Victor Basili, Adam Trendowicz, Martin Kowalczyk, Jens Heidrich, Carolyn Seaman, Jürgen Münch, Dieter Rombach. Aligning Organizations through Measurement. Springer-Verlag, 2014.
    [Bibtex] [url]
    @book{AOTM,
      author = {Victor Basili, Adam Trendowicz, Martin Kowalczyk, Jens Heidrich, Carolyn Seaman, Jürgen Münch, Dieter Rombach},
      title = {Aligning Organizations through Measurement},
      publisher = {Springer-Verlag},
      url ={http://www.springer.com/computer/swe/book/978-3-319-05046-1},
      year  = {2014}
    }
  • [PDF] [DOI] Alexander-Derek Rein, Jürgen Münch. Feature Prioritization Based on Mock Purchase: A Mobile Case Study. In Proceedings of the Lean Enterprise Software and Systems Conference (LESS 2013, Galway, Ireland, December 1-4), volume 167 of LNBIP, pages 165-179. Springer Berlin Heidelberg, 2013.
    [Bibtex] [doi] [url] [pdf]
    @inproceedings{LESS2013b,
      author    = {Alexander-Derek Rein, Jürgen Münch},
      title     = {Feature Prioritization Based on Mock Purchase: A Mobile Case Study},
      booktitle = {Proceedings of the Lean Enterprise Software and Systems Conference (LESS 2013, Galway, Ireland, December 1-4)},
      publisher = {Springer Berlin Heidelberg},
    series = {LNBIP},
    volume = {167},
    pages = {165-179},
      year      = {2013},
    doi = {10.1007/978-3-642-44930-7_11},
    url = {http://link.springer.com/chapter/10.1007/978-3-642-44930-7_11}
    }
  • [PDF] Fabian Fagerholm, Nilay Oza, Jürgen Münch. A Platform for Teaching Applied Distributed Software Development: The Ongoing Journey of the Helsinki Software Factory. In Proceedings of the 3rd International Workshop on Collaborative Teaching of Globally Distributed Software Development (CTGDSD 2013), pages 1-5, San Francisco, United States, May 25 2013.
    [Bibtex] [pdf]
    @inproceedings{FOM13,
    author = {Fabian Fagerholm, Nilay Oza, Jürgen Münch},
    title = {A Platform for Teaching Applied Distributed Software Development: The Ongoing Journey of the Helsinki Software Factory}, 
    booktitle = {Proceedings of the 3rd International Workshop on Collaborative Teaching of Globally Distributed Software Development (CTGDSD 2013)}, 
    year = {2013},
    month = {May 25},
    pages = {1-5},
    address = {San Francisco, United States}
    }