An Empirical Study on the Correctness and Effort to Integrate Feature Models

Abstract

Feature model integration is pivotal in software development, particularly in evolving software product lines through new feature accommodations. Despite its significance, the influence of developers’ experience on integration efforts and correctness still needs to be more adequately understood. This study conducted a controlled experiment with 25 participants (18 students and seven professionals) following well-known guidelines to run empirical studies. Each participant addressed ten experimental tasks, encompassing 250 integration scenarios, to explore two research questions. The effort and correctness rate in integrating feature models were quantified, revealing that students exerted higher effort (29.23%) and achieved a higher number of correct integrations (39.53%) than professionals. Notably, this superiority lacked statistical significance. Additionally, this article highlights practical implications and noteworthy challenges for the scientific community, providing valuable insights for software development practices. The findings lay a foundation for future studies, delving into software development tasks where students and professionals may achieve comparable results. Finally, this study marks an initial step towards an ambitious agenda, empirically advancing the feature model integration field.