English

Bellwethers: A Baseline Method For Transfer Learning

Software Engineering 2018-01-23 v4

Abstract

Software analytics builds quality prediction models for software projects. Experience shows that (a) the more projects studied, the more varied are the conclusions; and (b) project managers lose faith in the results of software analytics if those results keep changing. To reduce this conclusion instability, we propose the use of "bellwethers": given N projects from a community the bellwether is the project whose data yields the best predictions on all others. The bellwethers offer a way to mitigate conclusion instability because conclusions about a community are stable as long as this bellwether continues as the best oracle. Bellwethers are also simple to discover (just wrap a for-loop around standard data miners). When compared to other transfer learning methods (TCA+, transfer Naive Bayes, value cognitive boosting), using just the bellwether data to construct a simple transfer learner yields comparable predictions. Further, bellwethers appear in many SE tasks such as defect prediction, effort estimation, and bad smell detection. We hence recommend using bellwethers as a baseline method for transfer learning against which future work should be compared

Keywords

Cite

@article{arxiv.1703.06218,
  title  = {Bellwethers: A Baseline Method For Transfer Learning},
  author = {Rahul Krishna and Tim Menzies},
  journal= {arXiv preprint arXiv:1703.06218},
  year   = {2018}
}

Comments

23 Pages

R2 v1 2026-06-22T18:49:23.236Z