Affiliation:
1. University of California at Irvine, USA
2. Czech Technical University, Czechia
3. Microsoft Research, USA
4. Northeastern University, USA
Abstract
Previous studies have shown that there is a non-trivial amount of duplication in source code. This paper analyzes a corpus of 4.5 million non-fork projects hosted on GitHub representing over 428 million files written in Java, C++, Python, and JavaScript. We found that this corpus has a mere 85 million unique files. In other words, 70% of the code on GitHub consists of clones of previously created files. There is considerable variation between language ecosystems. JavaScript has the highest rate of file duplication, only 6% of the files are distinct. Java, on the other hand, has the least duplication, 60% of files are distinct. Lastly, a project-level analysis shows that between 9% and 31% of the projects contain at least 80% of files that can be found elsewhere. These rates of duplication have implications for systems built on open source software as well as for researchers interested in analyzing large code bases. As a concrete artifact of this study, we have created DéjàVu, a publicly available map of code duplicates in GitHub repositories.
Publisher
Association for Computing Machinery (ACM)
Subject
Safety, Risk, Reliability and Quality,Software
Reference32 articles.
1. Orion: A Software Project Search Engine with Integrated Diverse Software Artifacts
2. The DaCapo benchmarks
3. Hudson Borges André C. Hora and Marco Tulio Valente. 2016. Understanding the Factors that Impact the Popularity of GitHub Repositories. (2016). http://arxiv.org/abs/1606.04984 Hudson Borges André C. Hora and Marco Tulio Valente. 2016. Understanding the Factors that Impact the Popularity of GitHub Repositories. (2016). http://arxiv.org/abs/1606.04984
Cited by
129 articles.
订阅此论文施引文献
订阅此论文施引文献，注册后可以免费订阅5篇论文的施引文献，订阅后可以查看论文全部施引文献
1. Code search engines for the next generation;Journal of Systems and Software;2024-09
2. Enhancing Function Name Prediction using Votes-Based Name Tokenization and Multi-task Learning;Proceedings of the ACM on Software Engineering;2024-07-12
3. Promoting open science in test-driven software experiments;Journal of Systems and Software;2024-06
4. Dataset: Copy-based Reuse in Open Source Software;Proceedings of the 21st International Conference on Mining Software Repositories;2024-04-15
5. AntiCopyPaster 2.0: Whitebox just-in-time code duplicates extraction;Proceedings of the 2024 IEEE/ACM 46th International Conference on Software Engineering: Companion Proceedings;2024-04-14