Automated competitor analysis using big data analytics

Abstract

Purpose Competitor analysis is a key component in operations management. Most business decisions are rooted in the analysis of rival products inferred from market structure. Relative to more traditional competitor analysis methods, the purpose of this paper is to provide operations managers with an innovative tool to monitor a firm’s market position and competitors in real time at higher resolution and lower cost than more traditional competitor analysis methods. Design/methodology/approach The authors combine the techniques of Web Crawler, Natural Language Processing and Machine Learning algorithms with data visualization to develop a big data competitor-analysis system that informs operations managers about competitors and meaningful relationships among them. The authors illustrate the approach using the fitness mobile app business. Findings The study shows that the system supports operational decision making both descriptively and prescriptively. In particular, the innovative probabilistic topic modeling algorithm combined with conventional multidimensional scaling, product feature comparison and market structure analyses reveal an app’s position in relation to its peers. The authors also develop a user segment overlapping index based on user’s social media data. The authors combine this new index with the product functionality similarity index to map indirect and direct competitors with and without user lock-in. Originality/value The approach improves on previous approaches by fully automating information extraction from multiple online sources. The authors believe this is the first system of its kind. With limited human intervention, the methodology can easily be adapted to different settings, giving quicker, more reliable real-time results. The approach is also cost effective for market analysis projects covering different data sources.