How to Design a Modular, Effective, and Interpretable Machine Learning-Based Real-Time System: Lessons from Automated Electrical Submersible Pump Surveillance

Abstract

Abstract Many machine learning (ML) projects do not progress beyond the proof-of-concept phase into real-world operations and remain economical at scale. Commonly discussed challenges revolve around digitalization, data, and infrastructure/tooling. However, there are other non-ML aspects that are equally if not more important towards building a successful system. This paper presents a general framework and lessons learned for building a robust, practical, and modular domain-centric ML-based system in contrast to purely "data-centric" or "model-centric" approaches. This paper presents the case study of a sophisticated "plug-and-play" real-time surveillance system for electrical submersible pumps (ESP) that has been successfully serving hundreds of wells of various configurations. The system has also been successfully tested to expand beyond advisory surveillance to include closed-loop-control for autonomous response to events. We discuss some of the intelligent design strategies that allow us to address various requirements and practical constraints, while still ensuring effective performance in the field. This paper also presents general learnings, design suggestions, and components that can be adapted for building similar ML-based systems for multivariate time-series type problems. The paper demonstrates with examples how building an artificial intelligence (AI) system with modular independent components can be more practical and effective in comparison to training large end-to-end deep learning models. These components can be independently tested, refactored, and even repurposed as libraries for other applications. We explain the significance of the first component, namely the data quality (DQ) engine, which is critical for any real-world engineering application dealing with the challenges of streaming field data. We discuss a second component, the reference engine, covering smart and practical ways in which first principles, subject-matter-expert knowledge, and memory-like features can be embedded into the system. Through the example of ESP surveillance, we differentiate performance of ML models from the performance of the overall system, and explain the complexities and trade-offs that go into tuning and evaluating such AI systems. We highlight the importance of designing observability into the core engine so each decision step within the system can be analyzed and explained. Such transparency is necessary for critical applications needing actionable insights and continuous improvements. The paper also shares a few other lessons from implementing surveillance at scale, and the ability to reuse components for robust, reliable closed-loop edge automation. Literature on ML system design is dominated by experiences from the technology sector, particularly consumer-facing applications. These are often challenging to adopt directly for high-risk, resource-constrained, and dynamic applications in upstream oil and gas. The insights from this paper will benefit data/ML professionals and help promote a greater appreciation by business leaders for what is required to build realistic real-time systems that incorporate ML models.