Identification of high-reliability regions of machine learning predictions in materials science using perovskite oxides as an example

Abstract

Abstract Progress in the application of machine learning (ML) methods to materials design is hindered by the lack of understanding of the reliability of ML predictions, in particular for the application of ML to small data sets often found in materials science. Using ML prediction of lattice parameter, formation energy and band gap of ABO3 perovskites as an example, we demonstrate that 1) similar to the composition-structure-property relationships, inclusion in the ML training data set of materials from classes with different chemical properties will not be beneficial and will decrease the accuracy of ML prediction; 2) Reliable results likely will be obtained by ML model for narrow classes of similar materials even in the case where the ML model will show large errors on the dataset consisting of several classes of materials, and 3) materials that satisfy all well-known chemical and physical principles that make a material physically reasonable are likely to be similar and show strong relationships between the properties of interest and the standard features used in ML. We also show that analysis of ML results by construction of a convex hull in features space that encloses accurately predicted systems can be used to identify high-reliability chemically similar regions and extract physical understanding. Our results indicate that the accuracy of ML prediction may be higher than previously appreciated for the regions in which the ML model interpolates the available data, and that inclusion of physically unreasonable systems is likely to decrease ML accuracy. Our work suggests that analysis of the error distributions of ML methods will be beneficial for the further development of the application of ML methods in material science.