The challenge of open-texture in law

Abstract

AbstractAn important challenge when creating automatically processable laws concerns open-textured terms. The ability to measure open-texture can assist in determining the feasibility of encoding regulation and where additional legal information is required to properly assess a legal issue or dispute. In this article, we propose a novel conceptualisation of open-texture with the aim of determining the extent of open-textured terms in legal documents. We conceptualise open-texture as a lever whose state is impacted by three types of forces: internal forces (the words within the text themselves), external forces (the resources brought to challenge the definition of words), and lateral forces (the merit of such challenges). We tested part of this conceptualisation with 26 participants by investigating agreement in paired annotators. Five key findings emerged. First, agreement on which words are open-texture within a legal text is possible and statistically significant. Second, agreement is even high at an average inter-rater reliability of 0.7 (Cohen’s kappa). Third, when there is agreement on the words, agreement on the Open-Texture Value is high. Fourth, there is a dependence between the Open-Texture Value and reasons invoked behind open-texture. Fifth, involving only four annotators can yield similar results compared to involving twenty more when it comes to only flagging clauses containing open-texture. We conclude the article by discussing limitations of our experiment and which remaining questions in real life cases are still outstanding.