Experimental Comparison of Editor Types for Domain-Specific Languages

Abstract

The editor type can influence the user experience for a domain-specific language, but empirical evaluation of this factor is still quite limited. In this paper, we present the results of our empirical study, in which we compare the productivity of users with different kinds of editors for the same domain-specific language. We chose the domain of quiz definitions and used three editors: a text editor with syntax highlighting and code completion developed with the Xtext framework, a projectional editor created using JetBrains MPS, and an existing form-based editor—Google Forms. The study was performed on 37 graduate students of computer science. The measured time was lower for the text editor than for the form-based editor, and the form-based editor’s time was lower than the projectional one’s; however, the results were statistically insignificant. The experiment was also complemented with a survey providing insight into the perception of different editor types by users.