A specialized data management system for parallel execution of particle physics codes

Abstract

The specialized data management system described in this paper was motivated by the need for much more efficient data management than a standard database management system could provide for particle physics codes in shared memory multiprocessor environments. The special characteristics of data and access patterns in particle physics codes need to be fully exploited in order to effect efficient data management. The data management system allows parameteric user control over system features not usually available to them, especially details of physical design and retrieval such as horizontal clustering, asynchronous I/O, and automatic distribution across processors. In the past, each physics code has constructed the equivalent of a primitive data management system from scratch. The system described in this paper is a generic system that can now be interfaced with a variety of physics codes.