Encountering Incomplete Temporal Information in Clinical Data Warehouses

Abstract

A clinical data warehouse (CDW) can be an important tool for the purposes of analysis and critical decision making in the medical field. Such a data repository integrates heterogeneous health data, including clinical, treatment and diagnostic data and laboratory test results from a variety of sources. Accurate data need to be stored and processed in a CDW with adequate computation capabilities and thus, time plays a crucial factor. A slowly changing dimension (SCD) is a dimension that changes slowly over time, either gradually or intermittently. This article introduces a new SCD type, Type BTA, where both valid time and transaction time are supported for providing a complete history of the dimensional data. With Type BTA, the history of an object can be captured through the changes as reflected in the CDW. Consequently, for the first time, the full history of retroactive and post-active changes can be preserved in a CDW. Specifically, Type BTA is implemented for a Clinical Data Warehouse using real cancer data, for which the advantages of this methodology are demonstrated and advocated.