Computationally Efficient Labeling of Cancer-Related Forum Posts by Non-clinical Text Information Retrieval

Abstract

AbstractModern societies produce vast amounts of digital data and merely keeping up with transmission and storage is difficult enough, but analyzing it to extract and apply useful information is harder still. Almost all research within healthcare data processing is concerned with formal clinical data. However, there is a lot of valuable but idle information in non-clinical data too; this information needs to be retrieved and activated. The present study combines state-of-the-art methods within distributed computing, text retrieval, clustering, and classification into a coherent and computationally efficient system that is able to clarify cancer patient trajectories based on non-clinical and freely available online forum posts. The motivation is: well informed patients, caretakers, and relatives often lead to better overall treatment outcomes due to enhanced possibilities of proper disease management. The resulting software prototype is fully functional and build to serve as a test bench for various text information retrieval and visualization methods. Via the prototype, we demonstrate a computationally efficient clustering of posts into cancer-types and a subsequent within-cluster classification into trajectory related classes. Also, the system provides an interactive graphical user interface allowing end-users to mine and oversee the valuable information.