Diabetes is a chronic metabolic disease currently affecting almost 415 million patients worldwide with an estimation of this reaching 642 million by the year 2040 [1]. Having diabetes is associated with substantially higher lifetime medical expenditures despite being associated with reduced life expectancy [2]. Optimal control of diabetes requires a high degree of self-management where individuals have the necessary knowledge, skill, and ability for diabetes self-care [3]. Self-management consists of a complex and dynamic set of processes and is deeply embedded in each patient’s unique situation [4]. Meeting the information needs of each patient is crucial in facilitating self-management.

Patients’ self-learning is an important component of self-management. For example, through self-learning modules, patients can gain more knowledge and be more knowledgeable about practice interventions regarding foot care, which is a widely neglected part of diabetes management [5]. Meanwhile, the Internet has become an important source of self-learning for patients. Many online health communities and forums have emerged as popular platforms for patients to ask questions and share information. However, the quality of health information on the Internet is highly variable [6]. It is crucial to provide expert-vetted information to patients. At the same time, there is an abundant supply of expert-vetted patient education resources that aim to help diabetic patients improve their diabetes self-management [7-9]; however, it is quite challenging for patients without a medical background to find relevant educational materials. A system that can automatically recommend such resources to patients based on their questions in an online forum would be one way to provide relevant expert-vetted education materials.

Retrieving relevant education materials for given questions can be regarded as an information retrieval task. Information retrieval refers to the task of retrieving information of any type from a collection of documents related to search queries. One classic information retrieval approach is based on keyword matching (ie, Boolean model) [10], where documents are represented as a set of terms and queries are represented as Boolean expressions. Another popular information retrieval approach is the ranking model. Unlike the Boolean model where terms are equally weighted, the ranking model ranks the result list in terms of relevance of documents with respect to an information need expressed in the query [10]. Ranking is usually to compute numeric scores of query/document pairs where numerous scoring algorithms have been used. For example, the vector space model (VSM) computes the similarity between a query vector and a document vector, where terms can be weighted using a term frequency-inverse document frequency (TF-IDF) model [11,12]. One common idea of information seeking is to come up with good queries by thinking of words that would likely appear in a relevant document. The language models directly model such ideas where a document is a good match to a query if the document is likely to generate such a query. For a query, the probabilistic language model approach computes a probabilistic language model and ranks documents based on the probability of the model generating the query. Semantic searching intends to improve searches by understanding the semantics in queries and document collections. Concept mapping is popularly used in semantic searches where keywords are mapped to concepts captured in terminological resources. In general English, WordNet is a popular terminology resource where terms are grouped into sets of synonyms according to their meanings and organized into hierarchies based on their semantic relations [13].

Recently, topic modeling, which discovers abstract topics in document collections, has become a frequently used technique in text mining. The most common topic modeling approach is Latent Dirichlet Allocation (LDA), which allows documents to have a mixture of topics. For example, Wang and Blei [14] used topic modeling to generate an interpretable latent structure for users and items, which can provide recommendations about both existing and newly published scientific articles. In information retrieval, topic modeling can be effective in enabling the incorporation of hidden semantics [15].

In the clinical domain, there are many information retrieval applications [16], including clinical decision support. For example, InfoRetriever was designed for family medicine providers to practice evidence-based medicine [17]. Information retrieval technology is also popularly used in patient education applications, such as the PERSIVAL system, which is based on individual patient records and provides personalized access to a distributed patient care digital library by retrieving and summarizing relevant education materials.

Here, we propose a system that leverages the latest information retrieval techniques, which recommends patient education materials for questions asked by patients online. The system aims to provide expert-vetted, patient-faced information to patients. A similar system has been proposed by Kandula et al [18] where, instead of patients questions, their system recommended relevant education materials based on medical records. In this study, we investigated the use of state-of-the-art information retrieval approaches to recommend diabetes education materials for questions available in an online diabetes forum.

An overview of our workflow of this study is presented in Figure 1. We designed a recommendation system using three retrieval models, including a topic modeling-based model, a semantic group-based model, and a VSM. To evaluate the performance of each model in the system, we assembled a gold standard dataset created manually for a randomly sampled subset of questions.

In summary, we investigated the use of the state-of-the-art information retrieval approaches to recommend diabetes education materials for questions available in an online forum for diabetes by leveraging a corpus assembled from diabetes education materials and a corpus assembled from an online forum. Our study shows that the language used in patient education materials is different from the language used in questions from an online forum. A topic modeling-based model has the potential to accurately recommend patient education material to a given question. Both topic modeling-based and semantic group-based models outperform the baseline VSM model. Network analysis illustrates that the network formed by topic modeling and the network formed by semantic groups are different and the combination of them may yield a better strategy.

Literature has shown that the language used by patients is different from the one used by clinicians [27]. Our study demonstrated that there is a language difference between patient education materials and questions in an online forum even though the target audiences of patient education materials are the patients. Patient educational materials are often produced internally by hospital staff without sufficient consideration of the patients intended to use them [28]. In our study, patient education materials tend to cover clinical and patient life topics, whereas patients tend to ask about disease-specific technology and treatment from the top words in Table 2. In addition, the semantic group of questions from diabetic patients corpus included mainly chemical drugs, physiology, devices, and gene aspects more than patient educational materials corpus, and these semantic groups also related to complication, treatment, and technology categories. There was consistency between the primary category distribution of questions from diabetic patients and their semantic groups. Therefore, analyzing online forums can identify information needs of patients and provide an opportunity to create patient-centric education materials.

The study demonstrated that topic modeling can mitigate the vocabulary difference between two corpora and achieve the best performance in recommending education materials to questions. In Figure 7, we found that the topic modeling-based model outperformed the other two models. Through topic modeling, topics and their probability distribution can be calculated for analyzing document similarity, which has been explored for document classification and personalized recommendation. For example, the iDoctor used LDA topic modeling for personalized and professionalized medical recommendation based on data available at crowd-sourced review websites [29] and Kandula et al’s [18] study also showed that the LDA topic modeling can better recommend patient education material to diabetic patients based on clinical notes. Our network analysis demonstrates that the topic modeling-based and semantic group-based models form two independent networks, which may imply that combining the two automated models has the potential to improve the recommendation.

Here, we only studied one disease and used our institutional patient education materials. More research is needed to see if our findings can be generalized. One direction for future work is to extend our study to other disease areas, other patient education material resources, and online forums.