This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
About 30% of people over 65 are subject to at least one unintentional fall a year. Fall prevention protocols and interventions can decrease the number of falls. To be effective, a prevention strategy requires a prior step to evaluate the fall risk of the subjects. Despite extensive research, existing assessment tools for fall risk have been insufficient for predicting falls.
The goal of this study is to present a novel web-based fall-risk assessment tool (FRAT-up) and to evaluate its accuracy in predicting falls, within a context of community-dwelling persons aged 65 and up.
FRAT-up is based on the assumption that a subject’s fall risk is given by the contribution of their exposure to each of the known fall-risk factors. Many scientific studies have investigated the relationship between falls and risk factors. The majority of these studies adopted statistical approaches, usually providing quantitative information such as odds ratios. FRAT-up exploits these numerical results to compute how each single factor contributes to the overall fall risk. FRAT-up is based on a formal ontology that enlists a number of known risk factors, together with quantitative findings in terms of odds ratios. From such information, an automatic algorithm generates a rule-based probabilistic logic program, that is, a set of rules for each risk factor. The rule-based program takes the health profile of the subject (in terms of exposure to the risk factors) and computes the fall risk. A Web-based interface allows users to input health profiles and to visualize the risk assessment for the given subject. FRAT-up has been evaluated on the InCHIANTI Study dataset, a representative population-based study of older persons living in the Chianti area (Tuscany, Italy). We compared reported falls with predicted ones and computed performance indicators.
The obtained area under curve of the receiver operating characteristic was 0.642 (95% CI 0.614-0.669), while the Brier score was 0.174. The Hosmer-Lemeshow test indicated statistical significance of miscalibration.
FRAT-up is a web-based tool for evaluating the fall risk of people aged 65 or up living in the community. Validation results of fall risks computed by FRAT-up show that its performance is comparable to externally validated state-of-the-art tools. A prototype is freely available through a web-based interface.
ClinicalTrials.gov NCT01331512 (The InChianti Follow-Up Study); http://clinicaltrials.gov/show/NCT01331512 (Archived by WebCite at http://www.webcitation.org/6UDrrRuaR).
About 30% of community-dwelling people aged 65 or more experience at least one unintentional fall a year [
Many preventive strategies have been proposed, and some of them have been shown to be effective [
Reviews of fall-risk assessment tools and their accuracy are available in the literature [
An impressive number of scientific publications have identified statistical correlation between the exposure to risk factors and the risk of falling, in terms of odds ratios. Moreover, several reviews and meta-analyses are available, thus providing a solid scientific base about fall-risk factors [
In our Fall-Risk Assessment Tool (FRAT-up), we faced the issue of representing the information available from scientific literature in a structured manner. In computer science, an ontology is a formal, explicit specification of a shared conceptualization [
The goal of FRAT-up is to provide a tool for the fall-risk assessment of subjects aged 65 or up and living in a community dwelling. The tool is mainly intended for two different health professional roles: (1) general practitioners (GPs) delivering primary care provisions, with no specific knowledge about falls, who need an assessment tool for evaluating subjects’ fall risk and possible early interventions, and (2) professionals involved in fall prevention and treatment, who need a tool for constantly assessing the fall risk in a reliable and quantitative manner. We identified the following requirements: (1) the assessment tool should identify people at high risk of falling, (2) the tool should exploit existing knowledge about fall-risk factors, (3) the tool should be sufficiently flexible to allow the use of different clinical tests for the estimate of each risk factor, and (4) the assessment tool should be robust with respect to the unavailability of complete information about the subject.
FRAT-up has been developed within the FARSEEING Project [
The FRAT-up fundamental hypothesis is to consider the fall risk as being directly related to the subject’s exposure to known risk factors. Thus, the starting point is the scientific literature that lists risk factors, together with quantitative information on their association with falls (usually in terms of odds ratios). However, such literature does not provide any structured definition of risk factors and related information. Hence, the first building block of the FRAT-up approach consists of a formal ontology listing risk factors and related data.
Once quantitative information is available through the FRAT-up risk factor ontology, we need to decide how (the exposure to) each risk factor contributes to the overall risk. Our approach is based on probabilities, while epidemiological studies on risk factors usually provide information in terms of odds ratios. Hence, the second building block is a mathematical transformation from odds ratios to probabilities under a few assumptions, as explained further in this section.
The third building block of FRAT-up is a Logic Programming with Annotated Disjunctions (LPAD) program that allows representation of the contribution of each risk factor in terms of probabilistic rules and probabilistic reasoning.
In FRAT-up, a fall-risk factor ontology has been defined, taking into account several domains. For example, the classification of risk factors by reversibility (surely reversible, subject specific reversible, or irreversible) and setting (community dwelling, acute care, etc) is shown in
Classification of risk factors by kind, reversibility, and setting. While the InCHIANTI dataset is about community dwellings, the ontology covers other settings, too.
Within the ontology, risk factors are distinguished as
The set of risk factors we include in the ontology comes from a well-established meta-analysis on known risk factors for falls in community-dwelling older people by Deandrea et al [
The ontology also includes the odds ratio for each risk factor, taken from Deandrea et al [
Additional data contained in the ontology are the risk factors’ prevalence and procedures to map estimators into factors. Complete information, including sources for quantitative data, is reported in
Risk factor names and types.
Name | Type |
age | scalar |
cognition impairment | dichotomous |
depression | dichotomous |
diabetes | dichotomous |
comorbidity | synergy |
dizziness and vertigo | dichotomous |
fear of falling | dichotomous |
female sex | dichotomous |
gait problems | dichotomous |
hearing impairment | dichotomous |
history of falls | dichotomous |
history of stroke | dichotomous |
instrumental disability | dichotomous |
living alone | dichotomous |
number of medications | scalar |
pain | dichotomous |
parkinson | dichotomous |
physical activity limitation | dichotomous |
physical disability | dichotomous |
poor self-perceived health status | dichotomous |
rheumatic disease | dichotomous |
urinary incontinence | dichotomous |
use of antiepileptics | dichotomous |
use of antihypertensives | dichotomous |
use of sedatives | dichotomous |
vision impairment | dichotomous |
walking aid use | dichotomous |
The FRAT-up risk-assessment algorithm is based on probability contributions from single risk factors. In the following, we show how we extract probabilities from odds ratios by means of a few mathematical steps.
Initially, we assume that each risk factor is dichotomous; we explain this further in the section on how to generalize to cases with scalar and synergy risk factors. Let
Equation 1:
We call
The conditional probability of
There is no single way of translating odds ratios to probabilities, since an exact function would require more information than what is conveyed by the odds ratios alone, so some assumptions are needed. We present a possible set of assumptions that leads to a univocal way of computing exposure contributions.
Definition of fall event.
Probability to fall from risk factor specific probabilities.
We assume that
Given the assumptions in Equation 1 and
Probability of factor specific fall event given exposure.
Probability to fall given exposures and contributions.
Odds ratio definition.
Approximated odds ratio.
Probability to fall with exposure to exactly 1 risk factor.
Contribution to fall probability from exposure to a single risk factor given odds ratio.
Probability to fall from risk factor odds ratios.
We assume to know
This model requires that we know for every risk factor if it is present or not. In the following section, we present the way FRAT-up deals with missing values.
For a general reference on how to get relative risk from odds ratio and the incidence of the outcome of interest in the unexposed group, see [
LPADs are logic programs [
where
Roughly speaking, for each clause containing a disjunction in its head, different instances are generated, each containing the clause with exactly one disjunct. The probability of a query would be given by the sum of all the probabilities of the instances whose models contain it.
We adopt the syntax of the
fall(X) : c0.
fall(X) : c1 :- e1(X).
fall(X) : c2 :- e2(X).
...
Where c0≡
The assessment tool should provide reliable information even when part of the subject’s data is missing. Missing values may arise when a test has not been (or cannot be) performed or the involved clinical professional does not consider its outcome decisive and reliable. In these cases, we have used the prevalence of the risk factors extending Code 1 as follows:
fall(X) : c0.
e1(X) : p1 :- u1(X).
fall(X) : c1 :- e1(X).
e2(X) : p2 :- u2(X).
fall(X) : c2 :- e2(X).
...
where u1(X), u2(X)… is true when the existence of the factor 1, 2… for subject
A scalar factor, with exposure levels from 0 (no exposure) to
Positive synergies (eg, comorbidities) between risk factors are well documented in the scientific literature. Since this would violate the causal independence assumption made before, we adjusted the model, following the Deandrea meta-analysis [
A synergy factor, representing the potential synergies between
The methodology that leads from risk factor odds ratio to LPAD rules is fully automatized. A working prototype has been produced and tested in the Java programming language (version 1.7); it may read risk factor odds ratios from an ontology or another source and outputs an LPAD program directly usable for risk assessment.
Synthetically (see
Steps in generating the LPAD rules.
FRAT-up discriminative performance and calibration have been tested on the InCHIANTI dataset (NCT01331512), where 1453 persons have been initially enrolled (1150 subjects aged 65 or more) and have undergone four consecutive visits globally covering a 9-year follow-up. It is a population-based epidemiologic study conducted in the Chianti region of Italy in two sites: Greve in Chianti (Area 1; 11,709 inhabitants; >65 years: 19.3%) and Bagno a Ripoli (Village of Antella, Area 2, 4704 inhabitants; >65 years: 20.3%). This study investigates age-related decline in mobility [
The InCHIANTI study started in September 1998 with the baseline assessment (first wave), which was completed in March 2000. Every 3 years, a follow-up assessment was performed. So, 3-year and 6-year follow-up assessments were performed respectively in 2001-2003 and 2004-2006 (second and third wave). A 9-year follow-up was then performed in 2007-2009 (fourth wave). The fifth wave is now ongoing.
At each wave, subjects were asked about the occurrence of any fall in the previous 12 months. In addition, clinical evaluation of the subjects was performed to collect information on fall-risk factors (other clinical variables were also collected, which are not of interest for this work [
Our study used the information about risk factors from the first three waves, considering only subjects aged 65 or up. By doing so, we obtained 2319 samples from 977 subjects (every subject can have up to three samples).
At each wave, the risk factors of each subject were used prospectively to calculate their risk of falling at the subsequent wave (eg, the risk factors from the clinical evaluation at baseline were used to calculate the future risk of falling, which was compared with the recorded information on the occurrence of any falls in the 12 months before follow-up 1, and so on).
The estimators present in the InCHIANTI dataset and the algorithms to derive the risk factors from them are listed in
The discriminative ability and calibration of FRAT-up were validated by means of receiver operating characteristic (ROC) curve, area under the ROC curve (AUC), Brier score, and the Hosmer-Lemeshow test [
The ROC curve can be seen in
ROC curve obtained on the InCHIANTI dataset.
Calibration plot; sample (N=2319) used for validation where divided in 10 deciles, according to their predicted risk. For each decile, the mean predicted risk and the observed proportion of positive cases (proportion of fallers) are shown on the X and Y axes, respectively. Bars indicate 95% confidence intervals.
The ability to discriminate between subjects who fall and subjects who do not fall, as operationalized by the AUC (0.642), compares favorably with other commonly used screening tools: a recent meta-analysis has estimated that the AUC of the TUG is between 0.54 and 0.59 [
FRAT-up overestimates the fall risk. Since this overestimation, as shown in
FRAT-up does exploit existing knowledge as it was built only from information derived from the literature, which was systematized in a meta-analysis. By doing so, it avoids overfitting and overoptimism, problems well known to affect predictive models [
Although the validation on the InCHIANTI dataset is based on a specific set of estimators, the architecture allows for the use of different estimators. The results of the validation have been obtained from the InCHIANTI dataset, where the percentage of missing values ranges from 0% on some variables (eg, sex and age) to 17% on vision impairment.
The interactive prototype of the FRAT-up algorithm is freely available online [
Screenshot of the Web-based interface.
FRAT-up is based on the simplifying assumption that the risk factors contribute independently to the probability of falling. Following Deandrea et al [
Our method showed robustness with respect to the missing values present in the InCHIANTI dataset. However, the extent to which the presence of missing values impacts the performance should be further investigated. In the future, FRAT-up will be tested on other datasets of different populations, possibly with different estimators for the risk factors, and compared with alternative risk assessment tools. Additionally, considering confidence intervals of the odds ratios could allow us in the future to assess the uncertainty associated with the fall-risk estimation.
Risk factors not reported in the meta-review by Deandrea et al [
Within the framework of a fall-prevention strategy, information would be useful on the indication of the modifiable risk factors of a specific subject and their quantitative impact on their risk. Practically, we foresee integration of the tool within electronic medical records, tools of general practitioners, as well as its adoption in public health bodies for population-wide evaluation.
The versatility of the presented solution will allow combining clinical information (that was used in this study) with other sources of data such as ambient sensor information or wearable sensors recording unsupervised long-term physical activity and/or quantitatively evaluating supervised or unsupervised physical performance by instrumented motor assessment [
An interesting extension of FRAT-up would be to implement it as an app for “smart” devices such as smartphones. The tool might be fed with rich sensor-based information and could be extended to provide “real-time” risk evaluation based on the subject’s current physical activity. Although from the technical viewpoint, such an extension would be easy and straightforward, using smartphone sensor data (in the fall-risk estimation) is still an open research issue.
Finally, since FRAT-up is based on a general methodology, it may be extended/applied in different ways, such as estimating fall risks in different settings (eg, acute care or nursing homes). Another extension would be to estimate outcomes other than falling, such as stroke risk, and more generally, estimate any risks directly related to the presence/absence of risk factors.
area under the ROC curve
Fall-Risk Assessment Tool
Logic Programming with Annotated Disjunctions
odds ratio
Performance Oriented Mobility Assessment
receiver operating characteristic
Timed Up and Go Test
The tool has been designed and implemented by LC, PP, LP, FC, and LC (Prof. Lorenzo Chiari) who coordinates the research group in the University of Bologna. SB and CB contributed to general discussion of the idea, the identification of the requirements, as well as the interpretation of the data and results. SB, as InCHIANTI principal investigator, has contributed the dataset.
The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/2007-2013) under grant agreement FARSEEING No. 288940.
The authors would like to thank Marco Colpo for his valuable help with the InCHIANTI dataset.
Clemens Becker has received consultation fees by Eli Lilly and Robert Bosch GmbH.
FRAT-up estimators, factors, and procedures to produce factor values starting from estimator values.