JMIR Publications

Journal of Medical Internet Research


Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 09.06.16 in Vol 18, No 6 (2016): Jun

This paper is in the following e-collection/theme issue:

    Original Paper

    Barriers and Facilitation Measures Related to People With Mental Disorders When Using the Web: A Systematic Review

    1Department of Medical Informatics, Biometry and Epidemiology – IBE, Chair for Public Health and Health Services Research, Research Unit for Biopsychosocial Health, Ludwig-Maximilians-Universität München, Munich, Germany

    2Blindness and Deafness Prevention, Disability and Rehabilitation (BDD), World Health Organization, Geneva, Switzerland

    Corresponding Author:

    Renaldo Bernard, MSc

    Department of Medical Informatics, Biometry and Epidemiology – IBE

    Chair for Public Health and Health Services Research, Research Unit for Biopsychosocial Health

    Ludwig-Maximilians-Universität München

    Marchioninistraße 17

    Munich, 81377


    Phone: 49 89 2180 78229

    Fax:49 89 2180 78230



    Background: Mental disorders (MDs) affect almost 1 in 4 adults at some point during their lifetime, and coupled with substance use disorders are the fifth leading cause of disability adjusted life years worldwide. People with these disorders often use the Web as an informational resource, platform for convenient self-directed treatment, and a means for many other kinds of support. However, some features of the Web can potentially erect barriers for this group that limit their access to these benefits, and there is a lack of research looking into this eventuality. Therefore, it is important to identify gaps in knowledge about “what” barriers exist and “how” they could be addressed so that this knowledge can inform Web professionals who aim to ensure the Web is inclusive to this population.

    Objective: The objective of this study was to provide an overview of existing evidence regarding the barriers people with mental disorders experience when using the Web and the facilitation measures used to address such barriers.

    Methods: This study involved a systematic review of studies that have considered the difficulties people with mental disorders experience when using digital technologies. Digital technologies were included because knowledge about any barriers here would likely be also applicable to the Web. A synthesis was performed by categorizing data according to the 4 foundational principles of Web accessibility as proposed by the World Wide Web Consortium, which forms the necessary basis for anyone to gain adequate access to the Web. Facilitation measures recommended by studies were later summarized into a set of minimal recommendations.

    Results: A total of 16 publications were included in this review, comprising 13 studies and 3 international guidelines. Findings suggest that people with mental disorders experience barriers that limit how they perceive, understand, and operate websites. Identified facilitation measures target these barriers in addition to ensuring that Web content can be reliably interpreted by a wide range of user applications.

    Conclusions: People with mental disorders encounter barriers on the Web, and attempts have been made to remove or reduce these barriers. As forewarned by experts in the area, only a few studies investigating this issue were found. More rigorous research is needed to be exhaustive and to have a larger impact on improving the Web for people with mental disorders.

    J Med Internet Res 2016;18(6):e157




    Mental disorders (MDs) are a significant public health issue owing to their high impact on people with these disorders, in terms of restrictions placed on their participation in all areas of life, family life and the wider society. Mental disorders affect almost 1 in 4 adults at some point during their lifetime [1] and coupled with substance use disorders are the fifth leading cause of disability adjusted life years worldwide [2]. People with mental disorders (PwMD) often experience similar impairments, activity limitations, and restricted participation in life events, even with the diversity in symptoms and etiology associated with these conditions [3]. Family members often provide care, which sometimes puts a strain on familial relationships, reduces opportunities for leisure, and negatively impacts finances due to time spent providing care instead of working [4]. The associated reduction in productivity from both affected persons and their family can translate to a decrease in contributions to the local economy [5]. In addition, having a large segment of the population subscribing to treatment and support services incurs considerable costs [5].

    The Web is often used as a source of support for PwMD and shows great promise for the reduction of the burden of mental disorders. Mental health–related Web browsing, primarily for information seeking, is common among PwMD [6,7]. Web-based mental health communities are known to supplement traditional mental health services [8] and act as an important factor in encouraging PwMD to seek professional help [9]. A recent meta-analysis has indicated that guided Web-based cognitive behavioral therapy may be as effective as the face-to-face equivalent for social anxiety disorder, panic disorder, spider phobia, and depressive symptoms [10]. Many other Web-based treatment and intervention options are increasingly being explored for other mental disorders (eg, posttraumatic stress disorder, eating disorders) [11] and populations including children (eg, Project CATCH-IT, MoodGYM) [12,13] with positive results.

    There are also features of the Web environment that could potentially limit how much PwMD who experience cognitive deficits can benefit from the Web. Using the Web is considered a very cognitively demanding activity requiring not only good knowledge and understanding of Web features (eg, search engines) but also the ability to quickly analyze, synthesize, evaluate, and apply presented information while avoiding inconsequential details (eg, adverts and untrustworthy information) that are abundant on the Web [14]. Several cognitive domains, including executive functioning, attention, and memory, are commonly impaired in PwMD [15]. These impairments may be linked to difficulties using the Web such as when performing Web searches, task switching, retaining and recalling information, and ignoring distractions (eg, adverts) to focus attention. Moreover, the Web has also been found to be relatively absent of nonverbal and social context cues (eg, gestures, facial expression) compared with off-line [16,17]. These cues are important for guiding behavior when interacting with others, and their absence could make social interaction difficult. Although Web users are normally able to skillfully compensate and overcome these “deficiencies” [18], sometimes even by capitalizing on them [19], it could be challenging for PwMD who experience cognitive deficits to do the same.

    People with mental disorders have received little attention from Web accessibility research despite increased inquiries into the difficulty others with cognitive impairment face on the Web. This research gap was highlighted over a decade ago [20,21], and more recently, there has been some indication that the gap still exists [22]. Current recommendations also prescribe the same treatment to address accessibility for PwMD and a myriad of other diverse conditions that fall under the broad heading of conditions associated with cognitive limitations (eg, intellectual disabilities, multiple sclerosis) [23].

    A comprehensive review of literature concerned with the barriers PwMD encounter when using the Web and/or the facilitation measures developed to address these barriers is needed to ensure that the Web is inclusive to this population. Available knowledge will support Web professionals in making well-informed choices about the removal of barriers affecting PwMD. If this is not possible, it may instead provide facilitation measures to accommodate this group. As a result, Web-based resources could be systematically evaluated for compliance with measures that are known to remove barriers or provide facilitation for PwMD. Identified gaps in knowledge about “what” barriers exist and “how” they could be addressed—based on a comparison and integration of what is known on the topic—is likely to encourage further research into these highlighted areas as well.

    The objective of this systematic review was to provide an overview of the existing evidence regarding the barriers PwMD experience when using the Web and facilitation measures used to address such barriers. Specific aims are to detail barriers and facilitation measures, how they were identified or developed, and related trends (ie, the extent of coverage for specific mental disorders or digital technologies, study designs used, publication recency, and research region).


    A systematic review was carried out to identify barriers PwMD encounter when using the Web and the recommended facilitation measures to remove or reduce these barriers.

    Search Strategy

    Search terms were broadly based on concepts relating to Web accessibility, mental disorders, and also digital technologies (see Multimedia Appendix 1). Digital technologies were included because knowledge about any barriers here would likely be also applicable to the Web. This was also a proactive measure to avoid having the review suffer from the paucity of research in the area as revealed by preliminary searches. Databases searched include MEDLINE, PsycARTICLES, CINAHL, Library, Information Science &Technology Abstracts, Computers & Applied Sciences Complete, Inspec, Web of Science Core Collection. Reference lists of included publications were also searched to avoid missing relevant publications not identified during the search of databases. There were no publication date restrictions to ensure that the review included as many studies as possible. There was also no restriction to empirical studies. Other types of publications such as international standards and guidelines are usually widely adopted and highly regarded and can be especially helpful when there is insufficient empirical evidence on a particular issue.

    Eligibility Criteria

    Included publications describe the difficulties PwMD encounter when using any digital technology or provide guidance on how to improve the accessibility of any digital technology for this group. All mental disorders were considered regardless of a formal diagnosis or not. All digital technologies such as computers, video games, mobile devices, and websites were also considered. Journal articles, gray literature, international and national standards and guidelines, reports, and conference proceedings written in the English language were considered for inclusion. Publications in the form of commentaries, letters to the editors, and editorials were excluded.

    Eligibility Assessment

    One reviewer (RB) screened all abstracts, and another (DH) screened 84% (1692/2013) selected at random. Both screenings were conducted independently to reduce the chance of reviewer bias and increase reliability [24]. Inconsistences in ratings—eligible, ambiguous, or excluded—were later discussed and resolved by consensus. One reviewer (RB) then appraised the full texts of abstracts rated as eligible.

    Data Extraction and Synthesis of Results

    Information extracted from studies was study characteristics—publication year, country, study design, methods and participants or target population (eg, mental disorders, age, gender, and education); barriers and facilitation measures—process used for the development of the facilitation measure and related mental disorders; and definitions of accessibility and disability. Data extracted from other documents—international standards and guidelines—did not include information about study designs and participants (eg, age and gender).

    The International Classification of Functioning, Disability and Health was used to define barriers and facilitation measures [25]. Factors (eg, small font, complicated language) that through their absence or presence limit functioning were identified as barriers. Conversely, factors (eg, legible font, simple language) that instead improve functioning through their absence or presence were identified as facilitation measures.

    Synthesis was performed by categorizing all findings and later summarizing facilitation measures recommended by studies. Data were first categorized according to the 4 foundational principles of Web accessibility: operable—user interface components and navigation must be easy and safe to use; understandable—information and the operation of a user interface must be easily interpreted accurately; perceivable—information and user interface components must be presentable to users in ways they can be sufficiently aware on these components; robust—content must be flexible enough that a wide range of user agents, including technologies that enable persons with disabilities to perform tasks that would be otherwise challenging (ie, assistive technologies), can interpret it reliably [26]. These 4 foundational principles were proposed by the World Wide Web Consortium (W3C) and form the necessary basis for anyone to gain adequate access to the Web. Results from studies came from 2 sources—expert opinion or empirical research—and they are labeled to denote these different sources. Facilitation measures from guidelines are also labeled for easy identification. Facilitation measures recommended by studies were later summarized into a set of minimal recommendations after the categorization of findings. Those from guidelines have already been aptly summarized elsewhere [27-29].


    A total of 16 publications were included in this review, comprising 13 studies reporting on the usability of various technologies [30-40] and Internet or computer use among PwMD [41,42] and 3 international guidelines [23,43,44], which were all developed by the W3C. These guidelines have been adopted by many governments and are also widely considered as the international standard for Web accessibility. A flow chart of the review process is presented in Figure 1.

    Figure 1. Flowchart of the review identification and selection process.
    View this figure

    Study and Guideline Characteristics

    Nine of the included studies [30-33,35,37-39,42] originated in the United States, 2 studies [34,40] in the United Kingdom, one [41] in Austria, and another [36] in Sweden as summarized in Table 1. Over 62% (10/16) of the included publications [32-34,37,38,42] were published within the last 5 years, and the earliest [35] was published in 1998.

    All 3 included guidelines were published by the W3C based in the United States. However, the guidelines are the result of collaboration among international experts. Two of the three included guidelines (User Agent Accessibility Guidelines 1.0 and Authoring Tool Accessibility Guidelines 1.0) were published over 12 years ago, and the third (Web Content Accessibility Guidelines 2.0) was published in 2008.

    Table 1. Characteristics of included publications.
    View this table

    Design and Methods

    Nine of the included studies investigated the usability of Web-based resources [30-34,37-40] and multimedia tools [35]. One study focused on Internet use [41], one on the use of digital technologies [36], one on the development of a mobile phone system [30], and another on computer use [42] among PwMD. Eight of the included studies used qualitative methods [31,33-36,40-42], 3 [37-39] adopted a quantitative approach, and 2 [30,32] used mixed methods. Seven studies used usability testing [30-33,37-39], 5 used interviews [32,33,35,41,42], 5 used focus groups [32,33,35,41,42], and single studies used observations [42], survey [30], and user testing.

    The 3 included guidelines [23,43,44] were primarily developed based on contributions over several years from experts involved in international working groups on varying aspects of Web accessibility [45].

    Sample Characteristics

    Sample sizes for included studies ranged from 5 to >100 (mean 48). Overall, 11 studies [30-33,36-42] reported the age of participants, which ranged from 18 to at least 75 years. Three studies [37,39,40] used the Diagnostic and Statistical Manual of Mental Disorders, 4th edition (DSM IV), 1 [41] used the International Classification of Diseases, 10th revision (ICD-10), and the remaining studies did not mention the use of a classification of mental disorders. Samples including people with schizophrenia (69%) [30,32,35-39,41,42] were most common among the 13 included studies, followed by samples where participants were affected by depression (62%) [31,32,34-36,38,39,42], schizoaffective disorder (46%) [30,37-39,41,42], anxiety disorders (38%) [31,34,36,39,42], and bipolar disorder (38%) [32,36,38,40,42]. Single studies reported that participants had severe mental illness (SMI) (eg, schizophrenia, schizoaffective disorder, bipolar disorder, and major depression) [33], mental disorders [36], psychotic disorders [39], and substance use disorder [39] but did not state any particular mental disorder. Most studies considered more than 1 mental disorder except [33], which focused on schizophrenia and [40] on bipolar disorder.

    All 3 included guidelines were developed to give guidance on how to remove and reduce barriers experienced by people with a range of disabilities including auditory, cognitive, and neurological, physical, speech, and visual disabilities. Extracted guidelines were identified by the authors of the guidelines as being relevant to cognitive and neurological disorders [46]. These disorders include attention-deficit hyperactivity disorder, autism spectrum disorder, intellectual disabilities, learning disabilities, memory impairments, multiple sclerosis, perceptual disabilities, seizure disorders, and mental disorders. No particular mental disorder was specified.

    Digital Technology

    As summarized in Table 1, websites were the most studied digital technology, followed by single studies each investigating either computers [42] or multimedia applications [35]. Only three studies [42,30,36] investigated more than 1 technology, viz computers and websites, mobile phone and websites, and several digital technologies, respectively. The 3 included guidelines target websites (ie, Web Content Accessibility Guidelines 2.0), user agents (ie, any software that retrieves, renders, and facilitates end user interaction with Web content; User Agent Accessibility Guidelines 1.0) and Web authoring tools (Authoring Tool Accessibility Guidelines 1.0).

    Scope of Barriers and Facilitation Measures Related to Digital Technology Usage by PwMD

    Included studies revealed 42 barriers and 59 facilitation measures. These are summarized in Tables 2 and 5. Four studies [31,32,35,37] did not mention any barriers and 2 [36,41] no facilitation measures. Four studies [30,33,34,38] recommended facilitation measures to address barriers, and only 25 of these pairings were identified.

    The 3 included guidelines recommended 30 facilitation measures and did not explicitly report any barriers. However, the W3C has published several barriers on its website that people with cognitive and neurological disabilities including mental health disabilities face when using the Web. Examples of these barriers include complex navigation mechanisms, page layouts that are difficult to understand and use, and moving, blinking, or flickering content, and background audio that cannot be turned off [46].

    Of the 131 identified barriers and facilitation measures, 63 were relevant to depression (48%), 54 to schizophrenia (41%), 48 to anxiety disorders (37%), 39 to bipolar disorder (30%), 37 to mental disorders (28%), 35 to schizoaffective disorder (27%), 11 to SMI (8%), and 3 to substance abuse and psychotic disorders equally (2%). Most of the 42 identified barriers were relevant to people with depression (64%), followed by those with an anxiety disorder (62%), schizophrenia (50%), bipolar disorder (40%), schizoaffective disorder (31%), mental disorders (17%), SMI (12%), and substance use disorder and other psychotic disorders equally (2%). Identified facilitation measures (n=89) mostly targeted people with depression (40%), schizophrenia (37%), mental disorders (34%), and anxiety, bipolar disorder and schizoaffective disorder equally (25%). SMI (7%) and substance use disorder and other psychotic disorders equally (2%) accounted for a small portion of the identified facilitation measures.

    All barriers identified were revealed by research findings. Identified facilitation measures were proposed directly from research findings (n=31) [30,33,37-40], by international working groups of experts in the area of accessibility (n=30) [23,43,44] and expert opinion of researchers conducting studies (n=28) [31,32,34,35,42].

    Table 2. Barriers and facilitation measures categorized by the ‘perceivable’ foundational principle of Web accessibility.
    View this table
    Table 3. Barriers and facilitation measures categorized by the ‘understandable’ foundational principle of Web accessibility.
    View this table
    Table 4. Barriers and facilitation measures categorized by the ‘operable’ foundational principle of Web accessibility.
    View this table
    Table 5. Barriers and facilitation measures categorized by the ‘robust’ foundational principle of Web accessibility.
    View this table

    Synthesis of Results

    Categorization of Results by Foundational Principles of Web Accessibility

    The identified barriers and facilitation measures were categorized according to the foundational principles of Web accessibility that was proposed by the W3C and are summarized in Tables 2 and 5 —additional tables organized by categories can be requested. Each identified barrier and facilitation measure was sorted into multiple categories if applicable. The barriers resulted in 3 categories as none were assigned to the robust category: operable (n=26); understandable (n=16); perceivable (n=4). The facilitation measures resulted into 4 categories: operable (n=35); understandable (n=49); perceivable (n=26); and robust (n=8).

    Some studies paired a barrier with a corresponding facilitation measure, and other studies did not. The former was categorized based on the barrier, and the latter was categorized based on the specific barrier or facilitation measure that was not paired. Linking barriers that were not paired with a corresponding facilitation measure was beyond the scope of this review. A synthesis of Tables 2 and 5 is presented in the following section.


    Identified barriers and facilitation measures (n=61) in this category gave most coverage to depression (49%), followed by bipolar disorder (43%), anxiety (41%), schizophrenia (39%), mental disorders (34%), schizoaffective disorder (20%), SMI (16%), and substance use disorder and other psychotic disorders equally (7%).

    Barriers reported by included studies are primarily related to poorly designed navigational elements (eg, content filters), difficulties with fine motor coordination (eg, clicking small radio buttons, operating computer mouse, scrolling), poorly designed pages with time-limited response forms, too much information, and unoptimized components that contribute to slow webpage loading times.

    Facilitation measures derived from empirical evidence gave guidance on design involving a reduction in the number of clicks needed to select options, an increase in buttons sizes, and websites that feature a shallow hierarchical structure and allows for personalization. Facilitation measures based on the expert opinion of researchers conducting studies suggest that websites should incorporate efficient content filters with intuitive navigation and permit users to browse at their pace.

    Most facilitation measures recommended by the 3 included guidelines were focused on increasing users’ control. This involved providing users with enough time, alternative methods and information presentation styles, and instruction to interact with content. Other measures recommended that authoring tools must be accessible, promote accessible practices, and support the creation of accessible content.


    Most of the 64 identified barriers and facilitation measures in this category addressed depression (61%), schizophrenia (45%), anxiety (41%), mental disorders (34%), schizoaffective disorder (31%), and bipolar disorder (27%). However, SMI (3%) received considerably less coverage, and no barriers and facilitation measures were recorded for substance use disorder and other psychotic disorders in this category.

    Included studies revealed barriers that included the use of complicated and excessive content, distracting and confusing design, and complex and overindulgent website functions (eg, excessive advertising and complicated purchasing processes). Facilitation measures derived from empirical evidence heavily focus on increasing the clarity of website content by ensuring only necessary information is shared and provided at a low reading level with no abbreviations and unfamiliar phrasing. Facilitation measures based on expert opinion focus more on the presentation and organization of website content. For example, they recommend the usage of alternative information formats, explicit labels that use concrete sentences to describe content and instructions, organizing content by importance, and forming meaningful content groups.

    Facilitation measures from the 3 included guidelines recommend ways to help make content readable and understandable by ensuring abbreviations are expanded, reading level is appropriate, and providing explanations for any jargon used among other things. It was also recommended that several features should be incorporated into Web authoring tools: accessibility solutions in the design, mechanisms to correct inaccessible content and those that support accessible authoring practices.


    Most of the 30 identified barriers and facilitation measures in this category targeted people with mental disorders (40%), depression (33%), anxiety and schizophrenia equally (30%), schizoaffective disorder (27%), bipolar disorder (17%) substance use disorder, and other psychotic disorders (3%). No barriers and facilitation measures were recorded for SMI in this category.

    Identified barriers point to difficulties with reading small font, recognizing icons, and locating information. Facilitation measures derived both from empirical evidence and the expert opinion of researchers conducting studies recommend that links and other navigational elements should be easily recognizable, and use of images must be purposeful.

    Facilitation measures recommended by the 3 included guidelines were predominantly focused on providing alternative content options and personal configurations for content. Other measures, all originating from the Authoring Tool Accessibility Guidelines1.0, generally recommend that authoring tools and practices must be accessible and support the creation of accessible content.


    This category only contains facilitation measures from 1 of the 3 included guidelines, and no barriers were identified. All identified facilitation measures target PwMD. Recommended facilitation measures largely promote compatibility between user agents, authoring tools and Web content, and assistive technologies. The suggested methods to do this involve providing ways of checking and correcting inaccessible content within authoring tools and mainly adhering to standard markup, relevant W3C recommendations, and operating environment conventions.

    Summary of Facilitation Measures Recommended by Studies

    Facilitation measures recommended by studies were summarized into a group of 20 from 59 recommendations and are summarized in Table 6. Table 6 does not list or arrange summarized facilitation measures in any particular order. Nine of the summarized facilitation measures were the result of empirical work and 11 from the expert opinion of researchers.

    Table 6. Summary of facilitation measures recommended by studies.
    View this table


    Principal Findings and Comparison With Prior Work

    The 13 studies that could be included in this review support preexisting views [20,21] that there is little research on the barriers PwMD experience when using digital technology and facilitation measures used to address such barriers. Despite being few, included studies and guidelines give valuable insight into what is known and where knowledge gaps lie.

    Barriers People With Mental Disorders Encounter When Using Digital Technologies

    People with mental disorders encounter a wide range of barriers when using the Web that makes it difficult for them to perceive, understand, and operate this tool along with content contained therein. Most barriers result from distracting and confusing design, complicated content and website functions, an overabundance of information, and a high-demand for good fine-motor skills and rapid information processing. Persons affected by other conditions associated with cognitive dysfunction have also been known to experience many of these barriers as indicated by Web design guidelines [47].

    However, included barriers were related to neurocognitive dysfunction—impaired attention, processing and responding to information slowly and problem-solving—and none were associated with sociocognitive deficits—impaired affect regulation and difficulty processing emotional cues. This is possibly due to affective measurements being overlooked by researchers of included studies.

    Barriers were predominantly identified using qualitative research methods and to a lesser extent mixed and quantitative methods. Identified barriers were often not well stated—not including details about the particular user category affected, disability type, hindered activity or task, and how it is hindered—and there was no indication of how restrictive barriers were or how often particular groups of participants encountered them. This can contribute to the development of tentative and inconclusive recommendations that may not be helpful.

    Recommended Facilitation Measures

    Studies recommended facilitation measures that contribute towards ensuring the use of intuitive navigation, correctly functioning features, simple language, explicit, consistent and easy-to-detect website components, organized content, a flat hierarchical content structure, multimedia formats, and easy-to-operate functions. Facilitation measures recommended by included guidelines focused on improvement strategies that ensure websites are sufficiently operable, understandable, perceivable, and robust. Given the overlap in barriers, it was correctly anticipated that identified facilitation measures would also be mostly in agreement with recommendations for other conditions associated with cognitive deficits.

    Facilitation measures were largely developed based on the opinion of researchers conducting studies and consensus among members of international working groups of experts in the area of accessibility. Some researchers [38] disagree with this approach because it does not involve empirical research with people affected by the particular condition when finding ways to meet their needs. However, facilitation measures derived from empirical work were similar to those based on the opinion of researchers conducting studies. Nonetheless, as shown in Results section, more focus was placed in different areas for 2 of the 3 principles under which facilitation measures were categorized. Facilitation measures recommended by included guidelines addressed problem areas, whereas other facilitation measures targeted specific barriers.

    Facilitation measures were seldom linked to barriers. For example, no facilitation measures recommended by included guidelines had barriers associated with them. Consequently, many facilitation measures were recommended without validation and in a way that makes future validation difficult. This poses a challenge when selecting facilitation measures to address a particular barrier and attempting to increase the effectiveness of a particular facilitation measure.

    Coverage of Mental Disorders

    As schizophrenia is associated with more severe cognitive deficits than other conditions [48,49] and many participants were also recruited from institutional settings, it was foreseeable that most studies in the area would involve people affected by these 2 conditions. Good cognitive ability is very important when using the Web [38], and the deficits associated with these conditions can put this population at high risk of encountering barriers when using digital technologies such as the Web. Although people affected by depression, anxiety, and bipolar disorder are believed to experience less severe cognitive deficits than those affected by schizophrenia [15], these conditions received similar coverage by included studies. This is possibly due to these conditions being common and the debilitating impact they could still have on the lives of people affected.

    Coverage of Digital Technologies

    The overwhelming focus on websites out of many digital technologies demonstrates the heavy importance placed on the Web for its usefulness for PwMD. It also acknowledges that there is a need to further optimize Web-based resources. A single 1998 study [35] did not focus on websites but on a multimedia application. This is not surprising as the Web was not widely adopted during that time, but such applications were common.

    Types and Suitability of Study Designs

    Qualitative methods were suitably adopted for most included studies because they sought to describe and explore technology usage and design for PwMD. The 3 other studies [37-39] investigated the effectiveness of design elements for PwMD and appropriately used quantitative usability testing methods.

    It is acknowledged that more granular analysis and reporting of results by mental disorders in studies that involved people with more than 1 MD could potentially reveal a slightly different result. All studies except 3 [37,39-41] noted the classification of MD used when recruiting participants, and this makes it challenging to perform comparisons between results of similar studies and mental disorders and to confidently link results to classifications.

    Included studies raise concerns about a bias toward Western culture owing to an absence of research conducted with participants from other cultures. Multicountry studies (eg, [50,51]) have established that culture helps shape technology usage to a great extent.

    Participants in included studies ranged widely in age from 18 to over 75 years, and the experiences between younger and older participants were rarely compared or separated. It is important to account for age because it plays a significant role in determining the types of barriers individuals experience when using technology [52,53].

    Recency of Research

    Findings show that more accessibility and usability research involving PwMD have been done in the last 5 years (10) compared with previous times (3). Considerably more research was done during the same period as revealed by a keyword search of several databases (ie, MEDLINE, PsycARTICLES, CINAHL, Library, Information Science and Technology Abstracts, Computers and Applied Sciences Complete, and ACM Digital Library) for Web or information and communications technology or digital accessibility or usability and visual (139), mobility (64), cognitive and learning (34), and auditory (17) impairments. This suggests that activity in the area is increasing but not at a rate comparable to similar research done with other populations. Included accessibility guidelines were dated. However, version 2.0 updates for Authoring Tool Accessibility Guidelines and User Agent Accessibility Guidelines are almost stable and referenceable versions that will likely be W3C Recommendations and new Web standards [54,55].


    Although the literature search was conducted in many databases, results were limited to publications in English. However, no publications were later excluded based on this restriction. Included publications were not limited to those involving empirical work because preliminary searches indicated a paucity of research focusing on the area. As a result, international guidelines were included in the review. However, these guidelines are based on consensus among many experts and not empirical work, which allows for more valid conclusions. Moreover, although identified barriers found in studies were the result of empirical work, not all facilitation measures identified by studies were empirically validated. Nonetheless, as mentioned in Discussion section, empirically derived facilitation measures were similar to those based on the expert opinion of researchers conducting studies and were not in conflict with facilitation measures recommended by international guidelines.

    Most of the included studies did not use a structured diagnostic classification (eg, ICD or DSM), and this has repercussions for our conclusions being tied to a diagnosis. For instance, it cannot be said unequivocally that persons with a particular diagnosis (eg, depression) experience a certain barrier as reported by those studies that did not use a structured diagnostic classification. Care was also taken to avoid making strong conclusions based on the small number of included studies (13), and it is advised that findings should be interpreted with this in mind.

    Implications and Recommendations for Practice and Future Research

    Web professionals can now consult a full compilation of research and guidelines–based barriers and facilitation measures relevant to PwMD when developing and optimizing Web-based resources. This will raise awareness of PwMD’s needs when using the Web among Web professionals and potentially stimulate further discussion and action within the profession.

    The body of research is in need of significant development, and it is too early to make meaningful conclusions on any particular MD, especially based on high-risk symptomatology. For future research, priority should be given to investigating all mental disorders initially. More research in the area is therefore required especially for mood, anxiety, dissociative, somatic, eating, sleep, impulse control, and personality disorders as these have attracted little or no attention.

    In agreement with [56-58], an increased effort is needed to investigate the accessibility of technological innovations and health systems. This should be done in a more systematic way with clinically diagnosed samples to obtain conclusive evidence about what barriers exist and how they can be removed. This would involve ensuring each barrier is well stated along with an indication of the level of restriction it causes and frequency of occurrence among the particular user group. Validating strategies targeting the removal of barriers before recommending them as facilitation measures would also be helpful.

    Additional actions could be taken by researchers to further develop this area of work. Incorporating valid measures for sociocognitive impairment allows for a more comprehensive evaluation of accessibility for PwMD. It would be important to know if there are cultural differences in the barriers encountered, the level of restriction a particular barrier causes, and/or the frequency of its occurrence. Accessibility studies could also consider a wider range of websites—social networking, e-commerce, education, health—and not just websites targeting PwMD to ensure all aspects of Web usage are investigated.


    Indeed, PwMD encounter barriers on the Web, and attempts have been made to remove or reduce these barriers. To the best of our knowledge, these results represent the first attempt to consolidate information on all barriers and facilitation measures investigated for PwMD when using digital technologies in a systematic way. However, it must be taken into consideration that only 13 studies and 3 guidelines meeting the inclusion criteria were identified. These findings also highlight the dire need for more rigorous research to be exhaustive and to have a larger impact on improving the Web for PwMD.


    The research leading to these results has received funding from the People Programme (Marie Curie Actions) of the European Union’s Seventh Framework Programme FP7/2007 - 2013 under REA grant agreement no. 316795. The views expressed in this paper are those of the authors and do not necessarily represent views or policies of the World Health Organization.

    The authors specially thank Dr. Heinrich Gall for developing the software app used during this study and providing support for it when needed. They also thank Mr. Daley Holloway for his generous support during screening. All authors participated in each step of this work.

    Conflicts of Interest

    None declared.

    Multimedia Appendix 1

    Summary of search concepts and terms.

    PDF File (Adobe PDF File), 45KB


    1. Kessler RC, Angermeyer M, Anthony JC, Demyttenaere K, Gasquet I, Gluzman S, et al. Lifetime prevalence and age-of-onset distributions of mental disorders in the World Health Organization's World Mental Health Survey Initiative. World Psychiatry 2007 Oct;6(3):168-176 [FREE Full text] [Medline]
    2. Whiteford HA, Degenhardt L, Rehm J, Baxter AJ, Ferrari AJ, Erskine HE, et al. Global burden of disease attributable to mental and substance use disorders: findings from the Global Burden of Disease Study 2010. Lancet 2013 Nov 9;382(9904):1575-1586. [CrossRef] [Medline]
    3. Cieza A, Anczewska M, Ayuso-Mateos JL, Baker M, Bickenbach J, Chatterji S, et al. Understanding the Impact of Brain Disorders: Towards a 'Horizontal Epidemiology' of Psychosocial Difficulties and Their Determinants. PLoS One 2015;10(9):e0136271 [FREE Full text] [CrossRef] [Medline]
    4. Navidian A, Zaheden F. Burden experienced by family caregivers of patients with mental disorders. Pakistan Journal of Psychological Research 2008;23(1):19-28.
    5. World Health Assembly 65. WHO. 2012. Global burden of mental disorders and the need for a comprehensive, coordinated response from health and social sectors at the country level   URL: [WebCite Cache]
    6. Kalckreuth S, Trefflich F, Rummel-Kluge C. Mental health related Internet use among psychiatric patients: a cross-sectional analysis. BMC Psychiatry 2014;14:368 [FREE Full text] [CrossRef] [Medline]
    7. Chen J, Zhu S. Online Information Searches and Help Seeking for Mental Health Problems in Urban China. Adm Policy Ment Health 2015 May 16:NA Epub ahead of print(forthcoming). [CrossRef] [Medline]
    8. Kummervold PE, Gammon D, Bergvik S, Johnsen JK, Hasvold T, Rosenvinge JH. Social support in a wired world: use of online mental health forums in Norway. Nord J Psychiatry 2002;56(1):59-65. [CrossRef] [Medline]
    9. Powell J, McCarthy N, Eysenbach G. Cross-sectional survey of users of Internet depression communities. BMC Psychiatry 2003 Dec 10;3:19 [FREE Full text] [CrossRef] [Medline]
    10. Andersson G, Cuijpers P, Carlbring P, Riper H, Hedman E. Guided Internet-based vs. face-to-face cognitive behavior therapy for psychiatric and somatic disorders: a systematic review and meta-analysis. World Psychiatry 2014 Oct;13(3):288-295 [FREE Full text] [CrossRef] [Medline]
    11. Kuester A, Niemeyer H, Knaevelsrud C. Internet-based interventions for posttraumatic stress: A meta-analysis of randomized controlled trials. Clin Psychol Rev 2016 Feb;43:1-16. [CrossRef] [Medline]
    12. Van Voorhees B, Vanderplough-Booth K, Fogel J, Gladstone T, Bell C, Stuart S, et al. Integrative internet-based depression prevention for adolescents: a randomized clinical trial in primary care for vulnerability and protective factors. J Can Acad Child Adolesc Psychiatry 2008 Nov;17(4):184-196 [FREE Full text] [Medline]
    13. O'Kearney R, Kang K, Christensen H, Griffiths K. A controlled trial of a school-based Internet program for reducing depressive symptoms in adolescent girls. Depress Anxiety 2009;26(1):65-72. [CrossRef] [Medline]
    14. Johnson GM. Functional Internet Literacy: required cognitive skills with implications for instruction. E-Learning 2007;4(4):433-441. [CrossRef]
    15. Millan MJ, Agid Y, Brüne M, Bullmore ET, Carter CS, Clayton NS, et al. Cognitive dysfunction in psychiatric disorders: characteristics, causes and the quest for improved therapy. Nat Rev Drug Discov 2012 Feb;11(2):141-168. [CrossRef] [Medline]
    16. Short J, Williams E, Christie B. The social psychology of telecommunications. London: Wiley; 1976.
    17. Lea M, Spears R. Love at first byte? Building personal relationships over computer networks. In: Wood JT, Duck S, editors. Under-studied relationships: off the beaten track. Thousand Oaks, CA: SAGE Publications; 1995.
    18. WALTHER JB. Interpersonal Effects in Computer-Mediated Interaction: A Relational Perspective. Communication Research 1992 Feb 01;19(1):52-90. [CrossRef]
    19. WALTHER JB. Computer-Mediated Communication: Impersonal, Interpersonal, and Hyperpersonal Interaction. Communication Research 1996 Feb 01;23(1):3-43. [CrossRef]
    20. Mariger H. Cognitive Disabilities and the Web: Where Accessibility and Usability Meet?. 2006.   URL: [accessed 2015-12-18] [WebCite Cache]
    21. Hudson R, Weakley R, Firminger P. An Accessibility Frontier: Cognitive disabilities and learning difficulties. 2004.   URL: http:/​/usability.​​2004/​12/​an-accessibility-frontier-cognitive-disabilities-and-learning-difficulties-2004/​ [accessed 2015-12-18] [WebCite Cache]
    22. Bernard R, Sabariego C, Baldwin D, Abou-Zahra S, Cieza A. BETTER-project: web accessibility for persons with mental disorders. In: Kurosu M, editor. Human-Computer Interaction, Users and Contexts 17th International Conference, HCI International. Switzerland: Springer International Publishing; 2015:25-34.
    23. World Wide Web Consortium. 2008. Web Content Accessibility Guidelines (WCAG) 2.0   URL: [accessed 2015-12-18] [WebCite Cache]
    24. Denison HJ, Dodds RM, Ntani G, Cooper R, Cooper C, Sayer AA, et al. How to get started with a systematic review in epidemiology: an introductory guide for early career researchers. Arch Public Health 2013;71(1):21 [FREE Full text] [CrossRef] [Medline]
    25. World Health Organization. International classification of functioning, disability and health: ICF. Geneva: World Health Organization; 2001.
    26. World Wide Web Consortium. Introduction to Understanding WCAG 2   URL: [accessed 2016-01-27] [WebCite Cache]
    27. World Wide Web Consortium. WCAG 2 at a Glance. 2012.   URL: [accessed 2016-03-03] [WebCite Cache]
    28. World Wide Web Consortium. 2014. ATAG at a Glance   URL: [accessed 2016-03-03] [WebCite Cache]
    29. World Wide Web Consortium. 2005. User Agent Accessibility Guidelines (UAAG) Overview   URL: [accessed 2015-12-18] [WebCite Cache]
    30. Ben-Zeev D, Kaiser SM, Brenner CJ, Begale M, Duffecy J, Mohr DC. Development and usability testing of FOCUS: a smartphone system for self-management of schizophrenia. Psychiatr Rehabil J 2013 Dec;36(4):289-296 [FREE Full text] [CrossRef] [Medline]
    31. Bulger J. Jenniferbulger. 2002. A usability study of mental health websites with an emphasis on homepage design: Performance and preferences of those with anxiety disorders   URL:'s_paper.pdf [accessed 2015-12-18] [WebCite Cache]
    32. Brunette MF, Ferron JC, Devitt T, Geiger P, Martin WM, Pratt S, et al. Do smoking cessation websites meet the needs of smokers with severe mental illnesses? Health Educ Res 2012 Apr;27(2):183-190 [FREE Full text] [CrossRef] [Medline]
    33. Ferron JC, Brunette MF, McHugo GJ, Devitt TS, Martin WM, Drake RE. Developing a quit smoking website that is usable by people with severe mental illnesses. Psychiatr Rehabil J 2011;35(2):111-116. [CrossRef] [Medline]
    34. Good A, Sambhanthan A. Accessing web based health care and resources for mental health: interface design considerations for people experiencing mental illness. In: Marcus A, editor. Accessing web based health careresources for mental health: interface design considerations for people experiencing mental illness. Switzerland: Springer International Publishing; 2014:978-973.
    35. Jimison HB, Sher PP, Appleyard R, LeVernois Y. The use of multimedia in the informed consent process. J Am Med Inform Assoc 1998;5(3):245-256 [FREE Full text] [Medline]
    36. Johansson S, Gulliksen J, Lantz A. Cognitive Accessibility for Mentally Disabled Persons. In: Abascal J, Barbosa S, Fetter M, Gross T, Palanque P, Winckler M, editors. Human-Computer Interaction – INTERACT 2015. Switzerland: Springer International Publishing; 2015:418-435.
    37. Rotondi AJ, Eack SM, Hanusa BH, Spring MB, Haas GL. Critical design elements of e-health applications for users with severe mental illness: singular focus, simple architecture, prominent contents, explicit navigation, and inclusive hyperlinks. Schizophr Bull 2015 Mar;41(2):440-448. [CrossRef] [Medline]
    38. Rotondi AJ, Sinkule J, Haas GL, Spring MB, Litschge CM, Newhill CE, et al. Designing websites for persons with cognitive deficits: Design and usability of a psychoeducational intervention for persons with severe mental illness. Psychol Serv 2007 Aug;4(3):202-224 [FREE Full text] [CrossRef] [Medline]
    39. Rotondi AJ, Spring M, Grady J, Simpson R, Luther J, Abebe KZ, et al. Use of a Fractional Factorial Experiment to Assess the E-healthcare Application Design Needs of Persons With Dual Diagnosis. Journal of Dual Diagnosis 2012 Nov;8(4):277-282. [CrossRef]
    40. Todd NJ, Jones SH, Lobban FA. What do service users with bipolar disorder want from a web-based self-management intervention? A qualitative focus group study. Clin Psychol Psychother 2013;20(6):531-543. [CrossRef] [Medline]
    41. Schrank B, Sibitz I, Unger A, Amering M. How patients with schizophrenia use the internet: qualitative study. J Med Internet Res 2010;12(5):e70 [FREE Full text] [CrossRef] [Medline]
    42. Black AC, Serowik KL, Schensul JJ, Bowen AM, Rosen MI. Build a better mouse: directly-observed issues in computer use for adults with SMI. Psychiatr Q 2013 Mar;84(1):81-92 [FREE Full text] [CrossRef] [Medline]
    43. Treviranus J, McCathieNevile C, Jacobs I, Richards J. World Wide Web Consortium. 2000. Authoring Tool Accessibility Guidelines 1.0   URL: [accessed 2015-12-18] [WebCite Cache]
    44. Jacobs I, Gunderson J, Hansen E. World Wide Web Consortium. 2002. User Agent Accessibility Guidelines 1.0   URL: [accessed 2015-12-18] [WebCite Cache]
    45. World Wide Web Consortium. 2008. How WAI Develops Accessibility Guidelines through the W3C Process: Milestones and Opportunities to Contribute   URL: [accessed 2015-12-18] [WebCite Cache]
    46. World Wide Web Consortium. 2012. Diversity of Web Users: How People with Disabilities Use the Web   URL: [accessed 2015-12-18] [WebCite Cache]
    47. Friedman M, Bryen D. Web accessibility design recommendations for people with cognitive disabilities. Technology & Disability 2007;19(4):205-212.
    48. Shrivastava AK, Johnston ME. Cognitive neurosciences: A new paradigm in management and outcome of schizophrenia. Indian J Psychiatry 2010 Apr;52(2):100-105 [FREE Full text] [CrossRef] [Medline]
    49. Bora E, Yucel M, Pantelis C. Cognitive functioning in schizophrenia, schizoaffective disorder and affective psychoses: meta-analytic study. Br J Psychiatry 2009 Dec;195(6):475-482 [FREE Full text] [CrossRef] [Medline]
    50. Calhoun KJ, Teng JTC, Cheon MJ. Impact of national culture on information technology usage behaviour: An exploratory study of decision making in Korea and the USA. Behaviour & Information Technology 2010 Nov 08;21(4):293-302. [CrossRef]
    51. Park C, Jun J. A cross‐cultural comparison of Internet buying behavior. International Marketing Review 2003 Oct;20(5):534-553. [CrossRef]
    52. Díaz-Bossini J, Moreno L. Accessibility to Mobile Interfaces for Older People. Procedia Computer Science 2014;27:57-66. [CrossRef]
    53. de Lara SMA, Fortes RPDM, Russo CM, Freire AP. A study on the acceptance of website interaction aids by older adults. Univ Access Inf Soc 2015 Jun 24:1-16. [CrossRef]
    54. World Wide Web Consortium. 2013. Authoring Tool Accessibility Guidelines (ATAG) Overview   URL: [accessed 2015-12-18] [WebCite Cache]
    55. World Wide Web Consortium. 2005. User Agent Accessibility Guidelines (UAAG) Overview   URL: [accessed 2015-12-18] [WebCite Cache]
    56. Davies EB, Morriss R, Glazebrook C. Computer-delivered and web-based interventions to improve depression, anxiety, and psychological well-being of university students: a systematic review and meta-analysis. J Med Internet Res 2014;16(5):e130 [FREE Full text] [CrossRef] [Medline]
    57. Eysenbach G. The law of attrition. J Med Internet Res 2005;7(1):e11 [FREE Full text] [CrossRef] [Medline]
    58. Renton T, Tang H, Ennis N, Cusimano MD, Bhalerao S, Schweizer TA, et al. Web-based intervention programs for depression: a scoping review and evaluation. J Med Internet Res 2014;16(9):e209 [FREE Full text] [CrossRef] [Medline]


    MD: mental disorders
    PwMD: people with mental disorders
    SMI: severe mental illness
    USA: United States of America
    W3C: World Wide Web Consortium

    Edited by G Eysenbach; submitted 18.12.15; peer-reviewed by A Rotondi, J Ferron, M Brunette; comments to author 24.01.16; revised version received 01.04.16; accepted 18.04.16; published 09.06.16

    ©Renaldo Bernard, Carla Sabariego, Alarcos Cieza. Originally published in the Journal of Medical Internet Research (, 09.06.2016.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.