This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.
The Internet Gaming Disorder Scale–Short-Form (IGDS9-SF) is among the best with regard to its psychometric properties. Therefore, clinical psychologists are likely guided to use the IGDS9-SF if they want to assess or screen the disordered gaming in their practice. However, the information, especially psychometric evidence, concerning the IGDS9-SF has not been fully examined and summarized.
This systematic review evaluated the psychometric properties of different language versions of the IGDS9-SF and assessed its methodological quality in order to improve the clinicians’ understanding of the IGDS9-SF and facilitate its use.
Systematic literature searches were carried out using
In total, 21 studies comprising 15 language versions of the IGDS9-SF were included. Overall, the IGDS9-SF showed adequate internal consistency (although some items did not have satisfactory item-total correlation [IT]), excellent criterion validity, and the ability to distinguish different subgroups with measurement invariance being supported across gender and age. In terms of factor structure, the IGDS9-SF was shown to have a unidimensional factor structure across all 21 studies.
Although there is insufficient evidence regarding the responsiveness and properties of the IGDS9-SF using item response theory, the existing evidence supports its use in assessing disordered gaming among individuals.
In the era of technology, internet use has become one of the essential components of everyone’s life [
Despite the many positive outcomes associated with online gaming, such as decreased loneliness and promotion of psychological well-being [
Additionally, the 11th revision of the
Ever since the emergence of the first internet addiction scale in 1998 [
Currently, there are at least seven psychometric assessment tools for IGD that have been developed using DSM-5 criteria [
Among the aforementioned psychometric instruments, the IGDS9-SF emerges as a robust and concise psychometric tool for assessing individuals with IGD, with a recent study identifying this tool as having great support in relation to its sound psychometric properties and significant advantages in comparison to most existing tools for IGD [
In the current literature, the IGDS9-SF has been translated into 17 languages: Chinese, with three sublanguages of traditional Chinese in Hong Kong [
Although the psychometric properties of the IGDS9-SF have been examined among different populations, to the best of our knowledge, there is no systematic review of the IGDS9-SF reporting its psychometric characteristics in depth. Dispersed information in the extant literature regarding the IGDS9-SF with varying sample sizes and across different countries makes it timely to investigate whether clinicians should be adopting the IGDS9-SF for assessment of IGD. More specifically, there are a number of questions that are best answered by carrying out a systematic review of the psychometric properties of the IGDS9-SF.
First, the IGDS9-SF may have different psychometric features across different language versions, and it is unclear whether the existing psychometric evidence for different language versions of the IGDS9-SF is equivalent. Second, prior psychometric testing studies on the IGDS9-SF need to be evaluated for their methodological quality.
Without formal assessment of the quality of previous studies on the IGDS9-SF, the results pertaining to its psychometric properties may be biased. For the sake of improving clinicians’ understanding and facilitating the use of IGDS9-SF across other contexts beyond research settings, this systematic review incorporated different items of evidence concerning the psychometric features of the IGDS9-SF across a wide range of populations. More specifically, if the psychometric properties of the IGDS9-SF are found to be supported across different contexts, clinicians can use the IGDS9-SF criteria to exchange their expert opinions using the same signs, symptoms, and components. For example, a clinician in Taiwan can use IGDS9-SF scores to assess the IGD severity level of a disordered gamer, and this information can be well understood and correctly interpreted by clinicians in other countries.
Furthermore, when carrying out psychological assessment, clinicians are required by professional governing bodies (eg, the American Psychological Association, the British Psychological Society, the Australian Psychological Society) to adopt valid and reliable psychometric tools to support the adoption of evidence-based practices. Therefore, this review is of importance to clinicians working with disordered gamers.
This review followed the recommended flow of the Preferred Reporting Items of Systematic Reviews and Meta-analyses (PRISMA) guidelines [
A literature search was carried out using
A total of 2533 journal articles were identified. Duplicates (n=200) were removed using EndNote. The titles and abstracts of the remaining journal articles (n=2333) were screened for eligibility by two authors independently (ie, the same two authors who screened all 2333 journal articles). Of these, 2286 articles did not focus on the IGDS9-SF and were removed. Full texts of all potential articles were then retrieved (n=47) and screened using the same procedure. Of these, 26 were removed because of the following reasons: (1) the study did not test the psychometric properties of the IGDS9-SF (n=22), (2) the study was a non-peer-reviewed conference paper (n=3), or (3) the study was a review paper (n=1). The remaining 21 studies were evaluated and analyzed in this systematic review.
The review included only English-language studies of all types of research design under the condition that they (1) reported at least one psychometric property of the IGDS9-SF (eg, internal consistency, reliability, measurement error, content validity, construct validity, criterion validity, or responsiveness), as defined by COSMIN, and (ii) aimed at testing the psychometric properties of the IGDS9-SF. Exclusion criteria were nonrefereed studies, review studies, conference proceedings, dissertations, commentaries, editorials, or letters to journal editors. The aforementioned inclusion and exclusion criteria were applied to all paper titles and abstracts, and screening of full texts.
The psychometric properties of the included studies were evaluated by five authors (TYJC, SWTM, LYN, LYJP, and YLEW), and each study was independently assessed by any two of the five authors using the COSMIN Risk-of-Bias checklist [
Ranging from 3 to 13 items, each property was scored on a 4-point scale with four predefined options: very good (V), adequate (A), doubtful (D), and inadequate (I) [
The quality of the statistical results of each reported measurement property of the IGDS9-SF was rated against the updated criteria for good measurement properties based on Terwee et al [
Of the 2533 identified studies, 200 were duplicates and 2333 were screened for abstracts (
Flowchart of study selection [
In terms of sample characteristics across all eligible studies, nine studies included gamers [
Characteristics of included studies and psychometric properties of the IGDS9-SFa not included in COSMINb.
Author/country | Study design | Population | Sample size | Age (years) | Testing period | Others (not included in COSMIN) |
Chen et al [ |
Cross-sectional | University students | Hong Kong 304 (67.4% female) |
Hong Kong |
3 months (for testing temporal invariance) | Skewness=0.21-2.41 |
Beranuy et al [ |
Cross-sectional | Vocational training students | 535 (21.5% female) | 18.35 ± 2.13 | N/Ac | ITd=0.47-0.67 |
de Palo et al [ |
Cross-sectional | Albania and Italy: People from schools, universities, and gaming halls |
Overall 1411 (36.4% female) |
Overall 25.94 ± 8.91 |
N/A | No item showed absolute values of skewness >2 or values of kurtosis >7 |
Evren et al [ |
Cross-sectional | University students, active internet gamers, and ESL Turkey amateur e-sport players | 457 (62.4% female) | N/A | N/A | IT=0.663-0.826 |
Gomez et al [ |
Cross-sectional | Internet gamers | 868 (39.7% female) | Overall 28.64 ± 8.79 |
N/A | N/A |
Leung et al [ |
Cross-sectional | University students | Hong Kong 306 (67.6% female) |
Hong Kong 24.08 ± 5.06 |
N/A | IT=0.68-0.85 |
Monacis et al [ |
Cross-sectional | Students from Italian schools, universities, and gaming hall | 687 (45.4% female) | 21.62 ± 3.90 | N/A | No item showed absolute values of skewness >2 or values of kurtosis >7 |
Pontes and Griffiths [ |
Cross-sectional | English-speaking gamers from 58 different countries | 1060 (14.9% female) | 27 ± 9.02 | N/A | Floor effect=3.8%-5.6% |
Pontes and Griffiths [ |
Cross-sectional | Students in sixth, seventh, eighth, and ninth grades of a major located in the Algarve | 509 (47.9% female) | 13 ± 1.64 | N/A | No item showed absolute values of skewness >3 or values of kurtosis >9 |
Pontes et al [ |
Cross-sectional | English-speaking gamers from English-speaking online gaming forums that are popular among gamers | USA 405 (38% female) |
USA 32.57 ± 11.33 |
N/A | N/A |
Pontes et al [ |
Cross-sectional | Students in eighth grade | 1071 (49.8% female) | 13.44 ± 0.59 | N/A | No item showed absolute values of skewness >3 or values of kurtosis >9 |
Schivinski et al [ |
Cross-sectional | Gamers | 3377 (17.4% female) | 20 ± 4.3 | N/A | Skewness=–0.08-1.51 |
Severo et al [ |
Cross-sectional | High school and college students | 555 (42.5% female) | 20.3 ± 5.4 | N/A | No item showed absolute values of skewness >3 or values of kurtosis >9 |
Stavropoulos et al [ |
Longitudinal | Emerging adults from the general community who played massively multiplayer online (MMO) games | Australia 61 |
Australia |
60-90 days | N/A |
Wu et al [ |
Cross-sectional | High school adolescents | 2363 (35.2% female) | 15.6 ± 1.2 | 2 weeks (for testing test-retest reliability) | IT=0.54-0.74 |
Yam et al [ |
Cross-sectional | University students | 307 (67.6% female) | 21.64 ± 8.11 | N/A | IT=0.527-0.724 |
Stavropoulos et al [ |
Cross-sectional | Internet gamers | Australia 171 (23.4% female) |
Australia 25.72 ± 5.52 |
N/A | N/A |
Arıcak et al [ |
Cross-sectional | Group 1: Students learning in English in a private institution |
Group 1, 35 |
Group 1, 12.50 ± 1.20 |
Group 4: 2 weeks (for testing test- retest reliability) | Floor effect=9% |
Chen et al [ |
Cross-sectional | Primary school children in fourth to sixth grades | 1108 (51.7% female) | 10.37 ± 0.95 | N/A | IT=0.55-0.76 |
Kim and Ko [ |
Cross-sectional | Korean internet game users from major online gaming forums, universities, counseling centers, and libraries located in the greater Seoul area, Gyeonggi and Chungcheong Provinces of Korea | 594 (29.6% female) | 23.5 ± 6.29 | N/A | IT=0.49-0.68 |
T’ng and Pau [ |
Cross-sectional | Youth who played MOBAe | 1050 (25.1% female) | 21.96 ± 2.37 | N/A | No item showed absolute values of skewness >2 or values of kurtosis >7 |
aIGDS9-SF: Internet Gaming Disorder Scale–Short-Form.
bCOSMIN: COnsensus-based Standards for the selection of health status Measurement Instrument.
cN/A: not available.
dIT: item-total correlation
eMOBA: multiplayer online battle arena
Summarized psychometric properties of the IGDS9-SFa in the analyzed studies.
Psychometric property included in COSMINb | N | Meth qualc | Result (ratingd) |
Structural validity | 19,049 | I |
One factor (+) |
Internal consistency | 19,049 | V |
.810-.963 (+) |
Cross-cultural validity | 7352 | I |
Age (+) full invariance Gender (?) partial invariance Time on gaming (+) partial invariance Country (–) partial invariance |
Reliability | 2962 | D |
ICCe=.94 Pearson correlation=.756-.87 |
Measurement error | 2962 | D |
0.16-2.27 (?) |
Criterion validity | 457 | V |
r=.988 (+) |
Concurrent validity | 12,323 | V |
Absolute r=.00-.556 Absolute β=.103-.663 |
Convergent validity | 6149 | V |
Absolute r=.06-.827 |
Discriminative validity | 1142 | D |
Significant difference found in age and gender |
aIGDS9-SF: Internet Gaming Disorder Scale–Short-Form.
bCOSMIN: Consensus-based Standards for the selection of health status Measurement Instrument.
cCOSMIN score after removing the sample size item from the rating: V, very good; A, adequate; D, doubtful; I, inadequate; N, not applicable.
dQuality score of the measurement property: +, sufficient; -, insufficient; ?, indeterminate.
eICC: intraclass correlation coefficient.
All studies reported the structural validity of the IGDS9-SF for the 15 versions (n=19,049). Of the 21 studies, 14 demonstrated very good methodological quality and had a positive rating for the quality of statistical findings; 6 studies with good to excellent methodological quality showed an indeterminate rating on the quality of statistical findings due to the absence of the standardized root-mean-square residual (SRMR; Supplementary Table S1 in
In addition, one study performed Rasch analysis to test the person separation reliability, person separation index, item separation reliability, and item separation index of the Persian version of the IGDS9-SF. The results showed doubtful methodological quality and a positive rating for the quality of statistical findings (person separation reliability=.86, person separation index=2.50, item separation reliability=1.00, item separation index=28.79). The study also reported an acceptable range of infit (0.79-1.37) and outfit mean square (0.74-1.34), as well as the range of item difficulties (–1.06-1.57).
All studies evaluated the internal consistency of the IGDS9-SF for the 15 versions (n=19,049). All studies had very good methodological quality and showed a positive rating for the quality of statistical findings concerning internal consistency (Cronbach α=.810-.963 and person separation reliability=.86; see Supplementary Table S1 in
Of the 21 studies, 9 examined the measurement invariance of the IGDS9-SF across different factors (n=7352; see Supplementary Table S2 in
In addition, four studies reported the measurement invariance across multiple countries, and one study supported full invariance across the traditional Chinese (Hong Kong and Taiwan) versions. Partial metric and scalar invariance were supported across Albania, the United States, the United Kingdom, and Italy but neither across the United States, the United Kingdom, and India nor across the United States, the United Kingdom, and Australia.
Furthermore, one study examined the differential item functioning (DIF) contrast across gender and time spent gaming using Rasch analysis; three DIF items were found, namely Item 4, “Fail to control or cease gaming activities” concerning gender (DIF contrast=–0.55); item 4 concerning time spent online gaming per week (DIF contrast=–0.67); and item 9, “Jeopardize or lose an important thing because of gaming activity” concerning time spent online gaming per week (DIF contrast=.61).
Reliability of the IGDS9-SF was evaluated in four studies on four versions (n=2962). The studies investigated the test-retest reliability of the IGDS9-SF (ie, traditional Chinese [Hong Kong and Taiwan], Persian, and Turkish versions), and all demonstrated fair methodological quality. All studies also demonstrated a positive rating for the quality of statistical findings (Supplementary Table S2 in
The above-mentioned four studies on four versions also examined the measurement errors in using the IGDS9-SF (n=2962; see Supplementary Table S3 in
One study on the Turkish version evaluated criterion validity and showed very good methodological quality (Supplementary Table S3 in
Of the 21 studies, 11 using 11 versions evaluated the concurrent validity of the IGDS9-SF (Supplementary Table S4 in
Of the 21 studies, 10 evaluated the convergent validity of the IGDS9-SF. Very good methodological quality was demonstrated in seven studies: traditional Chinese (Hong Kong and Taiwan), simplified Chinese, Spanish, Italian, European Portuguese, South American Portuguese, and Korean versions of the IGDS9-SF. Another three studies on the traditional Chinese (Hong Kong and Taiwan), English, and Turkish versions showed inadequate methodological quality. All 10 studies had a positive rating for the statistical quality (Supplementary Table S4 in
In addition, two studies evaluated the discriminative validity of the IGDS9-SF (Supplementary Table S5 in
Floor and ceiling effects of the IGDS9-SF were reported in six studies on the English, traditional Chinese (Hong Kong), simplified Chinese, Persian, Turkish, and Malay versions. The study on the English version of the IGDS9-SF reported more gamers at floor-level scores (3.8%-5.6%) than at the ceiling level (0.2%-0.8%). The study on the Persian version of the IGDS9-SF reported a 1.8% ceiling effect and a 0.8% floor effect, whereas studies on the traditional Chinese (Hong Kong) and simplified Chinese versions reported no ceiling effect but a relatively high floor effect (21%-24.6%). Another study on the Turkish version did not report any ceiling effect of the IGDS9-SF but reported a 9% floor effect. A study on the Malay version of the IGDS9-SF reported a 1.7% ceiling effect and a 0.1% floor effect. Except for the studies on the traditional and simplified Chinese versions, the distribution of scoring on the IGDS9-SF in the related studies indicated acceptable floor and ceiling effects (ie, an effect less than 15%) [
IT was reported in eight studies and ranged from 0.342 to 0.86 (
Two studies on the traditional Chinese (Hong Kong) and Turkish versions also had the lowest IT correlation on item 7 (IT=.663-.68). Item 9 (“Have you jeopardized an important relationship, job, or educational or career opportunity because of your gaming activity?”) demonstrated the lowest IT in the South American Portuguese, traditional Chinese (Hong Kong), and simplified Chinese versions (IT=.342-.55), while the item with the lowest IT in the Persian version was item 4 (“Do you systematically fail when trying to control or cease your gaming activity?”) (IT=.54).
The quality of methodology and the quality of statistical findings for each psychometric property of the IGDS9-SF were evaluated for all 21 eligible studies. The evidence regarding the psychometric properties of the IGDS9-SF was summarized based on the existing evidence. In general, the IGDS9-SF demonstrated good internal consistency, although some items did not have satisfactory IT, especially items 7 (Spanish, Chinese, South American Portuguese, Turkish and Persian), 8 (Spanish, Chinese, and South American Portuguese), and 9 (Chinese, South American Portuguese, Turkish, and Persian). However, the IGDS9-SF has excellent criterion validity, as evidenced by the strong correlation with the 27-item IGDS [
All nine items of the IGDS9-SF demonstrated satisfactory IT (greater than 0.4) [
In relation to compromising occupation/education or a significant relationship due to the involvement in gaming (ie, item 9), a simpler criterion that only covers occupational/educational aspects is more appropriate than concurrently conflating relationships and occupational/educational loss [
The behavior of escaping from adverse moods by gaming (ie, item 8) is reported as a criterion that is unable to differentiate between disordered and nondisordered gamers [
The IGDS9-SF demonstrated a strong correlation with the 27-item IGDS [
In addition to criterion validity, the concurrent validity of the IGDS9-SF was supported by the positive correlation between the IGDS9-SF score and the hours spent on online activities (eg, smartphone use, social media use, and gaming), with the correlations between the IGDS9-SF and gaming frequency being the most significant [
In addition, a positive association was found between IGDS9-SF scores and three psychological symptoms (ie, depression, anxiety, and stress) [
Although the IGDS9-SF is positively associated with other similar measures (eg, Mobile Phone-Related Experiences Questionnaire [CERM] and Online Gambling Disorder Questionnaire [OGD-Q]) [
All 21 studies demonstrated the unidimensional structure of the IGDS9-SF and were supported by both confirmatory factor analysis and Rasch analysis. All factor loadings reported across the studies reviewed were satisfactory. Moreover, measurement invariance across gender [
In addition to the most commonly used classical test theory (CTT) in the included studies, only two included studies used item response theory (or Rasch analysis) [
This systematic review had several strengths. First, a comprehensive search strategy was adopted to identify and evaluate potential studies. Second, the review evaluated studies with an accepted set of criteria, that is, both methodological quality (COSMIN) and statistical quality. Moreover, all 21 eligible studies included for evaluation in the systematic review had the strength of having good sample sizes (ie, N>100) based on the COSMIN risk-of-bias checklist. Therefore, this review took advantage of the good sample sizes of the studies evaluated to increase the accuracy in the summarized psychometric properties of the IGDS9-SF.
This review also had potential limitations. In the study selection stage, the language was limited to only English and each included study was assessed by only two authors (and these authors were not necessarily the same two for each paper). More specifically, although consensus was achieved in accordance with COSMIN guidelines, the authors who rated the quality of each study may have used slightly varied rating styles. Nevertheless, the COSMIN guidelines provided clear and concrete instructions for the evaluation of the studies, so the use of different authors in the study evaluation made it unlikely that this led to serious bias. Additionally, evaluation of the results was verified by the corresponding author, who is an expert psychometrician, which further minimized the possibility of evaluation bias. Some psychometric properties of the IGDS9-SF (eg, responsiveness) were not evaluated in any of the eligible studies. More specifically, responsiveness as an important psychometric property to understand whether an instrument is sensitive in detecting change was not carried out in any of the 21 studies. Further studies investigating responsiveness are therefore required. Finally, this systematic review did not use the Google Scholar database to supplement the literature search concerning the psychometric properties of the IGDS9-SF. Therefore, some studies may have been missed in this systematic review.
The IGDS9-SF was designed both as a brief screening tool to assist clinicians in assessing IGD severity as well as for use in epidemiological studies [
In sum, the IGDS9-SF is a time-efficient and psychometrically sound tool that can help clinicians screen for potential patients who may need more detailed clinical evaluation regarding their gaming behaviors. The use of the IGDS9-SF may also help in busy clinical settings by saving evaluation time in diagnosing and assessing IGD in patients. Indeed, a recent review of all screening instruments currently available for disordererd gaming (n=32) reported that the IGDS9-SF was among the best tools with regard to its psychometric properties when compared with all the others similar tools [
This systematic review summarized and reviewed evidence from various populations regarding the IGDS9-SF with regard to its structural validity, internal consistency, cross-cultural validity/measurement invariance, reliability, measurement error, criterion validity, convergent validity, and discriminative or known-group validity. Overall, there was strong evidence demonstrating that the IGDS9-SF has good internal consistency and excellent criterion validity for wide-ranging populations. Responsiveness and reliability as two important psychometric properties of the IGDS9-SF require more evidence because few studies have evaluated them. Regarding the psychometric evidence of different language versions, it was found that all versions of the IGDS9-SF present strong psychometric properties, except for the concurrent validity of the traditional Chinese (Hong Kong and Taiwan), simplified Chinese, Polish, and Persian versions. As such, the psychometric properties of the IGDS9-SF are robust, and the findings regarding its cross-cultural psychometric features in different language versions have been further clarified. Therefore, the IGDS9-SF can be used widely within clinical and research settings.
Database search.
Results of studies on measurement properties.
American Psychiatric Association
Mobile Phone-Related Experiences Questionnaire
Consensus-based Standards for the selection of health status Measurement Instrument
classical test theory
differential item functioning
Electronic Sports League
internet gaming disorder
Internet Gaming Disorder Scale–Short-Form
Internet Gaming Disorder Test
item-total correlation
minimal important change
massively multiplayer online
multiplayer online battle arena
Online Gambling Disorder Questionnaire
patient-reported outcome measure
root-mean-square error of approximation
standard error of measurement
standardized root-mean-square residual
The authors acknowledge financial support from the Ministry of Science and Technology, Taiwan (MOST 110-2410-H-006-115), the 2021 Southeast and South Asia and Taiwan Universities Joint Research Scheme (NCKU 31), and the Higher Education Sprout Project, Ministry of Education to the Headquarters of University Advancement at National Cheng Kung University (NCKU). MDG and HMP are the developers of the IGDS9-SF.
LYJP, TYJC, SWTM, LYN, YLEW, C-YL, and C-WC designed the study and wrote the protocol. SWTM and YLEW conducted literature searches and screened the eligibility. TYJC, LYN, and LYJP were involved in the discussion of eligibility. TYJC, SWTM, LYN, LYJP, and YLEW provided summaries of previous research studies. LYJP, TYJC, SWTM, LYN, YLEW, C-YL, and C-WC wrote the first draft of the manuscript. C-YL and AHP registered the systematic review. AHP, MDG, and HMP critically reviewed the manuscript. HMP helped in drafting and revising the final version of the manuscript. All authors have approved the final manuscript.
None declared. The Internet Gaming Disorder Scale–Short-Form is not a commercial instrument. Therefore, both MDG and HMP have no conflicts of interest.