Published on in Vol 22, No 9 (2020): September

Preprints (earlier versions) of this paper are available at, first published .
Comparison of a Mobile Health Electronic Visual Analog Scale App With a Traditional Paper Visual Analog Scale for Pain Evaluation: Cross-Sectional Observational Study

Comparison of a Mobile Health Electronic Visual Analog Scale App With a Traditional Paper Visual Analog Scale for Pain Evaluation: Cross-Sectional Observational Study

Comparison of a Mobile Health Electronic Visual Analog Scale App With a Traditional Paper Visual Analog Scale for Pain Evaluation: Cross-Sectional Observational Study

Original Paper

1School of Health Sciences, Faculty of Health and Medicine, The University of Newcastle, Ourimbah, Australia

2School of Biomedical Sciences and Pharmacy, Faculty of Health and Medicine, The University of Newcastle, Ourimbah, Australia

3Facultat de Ciències de la Salut de Manresa, Universitat de Vic-Universitat Central de Catalunya, Manresa, Spain

4Department of Physical Therapy, Escola Universertària de la Salut i l’Esport (EUSES), University of Girona, Salt, Girona, Spain

5MIXESTAT SL, Barcelona, Spain

6Priority Research Centre Health Behaviour, Hunter Medical Research Institute, Newcastle, Australia

*all authors contributed equally

Corresponding Author:

Andrea Coda, BSc, Hons, PhD

School of Health Sciences

Faculty of Health and Medicine

The University of Newcastle

Health Precinct, BE154, PO Box 127

Ourimbah, 2258


Phone: 61 0243484507


Background: Accurate quantification of pain in a clinical setting is vital. The use of an electronic pain scale enables data to be collected, analyzed, and utilized much faster compared with traditional paper-based scales. The advancement of smart technology in pediatric and adult pain evaluation may offer opportunities to introduce easy-to-use and reliable pain assessment methods within different clinical settings. If promptly introduced within different pediatric and adult pain clinic services, validated and easily accessible mobile health pain apps may lead to early pain detection, promoting improvement in patient’s quality of life and leading to potentially less time off from school or work.

Objective: This cross-sectional observational study aimed to investigate the interchangeability of an electronic visual analog scale (eVAS) app with a traditional paper visual analog scale (pVAS) among Australian children, adolescents, and adults for pain evaluation.

Methods: Healthy participants (age range 10-75 years) were recruited from a sporting club and a secondary school in Melbourne (Australia). The data collection process involved application of pressure (8.5 kg/cm2) from a Wagner Force Dial FDK 20 to the midpoint of the thumb. The pressure was applied twice with a 5-minute interval. At each pressure application, participants were asked to randomly record their pain perception using the “eVAS” accessible via the “Interactive Clinics” app and the traditional pVAS. Statistical analysis was conducted to determine intermethod and intramethod reliabilities.

Results: Overall, 109 healthy participants were recruited. Adults (mean age 42.43 years, SD 14.50 years) had excellent reliability, with an intraclass correlation coefficient (ICC) of 0.94 (95% CI 0.91-0.96). Children and adolescents (mean age 13.91 years, SD 2.89 years) had moderate-to-good intermethod and intramethod reliabilities, with an ICC of 0.80 (95% CI 0.70-0.87) and average ICC of 0.80 (95% CI 0.69-0.87), respectively.

Conclusions: The eVAS app appears to be interchangeable compared with the traditional pVAS among children, adolescents, and adults. This pain evaluation method may offer new opportunities to introduce user-friendly and validated pain assessment apps for patients, clinicians, and allied health professionals.

J Med Internet Res 2020;22(9):e18284



Pain is a complex and multifactorial phenomenon that can negatively impact a patient’s health-related quality of life [1]. Pain outcome measures are commonly used to assess the severity of symptoms in children, adolescents, and adults [2]. Traditionally, symptom progression has been recorded using the visual analog scale (VAS), Wong Baker scale, numeric rating scale, verbal rating scale, and faces pain scale-revised [3-6]. These tools have been extensively validated as appropriate measures for assessing pain and are commonly used daily by allied health professionals (AHPs).

Evidence suggests there are limitations associated with the more traditional paper pain outcome measures that are still commonly used in various clinical settings [7]. Drawn face scales may result in incorrect recordings if a child experiences difficulty in distinguishing between the feeling of pain and the emotional state, and smiling faces could result in overestimation of pain intensity [4]. These limitations are mostly based on paper pain scales being cumbersome, occasionally complex to use, and at risk of possible practitioner error [8].

The continuous growth of mobile health (mHealth) offers unparalleled opportunities to address issues related to health systems and accessing accurate, reliable, and frequent health data [9,10]. The introduction of smart technology in pediatric and adult pain evaluation may offer opportunities to implement tailored pain assessments within different clinical settings. Recently, novel technologies have emerged that utilize smart devices to improve the existing traditional pain outcome measures [11-16]. Few of these novel technologies have been examined for reliability and validity among children, adolescents, and adults, with most employing small sample sizes. There is growing evidence to suggest that electronic pain outcome measures are interchangeable with existing traditional pain outcome measures, but more mHealth and eHealth research is needed to test the validity and reliability of the electronic VAS (eVAS) among children and adolescents [17]. By evaluating the validity of mHealth and eHealth interventions available to patients and clinicians, we can equip AHPs with a more effective tool to measure symptom progression. This study adheres to the standards of digital health interventions set by the World Health Organization (WHO) in 2016 [18] and the Australian Government Health Authorities’ guidelines with regard to how to validate new digital mHealth systems for the benefit of patients, clinicians, and the community [19]. These include monitoring and evaluating linear stages of development from prototype through to pilot studies showing efficacy, demonstration of effectiveness, scaling up, and integration into the clinical environment [18]. The traditional paper VAS (pVAS) was used owing to its well-established validity across all age groups [20]. This cross-sectional observational study aimed to investigate the interchangeability between an eVAS app and a traditional pVAS among children, adolescents, and adults. This may validate the use of the eVAS in a clinical setting, enhancing the collection, analysis, and dissemination of data in line with the eHealth and mHealth pathways envisaged by health authorities worldwide.


Healthy participants were recruited through a convenient sample of participants in Melbourne (Victoria, Australia) from John Monash Science School (Clayton) and KBH Brumbies Hockey Club (Mont Albert North). English speaking children and adolescents (age range 10-18 years) and adults (age range 18-75 years) were eligible to participate in the study. A mean age of 9.8 years appears to be suitable to evaluate the concept of experienced pain [3]; therefore, this finding was utilized to inform the age group for this mHealth trial, despite the possibility to reliably use the traditional pVAS from the age of 7 years [20]. Participants were excluded if they were diagnosed with neurological disorders, were receiving medication that would alter pain perception or threshold, and had severe visual impairments that may prevent viewing the pVAS and eVAS.

Prior to consent approval, a participant information sheet was supplied to potential participants, and they were made aware of the procedure and the time commitments required to take part in the study. The participant information sheet was adapted according to age (adults, and children and adolescents) including more visual aids in the children’s version. Ethical approval was obtained from the University of Newcastle Human Research Ethics Committee (Dev-005638). Approval was also sought from and granted by the Victorian Schools and the Department of Education (Victoria) (2018_00373). Participants’ gender and age were recorded and their identities were completely anonymized, with a unique ID assigned to each participant.

Measuring Instruments

In order to apply a standardized pressure to the participant’s thumb, a Wagner Force Dial FDK 20 with a 1 cm2 circular rubber end was adopted for each data collection, and the same data collector (AT) completed all measurements. The setup of the collection included a simple table and chair. The participant sat on the chair with the thumb on the edge of the table and other fingers underneath. A vertical pressure (8.5 kg/cm2) using the Force Dial was applied to the midpoint of the thumb for 3 seconds (Figure 1).

Figure 1. Wagner Force Dial pressure applied on the participant’s thumb.
View this figure

The chosen amount of pressure (kg/cm2) applied to the participant’s thumb was previously successfully used by Escalona-Marfil et al and provides light enough pressure to mimic symptoms of mild pain while not being too extensive as to generate stronger pain or skin damage [17,21]. After application of pressure, the participant was asked to randomly complete either the pVAS (Figure 2) or eVAS (Figure 3).

Figure 2. Standard paper visual analog scale that was printed on a white background A4-sized paper.
View this figure
Figure 3. The electronic visual analog scale accessible via the Interactive Clinics app developed by BitGenoma Ltd, Spain.
View this figure

The order of measurement using the pVAS and eVAS was block randomized (groups of 10). Sequential allocation was achieved using a freely available random number generator [22]. Allocation concealment for the eVAS and pVAS sequence was achieved by using sequentially numbered, opaque, and sealed envelopes. Both sequential generation and allocation concealment were conducted by an independent research team member (AC) who was not involved during the data collection process and did not have any prior or ongoing contact with enrolled participants.

With regard to the pVAS, participants were asked to draw a vertical line that corresponded with their symptom level, with the left side corresponding to “no pain” and the right side corresponding to “the worst pain imaginable” [21]. The traditional pVAS was a 12 × 7.5-cm white paper sheet with a 10-cm horizontal line drawn and two vertical 6-cm lines drawn either side. The eVAS was accessible via the “Interactive Clinics” app, and all recordings were conducted on a 7-inch (17.8 cm) Samsung Galaxy Tab 3 with Android operating system (v5.1.1), displaying a 13.5-cm straight horizontal line on a white background. When using the eVAS, the tablet was placed horizontally on the table at all times and each participant was asked to place one finger on the line on the screen [17]. In order to prevent bias, the data collector (AT) wiped the tablet screen completely between measurements to prevent the participant from being able to see the position of the fingerprint previously placed on the tablet’s screen.

After a period of 5 minutes, the sequence of data collection was reversed for each individual. Data collection only took approximately 10-15 minutes in total for each participant, and no follow-up was required. The use of standardized pressure application with 1-minute intervals was reliable for absolute pressure thresholds in multiple studies [23,24]. Additionally, simple pressure algometry is a repeatable measure of pain threshold [25,26]. A 5-minute interval was introduced to reduce possible risk of reacting pre-emptively to stimuli and to prevent temporal summation that could have impacted the quality of the assessment. For logistical reasons, the data collector (AT) was not blinded to this process. However, as soon as the values on the pVAS and eVAS were recorded, the tablet or the paper was immediately withheld from the participant in order to prevent any possible modifications to the data entered by the participant. The results gathered from the pVAS were extrapolated (by AT) using a standard plastic ruler, whereas the eVAS value was automatically calculated by the software.

Statistical Analysis

For a 5% two-sided t test with α=.05 and 80% power in an observational cross-sectional study with one intervention observation and a moderate effect size, it was estimated that a total of 100 subjects would be required [27,28]. The study was overpowered to an estimated 110 subjects (55 aged 10-18 years and 55 aged 18-75 years) to allow for a 9% dropout rate during the data collection period. Summary statistics for eVAS and pVAS results were calculated by splitting the measurement and method. Two approaches have been used to evaluate the agreement of the two methods (intermethod and intramethod reliabilities by means of intraclass correlation coefficients). STATA 15 (StataCorp LLC) was utilized for statistical analysis [29]. The independent statistician was blinded to the eVAS and pVAS allocation concealment and to the participant identity.

Intermethod and Intramethod Agreement Analysis

A mixed factorial model was employed to derive two intraclass correlation coefficients according to Shrout-Fleiss reliability fixed set [29,30]. One coefficient was a measure of intermethod reliability (ρ) estimated by the intraclass correlation coefficient (ICC). This coefficient was defined as the correlation between VAS values from different methods in the same subject and same replication. The other intraclass coefficient (γ) estimated by the average ICC (ICCa) was used as a measure of intramethod reliability. This was defined as the correlation between VAS values in the same method and same subject. A two-way balanced mixed analysis of variance model without interaction, a random subject effect, and a fixed method effect was fitted to estimate ICCs. The mean of squares for subjects, subject-method interactions, and errors from components of variance were also calculated [31]. Statistical inference of the ICCs was performed with CIs. In order to improve reliability coefficients, 95% CIs were calculated from the estimated sum of squares. For both intermethod reliability and intramethod reliability, the ICCs were higher than 0.8. In order to specify the precision of the estimated ICC, the length of the 95% CI was expressed as a function of the ICC value. Given that it was not possible to increase the number of methods to evaluate the VAS, the number of subjects was increased. In children and adolescents, with 94 ratings per method (47 subjects with two replicates per subject) and an anticipated ICC value of at least 0.8, an acceptable length for the 95% CI will be less than or equal to 0.2. In adults, with 124 ratings per method (62 subjects with two replicates per subject) and an anticipated ICC value of at least 0.8, an acceptable length for the 95% CI will be less than or equal to 0.1. Good agreement among methods was evaluated by plotting both methods against subjects and performing a Wilcoxon rank-sum test. According to Portney and Watkins, ICC values are classified as follows: ICC < 0.5, poor; 0.5 ≤ ICC ≤ 0.75, moderate; 0.75 < ICC ≤ 0.9, good; and ICC > 0.9, excellent [30].

Participant Characteristics

A total of 109 participants were included in the study. The study population consisted of 47 children and adolescents (mean age 13.9 years, SD 2.89 years; range 10-18 years; 16 female and 31 male participants) and 62 adults (mean age 42.44 years, SD 14.50 years; range 19-73 years; 37 female and 25 male participants) (Table 1 and Table 2, respectively). No participants were lost to the analysis.

Table 1. Summary statistics (visual analog scale measurements) for children and adolescents.
VariableFirst measureSecond measure
Mean value1.6925531.6574471.7746811.642979
SD value0.9773311.0391771.039741.000408
Minimum value0.
Maximum value3.654.004.424.00

aeVAS: electronic visual analog scale.

bpVAS: paper visual analog scale.

cP50: middle estimate.

Table 2. Summary statistics (visual analog scale measurements) for adults.
VariableFirst measureSecond measure
Mean value1.7383871.6903231.8198391.759677
SD value1.5506111.5717231.7484861.743415
Minimum value0.
Maximum value7.918.108.268.50

aeVAS: electronic visual analog scale.

bpVAS: paper visual analog scale.

cP50: middle estimate.

Analysis of the eVAS and pVAS in Children, Adolescents, and Adults

Tables 1 and 2 show summary statistics for VAS measurements by measurement order and instrument (eVAS and pVAS) for the child and adolescent group and adult group, respectively. Differences between methods for median values ranged from 0.05 to 0.15 in children and adolescents and from 0.035 to 0.145 in adults. In Figures 4 and 5, the scatter plots for eVAS compared with pVAS are displayed for the child and adolescent group and the adult group, respectively, showing agreement between the two methods.

It is possible to observe in adults, the dispersion from values of VAS around 3, which is corrected when taking natural logarithms into account (Figure 6).

Figure 4. Scatter plot of the data in children and adolescents. Points are represented by subject number. eVAS: electronic visual analog scale; pVAS: paper visual analog scale.
View this figure
Figure 5. Scatter plot of the data in adults. Points are represented by subject number. eVAS: electronic visual analog scale; pVAS: paper visual analog scale.
View this figure
Figure 6. Scatter plot of the data (log) in adults. Points are represented by subject number. eVAS: electronic visual analog scale; pVAS: paper visual analog scale.
View this figure

Figures 7 and 8 show the agreement between the methods for children and adolescents, and adults, respectively.

The two-sample Wilcoxon rank-sum test for comparing methods was not significant (children and adolescents, P=.48; adults, P=.73). The normality of residuals of the model for the child and adolescent group and the adult group showed a centered distribution (children and adolescents: Shapiro-Francia test, P=.05 and Shapiro-Wilks test, P=.06; adults: Shapiro-Francia test, P=.13 and Shapiro-Wilks test, P=.21).

In children and adolescents, the intermethod reliability estimated by ICC reached the value of 0.80 (95% CI 0.70-0.87), indicating moderate-to-good reliability. The intramethod reliability estimated by ICCa reached the value of 0.80 (95% CI 0.69-0.87), indicating moderate-to-good reliability. For both coefficients, the length of the interval was less than 0.2.

Figure 7. Rating data for the two methods in children and adolescents. eVAS: electronic visual analog scale; pVAS: paper visual analog scale.
View this figure

In adults, the intermethod reliability estimated by ICC reached the value of 0.94 (95% CI 0.91-0.96), indicating excellent reliability. The intramethod reliability estimated by ICCa reached the value of 0.94 (95% CI 0.91-0.96), indicating excellent reliability [32]. For both coefficients, the length of the interval was 0.1 or less.

Figure 8. Rating data for the two methods in adults. eVAS: electronic visual analog scale; pVAS: paper visual analog scale.
View this figure

Principal Findings

In the adult population, this study supported the interchangeability of the eVAS and pVAS owing to excellent intermethod and intramethod reliabilities as determined by an ICC value of 0.94 (95% CI 0.91-0.96) and ICCa value of 0.94 (95% CI 0.91-0.96), respectively. This supports previous findings by Bird et al who also reported excellent reliability in older adults (age range 65-85 years), using an apple iPad eVAS [11]. The eVAS also demonstrated excellent reliability, with regard to both individual ICC 0.90 (95% CI 0.82-0.95) and ICCa 0.97 (95% CI 0.95-1.0). Within a child and adolescent population, this study recognized the interchangeability of the eVAS and pVAS owing to moderate-to-good intermethod and intramethod reliabilities as determined by an ICC value of 0.80 (95% CI 0.70-0.87) and ICCa value of 0.80 (95% CI 0.69-0.87), respectively. Although this methodology has never been performed within this age range, Sánchez-Rodríguez et al reported that the mobile app “Painometer,” which includes a 100-mm VAS scale, was concordant with its traditional counterparts from ages 12 to 19 years [13].

There are multiple methodological strengths of this study. Specifically, as previously recommended by Escalona-Marfil et al, this trial included block randomization during data collection for the eVAS and pVAS allocation sequence and also introduced a 5-minute interval between measures to reduce possible pain recall bias. This study suggested that fingerprints might remain visible on the screen during the eVAS recordings [17]. The current Australian study prevented traceable fingerprints by wiping clean the tablet’s screen at the end of each recording. For the first time, this study reports the interchangeability of using eVAS compared with pVAS in pediatric participants from 10 years of age.

Occasionally, when using the traditional pVAS, patients may draw the line before the zero (0) or after the 100 point; therefore, these scores become invalid. Instead, the introduction of the eVAS app as part of regular pain evaluation prevents patients from recording their pain level outside the pVAS line, subsequently avoiding invalid scores.

For clinicians and researchers, especially those involved in community nursing or domiciliary visits, the time, cost, and space savings of data storage using the eVAS may be considerable when compared with the traditional pVAS, where manual transcription into clinical notes is required. The proposed eVAS app allows for automatic calculation of the VAS score, preventing possible human errors while using a ruler.

Patients may draw multiple lines on the pVAS or use inappropriate pens or thick highlighters. This can become confusing for clinicians to thoroughly interpret and record the intended results. A recent scoping review of systematic reviews highlighted that mHealth and hand-held electronic devices allow for accurate and complete medical documentation, providing instant access to reliable health data that may support clinical decision making [33]. Novel mHealth tools, such as the eVAS app, may make the work of health professionals even more efficient and increase reliability during their clinical assessments.

The eVAS app allows for the objective monitoring and recording of patients’ pain levels. This mHealth tool might become advantageous for those patients living in geographically remote areas, where limited access to specialists is apparent. Patients and parents/caregivers may not always be required to visit the hospital, consequently saving the time and money required to travel long distances from rural areas. If promptly introduced within different pediatric and adult pain clinic services, the eVAS may support early pain detection, preventing incidences of unnecessary prolonged pain, with a consequent improvement in the patient’s quality of life. This improved clinical management of pain may also lead to a reduction in absence from school or work.

Although not utilized within this study for data analysis, the eVAS app is capable of recording the time and day when the measures are taken. This important feature could be integrated within clinical settings and automatically reported within patient clinical records to highlight any diurnal variation in pain perception. Future trials could therefore also investigate the possible fluctuation of pain within a day. Notably, this may provide greater understanding of the complex nature of pain in response to environmental conditions or treatment plans [34].

This study further adds to the growing body of evidence that supports the use of digital technologies in health care. eHealth and mHealth have already been extensively used as tools for education, diagnosis, and management of pathologies such as diabetes [35,36], pediatric rheumatology [37], polycystic ovarian syndrome [38], and alcoholism [39]. At present, there are limited approaches available that combine evidence-based practice with health apps [40]. Portelli et al reported that most of the current apps available for pain management are rarely supported by an evidence base and may be misleading with their claims [41]. In addition, there are still limited regulations regarding data privacy for information collected from these apps [42]. Alarmingly, Blenner et al highlighted that many apps for diabetes management sold data to third parties without disclosure, even with a privacy policy stating that data were not going to be shared for commercial benefit [43]. This indicates the importance of further high-quality research into mHealth and eHealth regarding data privacy. More effort is also needed with regard to educating patients and practitioners in the use of apps that fully adhere to the guidelines clearly set by the WHO in evaluating digital health outcomes [18,44,45].


There are some limitations that should be considered while interpreting the findings of this study. First, despite a recruitment effort, a balanced number of children and adolescents (n=47, 43%) and adults (n=62, 57%) was not obtained. This was due to logistical school issues in obtaining signed consent forms from parents during busy school terms. The relatively smaller pediatric sample size may have had an impact on the overall ICC values obtained from the child and adolescent population, in comparison with the adult population. Second, it should be noted that data were collected in a convenient sample of people from the community who were not exposed to high levels of pressure pain with the Wagner Force Dial. Future studies may consider testing the eVAS with different intensities of pressure that would be deemed ethically acceptable. Third, a 1-minute interval is typically considered a suitable time gap to measure pain generated by pressure application [22,23]. During this trial, a 5-minute gap was adopted to record the pain generated by the pressure application (8.5 kg/cm2) on the participant’s thumb. The 5-minute gap was chosen to reduce any possible pain recall, especially among pediatric participants and to allow full recovery of sensory function of the thumb. There is possible anchoring bias on repetition of the two tests within a 5-minute interval. A 1-minute interval is deemed appropriate to assess a noxious stimulus without temporal summation [22,23,26]. Within the concept of pain measurement, recall bias may include a psychosocial aspect acknowledging that pain may be amplified or reduced after an extended period of time [46]. Additionally, pain perception within a population may fluctuate day to day [47]. However, for the purpose of this cross-sectional study and for determining interchangeability between the pVAS and eVAS, it is imperative that a single controlled stimulus is used and that measurements are undertaken within the same environment to eliminate confounding. Finally, although the child and adolescent population had slightly lower reliability relative to the adult population, this evidence supports the use of the eVAS in the pediatric population. A possible cause for the lower reliability relative to the adult population is the difference in scale length across platforms. The eVAS line width was 13.5 cm compared with 10 cm in the pVAS. Conceptual understanding of scales may differ between adult, and child and adolescent groups [48]. To encourage consistent conceptual understanding of the study in a large age range (10-75 years), the participants were made aware of the difference in sizes of the scales and asked to mark in a ratio. It is plausible that owing to the possible limited understanding of spatial ratio, there might be an impact on the results from younger participants [48].

In conclusion, the use of technology by children, adolescents, and adults is growing and is evident across multiple settings [49,50]. This study highlights the need for further investigation regarding the transferability of an eVAS pain app to different smartphone and tablet screen sizes that are already largely accessible within the community.

Clinical Implications

Monitoring and evaluating digital health interventions can be challenging, but have become requirements within the mHealth and eHealth fields [18]. This study specifically supports the adoption of these easy-to-use and validated pain assessment mHealth methods that have excellent reliability in adults and moderate-to-good reliability in children and adolescents. The emerging field of digital health presents an evolving cultural shift within health care settings. The growing use of digital mHealth has the potential to improve pain management. eHealth and mHealth have the ability to improve adherence to pain reporting [51,52], allow real-time data capture [53], and improve communication between practitioners and patients [54].


This study provides supporting evidence on the interchangeability of the eVAS and pVAS in child and adolescent, and adult populations. The introduction of similar validated eVAS pain apps may greatly increase the quality of reliable data accessible to clinicians, thereby improving the well-being of symptomatic patients. Most importantly, the use of mHealth in pain management may also facilitate timely clinical decisions, improve patients’ self-management and overall awareness in the progression of their pain levels, and become an integrated approach consistent with the eHealth goals of the WHO and Australian Health Authorities [18,19]. Further research is needed on the use of these pain apps among symptomatic children, adolescents, and adults to ascertain the possible impacts of this new technology in these populations [55,56].


The authors would like to thank John Monash Science School (Clayton, Victoria) and KBH Brumbies Hockey Club (Mont Albert North, Victoria) for their generous support of this project. In particular, we would like to thank KBH Brumbies Hockey Club president Mike Hanlon and coach Daniel Brennan; the principal team of JMSS Peter Corkill, Sally Cheah-Johnson, and Andrew Chisholm; and JMSS teaching staff Adriana Abels and Peter Thompson for their enthusiasm and support.

Conflicts of Interest

None declared.

  1. Rabbitts JA, Holley AL, Groenewald CB, Palermo TM. Association Between Widespread Pain Scores and Functional Impairment and Health-Related Quality of Life in Clinical Samples of Children. The Journal of Pain 2016 Jun;17(6):678-684. [CrossRef]
  2. Hersh AO, Salimian PK, Weitzman ER. Using Patient-Reported Outcome Measures to Capture the Patient's Voice in Research and Care of Juvenile Idiopathic Arthritis. Rheumatic Disease Clinics of North America 2016 May;42(2):333-346. [CrossRef]
  3. Shields BJ, Cohen DM, Harbeck-Weber C, Powers JD, Smith GA. Pediatric Pain Measurement Using a Visual Analogue Scale: A Comparison of Two Teaching Methods. Clin Pediatr (Phila) 2016 Jul 02;42(3):227-234. [CrossRef]
  4. Hicks CL, von Baeyer CL, Spafford PA, van Korlaar I, Goodenough B. The Faces Pain Scale – Revised: toward a common metric in pediatric pain measurement. Pain 2001;93(2):173-183. [CrossRef]
  5. Tomlinson D, von Baeyer CL, Stinson JN, Sung L. A systematic review of faces scales for the self-report of pain intensity in children. Pediatrics 2010 Nov;126(5):e1168-e1198. [CrossRef] [Medline]
  6. Ruskin D, Lalloo C, Amaria K, Stinson JN, Kewley E, Campbell F, et al. Assessing Pain Intensity in Children with Chronic Pain: Convergent and Discriminant Validity of The 0 To 10 Numerical Rating Scale in Clinical Practice. Pain Research and Management 2014;19(3):141-148. [CrossRef]
  7. Zisk-Rony RY, Lev J, Haviv H. Nurses' Report of In-hospital Pediatric Pain Assessment: Examining Challenges and Perspectives. Pain Management Nursing 2015 Apr;16(2):112-120. [CrossRef]
  8. Quinn BL, Sheldon LK, Cooley ME. Pediatric Pain Assessment by Drawn Faces Scales: A Review. Pain Management Nursing 2014 Dec;15(4):909-918. [CrossRef]
  9. Davis TL, DiClemente R, Prietula M. Taking mHealth Forward: Examining the Core Characteristics. JMIR Mhealth Uhealth 2016 Aug 10;4(3):e97 [FREE Full text] [CrossRef] [Medline]
  10. Ryu S. Book Review: mHealth: New Horizons for Health through Mobile Technologies: Based on the Findings of the Second Global Survey on eHealth (Global Observatory for eHealth Series, Volume 3). Healthc Inform Res 2012;18(3):231. [CrossRef]
  11. Bird M, Callisaya ML, Cannell J, Gibbons T, Smith ST, Ahuja KD. Accuracy, Validity, and Reliability of an Electronic Visual Analog Scale for Pain on a Touch Screen Tablet in Healthy Older Adults: A Clinical Trial. Interact J Med Res 2016 Jan 14;5(1):e3. [CrossRef]
  12. Stinson JN, Lalloo C, Harris L, Isaac L, Campbell F, Brown S, et al. iCanCope with Pain™: User-Centred Design of a Web- and Mobile-Based Self-Management Program for Youth with Chronic Pain Based on Identified Health Care Needs. Pain Research and Management 2014;19(5):257-265. [CrossRef]
  13. Sánchez-Rodríguez E, de la Vega R, Castarlenas E, Roset R, Miró J. An App for the Assessment of Pain Intensity: Validity Properties and Agreement of Pain Reports When Used with Young People. Pain Med 2015 Oct 01;16(10):1982-1992. [CrossRef]
  14. Sun T, West N, Ansermino JM, Montgomery CJ, Myers D, Dunsmuir D, et al. A smartphone version of the Faces Pain Scale-Revised and the Color Analog Scale for postoperative pain assessment in children. Paediatr Anaesth 2015 Oct 28;25(12):1264-1273. [CrossRef]
  15. Stinson JN, Stevens BJ, Feldman BM, Streiner D, McGrath PJ, Dupuis A, et al. Construct validity of a multidimensional electronic pain diary for adolescents with arthritis. Pain 2008 Jun;136(3):281-292. [CrossRef] [Medline]
  16. Luca NJ, Stinson JN, Feldman BM, Benseler SM, Beaton D, Campillo S, et al. Validation of the Standardized Universal Pain Evaluations for Rheumatology Providers for Children and Youth (SUPER-KIDZ). J Orthop Sports Phys Ther 2017 Oct;47(10):731-740. [CrossRef]
  17. Escalona-Marfil C, Coda A, Ruiz-Moreno J, Riu-Gispert L, Gironès X. Validation of an Electronic Visual Analog Scale mHealth Tool for Acute Pain Assessment: Prospective Cross-Sectional Study. J Med Internet Res 2020 Feb 12;22(2):e13468 [FREE Full text] [CrossRef] [Medline]
  18. World Health Organization. Monitoring and evaluating digital health interventions: a practical guide to conducting research and assessment. Geneva, Switzerland: World Health Organization; 2016.
  19. eHealth Strategy for NSW Health: 2016-2026. NSW Government.   URL: [accessed 2020-09-02]
  20. Shields BJ, Palermo TM, Powers JD, Grewe SD, Smith GA. Predictors of a child's ability to use a visual analogue scale. Child Care Health Dev 2003 Jul;29(4):281-290 [FREE Full text] [CrossRef] [Medline]
  21. Breivik H, Borchgrevink P, Allen S, Rosseland L, Romundstad L, Breivik Hals E, et al. Assessment of pain. British Journal of Anaesthesia 2008 Jul;101(1):17-24. [CrossRef]
  22. RANDOM.ORG.   URL: [accessed 2019-01-01]
  23. Cole LJ, Farrell MJ, Gibson SJ, Egan GF. Age-related differences in pain sensitivity and regional brain activity evoked by noxious pressure. Neurobiology of Aging 2010 Mar;31(3):494-503. [CrossRef]
  24. Balaguier R, Madeleine P, Vuillerme N. Intra-session absolute and relative reliability of pressure pain thresholds in the low back region of vine-workers: effect of the number of trials. BMC Musculoskelet Disord 2016 Aug 18;17(1). [CrossRef]
  25. Buchanan HM, Midgley J. Evaluation of pain threshold using a simple pressure algometer. Clin Rheumatol 1987 Dec;6(4):510-517. [CrossRef]
  26. Frank L, McLaughlin P, Vaughan B. The repeatability of pressure algometry in asymptomatic individuals over consecutive days. International Journal of Osteopathic Medicine 2013 Sep;16(3):143-152 [FREE Full text] [CrossRef]
  27. Noordzij M, Tripepi G, Dekker FW, Zoccali C, Tanck MW, Jager KJ. Sample size calculations: basic principles and common pitfalls. Nephrology Dialysis Transplantation 2010 Jan 12;25(5):1388-1393. [CrossRef]
  28. Gignac GE, Szodorai ET. Effect size guidelines for individual differences researchers. Personality and Individual Differences 2016 Nov;102:74-78. [CrossRef]
  29. Stata: Software for Statistics and Data Sciences.   URL: [accessed 2019-05-02]
  30. Portney L, Watkins M. Foundations of Clinical Research: Applications to Practice. London, England: Pearson; 2000.
  31. Gwet K. Handbook of Inter-Rater Reliability: The Definitive Guide to Measuring the Extent of Agreement Among Raters. California, United States: Advanced Analytics; 2014.
  32. Koo TK, Li MY. A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research. J Chiropr Med 2016 Jun;15(2):155-163 [FREE Full text] [CrossRef] [Medline]
  33. Mickan S, Tilson JK, Atherton H, Roberts NW, Heneghan C. Evidence of effectiveness of health care professionals using handheld computers: a scoping review of systematic reviews. J Med Internet Res 2013;15(10):e212 [FREE Full text] [CrossRef] [Medline]
  34. Hawker GA, Mian S, Kendzerska T, French M. Measures of adult pain: Visual Analog Scale for Pain (VAS Pain), Numeric Rating Scale for Pain (NRS Pain), McGill Pain Questionnaire (MPQ), Short-Form McGill Pain Questionnaire (SF-MPQ), Chronic Pain Grade Scale (CPGS), Short Form-36 Bodily Pain Scale (SF-36 BPS), and Measure of Intermittent and Constant Osteoarthritis Pain (ICOAP). Arthritis Care Res (Hoboken) 2011 Nov;63 Suppl 11:S240-S252. [CrossRef] [Medline]
  35. Cafazzo JA, Casselman M, Hamming N, Katzman DK, Palmert MR. Design of an mHealth app for the self-management of adolescent type 1 diabetes: a pilot study. J Med Internet Res 2012 May;14(3):e70 [FREE Full text] [CrossRef] [Medline]
  36. Coda A, Sculley D, Santos D, Girones X, Acharya S. Exploring the Effectiveness of Smart Technologies in the Management of Type 2 Diabetes Mellitus. J Diabetes Sci Technol 2017 May 25;12(1):199-201. [CrossRef]
  37. Coda A, Sculley D, Santos D, Girones X, Brosseau L, Smith DR, et al. Harnessing interactive technologies to improve health outcomes in juvenile idiopathic arthritis. Pediatr Rheumatol Online J 2017 May 16;15(1):40 [FREE Full text] [CrossRef] [Medline]
  38. Boyle JA, Xu R, Gilbert E, Kuczynska-Burggraf M, Tan B, Teede H, et al. Ask PCOS: Identifying Need to Inform Evidence-Based App Development for Polycystic Ovary Syndrome. Semin Reprod Med 2018 Dec;36(1):59-65. [CrossRef] [Medline]
  39. Cohn AM, Hunter-Reel D, Hagman BT, Mitchell J. Promoting behavior change from alcohol use through mobile technology: the future of ecological momentary assessment. Alcohol Clin Exp Res 2011 Dec;35(12):2209-2215 [FREE Full text] [CrossRef] [Medline]
  40. Lalloo C, Jibb LA, Rivera J, Agarwal A, Stinson JN. "There's a Pain App for That": Review of Patient-targeted Smartphone Applications for Pain Management. Clin J Pain 2015 Jun;31(6):557-563. [CrossRef] [Medline]
  41. Portelli P, Eldred C. A quality review of smartphone applications for the management of pain. British Journal of Pain 2016 Jul 08;10(3):135-140. [CrossRef]
  42. Rosenfeld L, Torous J, Vahia IV. Data Security and Privacy in Apps for Dementia: An Analysis of Existing Privacy Policies. Am J Geriatr Psychiatry 2017 Aug;25(8):873-877. [CrossRef] [Medline]
  43. Blenner SR, Köllmer M, Rouse AJ, Daneshvar N, Williams C, Andrews LB. Privacy Policies of Android Diabetes Apps and Sharing of Health Information. JAMA 2016 Mar 08;315(10):1051-1052. [CrossRef] [Medline]
  44. Zaidan BB, Haiqi A, Zaidan AA, Abdulnabi M, Kiah ML, Muzamel H. A security framework for nationwide health information exchange based on telehealth strategy. J Med Syst 2015 May 3;39(5):51. [CrossRef] [Medline]
  45. World Health Organization. Classification of digital health interventions v1.0: a shared language to describe the uses of digital technology for health. Geneva, Switzerland: World Health Organization; 2018.
  46. Isselée H, De LA, Bogaerts K, Lysens R. Long-term fluctuations of pressure pain thresholds in healthy men, normally menstruating women and oral contraceptive users. Eur J Pain 2001;5(1):27-37. [CrossRef] [Medline]
  47. Hutchings A, Calloway M, Choy E, Hooper M, Hunter DJ, Jordan JM, et al. The Longitudinal Examination of Arthritis Pain (LEAP) study: relationships between weekly fluctuations in patient-rated joint pain and other health outcomes. J Rheumatol 2007 Nov;34(11):2291-2300. [Medline]
  48. Sophian C. Perceptions of proportionality in young children: matching spatial ratios. Cognition 2000 May 15;75(2):145-170. [CrossRef] [Medline]
  49. Edwards S, Henderson M, Gronn D, Scott A, Mirkhil M. Digital disconnect or digital difference? A socio-ecological perspective on young children’s technology use in the home and the early childhood centre. Technology, Pedagogy and Education 2016 Apr 06;26(1):1-17. [CrossRef]
  50. Digital lives of older Australians. Australian Communications and Media Authority. 2020.   URL: [accessed 2020-06-09]
  51. Stone AA, Broderick JE, Schwartz JE, Shiffman S, Litcher-Kelly L, Calvanese P. Intensive momentary reporting of pain with an electronic diary: reactivity, compliance, and patient satisfaction. Pain 2003 Jul;104(1-2):343-351. [CrossRef] [Medline]
  52. Palermo TM, Valenzuela D, Stork PP. A randomized trial of electronic versus paper pain diaries in children: impact on compliance, accuracy, and acceptability. Pain 2004 Feb;107(3):213-219. [CrossRef] [Medline]
  53. Stinson JN, Jibb LA, Nguyen C, Nathan PC, Maloney AM, Dupuis LL, et al. Construct validity and reliability of a real-time multidimensional smartphone app to assess pain in children and adolescents with cancer. Pain 2015 Dec;156(12):2607-2615. [CrossRef] [Medline]
  54. Jacob E, Pavlish C, Duran J, Stinson J, Lewis MA, Zeltzer L. Facilitating pediatric patient-provider communications using wireless technology in children and adolescents with sickle cell disease. J Pediatr Health Care 2013 Jul;27(4):284-292 [FREE Full text] [CrossRef] [Medline]
  55. Jamison RN, Jurcik DC, Edwards RR, Huang C, Ross EL. A Pilot Comparison of a Smartphone App With or Without 2-Way Messaging Among Chronic Pain Patients: Who Benefits From a Pain App? Clin J Pain 2017 Aug;33(8):676-686 [FREE Full text] [CrossRef] [Medline]
  56. Sánchez-Rodríguez E, Castarlenas E, de la Vega R, Roset R, Miró J. On the electronic measurement of pain intensity: Can we use different pain intensity scales interchangeably? J Health Psychol 2017 Nov 09;22(13):1658-1667. [CrossRef] [Medline]

AHP: allied health professional
eVAS: electronic visual analog scale
pVAS: paper visual analog scale
VAS: visual analog scale
WHO: World Health Organization

Edited by G Eysenbach; submitted 17.02.20; peer-reviewed by M Lozano-Lozano, R Cruz Martínez; comments to author 28.03.20; revised version received 09.07.20; accepted 26.07.20; published 17.09.20


©Alexandra Turnbull, Dean Sculley, Carles Escalona-Marfil, Lluís Riu-Gispert, Jorge Ruiz-Moreno, Xavier Gironès, Andrea Coda. Originally published in the Journal of Medical Internet Research (, 17.09.2020.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.