The Rey-Osterrieth Complex Figure Test (RCFT) is widely used to examine the visuoconstruction and visual memory function of patients with brain injuries or cognitive impairment such as Alzheimer disease (AD) [1]. The conventional scoring system of RCFT focuses on scoring the final finished drawing of the figure by assessing the shape and positional accuracy of its elements. There are only a few scoring methods that quantify the drawing sequences of elements [2-4]. However, patients with cognitive impairment due to brain injuries show different patterns in the drawing process and errors on the final finished drawing compared with normal controls (NC; ie, individuals with normal cognition) [5]. For example, some patients with brain injuries complete a figure by adding one detailed part after another instead of starting with an overall outline and adding local or detailed parts. Furthermore, the number, length, and speed of strokes made by patients with brain injuries may differ from that of NCs. Therefore, it is necessary to establish a scoring system using a digital device that records not only the spatial arrangement of the finished drawing but also all the details of drawing processes, including the sequence in drawing the parts of the figures. This approach will augment our understanding of the organizing strategy and executive functions of patients with brain injuries. Additionally, it will increase our understanding of the structural and design integrity of drawing figures.

A digital pen and tablet can be optimal tools for observing and acquiring data for the drawing process. Recent studies have used a digital device to evaluate the visuoconstructional abilities and executive functions of patients with AD. These researchers used a clock drawing test [6,7] or a 3D house copy test [8] to quantitatively evaluate cognitive function in patients with AD. It was possible to differentiate patients with mild cognitive impairment (MCI) and AD from NCs by assessing physical parameters such as kinetics (eg, time, velocity) or device-human interaction values (eg, pressure, number/density of strokes). Compared with NCs, even patients with MCI who received normal scores on the clock drawing test showed prolonged transition times associated with drawing performances [6]. The kinetics and sequence-based schemes from digital drawing tests have successfully demonstrated changes in cognitive progress. Previously, only 1 study has analyzed RCFT using a digital device; however, the group did not assess the drawing procedure but focused on recognizing the outline and details in relation to the conventional scoring system [9].

To implement the scheme in a digital device, we first decided to choose an abstract figure such as the RCFT rather than concrete objects such as a clock or a house. Clocks and houses elicit semantic knowledge, which primarily involves the ventral visual pathway (what pathway). Using complex figures on the other hand, such as the RCFT, involves the dorsal visual pathway (where pathway) and requires visuospatial working memory before translation into a motor program and execution of plan [10]. Second, we simplified the Rey figure to reduce not only the drawing time but also the number of strokes required to complete the figure. Arranging too many structures in a limited space may result in stroke overlap when drawing, which may prevent the digital device from identifying each stroke.

In this study, we hypothesized that the movement kinetics (pen tip trajectory of the digital pen) and digitized spatial information acquired from the simplified RCFT would differ between NCs and patients with AD. Several previous studies have reported that patients with early-onset AD (EOAD) have significant visuospatial or visuoconstructive difficulties compared with patients with late-onset AD (LOAD) [11-13]. We therefore hypothesized that patients with EOAD would show more pronounced changes in movement kinetics and digitized spatial information compared with patients with LOAD.

Participants were selected from those who visited the Memory Disorders Clinic at the Samsung Medical Center in Seoul, Republic of Korea, between March 1 and December 1, 2017. The study group comprised 17 patients with EOAD, 21 patients with LOAD, and 17 NC individuals. We consecutively recruited participants who satisfied the following criteria: (1) normal visual acuity, (2) >6 years of education, (3) completion of a standardized neuropsychological battery called the Seoul Neuropsychological Screening Battery (SNSB) [14,15] and the Mini-Mental State Examination (MMSE) test [16], and (4) magnetic resonance imaging (MRI) results. Participants were classified via rigorous diagnostic methods, including multiple tests (eg, blood test, MRI/positron emission tomography scans) and clinical consensus of neurologists, neuropsychologists, and radiologists. All patients with AD fulfilled the criteria proposed by the National Institute of Neurological and Communicative Disorders and Stroke and the Alzheimer’s Disease and Related Disorders Association (NINCDS-ADRDA) [17]. Patients with EOAD were defined as those whose first symptoms occurred at an early age (>45 years and <65 years of age). Individuals with moderate or severe vision loss (visual acuity <0.3) or those with very low MMSE scores with lower cutoffs of 10 or education levels <6th grade were not included in this study.

All participants underwent a standardized neuropsychological battery called the SNSB [14,15], which consists of tests for attention, language, calculation, visuospatial, memory, and frontal/executive functions. The MMSE and Clinical Dementia Rating tests were also carried out to evaluate general cognition. The standardized tests used for this study were as follows: attention was assessed using the backward digit span and letter cancellation tests; language was assessed using the Korean version of the Boston Naming Test; calculation was assessed using 3 items comprising each for addition, subtraction, multiplication, and division; visuospatial function was assessed using the RCFT; memory function was assessed using immediate and delayed recall of the Seoul Verbal Learning Test and RCFT; frontal-executive function was assessed using the phonemic and semantic Controlled Oral Word Association Test and the Stroop word/color reading test. Each score was converted into a standardized Z-score based on age- and education-adjusted norms.

We modified the original RCFT into a simpler version while maintaining the main frame of the figure. Modification was performed by 1 neurologist (experience >10 years) and 2 neuropsychologists (average experience >20 years) to balance the numbers of global and local components. Although the original RCFT comprises 18 components (4 global and 14 local components) [18], our simplified RCFT comprised 4 global and 4 local components, as illustrated in Figure 1. The large rectangle with horizontal, vertical, and diagonal crosses was maintained, but other regional or local features were simplified except for the 4 horizontal lines in the upper left panel. For the local features, the components in the original RCFT were changed as follows: (1) the diamond was replaced by a square, (2) the circle with 3 dots was replaced by double circles, (3) the 5 parallel lines were replaced by 3 triangles, (4) and the 4 parallel lines were replaced by 4 arrows. We added arrows to these lines such that the participants would be more attentive to the local features. Some overlapping lines and detailed components outside of the outlines were eliminated in the simplified version. For the global components, the side of the large triangle attached to the large rectangle and the horizontal line of the large triangle were excluded. Of the local components, the following were removed: the vertical cross, the small triangle above the large rectangle, the horizontal cross, the square attached to the large rectangle, the small rectangle, the small horizontal line above the small rectangle, the vertical line within the side of the large triangle, and the small vertical line within the large rectangle.

Each of the 8 components in the simplified RCFT was scored separately (Meyers and Meyers protocol) in terms of accuracy and placement [18]. Component scores were assigned as 2 (accurately drawn, correctly placed), 1 (accurately drawn, incorrectly placed or inaccurately drawn, correctly placed), 0.5 (inaccurately drawn, incorrectly placed but recognizable), or 0 (inaccurately drawn, incorrectly placed, unrecognizable). Thus, the possible range of raw scores was 0.0-16.0. A low score suggested impaired visuoperceptual or visuoconstructive functions.

The experimental equipment for measuring the performance of copying a figure consisted of an X-Y digital tablet (size: 12-inch, resolution: 2160 × 1440; Samsung Galaxy Book 12, Samsung Electronics) and a digital pen (nib diameter: 0.7 mm, pressure: 4096; S-pen, Samsung Electronics). Participants performed the simplified RCFT on a tablet in portrait orientation (Figure 1). We defined the upper half of the working area as perceptual space and the lower half as working space. The simplified RCFT was presented in the perceptual space. An eraser tool was not included in the interface to observe the basal level of copying. Before the task, the experimenter delivered the following instructions to the participants: “When the task begins, a figure will appear on the upper half of this tablet. Please copy the figure with this pen. Try to use the empty area below as much as possible. This pen does not have an eraser. You cannot change your drawing. If you made any mistakes, please disregard them and continue with carrying out the task. Take your time, and please let me know when you are finished.” The trajectories of the drawings were recorded on the X-Y digital tablet at a sampling frequency of 60 Hz, and the x- and y-axes were horizontal and vertical planes, respectively.

The digital pen-touch point responses were recorded by the X-Y digital tablet as Cartesian coordinates in pixel units based on placement on the screen, movement, and removal from the screen over time. The range of positions was (0, 0) to (2160, 1440), which corresponded to the range of display in pixels. We normalized the position from a range of –1 to 1 with respect to the origin (0, 0) at the center of the display. The vertical ranges of perceptual and working spaces corresponded to 0 to 1 and –1 to 0, respectively. To consider the robustness of the automatic evaluation, we focused on analyzing parameters such as pen stroke, occupied drawing area (drawing boundary), and copying results (similarity between the original figure).

We analyzed 3 features: (1) digital pen stroke, which was defined as continuous movement of the digital pen while maintaining contact with the writing tablet; (2) drawing boundary, defined as the extremum coordinates; and (3) copying results (similarity of the drawing to the original). These features were selected based on their robustness to automatic evaluation.

For statistical analysis of the demographic and cognitive profile data, the Kolmogorov-Smirnov test was used to conduct normality tests. The Kruskal-Wallis test was used to examine the statistical significance between groups at a significance level of P=.05 because variables did not follow a normal distribution. We performed post hoc comparisons using the pairwise Mann-Whitney U test with Bonferroni correction. We used the chi-square test or Fisher exact test for categorical variables, followed by a Bonferroni post hoc analysis. To validate the simplified Rey figures, we used Pearson correlation to compare conventional RCFT scores and simplified RCFT scores.

For statistical analysis of digital pen data, the Lilliefors test was used to assess the normality and the Kruskal-Wallis test was again used to examine the statistical significance between groups at a confidence level of P=.05. We performed post hoc comparisons using the pairwise Mann-Whitney U test with a Dunn-Šidák correction by adjusting the P value of each pairwise test as p=1–(1–p_o)³, where 3 reflects the number of groups. We compared the immediate/recall/recognition scores of the original RCFT and our digital metrics from the simplified RCFT using Pearson correlation to determine whether there are any data linked to memory function. All hypothesis tests were two-tailed. MATLAB R2018b was used for the calculations.

The demographic and cognitive profiles of the participants are summarized in Table 1. Age (P<.001) and education (P=.10) differed significantly among NCs and participants with EOAD and LOAD. The prevalence of apolipoprotein E4 carriers among NCs (0/13, 0%) was significantly lower than that among patients with EOAD (10/16, 63%) or LOAD (9/20, 45%). To investigate differences in neuropsychological scores among the 3 groups, we used Z scores based on age- and education-adjusted norms. Both patients with EOAD and LOAD showed poor performance in attention, language, visuospatial function, memory, and frontal/executive functions compared with NCs (Table 1).

Before the analysis, digital data were reviewed carefully, and data that were not appropriate were excluded from the analysis. A total of 3 participants were excluded because of mismatched digital pen data between attach/detach and movement pairs; 8 participants were excluded because of missing attach movements in coordinate data; 6 participants were excluded because of mismatched digital pen data between result image and pen movement. Digital data from a total of 11 NCs, 11 patients with EOAD, and 16 patients with LOAD were analyzed.

Table 1

Clinical characteristics.

Table 2 shows the median and interquartile range raw copy scores of the original and simplified RCFT results from NCs, patients with EOAD, and patients with LOAD. The NCs showed significantly higher scores than the 2 AD groups in both the simplified (EOAD: P<.001; LOAD: P<.001) and original RCFT tests (EOAD: P<.001; LOAD: P<.001). There was no significant difference between patients with EOAD and patients with LOAD in the simplified or original RCFT. There was a significant linear relationship between the conventional RCFT and the simplified RCFT score (r=0.884; P<.001; Figure 5).

We compared the immediate/recall/recognition scores of the original RCFT and the digital data acquired from our simplified RCFT to examine whether any of our digital data could be associated with memory function using Pearson correlation. A positive correlation (r=0.353; P=.04) was observed between immediate recall scores of the original RCFT and the coefficients of shape similarity. In addition, immediate recall scores of the original RCFT and vertical positions of the simplified RCFT on the y-axis negatively correlated as follows: vertical position of the center of the whole area (r=–0.341; P=.045), vertical position of the center of the skeletal area (r=–0.336; P=.048), vertical position of the center of the mass (r=–0.339; P=.047), vertical position of the top edge of the whole area (r=–0.343; P=.04), and location of the top edge of the skeleton area (r=–0.350; P=.04). However, there were no significant correlations between recall/recognition scores of the original RCFT and the digital data from the simplified RCFT.

The organizing abilities of the participants were evaluated by analyzing the copying sequences. We converted the original drawing of the simplified Rey figure to pseudocolored images coded by a series of colors in the order of strokes (Multimedia Appendices 1-3). Specifically, the total number of strokes made by each participant was broken down into segments according to the order of strokes, and each segment was coded with a series of colors ranging from red to blue. For instance, when an individual made 40 strokes in total, the first 4 strokes that accounted for the first 10% (4/40) were assigned a red color and the next 10% (4/40) as yellow and so on. Most of the participants drew the global features first and then added the local features, as exemplified in Figure 6. However, some patients started drawing local features such as the double circle or the triangles, as shown in the representative figures (Figure 6) during the middle of drawing global features. In addition, 1 patient with EOAD (number 5 in Multimedia Appendix 2) completed the global structure last. Nonetheless, the tendency of drawing global elements in the beginning was preserved across all subjects. In the unorganized copy as well, participants started with the large box (Figure 6) and then moved onto the local elements.

Clinical interpretation of copying performance in the RCFT was partly based on individual performance in copying. Therefore, we assessed the number, length, and speed of sequential line strokes (Table 3).

To scale the similarity between the original and copied figures, we applied a 2D cross-correlation analysis and calculated its coefficients and shifts in space. Although the conventional rating evaluates the accuracy of the shape and spatial arrangement of parts, the coefficient signifies the overall similarity between the 2 images (Table 3). To check the validity of this new similarity coefficient, we used Pearson correlation to compare the visual rating scores of the simplified RCFT and the similarity coefficients. According to the analysis, the coefficients of the shape similarity positively correlated to the visual rating (r=0.779; P<.001).

For both AD groups, the coefficient representing pattern matching was significantly lower than that of NC individuals (pairwise Mann-Whitney U test: P=.004 for EOAD vs NCs and P=.002 for LOAD vs NCs; Figure 4). However, there was no statistically significant difference between patients with EOAD and patients with LOAD (P=.86). Compared with the target stimulus, several participants drew images that were larger or smaller in size; thus, we normalized the copied figures and then performed a 2D cross-coefficient analysis. We found that the similarity was significantly lower in the patients with LOAD compared with that of the NC individuals even after rescaling the image sizes (P=.04). However, the significance of the difference between the patients with EOAD and the NC individuals disappeared after adjusting for image size (Figure 4).

This study introduced a novel analytic method that uses a digital pen and tablet to evaluate a completed drawing and movement kinetics involved in the drawing process to compare the visuoconstructional abilities of patients with AD and NC individuals. First, we created a simplified RCFT and compared the scores of the simplified Rey figure with those of the original to validate our simplified RCFT. We then compared the 2 AD groups with the NC individuals in terms of the simplified RCFT-derived digital metrics that were analyzed based on 3 aspects: (1) the number, speed, length, and sequence of pen strokes; (2) the spatial arrangement of the drawing; and (3) the similarity between the target picture and the drawing of the participants. Significant differences were noted between the NC and the 2 AD groups. Taken together, our findings suggest that movement kinetics in the drawing process and scores acquired from the finished drawings can serve as useful biomarkers to investigate visuoconstructional dysfunction in AD [20].

We created a simplified RCFT to reduce drawing time and the number of strokes required to complete the picture. In the limited space, reducing the number of strokes makes it easier for the strokes to be identified by the digital device. Compared with the original, a high correlation in the simplified RCFT score was observed in both the whole and the separate groups (NC and the 2 AD groups; Figure 5; Table 2). In line with previous studies that reported strong correlations between simpler and original RCFT in normal adults [21] and patients with AD [22], our findings suggest that the simplified and original RCFTs may be comparable in evaluating visuoconstructional dysfunction in patients with AD. It was interesting to note that there were outliers among those with lower original RCFT scores: scores of 0 to 10 on the original yielded various scores of 1 to 14 on the simplified version (Figure 5). This could have occurred because patients with substantial visuospatial impairment may become overwhelmed when encountering a complex figure so that they eventually become less motivated to complete a figure. However, the same individuals would be relatively more willing to complete a figure if it was less complicated. These findings may also have implications that our simplified figure may be more appropriate for individuals with mild to moderate than severe impairment of cognition or visuospatial abilities.

Regarding the digital pen stroke data analysis, we first compared the 2 AD groups (EOAD and LOAD) with the NC individuals in terms of temporal appearance of long stokes. Overall, the AD groups used fewer long strokes than the NC group. In addition, both the AD groups drew long strokes later in the drawing process compared with that of the NC group. This suggested that the AD groups tended to draw local features earlier and used shorter strokes to finish a figure than the NC individuals. A previous study reported that more than half of the NC individuals first drew the global features of RCFT and then the details or local features subsequently [23]. Other studies suggest that piecemeal approaches in drawing are indicative of brain disease and that they also reflect the inability of an individual to process global information equivalent to that of NC individuals [18].

Additionally, our digital device enabled us to track the sequences of the drawing by color coding the strokes. In the pencil and paper version of the RCFT, neuropsychologists often track copying sequences by giving the patients a series of color pencils. For example, the red color pencil for the first 10 seconds and the yellow pencil for the next 10 seconds, and so on [2]. Similarly, in our digitized images, we assigned colors to the strokes in a certain order. We found that all NC individuals maintained a similar sequence of drawing, starting with the large box and then drawing the diagonal or orthogonal axes. Most patients who successfully copied all the elements also followed the sequence of the NC individuals. Unconventional approaches to drawing, such as drawing details before finishing the global structure or completing the global structure at the end, were identified from some of the patients. Such behavior, however, was not characterized as unimodal features but rather as a portrayal of an individual’s drawing process. Color coding strokes following the completion of a test may be advantageous over using color pencils because switching color pencils will interrupt the drawing process and thus affect the overall memory processing of the participants.

The first and second methods employed in the pen stroke analysis, as described earlier, suggest that analysis using a digital device can allow quantification of the global versus the local nature of drawing in patients with AD. Regarding the anatomical substrates of global and local processing, it is known that the left hemisphere is more specialized in processing local rather than global features, and vice versa for the right hemisphere. This construct, however, is not consistent with our findings as typical AD involves the bilateral temporoparietal area in a symmetrical fashion. Rather, the global element first and local element later strategy may be related to the organizing abilities of individuals that can be mediated by prefrontal cortices, which is one of the brain regions affected in the early stage of AD. Another explanation could be the visual perception hypothesis. Previous studies have reported that the visual perception of normal individuals is a sequential process in which global features are perceived before local features. Another study also had patients with AD read Navon figures (large global digits composed of smaller local digits) and found that patients showed impairment in reading global figures [24]. Therefore, the preference for drawing local features from the AD groups in our study may indicate a mild form of simultanagnosia, which is part of the Balint syndrome.

Another feature we noticed from our pen stroke analysis was that not only the average length of strokes was significantly shorter in patients with LOAD compared with NC individuals, but the average speed of the longest stroke was also significantly slower in patients with LOAD compared with NC individuals (Figure 2). Age-related changes could have contributed to the results because the older the group was, the longer it took for the participants to complete a stroke (shortest to longest in time: EOAD<NC<LOAD). Statistically significant differences were only identified when comparing the NC and LOAD groups. However, we observed another movement kinetics finding that was inconsistent with our age account. Both patients with EOAD and patients with LOAD had longer pause and elapsed times to initiate long strokes compared with the time that NC individuals took (Figure 2). Therefore, decreased length/speed and increased transition and pause time might be related to Parkinsonism or psychomotor speed slowing in both groups of patients with AD. Unfortunately, we did not evaluate the Unified Parkinson’s Disease Rating Scale (UPDRS) score. Therefore, although we excluded patients with Parkinsonism or degenerative disorders that accompany Parkinsonism, such as progressive supranuclear palsy or Lewy body dementia, while enrolling participants, we cannot eliminate the possibility of the Parkinsonism factored in our data.

On the basis of our analysis of the position of the drawing area, patients with AD tended to draw the figures closer toward the perception space containing the target figure. This upward deviation is indicative of the closing-in phenomenon. Previous studies have analyzed copied figures of the alternating square and triangle task and showed that as patients drew from left to right, the drawing approached the target, resulting in a sloping image. The angle of the slope was steeper in patients with AD and vascular dementia in comparison with that of NCs [25,26]. Other studies used the RCFT to analyze the closing-in phenomenon but relied on visual rating rather than quantification [27,28]. To the best of our knowledge, this is the first study to quantify the closing-in phenomenon using the RCFT. We analyzed the x and y coordinates of the drawing area and the input data ratio of the drawing to quantify the closing-in phenomenon (Figure 3). We have successfully demonstrated the difference between patients with AD and normal individuals. The underlying mechanisms of the closing-in phenomenon have not been fully elucidated. Previous studies have explained that the closing-in phenomenon is a part of constructional apraxia or primitive behaviors [26]. Another study analyzed the severity of closing-in as a function of figure complexity and suggested that the closing-in phenomenon is related to the patients’ compensatory strategies to overcome visuospatial dysfunction or visuospatial working memory deficits [19,29,30]. The higher immediate recall scores of the original RCFT associated with higher vertical positions of the simplified RCFT observed in this study also strongly support that the closing-in phenomenon is related to the visuospatial working memory deficit. Overall, the results of this study provide supporting evidence that using a digital device enables quantification of the closing-in phenomenon.

Our 2D cross-correlation analysis calculated a coefficient reflecting the overall similarity between the target picture and the drawings of the participants. The coefficient of both AD groups was significantly lower compared with that of the NCs, although the significance of the difference between the patients with EOAD and the NC individuals disappeared after adjusting the image size. Furthermore, there was a significant correlation between the similarity coefficients and the visual rating scores of the simplified RCFT. Therefore, automatic evaluation of similarity indices, which is not possible when using pen and paper, is another advantage of our analytic method that involves a digital device.

Contrary to our expectations, there was no significant difference between patients with LOAD and patients with EOAD, except that unlike patients with LOAD, patients with EOAD showed signs of leftward deviation when drawing the entire figure. Clinically, visuospatial dysfunction and apraxia are more prominent in patients with EOAD than in patients with LOAD [11,12]. A previous study also showed that 33% of patients with EOAD showed nonmemory symptoms, whereas only 6% of the patients with LOAD showed these symptoms [12]. These nonmemory symptoms included language and executive and visuoconstructional functions [31]. Another study that used RCFT to assess visuoconstructional abilities showed that the EOAD group was significantly more impaired than the LOAD group [11]. In contrast to these previous studies, there was no difference between the EOAD and the LOAD groups when analyzing various visuospatial parameters, except the horizontal position of the entire figure. The large variation in the EOAD group could have accounted for the lack of statistical significance.

Thus far, we have addressed 3 aspects of digital metrics derived from the simplified RCFT. It is known that there are several components in human visuospatial function such as visuoperceptual, visuoconstructional, or visuospatial working memory abilities [30,32,33]. Therefore, it may be important to discuss which visuospatial abilities are related to our digital metrics. First, pen stroke data may be linked to motor skills and strategies for the construction of visual components [33]. Second, analysis of the position of the drawing area may be related to the closing-in phenomenon, which reflects visuoperceptual or visuospatial working memory deficits [30]. Third, a 2D cross-correlation analysis may be associated with general visuospatial abilities that include both perception and construction attributes of visuospatial function [32]. Additionally, the positive correlation observed between the immediate recall scores of the original RCFT and the coefficients of the shape similarity from our digital version also support the working memory deficit hypothesis.

One limitation of this study is the sample size. The relatively small sample size of the participants may restrict the validity and reliability of our results. However, to overcome this limitation, we enrolled participants who had undergone imaging and neuropsychological battery tests. The small sample size may also explain the lack of difference in visuoconstructional dysfunctions between patients with EOAD and patients with LOAD.

We performed an analysis using a simplified RCFT that encompassed not only the quantification of the final figure but also the drawing process, such as pen strokes, spatial arrangement, and similarities, by using digital drawing data. Analyzing the drawing sequences of global and local elements as well as the number, length, and speed of strokes may only be possible through digital devices. Furthermore, our digitized version enabled automatic quantification of the degree of similarity and position of the drawing with respect to the target picture. Therefore, our digital metrics can complement the conventional visual rating of RCFT, which has been administered via pen and paper. Future standardization studies involving many patients and NCs are warranted to investigate whether our digitized, simplified RCFT would be useful in clinical and research settings.