Clinical neuropsychologists have traditionally developed and validated parsimonious assessment tools using basic technologies (ie, pencil and paper protocols, general linear model). Advances have predominantly occurred in expanded normative standards throughout the history of this profession [1]. Although these low-dimensional tools are well-validated assessments of basic cognitive constructs, they have limited presentation (static 2D stimuli) and logging capabilities (which require manual logging of responses). Moreover, low-dimensional approaches limit their statistical modeling (typically linear) to combinations of features relative to a set of weights for predicting the value of criterion variables. Some neuropsychologists may argue that the parsimony offered by low-dimensional tools reflects the reality of a much higher-dimensional deficit. However, low-dimensional tools may offer diminished interpretations of complex phenomena.

The preference for low-dimensional tools is apparent in surveys of assessments used by neuropsychologists [2,3]. This conservativism has resulted in neuropsychological assessments that have hardly changed since the original scales were established in the early 1900s [4,5]. Low-dimensional neuropsychological assessment tools place the neuropsychologist on par with the 19th century literary work on the nature of perception and dimensionality. Specifically, the narrator, A Square, resides in Flatland with residents (Flatlanders) whose perception is limited to 2 dimensions. After a discussion with a Stranger (a sphere), A Square comes to understand how complex and high dimensional the world is. Unfortunately, A Square is jailed for holding and communicating heretical beliefs [6]. For neuropsychologists, low-dimensional technologies have led us to search for simplified explanations of complex phenomena, which limits our ability to develop, validate, interpret, and communicate useful models of human neuropsychology. Recently, cognitive psychologists have called this the Flatland fallacy. They contend that the Flatland fallacy can be surmounted by formalizing psychological theories as computational models that have the capacity to make precise predictions about cognition and/or behavior (Figure 1) [7].

These authors explain that in the limited perspective of the Flatlanders’ view (bottom of the figure), a 3D object (sphere) seems to be fluctuating magnitudes (an expanding and reducing circle). However, the reality is that (top of the figure) this object is merely progressing through a lower‐dimensional plane. The low-dimensional perspective leads to a false understanding of reality. Similarly, neuropsychologists may incorrectly determine that the low level of dimensionality correctly describes neuropsychological or psychological phenomena. In fact, they may be missing the complexity and high dimensionality of neuropsychological phenomena.

Cognitive psychologists also contend that unitary cognitive constructs such as attention are limited and prevent psychologists from deepening the understanding of complex, or high-dimensional, phenomena. First, theories of unitary cognitive constructs are based on circular reasoning. Complex phenomena such as the conception of attention are explained by presumptive attentional systems. Instead, psychologists should model the parallel, reciprocal, and iterative interactions between context and neural or functional processes. This would enhance the characterization of physically executed actions [8]. Similarly, the analytical approach to psychology is problematic because (1) an exhaustive definition is proposed (eg, attention), (2) assumed subfunctions are identified (eg, selective, sustained, or divided attention) with separable functional and neuronal processes (or dedicated systems), and (3) research concentrates on specific tasks that purport to measure the theoretical subfunctions rather than underlying processes required to execute an efficient behavior in a particular situation [9]. Commonalities between subfunctions and other constructs (eg, working memory) are often not empirically distinguishable and by no means imply that the underlying functional and neural processes are different or separable. These authors propose that rather than being rigidly adherent to prior cognitive conceptual frameworks, psychologists should model mechanisms and processes (sensory, motor, and cognitive) that are found in several complex behaviors. These behaviors may run in parallel or interact across stimulus properties, time, and goals to achieve an outcome.

How do neuropsychologists move from low-dimensional neuropsychology to high-dimensional neuropsychology? The National Institutes of Health (NIH) offers initiatives for neuropsychologists interested in higher-dimensional tools, including (1) integrating neuroscience into behavioral and social sciences, (2) transformative advances in measurement science, (3) digital intervention platforms, and (4) large-scale population cohorts and data integration [10]. Similarly, the NIH Brain Research through Advancing Innovative Neurotechnologies (BRAIN) initiative seeks high-dimensional approaches to understand brain disorders (eg, Alzheimer disease, Parkinson disease, depression, and traumatic brain injury) and accelerating the technologies for high-dimensional modeling of how the brain records, processes, uses, stores, and retrieves vast quantities of information [11]. Neuropsychologists can enhance work conducted in NIH initiatives by offering interpretations of neuroscience findings based on clinical expertise.

After a brief consideration of the historic progression of neuropsychological assessment technologies, there is an elucidation of current NIH initiatives for the behavioral and social sciences as well as evaluations of current neuropsychological assessment technologies.

Neuropsychology has experienced a number of advances as it developed from a primarily qualitative practice to a more objective and evidence-based approach [12], with expanded normative standards [1], performance validity testing [13], and cross-cultural considerations [14]. Although these improvements have aided the investigation of neurocognitive functions, there are increasing discussions on the need to enhance the dimensionality of neuropsychological assessments and computational modeling [1,4,5,15-22].

The technological and theoretical development of neuropsychological assessment can be understood in terms of dimensional waves of technological adoption [5]. In Neuropsychology 1.0, neuropsychological assessments accentuate the development of low-dimensional and construct-driven (ie, simple stimulus presentations of stimuli to test abstract concepts like working memory) paper-and-pencil measures. In Neuropsychology 2.0, there is a technological move to automated administration, scoring, and in some instances the interpretation of low-dimensional stimulus presentations using computerized approaches (eg, NIH Toolbox and video teleconferencing) [23-26]. Concurrently, technological developments in neuroimaging have changed the role of neuropsychological assessments, from lesion localization to predictions about a patient’s ability to perform activities of daily living. Finally, Neuropsychology 3.0 reflects contemporary advances in high-dimensional (dynamic and ecologically valid simulation, logging, and modeling of everyday activities) tools.

Some neuropsychologists are hesitant to move from low-dimensional to high-dimensional tools because computerized assessments may introduce errors and/or decrease the reliability of the assessment process by means of automation [27]. Although there have been improvements in computational power and security, developers of high-dimensional technologies need to take appropriate actions to ensure proper implementation [28]. Furthermore, normative efforts are ongoing for high-dimensional assessments, and continued validation of advanced platforms and novel data analytic approaches is needed.

Three decades ago, clinical psychologists were urged to adopt progressively available advanced technologies [29]. Concurrently, in the 1980s, neuropsychologists started discussing the potential of computerized neuropsychological assessments [30]. It was subsequently pointed out that when compared with progress in our everyday technologies, psychological assessments had barely progressed throughout the 20th century (eg, Wechsler scales) [31]. Technologies found in neuropsychological testing can be compared with now obsolete black-and-white televisions, vinyl records, rotary-dial telephones, and the first commercial computer made in the United States (ie, Universal Automatic Computer I). Assessment technologies need to progress in ideas, not just new measures [31].

In the late 1990s, it was discussed how neuropsychology lagged behind (in absolute terms and in comparison with) other clinical neurosciences. Clinical neuropsychologists continued to use many of the same tools that have been developed decades earlier. Moreover, new tests that were coming out were not conceptually or substantively better than the ones from decades earlier (eg, Wechsler scales). Dodrill [32] pointed to the fact that in the 1970s, there was little difference in the technological progress of neurology and neuropsychology. This changed with the advent of computerized tomographic scanning, and neuropsychologists were no longer consulted for lesion localization. Neuroimaging advances allowed neurologists to better understand and treat neurologic pathophysiology [33]. Dodrill [32] suggests that if technological developments in neurology had been as slow as that found in neuropsychology, then neurologists would be limited to pneumoencephalograms and radioisotope scans. These procedures are deemed primeval by current neuroradiological standards.

To get an idea of where neuropsychology is today, basic searches were performed on July 29, 2020, to tally the number of technology publications per discipline. The first search included a PubMed search with the search terms “computer” AND (“neuropsychology” OR “neurology” OR “neuroscience”) from 1990 to 2019 (Figure 2). A second and third search using the terms “technology” and “neuroimaging” instead of “computer” revealed similar findings (Figures 3 and 4, respectively). Figures 2-4 show the number of publications by year that resulted from each of these 3 broad literature searches.

Findings from these basic searches suggest that high-dimensional technologies have vastly greater representations in neurology and the neurosciences. The inclusion of technologies is very recently increasing in neuropsychology but is explicitly not keeping pace with other neurosciences. Similarly, a survey of rates of neuropsychologists using computerized instruments revealed that only 6% of the 693 neuropsychology assessments were computerized [3]. The average respondent reported that they rarely used computerized tests. An increased likelihood of computerized assessment use was apparent for early career neuropsychologists.

Neuropsychologists can use deep learning algorithms that simulate the hierarchical structure of a person’s brain (healthy and damaged). Deep learning is a form of machine learning (ie, algorithms that learn from data to automatically perform tasks such as classification and prediction that can be nonlinear in nature) that processes data from lower dimensionality to increasingly higher dimensions. It is increasingly used to develop novel technologies, big data, and artificial intelligence [52]. Neuropsychologists can use deep learning to analyze studies with both traditional (low-dimensional paper and pencil) and high-dimensional simulation technologies (eg, virtual reality–based neuropsychological assessments, mixed reality, augmented reality). With deep learning, neuropsychologists could process the lower-dimensional data (paper-and-pencil tests). Next, they could move to increasingly higher-dimensional data (eg, from simulation technologies) and develop increasingly complex data-driven semantic concepts that are likely more representative of brain functioning than historical, theoretically based cognitive constructs (eg, working memory).

Probabilistic models and generative neural networks can be used to develop a unified framework for modeling neuropsychological functioning (nonclinical and clinical). Connectionist models such as these are understood to be a portion of the more general framework of probabilistic graphical models. Neuropsychological performances have been modeled as Bayesian computations (brain function expresses perceptions and actions as inferential processes). In this approach, neuropsychological deficits are false inferences arising from aberrant previous beliefs. Bayesian approaches can be used for computational phenotyping that uses graphical models implemented as stochastic processes that involve a randomly determined sequence of observations (each of which is considered as a sample of one element from a probability distribution) via generative neural networks [53]. Visual object recognition (eg, facial processing) can be used as an example. Selective lesions can be applied to computational models of visual object recognition to assess the impact of damage to various cortical regions (eg, early visual processing, extrastriate areas, anterior associative areas). New high-dimensional measures could be developed to assess visual agnosia and examine the appearance of category-specific deficits.

Deep learning architectures can also be used for modeling specific connection pathways in selective impairment. Stochastic decay (stochastic reduction of weight values that decreases responsivity to afferent signals) can be applied to synaptic strengths for examination of cognitive decline. Both global degradation of all network synapses and local degradation of inhibitory synapses from a given processing layer have been investigated. The findings revealed that although older participants accurately performed arithmetical tasks, they had impaired numerosity discrimination on trials requiring the inhibition of incongruent information. They also found that these results were related to poor inhibitory processes measured by standard neuropsychological assessments. The specific degradation of inhibitory processes resulted in a pattern closely resembling older participants’ performance [54]. The addition of computational modeling for the development, validation, and application of neuropsychological assessments represents a high-dimensional approach for neuropsychologists.

The NIH Toolbox is a battery of computerized neuropsychological assessments that uses item response theory (IRT) and CAT. With IRT, the NIH Toolbox has an alternative to classical test theory as it moves beyond group-specific norms [55]. In IRT, the probability of an item response is modeled according to the respondent’s position on the underlying construct of interest. This approach can be useful for providing item-level properties of each NIH Toolbox measure across the full range of each construct. Although neuropsychological measures tend to meet the reliability and validity requirements of classical test theory, the equivalence of item properties (eg, item difficulty and item discriminatory power) is often assumed across items. Consideration of item difficulty tends to be subsumed under independent variable manipulation (eg, cognitive load) to modify the marginal likelihood of correct responses in item subgroups. A limitation of this approach is that it does not match well with current item-level analyses found in neuroimaging assessments of brain activations following stimulus probes. For neuropsychological assessments to comport well with brain activation probes, item difficulty needs to be considered to avoid ceiling and floor effects in patient performances across clinical cohorts. IRT models offer the neuropsychologist both individual patient parameters and individual item characteristics that can be scaled along a shared latent dimension. Neuropsychological assessments would benefit from greater adoption of developments in IRT that emphasize the accuracy of individual items. Various IRT approaches have been applied as signal detection theory models that connect corresponding but discrete methods [56]. Combining IRT and signal detection delivers the measurement accuracy needed for robust modeling of item difficulty and examinee ability.

The NIH Toolbox CAT approach shortens testing time (by about half as long as low-dimensional paper-and-pencil measures). Through avoidance of floor or ceiling effects and concise item pools, CAT delivers equal (or greater) ability–level assessments [57,58]. Moreover, CAT offers enhanced efficiency, flexibility, and precision assessment of multiple domains of interest without adversely affecting participant burden. The application of IRT to CAT provides neuropsychologists with real-time assessment of item-level performance.

Neuropsychologists are increasingly interested in developing assessments that assess the patients’ real-world functions in a manner that generalizes to functional performance in everyday activities [38]. A function-led approach to neuropsychological assessments involves starting with directly observable everyday behaviors and proceeding backward to observe how a sequence of actions leads to a given behavior. Furthermore, a function-led approach examines how that behavior is disrupted. For example, a patient may have difficulty multitasking while using a global positioning system to navigate a simulated neighborhood in a driving simulator. High-dimensional technologies can be used to present dynamic and interactive stimuli in a 3D environment that includes automatic logging and computational modeling (eg, head movements, eye tracking, response latencies, patterns, etc) of a patient’s performance in everyday activities. High-dimensional neuropsychology tools are being developed and validated to simulate everyday functions (rather than abstract cognitive constructs) [5,41].

Given the drawbacks to experiments conducted in real-life settings (time consuming, require transportation, involve consent from local businesses, costly to use or build physical mock-ups, and difficult to replicate or standardize across settings) and difficulty in maintaining systematic control of real-world stimulus challenges, high-dimensional and function-led XR environments are being used by neuropsychologists.

Low-dimensional (paper-and-pencil and computer automated) neuropsychological tools only indirectly assess the patient’s ability to perform everyday activities [39,59]. VEs offer potential aids in enhancing the dimensionality and ecological validity of neuropsychological assessments through enhanced computational capacities for administration efficiency, stimulus presentation, automated logging of responses, and data analytic processing. Given the precise stimulus presentation and control of dynamic or high-dimensional perceptual stimuli, VEs offer neuropsychological assessments with enhanced ecological validity [5,60-62]. High-dimensional immersive VEs move beyond low-dimensional paper-and-pencil tests with static stimulus presentations in sterile testing rooms to simulated environments that replicate the distractions, stressors, and/or demands found in the real world.

Using passive data monitoring from everyday technologies (eg, smartphones, IoT), clinical scientists can collect real-time cognitive performance throughout the course of a day [10]. Each patient has a digital footprint that transpires from consistent use of everyday technologies. Coupling technologies with developments in measurement science allows for novel approaches to capture cognitions, affects, and behaviors [63]. Rapid progress in sensor technologies has led to objective and effective measures of behavioral performance, psychophysiology, and environmental contexts [64]. For example, machine learning has been employed to extract features from passive monitoring of mobile phone use. When comparing these features with performance on the psychomotor vigilance task, it was found that alertness deviations as small as 11% could be detected [65].

Another example of enhanced data monitoring can be found in the increased granularity in performance assessments and digital logging tools used in the Framingham Heart Study [66]. New developments in digital logging of verbal responses to cognitive stimuli allow for automated algorithms that can extract new language features (eg, speaker turn taking, speaking rate, hesitations, pitch, number of words, vocabulary). These features offer promise for predicting incident cognitive impairment [67]. Furthermore, low-dimensional pencils and pens can be upgraded with high-dimensional digital pens with associated software designed to measure pen positioning 75 times per second. Digital pens have a spatial resolution of ±0.002 inches. For example, digital pens are being used by neuropsychologists for assessing clock drawing performance [68,69]. Minute drawing elements such as pen strokes (eg, clock face, hand, digit) can be logged with greater than 84% accuracy [70]. The sensitivity of these high-dimensional technologies to minute drawing elements, decision-making latencies, and graphomotor characteristics may offer promise to greatly enhance lower-dimensional hand scoring of the Boston Process Approach. A review of the Boston Process Approach and neuropsychology-based technologies has been available recently [71].

Another transformative opportunity from the NIH OBSSR is the application of high-dimensional technologies to interventions [10]. Progress in neurocognitive rehabilitation has been enhanced by neuroimaging of plasticity of the brain. Similarly, a notable increase can be found in the use of noninvasive brain stimulation approaches that leverage neural plasticity for rehabilitation [72]. Neuropsychologists interested in rehabilitation emphasize the promotion of brain plasticity by increasing a patient’s capacity for performing everyday activities. The resource and labor intensiveness of interventions and the resulting limitations (reach, scalability, and duration) found in real-world assessment environments require interventions to be personalized at the start, adapted throughout treatment, and operationalized into coded databases for fidelity [10].

The NIH OBSSR strategic plan is also interested in big data, data analytics, and data integration techniques for developing collaborative knowledge bases [10]. Integrating neuropsychological data into large collaborative knowledge bases will allow neuropsychologists to either formalize cognitive ontologies or abandon cognitive ontologies for phylogenetically refined functional and neuronal processes that underlie all complex behaviors or more simplistically traditional neuropsychological tasks [8,32,38,105]. Formal designations of distinct sensory, motor, and cognitive entities can be established in terms of parallel, reciprocal, hierarchical, and/or spatiotemporal relations [7,18].

Consistent with critiques from cognitive psychology [8], a limitation of neuropsychological data integration is that low-dimensional neuropsychological assessments are made up of hypothetical interdimensional constructs inferred from research findings [19,38]. Evidence for poor test specificity is apparent in median correlations for common neuropsychological tests. It has been found that although the median correlation within domain groupings on a neuropsychological test was 0.52, the median correlation between groupings was 0.44 [32]. Therefore, the tests are not unambiguously domain specific. The median correlations should be notably higher within groupings and lower between groupings. A recent meta-analysis of relationships between the Wisconsin Card Sorting Test (WCST) and the Weschler Adult Intelligence Scale (WAIS) found a robust relationship between WCST performance and WAIS indices [106]. This is interesting because the WAIS was recently found to be the test most often administered by neuropsychologists and the WCST was the fifth most often administered [2]. Interestingly, the meta-analysis found that WCST scores were associated in comparable strength with both verbal and nonverbal domains from Wechsler Adult Intelligence Scale tests. Another issue is that there is considerable variation in some neuropsychological tests of the same domain (eg, various measures of go or no-go performance) [107]. The shared variance of tests of supposedly differing domains and the lack of consistency in tests of the same domain may decrease the capacity for accurate data integration.

Compounding this issue is the fact that current diagnostic frameworks found in the American Psychiatric Association’s Diagnostic and Statistical Manual of Mental Disorders (DSM) and the World Health Organization’s International Classification of Diseases (ICD) are dependent on presenting signs and symptoms. Moreover, they do not align with findings from genetics and clinical neuroscience [108,109].

Ontologies are formal specifications of entities found in a domain and their relations. An ontology contains designations of separate entities along with a specification of ontological relations among entities with representations via spatiotemporal (eg, preceded-by or contained-within) or hierarchical relations (eg, is-a or part-of). This provision of an objective, concise, common, and controlled vocabulary facilitates communication among domains. Neuropsychological assessment lags behind other clinical sciences in the development of formal ontologies [18,19].

As such, neuropsychologists have moved beyond the diagnostic taxonomies found in the DSM and ICD. These diagnostic taxonomies are not sufficient for biomarker research because they do not reflect relevant neurocognitive and behavioral systems. Instead, neuropsychologists interested in developing a common vocabulary for ontologies and collaborative knowledge bases should adopt the US National Institute of Mental Health’s Research Domain Criteria (RDoC) project. The RDoC aims to establish a classification system for mental disorders based on neuroscience and behavioral research [108,110,111].

Neuropsychologists interested in high-dimensional technologies have embraced the following NIH initiatives to advance scientific developments: (1) integration of neuroscience into behavioral and social sciences, (2) transformative advances in measurement science, (3) digital intervention platforms, and (4) large-scale population cohorts and data integration. Evidence that progress is occurring in neuropsychology exists; however, more work needs to be done. Much of this work involves adoption, development, and validation of novel technologies. Similarly, there is a need for a classification system (based on neuroscience and psychology research) that moves beyond low-dimensional emphases on unitary cognitive constructs specific to a purported functional or neuronal system. A high-dimensional classification instead embraces testable hypotheses of how an observed phenomenon is produced from fundamental underlying mechanisms or processes, the dynamics of those processes (eg, reciprocal, hierarchal, iterative), and the multiple functional or neuronal systems involved in several complex behaviors [8,34]. In more basic terms, neuropsychologists should theorize with verbs instead of nouns to serve scientific progress. Only then can neuropsychologists integrate data to develop meaningful ontologies and collaborative knowledge bases of high-dimensional neuropsychological phenomena. Computational modeling has great promise for achieving this endeavor.

High-dimensional neuropsychology requires substantial reform in the way the profession conducts training. High-dimensional training should be added to current trainings that emphasize primarily (in some programs it may be solely) low-dimensional neuropsychological tests (eg, paper-and-pencil tests) and methods (limited introduction to general linear modeling). Increased emphasis should be placed on technical skill development with high-dimensional technologies and data-driven inferential reasoning. Curricula in neuropsychology programs should be expanded to adapt to the recent technological advances that have led to exponential growth in the other sciences. This would require reimagining training in clinical psychology programs. If neuropsychologists of the future are to work with large collaborative knowledge bases and perform complicated computational modeling of big data, then they need at least basic training in areas traditionally associated with computer science (eg, computer programing) and informatics (algorithms and databases). As such, their basic statistical training would need to be enhanced to include data manipulation, predictive model generation, machine learning, natural language processing, graph theory, and visualization. Increased emphasis on training basic technical and computational skills will improve the ability of future neuropsychologists to participate in science.

A final note is the need for training in neuroethics. Neuroethics has been distinguished into 2 branches: (1) ethics of neuroscience—neuroethics as applied ethical reflection on the practices and technologies found in the neurosciences—and (2) neuroscience of ethics—what neuroscience can reveal about the nature of morality and morally relevant topics [112]. Neuroethics are important for the NIH BRAIN initiative. The NIH BRAIN project aims to examine the ways in which dynamic patterns of neural activity are transformed into cognition, emotion, perception, and action in health and disease [113]. The BRAIN initiative promotes the use of powerful new tools and technologies: (1) technologies for monitoring neural circuit activity and (2) technologies that enable the modulation of neural circuits [114]. As expected, the ethical concerns related to the medical and nonmedical use of neurotechnologies by neuropsychologists are profound. Neuroethics for neurotechnologies include a combination of principlist, deontological, and consequential ethical approaches to answer ethical quandaries [115,116]. Training in neuroethics and the ethical use of high-dimensional technologies will allow neuropsychologists to provide enhanced care for their patients.