This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
EDUCERE (“Ubiquitous Detection Ecosystem to Care and Early Stimulation for Children with Developmental Disorders”) is an ecosystem for ubiquitous detection, care, and early stimulation of children with developmental disorders. The objectives of this Spanish government-funded research and development project are to investigate, develop, and evaluate innovative solutions to detect changes in psychomotor development through the natural interaction of children with toys and everyday objects, and perform stimulation and early attention activities in real environments such as home and school. Thirty multidisciplinary professionals and three nursery schools worked in the EDUCERE project between 2014 and 2017 and they obtained satisfactory results. Related to EDUCERE, we found studies based on providing networks of connected smart objects and the interaction between toys and social networks.
This research includes the design, implementation, and validation of an EDUCERE smart toy aimed to automatically detect delays in psychomotor development. The results from initial tests led to enhancing the effectiveness of the original design and deployment. The smart toy, based on stackable cubes, has a data collector module and a smart system for detection of developmental delays, called the EDUCERE developmental delay screening system (DDSS).
The pilot study involved 65 toddlers aged between 23 and 37 months (mean=29.02, SD 3.81) who built a tower with five stackable cubes, designed by following the EDUCERE smart toy model. As toddlers made the tower, sensors in the cubes sent data to a collector module through a wireless connection. All trials were video-recorded for further analysis by child development experts. After watching the videos, experts scored the performance of the trials to compare and fine-tune the interpretation of the data automatically gathered by the toy-embedded sensors.
Judges were highly reliable in an interrater agreement analysis (intraclass correlation 0.961, 95% CI 0.937-0.967), suggesting that the process was successful to separate different levels of performance. A factor analysis of collected data showed that three factors, trembling, speed, and accuracy, accounted for 76.79% of the total variance, but only two of them were predictors of performance in a regression analysis: accuracy (
The EDUCERE DDSS is ready to use the regression equation obtained for the dependent variable “performance” as an algorithm for the automatic detection of psychomotor developmental delays. The results of the factor analysis are valuable to simplify the design of the smart toy by taking into account only the significant variables in the collector module. The fine-tuning of the toy process module will be carried out by following the specifications resulting from the analysis of the data to improve the efficiency and effectiveness of the product.
In early childhood, children learn by playing in natural settings such as the playground, home, or kindergarten. By playing, children develop various cognitive, perceptual, motor, linguistic, and communicative skills. When a child plays alone, he or she interacts with objects, usually toys, and performs various movements, such as picking or throwing objects, placing them in a row, or stacking them on top of one another. Not all these movements are simple; for example, building a tower with stackable cubes can be difficult for children between ages 2 and 3 years, especially if the cubes are small and the tower is high. Standardized developmental tests include items that ask children to make a tower by stacking cubes [
The research performed in the EDUCERE (Ecosistema de Detección Ubicua, atenCión y Estimulación tempRana para niños con trastornos del dEsarrollo; “Ubiquitous Detection Ecosystem to Care and Early Stimulation for Children with Developmental Disorders”) project focuses on psychomotor development and analyzes how toddlers build a tower of cubes by electronically recording all significant movements performed by the children. The tower of five stacking cubes is an example of the psychomotor behavior of toddlers aged between 2 and 3 years. The aim is to determine if children are capable of building the tower and to analyze how they do it in order to detect minimal delays in development, which may lead to preventive monitoring of some children or, when appropriate, to the implementation of an early attention program.
Ubiquitous computing and ambient intelligence could support innovative application domains, such as the detection of motor impairments within the home environment [
Further to this contextualization, the goal of the EDUCERE project [
Early detection of developmental problems is a critical matter to assure the wellness of children [
There are many childhood disorders, such as autism spectrum disorders, that can be detected by using information from the child’s movements in certain activities [
These approaches are partially intrusive because children have to use special wearable devices that are not part of their everyday routines. The goal of the EDUCERE project is to embed the measurement tools in ordinary objects, extending the Internet of things (IoT) paradigm to an “Internet of toys” experience by creating smart toys based on everyday objects equipped with low-cost sensors. This allows the acquisition of accurate information without interfering in children’s daily activities.
This approach to the IoT using toys as smart objects is not exclusive for the purposes presented in this paper. For instance, Wang et al [
The EDUCERE project aims to create a whole ecosystem of compatible smart toys that provide information that helps child development professionals to detect potential developmental problems. The first toys created for that purpose were a set of cubes that can be stacked to build a tower [
All toys include the ATMEL ATmega328p microcontroller by Atmel Corporation (San Jose, CA, USA). This controller is integrated with Arduino boards and is compatible with them, offering an easy and fast prototyping platform to develop the toy software through the Arduino integrated drive electronics. Each toy includes the sensors that add the needed functionality in each case. Every toy uses a NRF24 radio frequency adapter for communications with a data collector device. The NRF24L01+ by Nordic Semiconductor ASA (Oslo, Norway) is an ATmega328p-compatible 2.4 GHz radio frequency transmitter/receiver chosen because of its size, low cost, and low consumption.
Specifically, the stackable cubes consist of the preceding components and:
A 3.7 lithium-ion polymer battery. The main limitation for the battery was the maximum size of the cubes (2.5 cm per side), so 150 milliamp hour was the maximum capacity to be fitted in the available space. In addition, a protection circuit was included to ensure that the battery never offers less than 3.2 volts.
A set of 12 (two per cube face) light-dependent resistor sensors from Silonex Inc (Montreal, QC, Canada). These sensors allow for determining which face of the cube is covered at each moment.
A MPU-9150 InvenSense by Sunnyvale (San Jose, CA, USA) with a three-axis accelerometer and gyroscope. This sensor provides speed, acceleration, and shaking level values for each cube movement.
A tilt-based switch that enables a sleep mode to decrease power consumption.
Three light-emitting diodes (LEDs) and a buzzer for a visual and auditory user interface.
The cube printed circuit board construction (a-c) and the 3D printed external case (d).
The very strict size constraint in cube size (no more than 2.5 cm for each face) required designing a specific printed circuit board (PCB) shaped as a cube itself. The PCB is divided into six square faces and each face contains part of the printed circuit.
Cubes transmit the gathered data to a collector module that gives format and stores and encrypts the information. The collector module has been deployed in a Raspberry Pi board with a NRF24L01+ adapter that allows it to connect by itself to the toys. The collector provides a RESTful application program interface through a Wi-Fi access point that allows for managing the experiments and the data files obtained in these experiments using a native app in a tablet or a phone. The data files are cyphered using Advanced Encryption Standard (AES) on Cipher Block Chaining (CBC) mode with a one-use 128-bit key. The key is randomly generated for each experiment and is also encrypted with the RSA DDSS public key to ensure the information is only accessible in a secure Web server.
The DDSS of EDUCERE contains all administrative tasks to securely register children, professionals, and activities used during the experimentation scenario. The registry process is done in two steps:
Before beginning the process of experimentation with the explicit smart toy, a user with an administrative role will access the EDUCERE system to store the following information: the specific smart toy for the experimentation, the activities that can be performed with that smart toy, and the professional who will carry out the experiment.
After that, the administrative user reports to the professional so that they can begin the experimentation process. The professional must be authenticated in the EDUCERE DDSS and enter the information requested for each of the children who will be performing the experiment.
The person with the professional role must upload information about the children who will carry out the experiment with the smart toy. The professional enters name, date of birth, gender, the name of the professional who will carry out the experiment, and the name of the center where this experiment will be done. The professional must also record in EDUCERE the information about the activities that the child could do with the smart toy.
Components of the general architecture in EDUCERE developmental delay screening system.
EDUCERE mobile app use case diagram.
Several children participate at each screening session. In addition, the experimentation session can last for several days with the whole group.
Use cases of the mobile app consist of three main stages:
Experimental management: includes all use cases related to the experimentation process. Users log in to the mobile app to access, such as a known professional. In this case, only the log-in of the professional is required. Before starting experiments, the mobile app (previously connected to the EDUCERE Wi-Fi generated by the Raspberry Pi collector) synchronizes professionals and children with the collector. To start experiments, it is required to create or use an experiment session because all experiments will belong to a concrete session. The professional could also drop a session when the session is over. Once the professional has selected a session, the professional should configure the experiment, choosing a registered child and the cube activity. The experiment starts when the professional presses the “start experiment” button. The experiment stops when he or she presses the “stop experiment” button. Next, the professional can refuse it or store it. If the professional chooses to store it, the collector sends all data collected during the experiment in an encrypted file form. This file is saved in the internal storage of the mobile or tablet.
Server synchronization: there are two ways of synchronization between the mobile app and the EDUCERE DDSS server. Before the experiment starts, the mobile app connects with the EDUCERE DDSS server to download information about registered children and professionals. The mobile app uses this information to synchronize the professionals and children with the collector. After performing experiments, the mobile app connects with the EDUCERE DDSS server to upload experiment files with the collected measurements to the server.
Server authentication: to synchronize the mobile app with the EDUCERE DDSS server, the professional must be authenticated in the app (submitting log-in and password) to access to the server. This log-in and password are sent to the server to start synchronization.
The mobile app has been implemented in HTML5-Javascript for cross-platform development, using jQuery and the responsive Web app development framework Bootstrap. Through Cordova, a well-known open source mobile development framework, the wrapper is generated to run the app on different mobile platforms, such as Android or iOS. The result is a hybrid app executed via Web views. Cordova also provides a series of plug-ins to access the functionalities of the mobile device, such as internal storage, connection detection, and vibration, all of which are necessary for the development of the mobile app.
EDUCERE mobile app interfaces (in Spanish). The professional can select the toy for experiment (a) and show data from experiments performed (b).
Steps for interaction with the EDUCERE developmental delay screening system.
During the experimentation scenario, the records of the activities performed by the child with the smart toy are collected and sent to the EDUCERE DDSS to be persistently stored (see
Both in the school or home scenario, the child registered in the EDUCERE DDSS performs the activity indicated by the professional with the specific smart toy (rattles, balls, and cube towers). In this task, the professional assesses the child during the experimentation process and takes responsibility for the experiment. He/she uses a tablet to perform the process of starting the experiment, finalizing the experiment, and storing it. This is the way in which communication occurs between the professional and the collector, who manages the activity to be performed with the smart toy.
As
When opening the app, the displayed home screen helps to identify the professional who will perform the experiment. Once the professional provides identification, he or she selects the child who will perform the experiment. After that, the professional chooses the cube tower activity, which is the activity to be performed in which the child plays with a set of cubes.
Start experiment. At this point, it is necessary to synchronize the collector with the identifiers that are in the tablet corresponding to the child and professional (steps A and B in
After that, the child begins interacting with the toy and the information generated by the toy’s sensors is stored in the collector. When the child completes the activity with the smart toy, the professional indicates on the tablet that the experiment has ended.
At this step, the experiment data in the collector are transmitted to the tablet and securely stored.
Finally, when the mobile app detects a known Wi-Fi, it connects to it to synchronize with the EDUCERE DDSS server (always prevails over all Wi-Fi raspberry, to which you must connect to start the experiment). The tablet connects to a specific Web service in the server to synchronize and transfer information between the experiments stored in the tablet and the EDUCERE system databases on the server (step A in
A total of 65 toddlers (32 boys and 33 girls) from a public nursery school aged between 23 and 37 months (mean 29.02, SD 3.81 months) took part in a pilot trial in which they had to build a tower with five stackable cubes. The professional did not have clinical information about the toddlers who participated in the experiment. Parents signed an informed consent sent by EDUCERE project.
The toddlers were given five EDUCERE stackable cubes placed in a row on a template (
The observer sat to the left of the child who was in front of the table in the middle. During the experiment, the experimenter needed the following elements: (1) a collector module (described in Smart Toys Development subsection) to format, store, encrypt, and transmit information obtained from the cubes; (2) a tablet with a mobile app (as shown
Experimental scenario. Initially the five cubes are placed in a row on the template and the child is to build a tower with the cubes on the square in front.
The set up of the trials was fixed on the tables used by the toddlers. The professional made the tower with the stackable cubes in the marked position while saying, “Look, I’m going to make a tower with these cubes right here” (see
Four experts in child psychomotor development (one developmental psychologist, one physiotherapist, and two educators) viewed the recordings and selected an analyzable fragment and scored each child trial on a scale of 1 to 10 according to their performance. The experts had a meeting prior to viewing the videos in order to agree on assessment criteria (see
Each expert randomly viewed half of the videos and two experts individually rated each video. In this way, two experts scored each child’s performance. The experts made their assessments without having contact with one another to ensure their independence of judgment. Because the experts came from different professional backgrounds, it was expected that their assessments would not be identical, despite having reached agreement on the criteria.
The selected video fragments had their corresponding data recorded by the collector module. These data were included, together with the expert scores, in the statistical analysis.
The set of variables for each experiment were measured by using the sensors included in the toys (see Smart Toys Development subsection). The toy processes these variables starting from the values obtained from the sensors and stores them for each movement. The variables are the maximum acceleration during the movement, the maximum and mean speed, the time at which the maximum speed is reached within the movement, the time the movement took, and the number of shakings detected. The shakings are calculated from the graph of instantaneous accelerations by considering a shaking as data between two minimum values. There were four levels of shakings. In order to determine the level of shaking, the number of samples that fit the “mound” in the accelerations graph was taken into account. Using this classification scheme, the first level represented the smaller shakings (ie, the shorter “mounds”, where only one sample from the sensor was received before and after the maximum acceleration value) and the fourth level represented the bigger shakings (where four or more samples were received). This measurement and classification has been explained in more detail previously [
Once all the experiments were performed, the stored per-movement data were summarized in a per-experiment data file. The variables used in the analysis were these summarized values, including the performance scores determined by experts, which are detailed in
Summary of variables used in the analysis.
Variable name | Meaning | Dimensions/Range | How it is calculated? |
Performance scores | Scores of children performing the activity | 1-10 (10 being the best possible score) | It is assigned by experts while reviewing the experiment |
Number of movements | Total number of movements made with all the cubes during an experiment | 1-n (ideally five, one movement per cube) | A movement is any period of time in which the cube accelerometer sensor gives (after velocity calculation) a value high enough to determine the cube is moving (see [ |
Mean time of movement | Mean of the duration of each movement during an experiment | Milliseconds (msec) | The period of each movement is detected and stored and then the mean value of all these time values is calculated |
Mean speed of movement | Mean of all the mean speed values measured during an experiment in meters per second. | Meters per second (m/s) | The speed values during a movement are calculated by integrating the values obtained by the cube accelerometer; with all the instant values within a movement, the mean speed is calculated and this value is the mean of these means for all the experiment |
Mean of maximum speed | Mean of all the maximum speed values | m/s | For each speed value obtained during a movement, the maximum value is stored, then the mean of these values is calculated for the entire experiment |
Highest maximum speed | The maximum value of the maximum speeds | m/s | For all the maximum values stored during an experiment, the maximum value is selected |
Lowest maximum speed | The minimum value of the minimum speeds | m/s | For all the maximum values stored during an experiment, the minimum value is selected |
Maximum acceleration of movement | Mean of the maximum acceleration values | m/s2 | The accelerations are calculated directly from the values obtained in the accelerometer; the maximum value obtained for a movement is stored and, for this variable, the mean of these maximum values is calculated |
Highest maximum acceleration | The maximum value of the maximum accelerations | m/s2 | This variable represents the highest value of the maximum accelerations stored during an experiment |
Lowest maximum acceleration | The minimum value of the maximum accelerations | m/s2 | This variable represents the lowest value of the maximum accelerations stored during an experiment |
Mean of shaking (level 1)a | Mean of the number of shaking of level 1 | 1-n | Given the previous definition of shaking, this variable represents the mean of the level 1 shakings measured for each movement |
Mean of shaking (level 2)a | Mean of the number of shaking of level 2 | 1-n | Given the previous definition of shaking, this variable represents the mean of the level 2 shakings measured for each movement |
Mean of shaking (level 3)a | Mean of the number of shaking of level 3 | 1-n | Given the previous definition of shaking, this variable represents the mean of the level 3 shakings measured for each movement |
Mean of shaking (level 4)a | Mean of the number of shaking of level 4 | 1-n | Given the previous definition of shaking, this variable represents the mean of the level 4 shakings measured for each movement |
a The first level represents the smaller shakings (ie, the shorter “mounds”, where only one sample from the sensor is received before and after a maximum acceleration value) and the fourth level represents the bigger shakings (where four or more samples are received).
In order to analyze interrater agreement, a reliability analysis was conducted in IBM SPSS Statistics 23. We computed a intraclass correlation coefficient (ICC) analysis, following the Model ICC (1,k) in SPSS, in which 1 denotes that each participant is assessed by a different set of randomly selected raters, and k is the number of raters for every score. In this experiment, two experts out of four were randomly assigned half of the videos to rate. The other half of the videos were rated by the two remaining experts. This way, each of 65 videos had two ratings.
In the model, reliability was calculated by taking the mean of the two raters’ measurements across the 65 scores. The ICC for single measures was 0.961 (95% CI 0.937-0.976;
A factor analysis was conducted to group similar variables into dimensions [
The factor analysis showed that the first three factors together accounted for 76.784% of the total variance.
Rotated component matrix.
Variance and variables | Component | |||
1 | 2 | 3 | ||
Variance explained | 31.386% | 24.788% | 20.616%* | |
Number of movements | –.049 | .294 | –.782* | |
Mean time of movement (msec) | .983* | –.048 | .003 | |
Mean speed of movement (m/s) | .015 | .840* | .199 | |
Mean of max speed (m/s) | –.024 | .943* | –.009 | |
Highest maximum speed (m/s) | –.078 | .723* | –.572 | |
Lowest maximum speed (m/s) | .035 | .264 | .800* | |
Maximum acceleration of movement | –.083 | .809* | .139 | |
Highest maximum acceleration | –.090 | .642* | –.597 | |
Lowest maximum acceleration | –.044 | .447 | .784* | |
Mean of shaking 1 | .747* | –.021 | –.229 | |
Mean of shaking 2 | .896* | –.120 | –.047 | |
Mean of shaking 3 | .892* | –.024 | .173 | |
Mean of shaking 4 | .728* | .022 | .225 |
* Strongest correlations between variables and components (factors). Those in component 1 make up “trembling” factor, those in component 2 make up “speed” factor, and those in component 3 make up “accuracy” factor.
In
Component 1 presents high correlations with the variables mean time of movement and mean of shaking (1, 2, 3, and 4). We call this factor “trembling.”
Component 2 indicated high correlations with the variables mean speed of movement, mean maximum speed, highest maximum speed, maximum acceleration of movement, and highest maximum acceleration. We call this factor “speed.”
Component 3 links high correlations with the variables number of movements, lowest maximum speed, and lowest maximum acceleration. However, the correlation with number of movements was negative, as can be observed. This means that the number of movements varies in the opposite direction to that of the other significant variables of the component and, of course, opposite the factor. We call this factor “accuracy.”
In order to design the EDUCERE automatic system for the detection of delays in toddlers’ psychomotor development using the smart stackable cubes, it was necessary to describe an algorithm that included the significant factors, or independent variables, and a dependent variable “performance.”
Two multiple regression analyses were carried out to predict (1) the value of the variable performance based on the value of the three components obtained in the factor analysis, trembling, speed, and accuracy, and (2) the value of the variable “age” based on the same three components.
Multiple regression analyses: model summary.
Model | Dependent | Predictor | Adjusted |
SE of the estimate | ||
1 | Performance | Accuracy, speed, trembling | .517 | .267 | .231 | 1.556 |
2 | Age (months) | Accuracy, speed, trembling | .362 | .131 | .089 | 3.637 |
Multiple regression analyses: coefficients.
Model | Unstandardized coefficient, B (SE) | Standardized coefficient, beta | ||||||
(Constant) | 7.662 (0.193) | 39.698 | <.001 | |||||
Trembling | 0.050 (0.194) | 0.028 | 0.257 | .80 | ||||
Speed | –0.630 (0.194) | –0.355 | –3.239 | .002 | ||||
Accuracy | 0.665 (0.194) | 0.375 | 3.419 | .001 | ||||
(Constant) | 29.015 (0.451) | 64.315 | <.001 | |||||
Trembling | –0.152 (0.455) | –0.040 | –0.334 | .74 | ||||
Speed | 0.003 (0.455) | 0.001 | 0.007 | .99 | ||||
Accuracy | 1.372 (0.455) | 0.360 | 3.018 | .004 |
Based on
The coefficient for trembling (0.50) was not significantly different from zero (
The equation for the regression line in model 2 was: age = 29.015 – 0.152(trembling) + 0.03(speed) + 1.372(accuracy), with the standard error of the estimate=3.637.
Only the accuracy coefficient (1.372) was significant (
The regression equation obtained for the variable “performance” is the algorithm that will be the basis of the automatic detection of developmental delays. In order to obtain a design as efficient as possible, the design of the smart toy must be adjusted by reducing the amount of data from the collector module taking into account the nonsignificant results obtained in the statistical analyses.
From the factor analysis, we conclude that trembling explains the greater percentage of the variance (31.38%), but considering it as a possible predictor of the performance in the regression analysis, the results show a lack of significance, so variance is unrelated to performance. This implies that the sensors that provide measurements for the variables mean time of movement and mean of shaking (1, 2, 3, and 4) have to be reconfigured to obtain only the data of interest, those that the automatic system needs to classify toddlers’ psychomotor performances. Although the prediction power of this reduced set would be slightly lower than the obtained using the original one, the reduction is probably not enough to justify maintaining the complex design of the devices.
On the other hand, speed accounts for 24.78% of the variance, but it is also negatively related to performance. For this reason, the sensors that allow the collector module to obtain measurements of the mean speed of movement, the mean maximum speed, the highest maximum speed, the maximum acceleration of movement, and the highest maximum acceleration, must be kept in the smart toy because they were in the original design [
Finally, accuracy accounts for 20.61% of variance and is a predictor of performance, so sensors that collect data to measure the number of movements, the lowest maximum speed, and the lowest maximum acceleration should also be kept in the smart toy because they were in the original design [
Chronological age is not a direct indicator of the level of psychomotor development, although it is related to it. According to the results obtained in the regression analysis, the only factor that predicts children’s chronological age is accuracy. This result agrees with those of a study on dysgraphia [
In this research, we have detected at least three factors of interest to describe the level of psychomotor development of children: trembling, speed, and accuracy. This will be the starting point for further research that will focus on exploring the relationships of these factors to a set of motor behaviors of children in their natural settings, school, home, and playground [
The system is not designed to predict the age of the children, but it was of theoretical interest to know how age is related to the other variables observed. However, it is relevant to know that the variable accuracy, as described in this research, is related to the level of psychomotor development of children and is one of the aspects to be observed in predicting possible difficulties in school, such as dysgraphia [
The EDUCERE DDSS could benefit from larger sample sizes to “learn” to detect and classify delays of psychomotor development. Therefore, the following research work for the smart toy should be tested with more toddlers, with and without developmental delays, diagnosed or not. Furthermore, the data obtained from these new experiments will be used for a further validation of the results of the analysis presented in this paper because this new set of data could guarantee that the predictive power of the algorithm stays the same when separating training and test data.
Consequently, the next phase of the research will focus on the objective to establish the criteria for classification of psychomotor development delays and to describe the actions to be performed in each case, relying on the previously validated smart toys. These criteria do not correspond exactly to the conventional diagnostic criteria because the interest of this investigation is to detect slight delays that are usually unnoticed in the standardized tests, which is the added value of this research.
The final mission of the EDUCERE DDSS is starting to be achieved by providing parents, educators, psychologists, and pediatricians with accurate data about potential delays detected. Results obtained initially triggered some advisement to follow-up in some cases leading to messages such as, “the movements and child´s interaction are OK” or “let the child keep playing but visit the specialist in 3 months.”
Descriptions relevant to the experimental procedure.
developmental delay screening system
Ecosistema de Detección Ubicua, atenCión y Estimulación tempRana para niños con trastornos del dEsarrollo [Ubiquitous Detection Ecosystem to Care and Early Stimulation for Children with Developmental Disorders]
intraclass correlation coefficient
Internet of things
printed circuit board
This paper is part of research conducted under the EDUCERE project (Ubiquitous Detection Ecosystem to Care and Early Stimulation for Children with Developmental Disorders; TIN2013-47803-C2-1-R), supported by the Ministry of Education and Science of Spain through the National Plan for R+D+I (research, development, and innovation). Besides, we would like to thank the aid received for PhD students contracts of the Ministry of Economy of Spain (BES-2014-067912).
None declared.