교육용 로봇과 컴퓨터를 활용한 자폐범주성장애 아동의 AAC 동작어 상징 학습 효과

Effects of Robot and Computer-based Intervention on Learning Action Word Symbols of AAC for Children with Autism Spectrum Disorder

Article information

Commun Sci Disord Vol. 21, No. 4, 744-759, December, 2016
Publication date (electronic) : 2016 December 31
doi : https://doi.org/10.12963/csd.16344
aDepartment of Communication Disorders, Ewha Womans University, Seoul, Korea
bChildren’s Center for Developmental Support, Ewha Womans University, Seoul, Korea
cDepartment of Computer Science, KAIST, Daejeon, Korea
dSchool of Information Technology, SungShin Women’s University, Seoul, Korea
최은정a, 김영태,a, 연석정a,b, 김동준c, 홍기형d
a이화여자대학교 일반대학원 언어병리학과
b이화여자대학교 발달장애아동센터
c한국과학기술원 공과대학 전산학과
d성신여자대학교 IT학부
Correspondence: Young Tae Kim, PhD Department of Communication Disorders, Ewha Womans University, 52 Ewhayeodae-gil, Seodaemun-gu, Seoul 03760, Korea Tel: +82-2-3277-2120 Fax: +82-2-3277-2122 E-mail: youngtae@ewha.ac.kr

This work was supported by the National Research Foundation of Korea (NRF) funded by the Ministry of Education, Science, and Technology (NRF-2012-S1A5A2A-03034254).

본 연구는 2012년 정부(교육과학기술부)의 재원으로 한국연구재단의 지원을 받아 수행된 연구임(NRF-2012-S1A5A2A-03034254).

Received 2016 October 5; Revised 2016 November 17; Accepted 2016 November 25.

Abstract

배경 및 목적

본 연구에서는 AAC 동작어 상징 학습에 대한 교육용 로봇과 컴퓨터의 중재 효과를 비교해보고자 하였다.

방법

연구대상은 자폐범주성장애 아동 3명이며, 교대중재설계를 사용하여 AAC 동작어 상징 학습에 미치는 교육용 로봇과 컴퓨터 중재의 효과를 비교하였다. 연구에서 사용된 동작어 상징은 총 20개로 각 10개씩 두 세트로 나뉘어 기초선, 중재 그리고 일반화 단계에서 교육용 로봇과 컴퓨터를 통해 노출되었다.

결과

연구결과 교육용 로봇 중재에서는 대상 아동 3명 모두, 컴퓨터 중재에서는 대상 아동 3명 중 2명에게서 동작어 상징 이해도 향상이 나타났다. 일반화 단계에서는 모든 대상 아동이 두 가지 중재에서 동작어 상징 이해도 향상을 보였다. 대상 아동들이 보인 향상 정도를 비교하였을 때는 교육용 로봇 중재에서 컴퓨터 중재보다 전반적으로 더 높은 향상이 나타난 것을 알 수 있었다.

논의 및 결론

교육용 로봇과 컴퓨터 중재 모두가 동작어 상징 학습에 긍정적인 효과를 미쳤다. 그리고 두 가지 중재 효과를 비교했을 때는 교육용 로봇 중재가 컴퓨터 중재보다 더 높은 수준의 향상을 이끌어내는 것으로 나타났다.

Trans Abstract

Objectives

The purpose of this study was to investigate the effects of robot and computer-based intervention on learning action word symbols of augmentative and alternative communication (AAC) for three preschool children with autism spectrum disorder.

Methods

This study used an adapted alternating treatment design to compare the effects of robots and computers on learning action word symbols of AAC. Twenty symbols of action words in the ‘Ewha-AAC symbols set (Park et al., 2014; Yeon et al., 2016)’ were selected and divided into two groups: Set A and Set B. Symbols of Set A and Set B were exposed through a robot and computer during three phases, which were the baseline, experimental, and generalization phases.

Results

All three subjects who were trained using the educational robot showed improvements in comprehending action word symbols. However, only two out of the three subjects showed improvements in learning with the computer. In the case of generalization phase, all three subjects who were trained using the educational robot and the computer showed improvements. Comparing the improvement levels between the two interventions in both phases, the effectiveness of the educational robot was generally higher than that of the computer.

Conclusion

Both methods had positive effects on learning action word symbols of AAC, while the robot-based intervention showed higher improvement levels than the computer-based intervention.

Augmentative and alternative communication (AAC) is a communication method which utilizes different types of non-vocal communication to supplement or replace verbal interactions. It is composed of different components and formats which include techniques, strategies, supplementary tools, and symbols. Furthermore, the scope of AAC extends to all fields that help stimulate language ability and solve difficulties faced by people who have communication problems (Von Tetzchner, 1996).

AAC systems can be categorized into two parts: unaided communication systems and aided communication systems. Aided communication systems are efficient in conducting accurate and complex communication. Unlike aided communication systems, which require the use of physical tools, unaided communication systems can be used conveniently in everyday applications as they only require the use of facial expressions and body movements. However, the unaided communication systems also hold disadvantages for people who have difficulties or are unable to move certain body parts, like patients with brain injuries. Furthermore, people may misunderstand the body movements, or may not be able to communicate with one another if the interacting person is unaware of the system (Park, 2000).

However, the use of AAC systems does not directly imply smooth communication between subjects, as the users must first learn how to utilize various types of symbols and the meanings implied into them. Furthermore, users must be able to adequately select and utilize different methods such as direct input, eye-gazing, and scanning, and must procure the necessary ability to form AAC strategies needed to increase speed, create full sentences and increase message timing. Due to such reasons, the effective learning of symbols is as an essential factor in increasing its usage (Light, 1989; Light, Arnold, & Clark, 2003; Light, Roberts, DiMarco, & Greiner, 1998).

According to Bloom (1991), verbs play an important role in the child’s language development. Nouns may play an important role for children during the single word expression phase, but the combined use of nouns and different parts of speech are essential to acquire syntactic ability. Therefore, verbs play an important part in developing grammar and narration. Difficulties in expressing verbs terms are found in the AAC system due to the vocabulary-symbol interpretation phase. As verbs implicate movement, they are relatively harder to interpret than nouns, as it is necessary for these movements to be converted into stationary images. Such characteristics can cause AAC users to face difficulties in comprehending and utilizing the verb symbols.

A study has reported that the word finding ability of children with disabilities are affected by different parts of speech (Menyuk & Quill, 1985). This study indicates that learning words with relational meaning, such as verbs, prepositions, and adjectives are harder to learn than non-relational words. Thurber and Tager-Flusberg (1993) also stated that children with disabilities show a significantly lower word finding ability than normal children in regards to verbs. Such reports prove the difficulties faced by children with disabilities in learning verbs.

Computer-based interventions have been widely implemented in clinical settings. Also, together with the advancement of technology and the robot industry, the role of robots in our society expands day by day. In addition to such facts, the use of robots as supplemental tools is also expanding in the field of education. Currently, numerous educational robot developments and studies are being conducted. Therefore, considerations for the efficient use of computers and robots in educational and clinical settings have also increased.

Computers can also serve an important role. Not only can computer-based activities facilitate a broader range of educational activities to meet the various needs of children with mild learning disorders, but adaptive technologies now exist that can even enable children with severe disabilities to become active learners (Hasselbring, & Williams Glaser, 2000). Research on the utilization of computer-based interventions in speech and language pathology of children has been actively performed since the late 1980s (Kim, Kim, & Park, 2005). In the area of language therapy, various pictures are used by children to study words. Unlike traditional pictures that cannot show motion, computers can present pictures with motion so they can generate a higher intervention effect and attract children’s attention and concentration abilities. According to the research that introduced motion words using computer animations, the children’s verb expression accuracy improved and they showed a higher stimulus generalization toward pictures drawn with lines (Kim et al., 2005). Also, the research that targeted patients with aphasia showed that using computer programs for treatment helped the patients improve their comprehension and expressive abilities (Chung, Kim, Sim, Nam, & Kwon, 2003).

Recently, domestic studies have reported beneficial effects for language and social abilities through robot-based interventions. To evaluate the educational effects from using the robots, the multimedia medium was compared for analysis. It was found that the story-telling conducted by robots helped in increasing the children’s word recognition, comprehension, and interpretation abilities (Hyun, Kim, & Jang, 2008). Furthermore, according to the comparison between role play and robot utilized play, the robot utilized play was found to lead to more verbal and non-verbal interactions (Yong, Kim, Park, & Hyun, 2012). According to another study utilizing robots, the results showed that robots provide more optimistic effects in increasing the overall social ability of the children (Kim, & Lee, 2013). These studies prove the potential of robots as efficient supplements in the educational field.

Children with autism spectrum disorders have shown an interest in machine-like targets that are simple and operable. Thus, they enjoy interaction with the computer (Powell, 1996). However, the computer is fixed at a certain place, and in the form of interaction it is a method of unilateral stimulation (Powell, 1996). According to the study of Bernard-Opitz, Sriram, and Nakhoda-Sapuan (2001), there is a limitation to learning through the computer in daily life. However, a robot, instead of doing unilateral interaction through a screen, moves and provides two-way interactions (Robins, Daut-enhahn, & Dubowski, 2004). A robot has advantages such as various ranges of stimulations using software technologies, manipulation of specific shapes, and provisions for voluntarily induced and interested motives (Dautenhahn & Werry, 2004). Due to such facts, robots are currently being used as alternatives for intervention. Studies have emphasized the potential of robot-child interaction for inducing different types of responses from children with autism spectrum disorder (Lee, Kim, Yeon, Park, & Park, 2013). According to the study conducted by Pioggia et al. (2005), when interaction is stimulated through the use of robots, children with autism spectrum disorder showed increased levels of heart rate stability compared to normal people. This suggests means that children with autism spectrum disorder felt more comfortable with robots. Additionally, children with autism spectrum disorders stated that the robots seemed sad, and named them with names learned through story telling.

In accordance with such research, this study focused on exploring the effects of robot and computer-based interventions on learning the action word symbols of AAC for children with autism spectrum disorder. The experimental questions were as follows: (1) Are there any differences between the effect of robots and computers on learning action word symbols of AAC for children with autism spectrum disorder? (2) Will the effect of learning action word symbols of AAC generalize from individual words to words in a story context?

METHODS

Participants

Three preschool children with hospital-diagnosed autism spectrum disorder participated in the study. At first, four children participated in the study, but one of them was finally excluded because a stable baseline was not secured. Each subject was recommended by their therapists as having the ability to point and retain a proper posture for some period time. All of them had no experience with any type of AAC system or educational robots.

Each subject’s receptive vocabulary age on Receptive Expressive Vocabulary Test-Receptive (REVT-R; Kim, Hong, Kim, Jang, & Lee, 2009) was under 30 months. They have no physical difficulty using AAC as a result of Paradise AAC Assessment (PAA; Park, Kim, & Kim, 2008), and they are over perlocutionary level on Korean augmentative and alternative communication: assessment and intervention for special educators and speech-language pathologists (Kim, Park, Han, & Ku, 2016). Last, all of them understood the 20 action words used in the study. Characteristics of the participants are presented in Table 1.

Characteristics of the participants

Experimental materials

Assessment materials

Communication boards: Communication boards with four static cells were used in the assessment phase. Static AAC symbols of the action words were selected from the Ewha-AAC symbols set and placed into the cells (Park et al., 2014). Three out of the four cells were filled with foils that had been randomly selected from the 20 action word symbols, and the remaining cell was filled with an action word symbol designated as an answer. The subjects were asked to point to the correct answer among the four action word symbols provided.

Storybooks for assessment: Two stories, which contained the 10 action words from Sets A and B, were used in the assessment phase. The story with Set A was about a ‘small concert’ and consisted of nine sentences. The story with Set B was about ‘catching a ball’ and consisted of eight sentences. Afterward, two A4-sized storybooks were produced with these stories and nine colored pictures. The stories of Set A and B are presented in Appendix 1.

Intervention materials

Action words: Action words were selected from a list of high frequency action words acquired under age 3 by more than 75% of children (Pae & Kwak, 2011). Afterward 20 action words were selected which were suitable for robot motions. Every subject of this study understood these 20 action words. The 20 action words were randomly arranged into one of the two sets (A or B) and used from baseline to generalization. Robot-based intervention and computer-based intervention were implemented with different sets of words. The list of action words is presented in Table 2.

The list of action words

Storybooks for intervention: Two stories were provided to arouse interest in the participants. These stories were made up of the 10 action words from Sets A and B. The story with Set A was about a ‘birthday party’ and consisted of 10 sentences. The story with Set B was about ‘a kitten in the night’ and consisted of 10 sentences. Afterward, two A4-sized storybooks were produced with these stories and nine colored pictures for each storybook. The stories of Set A and B for intervention are presented in Appendix 2.

AAC symbols of action words: The 20 static AAC symbols of action words were selected from the Ewha-AAC symbols set (Park et al., 2014). Examples of the AAC symbols of the action words are presented in Appendix 3.

Educational robot: The robot provided a touchscreen for displaying the action word symbols and moving the head, arms, and wheels for showing the robot’s motions corresponding to the action word symbols, which were made with a combination of the robot moving its head, arms and wheels, in addition to facial expressions. The robot’s head was able to move up and down as well as left and right. The arms could move up and down separately or simultaneously. The wheels moved the robot forward and backward. The face was able to express emotions, such as happiness, disappointment, shyness and astonishment. In addition to a neutral expression, the face could appear sleepy. The face had lips that could simulate lip-syncing. The 20 motions of the robot for the action word symbols are listed in Table 3.

Robot’s motions for action word symbols

To evaluate the validity of the robot’s motions, 25 graduate students from the Department of Communication Disorders participated and were asked to score validity. Collected validity scores were averaged and used to counterbalance the level of difficulties between Sets A and B. The averaged validity scores are presented in Table 4.

Validity of robot's motions for action word symbols

Experimental design and procedures

Experimental design

This study employed a single subject, alternating-treatment design (ATD) to assess the effects of two interventions, robot-based, and computer-based, on learning action word symbols. The robot- and computer-based interventions were implemented alternately in order to compare the effects of both interventions on the learning of action word symbols. The learning sessions were conducted twice a week for 10 sessions.

ATD is a research method by which researchers conduct several interventions by turns with one subject and compare the efficiency among the interventions. In this manner, the application time among the interventions is balanced for comparison, and internal validity is established (Lee, Park, & Kim, 2000).

Experimental place and schedule

Every phase of the experiment was implemented at each subject’s house. Robot- and computer-based interventions were implemented alternately twice a week.

Experimental procedure

This study involved three phases: the baseline, experimental and generalization phases.

Baseline: In the baseline phase, the researcher presented the 10 action words from Set A and the other 10 action words from Set B. Then, the subjects were asked to point to the symbols that matched the action words verbally provided by the researcher. This phase was continued until stable baseline data for the three sessions was established.

Experimental phase: Each session of the experimental phase involved assessment and intervention. The assessment of this phase was conducted before intervention in the same way as the assessment of the baseline phase.

For intervention phases, the robot- and computer-based interventions were implemented alternately. Each intervention was implemented with a different set of words. At this point, in the case of Subject 1, Set A was implemented by the robot and Set B was implemented by computer. On the contrary, in the case of Subject 2 and Subject 3, Set A was implemented by computer, and Set B was implemented by the robot. The number of intervention sessions was basically 10, and the interventions were conducted under the closing criteria. The intervention would end early if the subjects gave all correct responses for three sessions for both conditions: action word symbols trained by robot- and computer-based AAC.

The procedure of the intervention was as follows. First, the storybooks containing the action words of Sets A and B were read aloud to the subjects. On each occasion that an action word was read aloud, the relevant action word symbols with accompanying verbal labels were displayed to the subjects. At this point, in the case of robot-based intervention, the action word symbols with the robot’s motions and verbal labels were displayed to the subjects. On the contrary, in the case of computer-based intervention, action word symbols with the researcher’s verbal labels were displayed. To sum up, robot-based intervention provided static action word symbols through the robot’ monitor, motion and verbal labels, while computer-based intervention provided static action word symbols through the computer monitor with the researcher’s verbal labels.

Generalization phase: Assessments were implemented before the baseline and after the experimental phases to figure out whether the effects of learning on action word symbols would be generalized from individual words to words in the context of stories. During this phase, subjects were asked to answer by pointing to symbols on communication boards about questions assessing the story, which were designed to have the action words as the answers.

Data analysis

Every session from the baseline to the generalization phases was recorded and scored. Each correct answer was scored as 1 point, and each incorrect answer was scored as 0 points.

To calculate the sizes of the effects of the interventions, the improvement rate difference (IRD) was calculated. IRD is calculated as the difference between the improved data points of a baseline and those in a treatment phase. The range of IRD scores is from 1.0 to –1.0. IRD scores of .50 and below indicate very small effects, while those of around .50 to .70 show moderate-size effects. IRD scores of around .70 to .75 indicate large effects, and IRD scores of higher than .75 are very large effects. Negative scores show that the results have deteriorated below those of the baseline phase (Parker, Vannest, & Brown, 2009).

Data reliability

To demonstrate the reliability of grading, a graduate student in the department of communication disorders was designated as a second observer, who was briefed on the study, the subjects and the method of assessment, and then scored by observing the recorded materials. Intervention was stopped after 90% accordance between observers occurred three times. Twenty percent of the recorded materials among the experimental phases were randomly selected, and all two sessions of the generalization phases were selected, scored and compared between the two observers. The intra-rater reliability was calculated by dividing the number of agreements by the number of agreements plus disagreements and multiplying the quotient by 100. The intra-rater reliability between the two observers was 96% for individual action word symbols, and 93% for action word symbols in the context of the stories.

Intervention fidelity

To ensure that the researcher consistently implemented robot and computer-based intervention, a speech-language therapist who has over 3 years of experience in speech therapy checked the researcher’s performance. The questions for evaluating the fidelity were as follows: (1) Did the robot and computer work well? (2) Were the robot and computer easily accessible to the subject? (3) Were the robot and computer arranged well for the subject to watch? (4) Was the material of story prepared properly? (5) Was the material of story used properly? (6) Were the symbols presented properly by the robot and computer?

Three sessions of interventions for each subjects were randomly selected and evaluated. Properly practiced items were scored as 1 point, and improperly practiced items were scored as 0 points. The score of intervention fidelity was calculated by dividing the score of intervention item which was implemented by the total score of item for intervention level which had to be implemented, and then multiplied by 100 into a percentage. The intervention fidelity for three subjects were all high: 100% on Subject 1, 94% on Subject 2, and 100% on Subject 3.

Social validity

To ensure that the robot and computer-based intervention on learning action word symbols was socially valid, the caregivers of the subjects were surveyed using a 5-point scale. The score 5 was the most positive response and the score 1 was the least positive response for each question. The questions were as follows: (1) Did the children enjoy the intervention with the robot? 2) (Did the children enjoy the intervention with computer? (3) Can the children express action words by using AAC? (4) Do you think the intervention was helpful for the children? As a result, the average social validity score for 4 items was 4.5 on Subject 1, 3.0 on Subject 2, and 4.75 on Subject 3.

RESULTS

Comparison between comprehension of action word symbols trained by robot and by computer

The results of the comparison between the comprehension of action word symbols trained by the robot and trained by the computer are presented in Table 5 and Figure 1. Two out of the three subjects (Subjects 1 and 2) had higher mean comprehension scores when trained by the robot in the intervention phase, although the means of scores trained by the computer were higher in the baseline phase. However, in the case of Subject 3, the mean of scores trained by the computer was higher in the intervention phase, although the mean of scores trained by the robot was higher in the baseline phase.

Results of comprehension scores for each subject

Figure 1.

Summary of comprehension scores of action word symbols.

In the case of Subject 1, both interventions showed improvements in the comprehension of action word symbols with a result of IRD 1 by the robot and of .88 by the computer. When comparing the efficiency between the two interventions on Subject 1 after intervention by the robot, the 4th, 6th, and 8th sessions showed higher scores and a stable trend with scores of more than 9. When the subject was trained by the computer, the 5th and 7th sessions showed higher scores and a relatively unstable trend when compared to robot intervention. Subject 1 showed the first perfect score at the 8th session in the case of robot intervention, and showed the first perfect score at the 9th session in the case of computer intervention. Perfect scores for both interventions were seen at the 9th, 10th, and 11th sessions. This result made the intervention end early because it had met the closing criteria, which is triggered by showing correct responses to all 20 action word symbols trained by both interventions in three consecutive assessments.

In the case of Subject 2, intervention by the robot showed improvements in the comprehension of the action word symbols with the IRD being .77, while intervention by the computer showed no improvements with the IRD being 0. The trend of the scores was relatively unstable in comparison with Subjects 1 and 3. When comparing the efficiency between the intervention by the robot and by the computer, the 4th, 7th, and 9th sessions showed the same scores for both interventions. However, the scores of the robot intervention were generally higher than those of the computer. Subject 2 showed the first perfect score at the 10th session in the case of robot intervention, and at the 13th session in the case of computer intervention.

In the case of Subject 3, both interventions showed improvements in the comprehension of the action word symbols with the IRD being 1 for interventions by both the robot and the computer. When comparing the efficiency between both interventions, all sessions, except for the 5th, showed the same scores. Subject 3 showed the first perfect score at the 5th session in the case of computer intervention and at the 6th in the case of robot intervention. Perfect scores were seen for both interventions at the 6th, 7th, and 8th sessions. This result made the intervention end early because it had met the closing criteria.

Comparison between generalization of action word symbols trained by robot and by computer

In the generalization phase, assessments were implemented before the baseline and after the experimental phases to observe whether the effects of learning action word symbols can be generalized from individual words to words in the context of stories. The scores of the action word symbols in the story context are presented in Table 6.

Results of generalization scores for each subject

In the case of Subject 1, the comprehension score of the action word symbols trained by either the robot or the computer showed 0 points for the assessment before the intervention at the generalization phase. On the contrary, for the assessment after the intervention at the generalization phase, the comprehension score trained by the robot showed more progress by 7 points, while that by the computer showed less progress by 4 points. This means that the action word symbols trained by the robot showed better progress in comprehension than by the computer, but Subject 1 showed less comprehension of the action word symbols at the generalization phase than in the last intervention session.

In the case of Subject 2, the comprehension score of the action word symbols trained by either the robot or the computer showed 1 point for the assessment before the intervention at the generalization phase, after which the score by either the robot or the computer showed 2 points. This means that the action word symbols trained both by the robot and the computer showed progress in comprehension, but the degree was low, as there was no difference between the two interventions.

In the case of Subject 3, the comprehension score trained by either the robot or the computer showed 0 points for the assessment before the intervention at the generalization phase. On the contrary, for the assessment after the intervention at the generalization phase, the comprehension score trained by the robot showed more progress by 5 points, while the training by the computer show-ed less progress by 1 point. This means that the action word symbols trained by the robot showed better progress in comprehension than by the computer. However, Subject 3, like the other subjects, showed less comprehension at the generalization phase than in the last intervention session.

CONCLUSION

In this study, the two methods of learning action word symbols with the robot and the computer were alternatively executed in order to determine their effectiveness in children with autism spectrum disorder. The conclusion is as follows.

Firstly, all three subjects who were trained using the educational robot showed big improvements in comprehending action word symbols. However, only two out of the three subjects showed improvements in learning with the computer. When comparing the improvement levels of effectiveness between the two interventions, the educational robot showed higher levels of effects on Subject 1 and Subject 2, while both interventions showed similar levels of effectiveness on Subject 3.

Secondly, all three subjects who were trained using the educational robot and the computer showed improvements in the generalization of comprehending action word symbols. When comparing the improvement levels of generalization between the two interventions, the effectiveness of the educational robot was noticeably higher than that of the computer on Subject 1 and Subject 3, while both interventions showed the same levels of generalization on Subject 2.

The following will discuss the implications of such study results and the limitations of the study.

Comprehension of individual action word symbols

This study showed that all of the subjects who were trained using the educational robot showed improvements in comprehending action word symbols. However, only two out of the three subjects showed improvements in learning with the computer. When comparing the improvement level of effectiveness between the two interventions, the educational robot showed higher levels of effects on Subject 1 and Subject 2, while both interventions showed similar levels of effectiveness on Subject 3.

One of the possible variables that affected the improvement of comprehending action word symbols through the educational robot and computer was the presentation of verbal labels. When children with the autism spectrum disorder acquire new words, presenting verbal labels is more effective than not presenting them (McDuffie, Yoder, & Stone, 2006). In Baldwin and Markman (1989), it was shown that pointing with the verbal labels was more likely to draw participants’ attention to the new object, compared to not using verbal labels. In this study, verbal labels were provided when action word symbols were exposed to the subjects, and it appears to have influenced the results in a positive way.

Next, utilizing assistive technologies, the robot and computer in this study, could be considered one reason for the improvement on learning action word symbols. This is because the assistive technology is an effective approach for enhancing the linguistic ability in children with autism spectrum disorder (Hong, 2008).

In this study, intervention through the educational robot showed higher improvement in action word symbol comprehension than intervention through a computer. Considering that one of the biggest differences between robot-based intervention and computer-based intervention was the existence of the robot’s motions, it might be possible that the robot’s motions helped the subjects understand or remember the action word symbols. Various difficulties have been found in expressing movement with symbols (Mineo, Peischl, & Pennington, 2008), and therefore action word symbols, which are expressed by static pictures only, cannot provide enough information and may cause confusion. In other words, the static pictures that were presented as action word symbols through the robot and computer screens might not be enough to express the meaning of the words. In this regard, the motions of the robot might have been effective in supplementing the static action word symbols.

A higher improvement rate by robot-based intervention also indicates that the subjects showed more interest in the educational robot. This is consistent with previous findings that elicited notable changes in disabled children with an approach using robots (Cook, Adams, Encarnação, & Alvarez, 2012; Robins, Dautenhahn, Te Boekhorst, & Billard, 2005). Two subjects preferred the educational robot over the computer during the intervention stage. Subject 1 showed high interest towards the robot’s movements. According to the caregiver’s report, the child showed attachment behaviors that were never shown before such as expecting the intervention session or looking for the robot. In the case of Subject 2, it was observed that there was a higher imitation ratio towards the robotic voice than the researcher’s voice. Unlike Subjects 1 and 2, Subject 3’s levels of interest or reaction toward the educational robot and the computer were almost identical, which seemed to be influenced by the child’s high preference for keyboards. Thus, no difference between preferences toward the two interventions seems to affect the similar levels of improvement.

The subjects’ comprehension of action word symbols was more flexible in the baseline stage and beginning stage of intervention. Their comprehension of action word symbols as well as concentration level during intervention rose towards the latter stages. This was due to the fact that the subjects’ rejection of the researcher or intervention task diminished over time. In the beginning stage of intervention, it was difficult to keep the children seated because they were not accustomed to the researcher and intervention task, which could have influenced the intervention results. Children with autism spectrum disorder have particular difficulties in adjusting to new tasks. Therefore, it is suggested for researchers to establish sufficient rapport with the child before the intervention and make the child execute preliminary tasks that have no influence on the intervention at the same time.

Through this study, it became clear that the robot-based intervention and the computer-based intervention are both effective methods for learning action word symbols, and that a higher level of improvement can be elicited with the educational robot. Using the educational robot to learn action word symbols for utilizing AAC may be effective, and it will be possible to use the robot-based intervention for communication through AAC (Jeon, Yeon, Kim, Song, & Kim, 2014).

Generalization of action word symbols

In order to investigate whether the effects of learning action word symbols were generalized from comprehending individual words to comprehending words in the context of stories, the level of comprehension of action word symbols before and after intervention was evaluated. As a result, when trained in action word symbols by using the educational robot and computer, all three subjects showed an improvement in comprehending action word symbols in the context of stories. When comparing the intervention effects of the educational robot and that of the computer, the educational robot showed a higher generalization for Subject 1 and Subject 3, while both methods showed similar levels of generalization for Subject 2. These results show that the comprehension of action word symbols can be generalized from individual action word symbols to the context of stories as well, in the same context as the robot eliciting higher results in individual action word symbols.

One more noticeable finding in the study was the improved level of comprehension. Regarding the symbols of individual action words, the robot-based intervention showed a higher improvement than the computer-based intervention, but a clearer improvement level was presented in the generalization stage. It is assumed that the level of difficulty of the assessment affected the result. In the stage of generalization, the assessment required answering questions about the stories, so it was more difficult than the assessment in the stage of intervention, which only tested comprehension of individual words. In other words, there was not much difference shown between the two interventions when the level of assessment difficulty was relatively low, while a more clear difference was shown when the level of assignment difficulty was relatively high.

Subject 2 showed a relatively low level of improvement in both interventions. This is due to the fact that sufficient intervention on the symbols of action words was not performed. Concerning the intervention result, both interventions were completed early by Subject 1 and Subject 3 when they received 100% for 3 consecutive sessions while Subject 2 received 100% for the first time on the 10th session. In addition, Subject 2 showed an unstable improvement trend in comprehension score compared to other subjects. In other words, Subject 2 had a lack of comprehension in action words compared to the other subjects, which affected generalization.

Limitations and suggestions for further research

The current study has several limitations. First, the sample size was small, so it is difficult to generalize the results of this study. Second, the levels of subjects were not widely and precisely considered. A follow-up study is needed to consider the levels of subjects in various aspects and reflect them in future studies more. Third, the tasks of this study were quite simple, so it might not be enough to determine the genuine effectiveness of both methods of intervention. It is necessary to examine the effects of interventions with more difficult tasks in future studies.

References

Baldwin, D.A., & Markman, E.M. (1989). Establishing word-object relations: a first step. Child Development, 60, 381–398.
Bernard-Opitz, V., Sriram, N., & Nakhoda-Sapuan, S. (2001). Enhancing social problem solving in children with autism and normal children through computer-assisted instruction. Journal of Autism and Developmental Disorders, 31, 377–384.
Bloom, L. (1991). Language development from two to three. Cambridge: Cambridge University Press.
Chung, Y.J., Kim, Y.T., Sim, H.S., Nam, K.C., & Kwon, M.S. (2003). Treatment efficacy: the computer as a treatment supplement for improving word comprehension and production in aphasic adults. Korean Journal of Communication Disorders, 8, 112–133.
Cook, A.M., Adams, K., Encarnação, P., & Alvarez, L. (2012). The role of assisted manipulation in cognitive development. Developmental Neurorehabilitation, 15, 136–148.
Dautenhahn, K., & Werry, I. (2004). Towards interactive robots in autism therapy: background, motivation and challenges. Pragmatics & Cognition, 12, 1–35.
Hasselbring, T.S., & William Glaser, C.H. (2000). Use of computer technology to help students with special needs. The Future of Children, 10, 102–122.
Hong, J.A. (2008). A review on high-tech assistive technologies in language education for the students with autism spectrum disorders. Special Education Research, 7, 5–25.
Hyun, E.J., Kim, S.Y., & Jang, S.K. (2008). Effects of a language activity using an “intelligent” robot on the linguistic abilities of young children. International Journal of Early Childhood Education, 28, 175–196.
Jeon, K.H., Yeon, S.J., Kim, Y.T., Song, S., & Kim, J. (2014). Robot-based augmentative and alternative communication for nonverbal children with communication disorders. In : Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing. (pp. 853–859.
Kim, G.C., & Lee, G.E. (2013). The effects of activities with teaching assistant robots on children’s social competence. Korean Journal of Early Childhood Education, 15, 29–49.
Kim, Y.T., Hong, G.H., Kim, K.H., Jang, H.S., & Lee, J.Y. (2009). Receptive & expressive vocabulary test (REVT). Seoul: Seoul Community Rehabilitation Center.
Kim, Y.T., Kim, Y.R., & Park, S.H. (2005). The effect of animation of CAI on the stimulus generalization to line drawing and the MLU. Journal of Speech and Hearing Disorders, 14, 81–96.
Kim, Y.T., Park, E., Han, S.K., & Ku, J.A. (2016). Korean augmentative and alternative communication: assessment and intervention for special educators and speech-language pathologists. Seoul: Hakjisa.
Lee, S.H., Park, E.H., & Kim, Y.T. (2000). Single subject research in educational and clinical settings. Seoul: Hakjisa.
Lee, S.J., Kim, Y.T., Yeon, S.J., Park, S.H., & Park, M.G. (2013). Experiences and demands of early childhood teachers regarding the use of assistive robots for the communication intervention. Korean Journal of Early Childhood Special Education, 13, 71–90.
Light, J. (1989). Toward a definition of communicative competence for individuals using augmentative and alternative communication systems. Augmentative and Alternative Communication, 5, 137–144.
Light, J., Arnold, K.B., & Clark, E.A. (2003). Finding a place in the “social circle of life”. Light, J. Communicative competence for individuals who use AAC: from research to effective practice. (pp. 361–397). Baltimore, MD: Paul H. Brookes Publishing.
Light, J., Roberts, B., DiMarco, R., & Greiner, N. (1998). Augmentative and alternative communication to support receptive and expressive communication for people with autism. Journal of Communication Disorders, 31, 153–180.
McDuffie, A.S., Yoder, P.J., & Stone, W.L. (2006). Labels increase attention to novel objects in children with autism and comprehension-matched children with typical development. Autism, 10, 288–301.
Menyuk, P., & Quill, K. (1985). Semantic problems in autistic children. Schopler, E., & Mesibov, G.B. Communication problems in autism. (pp. 127–145). New York, NY: Springer.
Mineo, B.A., Peischl, D., & Pennington, C. (2008). Moving targets: the effect of animation on identification of action word representations. Augmentative and Alternative Communication, 24, 162–173.
Pae, S.Y., & Kwak, K.J. (2011). Korean MacArthur-Bates Communicative Development Inventories (K M-B CDI) user’s guide and technical manual. Seoul: Mindpress.
Park, E. (2000). Issues in language development of augmentative and alternative communication users. Korean Journal of Communication Disorders, 5, 5–32.
Park, E.H., Kim, Y.T., & Kim, J.Y. (2008). Paradise AAC assessment. Seoul: Paradise Welfare Foundation.
Park, E.H., Kim, Y.T., Lim, J.H., Kim, K.Y., Yeon, S.J., Park, K.T., & Lee, Y.S. (2014). Development of Korean AAC program: culturally appropriate symbols and linguistic features. In : Paper presented at the 16th biennial conference of the International Society for Augmentative and Alternative Communication. Lisbon, Portugal.
Parker, R.I., Vannest, K.J., & Brown, L. (2009). The improvement rate difference for single-case research. Exceptional Children, 75, 135–150.
Pioggia, G., Igliozzi, R., Ferro, M., Ahluwalia, A., Muratori, F., & De Rossi, D. (2005). An Android for enhancing social skills and emotion recognition in people with autism. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 13, 507–515.
Powell, S. (1996). The use of computers in teaching people with autism. Shattock, P., & Linfoot, G. Autism on the Agenda. London: National Autistic Society.
Robins, B., Dautenhahn, K., & Dubowski, J. (2004). Investigating autistic children’s attitudes towards strangers with the theatrical robot: a new experimental paradigm in human-robot interaction studies. In : Proceedings of 13th IEEE International Workshop on Robot and Human Interactive Communication. (pp. 557–562.
Robins, B., Dautenhahn, K., Te Boekhorst, R., & Billard, A. (2005). Robotic assistants in therapy and education of children with autism: can a small humanoid robot help encourage social interaction skills? Universal Access in the Information Society, 4, 105–120.
Thurber, C., & Tager-Flusberg, H. (1993). Pauses in the narratives produced by autistic, mentally retarded, and normal children as an index of cognitive demand. Journal of Autism and Developmental Disorders, 23, 309–322.
Von Tetzchner, S. (1996). troduction to language development. Bjorck-Akesson, E., & Lindsay, P. Communication Naturally: Theoretical and methodological issues in augmentative and alternative communication: Proceedings of the 4th ISAAC Research Symposium. Vasteras: Malardalen University Press.
Yeon, S.J., Kim, Y.T., & Park, E. (2016). Transparency and name agreement of Korean Ewha-AAC symbols: nouns, verbs, and adjectives. AAC Research & Practice, 4, 45–63.
Yong, S.Y., Kim, K.A., Park, M.A., & Hyun, E.J. (2012). The comparison study of young children’s social interaction and playfulness in play situations: between teaching assistant robot situation and a role play situation. Korea Open Association for Early Childhood Education, 17, 23–48.

Appendices

Appendix 1.

Stories for assessment

csd-21-4-744-app1.pdf

Appendix 2.

Stories for intervention

csd-21-4-744-app2.pdf

Appendix 3.

Examples of action word symbols

csd-21-4-744-app3.pdf

Article information Continued

Figure 1.

Summary of comprehension scores of action word symbols.

Table 1.

Characteristics of the participants

Subject 1 Subject 2 Subject 3
CA (mo) 83 70 76
Sex M F M
Diagnosis ASD ASD ASD
REVT-R <30 months (score=19) <30 months (score=16) <30 months (score = 21)
K M-B CDI
 Receptive vocabulary 577 352 488
 Expressive vocabulary 428 321 377
Communication skill He can express what he wants by using a word or two word combination mostly including nouns. She can express what she wants by using a noun. He can express what he wants by using a word or two word combination mostly including nouns.
Motor ability No limitation for pointing with their fingers to a specific visual symbol among four given visual symbols.

CA=chronological age; ASD=autism spectrum disorder; REVT-R=Receptive Expressive Vocabulary Test-Receptive (Kim, Hong, Kim, Jang, & Lee, 2009); K M-B CDI=Korean MacArthur-Bates Communicative Development Inventories (Pae & Kwak, 2011).

Table 2.

The list of action words

No. Set A Set B
1 Dance Kiss
2 Blow Sleep
3 Look Look back
4 Walk Shake head
5 Lift Step aside
6 Push Give
7 Hit Speak
8 Cry Go out
9 Hurray Throw
10 Shake hands Laugh

Table 3.

Robot’s motions for action word symbols

No. Symbol Robot’s motions
Face Head Arms Wheels LED
1 Walk Normal Swing arms alternately Forward Arms
Wheels
2 Throw Normal Raise right arm and turn back Forward Right arm
Wheels
3 Give Normal Downside Raise arms to the middle Both sides of the head
Arms
4 Speak Lip-syncing Nod Both sides of the head
5 Look Normal Move diagonally up and down Both sides of the head
6 Shake hands Normal Raise right arm to the middle Right arm
7 Go out Normal Raise right arm to the middle Turn right and move forward Right arm
Wheels
8 Dance Happy Swing arms alternately Turn side to side Both sides of the head
Arms
Wheels
9 Laugh Happy Raise arms over the head Both sides of the head
Arms
10 Sleep Sleepy Downside Both sides of the head
11 Cry Disappointed Shake from side to side Swing arms alternately Both sides of the head
Arms
12 Shake head Normal Shake from side to side Both sides of the head
13 Hit Normal Raise right arm to the middle and turn back Right arm
14 Blow Astonished Upside Both sides of the head
15 Push Normal Raise right arm to the middle Forward Arms
Wheels
16 Step aside Normal Left side Wheels
17 Lift Normal Raise right arm over the head Right arm
18 Look back Normal Left side Wheels
19 Kiss Astonished Both sides of the head
Wheels
20 Hurray Normal Upside Raise arms over the head Both sides of the head
Arms

LED=light emitting diode.

Table 4.

Validity of robot's motions for action word symbols

No. Set A Set B Note
1 Blow Kiss Similar motion
2 Dance Sleep
3 Look Speak Similar motion
4 Walk Go out
5 Lift Step aside
6 Push Give Similar motion
7 Hit Throw Similar motion
8 Cry Shake head Similar motion
9 Hurray Look back
10 Shake hands Laugh
Mean 3.6 out of 5 3.4 out of 5

Table 5.

Results of comprehension scores for each subject

Subject 1
Subject 2
Subject 3
Robot Computer Robot Computer Robot Computer
Baseline 5.3 (4–7) 6 (4–7) 5 (4–7) 7.3 (7–8) 7 (6–8) 6 (4–8)
Intervention phase 9.5 (9–10) 9.1 (7–10) 8 (6–10) 7.3 (5–10) 9.6 (9–10) 9.8 (9–10)
IRD 1 .88 .77 0 1 1
Generalization phase 3.5 (0–7) 2 (0–4) 1.5 (1–2) 1.5 (1–2) 2.5 (0–5) .5 (0–1)

Values are presented as mean (range).

IRD=improvement rate difference.

Table 6.

Results of generalization scores for each subject

Subject 1
Subject 2
Subject 3
Robot Computer Robot Computer Robot Computer
Pre 0 0 1 1 0 0
Post 7 4 2 2 5 1
Mean (range) 3.5 (0–7) 2 (0–4) 1.5 (1–2) 1.5 (1–2) 2.5 (0–5) 0.5 (0–1)