The purpose of this study was to evaluate the effects a parent training program in Korea using the Teacher Performance Rate Accuracy Scale (TPRA) during training and video self-monitoring on the rates of accurate discrete trial Teaching (DTT) and children's correct responses. The study was conducted in an early intensive behavior intervention (EIBI) program of a public children's hospital in Korea. Two mothers who had children enrolled in the EIBI services participated in the study. A delayed multiple baseline across participants design was used. The training program consisted of individual coaching with therapist-conducted TPRAs and group instruction on parent self-monitoring using TPRA for self-evaluations. Results indicated that each parent's average rates of accurate DTT presentations increased and the rates of error presentation decreased after the intervention. As a result, their children's average percentage of correct responses increased during intervention and maintained after intervention. Results of this study suggest that this training program is promising and has potential for a large-scale implication in Korea.

Children with autism spectrum disorder (ASD) are challenged in various areas of functioning due to their deficits in social communication as well as repetitive movements and restricted patterns of interest (Diagnostic and statistical..., 2013). As a consequence, they often display problem behaviors, and their families need more education in effective parenting skills in order to meet the needs of their children. Parent training has long been recognized as an important factor affecting children's intervention outcomes and should be included as part of any early intervention program to promote generalization from treatment settings to home or community settings 12Educating children with..., 2001)most of the children were severely disturbed, having symptoms indicating an extremely poor prognosis. The children were treated in separate groups, and some were treated more than once, allowing for within-and between-subject replications of treatment effects. We have employed reliable measures of generalization across situations and behaviors as well as across time (follow-up.

Interventions derived from the principles of applied behavior analysis (ABA) are the most effective for individuals with ASD 345changes in intelligence and adaptive behavior scores were statistically significant in favor of the behavioral intervention group (effect sizes of 1.13 for Intelligence quotient (IQ. Discrete trial teaching (DTT) is a basic form of instruction based on the principles of ABA with the most empirical support 6) DTT is especially useful for teaching new forms of behavior (e.g., speech sounds or motor movements that the child previously could not make and is highly effective when implemented with fidelity 78. Effective early intervention programs in the form of early intensive behavioral intervention (EIBI) for children with ASD usually include DTT as an essential part of intervention in both structured as well as natural settings. The demands of training individuals to deliver DTT have increased rapidly, as the number of children diagnosed with ASD is rising in recent years. 9 estimated the prevalence of ASD in South Korea as 2.64%, or approximately 1 in 38 children, and concluded that children ASD was underdiagnosed in South Korea. The number of qualified professionals to provide ASD services does not meet the high demands, and therefore, parent training has also become a preferred treatment option 10). Currently, there are a total of 51 board certified behavior analysts across south Korea (Behavior Analyst Certification Board, 2018). The needs of many families of children with ASD in Korea remain unmet. Besides efficient use of professionals, parent training also has many benefits because parents are the most available human resources to their children, and training can empower them to educate their children. It is imperative to develop parent training programs and teach parents important skills, such as DTT, in order to increase the number of families receiving services in Korea.

Most professional or parent training on delivering DTT included multi-component programs with various effective instructional tactics. The components involved in DTT training programs consisted of written or vocal instructions, frequent quizzes to assess mastery, modeling or demonstrations, practice opportunities, and feedback on data collection and performance 11. The format of training delivery includes traditional face-to-face and computer-based self-training programs (e.g., 12. Video modeling was frequently utilized to present teaching demonstrations in these training programs (e.g., 1314.

One of the most important components in training programs pertains to how feedback on participants' performance is provided to strengthen fidelity of DTT delivery. For example, 15 used video feedback for the participants to self-evaluate their own performance on DTT from videos with a checklist. Similarly, a discrete trial teaching evaluation form (DTTEF) was designed and used to train direct therapists and parents to implement DTT accurately and reliably 1617. The research suggested the utility of an objective observation form containing elements of DTT in providing specific feedback and also enhancing fidelity of implementations.

In the school setting, 18 reported a Teacher Performance Rate and Accuracy (TPRA) evaluation form to evaluate teacher performance on delivering accurate three-term contingencies and their relationship with improved teacher performance as well as student learning. Specifically, a higher number of TPRA observations with specific feedback conducted by supervisors not only increased teacher productivity but also correlated to improved student performance for students with a variety of developmental disabilities 19. In a TPRA observation, the teacher and supervisor selected a student and one of the instructional programs with the definition of the target behavior. They determined reinforcers based on child's preference assessment and the reinforcement schedule and also identified antecedent/consequences for correct/incorrect student responses. The teacher then delivered instruction while the supervisor conducted a TPRA observation. Immediately following teacher instruction, the supervisor provided specific feedback while assessing their point-to-point agreement on student responses in the session. The TPRA was utilized for the purpose of training as well as performance evaluation for teachers, but whether the same form could be applied to parent training remains a question. 20 trained parent DTT implementations in a preschool setting using instructions, rehearsal, modeling and oral feedback. Parent fidelity increased from a low level during baseline to a high level after training was completed, and their children's correct responses also increased as a result of accurate parent DTT implementations. They videotaped the sessions for scoring of parent performance and did not use any observation form during observations. Besides using objective observation forms for training and evaluation of trainees, it is possible to teach trainees to use such a form and conduct self-evaluation from videos (e.g., 15. However, Belfiore et al. conducted staff training but not parent training. No study has been conducted to evaluate using an objective observation form with video self-monitoring as one of the components in teaching parents accurate implementations of DTT instruction. Compared to trainer-provided feedback or video modeling, video self-feedback also has many advantages, such as self-monitoring one's own instruction to facilitate accurate implementations as well as reducing the need for a trainer to be present for feedback.

The ultimate goal of parent training is to improve the child's learning. Parental involvement as one of the multi-components in early intervention adds one layer to ensure children's success in learning. Thus, when evaluating a parent training program, it is important to measure parent competence as well as the child's learning achievements in order to gain a complete picture.

Although ABA is an established treatment in western cultures, empirical studies of ABA for ASD conducted with the Korean population was non-existent prior to 2013 21. It is important for researchers to empirically examine ABA interventions for children with ASD in Korean culture. Given the fact that ASD is highly stigmatized in Korean culture, parents of children with ASD would rather obtain an inaccurate diagnosis, leave the children undiagnosed and untreated, or isolated the family from social interactions 219. Therefore, parent training becomes one of the priorities in early interventions programs for children with ASD. It is extremely important for researchers to take cultural contexts into consideration when developing early intervention and parent training programs. One study concerning parent training for children with ASD in Korea was found 22. The researchers used the TPRA form to evaluate accuracy of parent-implemented DTT using videos for self feedback. Results indicated that parents' accuracy of DTT implementation increased and their children's accurate responses of the instructional programs also increased. However, it was not clear whether the same effect can be generalized to other parents of children with ASD in Korea.

The present study was a replication of the above mentioned study conducted in response to the gap of limited clinical studies of ABA conducted in Korean culture and the lack of qualified ABA professional delivering services in Korea. The present study further replicated with two more parent-child dyads to employ an objective observation form with video self-monitoring in training parents DTT implementations in Korea. The main purpose was to evaluate whether the same effects of the parent training can be generalized to other Korean parents.

Two mother-child dyads (Omin-Itae and Minae-Chai) participated in the present study. Both mothers were in their late thirties, from middle-class families, and had college educations. Itae was diagnosed with ASD and developmental delays by a clinical psychologist using the Korean version of Bayley Scales of Infant Development II 23 and the Korean version of Childhood Autism Rating Scale (CARS) 24. Chai was diagnosed with ASD through psychological evaluations conducted with Social Communication Questionnaire 25, and the CARS.

Both children enrolled in a center-based early intensive behavior intervention (EIBI) program for six hours a day, four days a week. Each child received one-on-one instruction from a therapist. Their mothers stayed in the same room and were trained to deliver instruction to their children in the center. The children were two 4-year old boys. They both were toilet trained and had basic self-help skills. Itae's score on the Korean version of Psychoeducational Profile, Revised (PEP-R) 26 indicated that his mental age was equivalent to 10 months. His receptive and expressive language ability scores on Sequenced Language Scale for Infants, 4 th edition (SLSI-4) 27 were at 13 and 14 months respectively. At the time of the study, his instructional programs included discriminating and tacting common items in pictures, and mand training with one word.

Chai's PEP-R indicated his mental age was 26-28 months. The scores from SLSI-4 indicated his receptive language was 22 months and expressive language was 23 months. He was learning to tact actions, emotions, and answering social questions with full sentences.

The study was conducted in an EIBI program of a publicly funded children's hospital in a major city of Korea. All instruction was delivered in Korean. There were a total of three therapists and five children in the instruction room. The instruction was delivered either on the one-on-one format or as a group. The parents stayed with their children in the room during the study.

Curriculum. The CABAS® Preschool Inventory of Repertoires for Kindergarten (C-PIRK) 28) was used as a core curriculum to guide the assessment and development of individualized instruction programs for each child in the areas of academic, communication, language, cognitive, play and social skills. The instruction was delivered in the formats of discrete trial training (DTT) in structured and natural settings.

Teacher Performance Rate Accuracy Scale (TPRA). The TPRA used in this study was a modified version from 18 that has shown to be sensitive to the improvement of instructor performance and learner's achievements (see Table 1 for an example of a TPRA used in this study). The TPRA form provides accuracy of implementations on each component of the trial with timing, so the rates of accurate/inaccurate implementations can be calculated. In the present study, training therapists on using TPRA was provided by the first author prior to the onset of this study. The criterion was 100% accuracy and agreement for three consecutive TPRA observations. The therapists then conducted TPRA to assess parent performance throughout this study.

This example illustrates how TPRA was used to assess each component of DTT implementations. During the instruction for labeling actions, the mother obtained motivation and attention from her child. Then she presented an action picture and waited for 3 seconds. This comprised of an accurate presentation of the antecedent component in a trial, a check mark "∨" was recorded under antecedent. Suppose the child emitted a correct response following the presentation, a "+" was marked in the child's response component. Then the mother immediately reinforced the correct response, "R" was marked on the consequence component. If a child did not emit a correct response (a "-" was marked), the mother would have to implement a correction procedure to complete a trial (a C was marked in the consequence component). If the mother did not present a component accurately, the mark would be circled to indicate an error from the presenter. The ending of consequation (either reinforcement or a correction) concluded a trial.

Parent Performance Checklist (PPC). The PPC was conducted to evaluate the overall quality of DTT presentations in an instructional session, such as selection of appropriate reinforcers, appropriate use schedules of reinforcement, use of a variety of reinforcers, and use of appropriate prompts based on learner's history. The PPC was conducted after watching a 5-minute video presentation on each parent's DTT implementations. Each item was evaluated with a Likert scale of 0 to 4, representing Never, A little, Sometimes, and Always, respectively (see Table 2 for an example of PPC conducted in this study). A higher score on PPC indicates a relatively high quality of performance.

Electronic equipment. The equipment used included video camera for recording parents delivering DTT and a TV for the presentation of videos. Performances of parent presented DTT were video-taped stored in Windows Media Player files.

Response Definitions and Measurement Parent performance. Parent performance was measured by a) rate of accurate/inaccurate DTT presentations via TPRA and b) scores on PPC. The rate of accurate/inaccurate DTT presentations was calculated by dividing the total minutes of an observation with the number of accurate/inaccurate presentations on DTT components. The PPC has a total of nine items with 0 to 4 rating for each item. Thus, the score of PPA ranged from 0 to 36 with a higher score indicating a higher overall quality of presentations for each session.

Child performance. Child performance was defined by the percentage of child correct responses during parent-implemented DTT, regardless of the accuracy of parent implementations.

Collected data were analyzed primarily via visual examinations by detecting trend and variabilities of data path, level of responding, and immediacy of intervention effects 29. In addition to visual examinations, Percent Exceeding the Median (PEM) and Cohen's d were calculated to confirm that the differences in the level of responding across experiment phases were significant. The PEM was calculated through the following procedure: A median in a baseline phase was identified, and the data points above the median were calculated in the maintenance phases. The PEM was obtained using that number above median divided by the total number of data points in the maintenance condition. In general, a 90 % of PEM represents a highly effective intervention, an above 70 PEM indicates effective intervention, and if below 70, the effectiveness is unclear 30. One variation of Cohen's d scores is calculated using the formula d1 = (X A2 -X A1 )/S A1 suggested by 31. In this formula, X A2 and X A1 represent the mean of maintenance condition and baseline, respectively, and S A1 represents the standard deviation of baseline data. Over 0.8 of Cohen's d score represents a large intervention effect, 0.5 a moderate effect, and 0.2 a small effect 3233.

The present study used a delayed multiple baseline across two parent-child dyads 34, with a pre-experiment training, a baseline, an intervention, and a maintenance condition for each participant. All sessions of baseline, intervention, and maintenance were videotaped for the purpose of data collection and inter-observer agreement. Pre-Experiment training. A total of 12 weekly 2-hour parent education sessions on concepts and principles of behavior were provided in a group format prior to the experiment. During this training period, the mothers observed their children's instruction through a one-way mirror outside of the instruction room. Upon the completion of the group training sessions, the mothers sat next to the table where their children worked with their therapists, started recording data on their children's correct/incorrect responses, and calculated agreement with the therapists at the end of each session.

Baseline. During baseline, the mothers delivered DTT instruction for approximately 12 minutes within the 6-hour therapy per day. The therapists provided verbal instructions, modeling, coaching, and specific feedback after the instruction finished. The TPRA was not used to provide feedback for mothers but only used to collect data from the videos during baseline.

Intervention. The intervention condition consisted of individual coaching sessions with TPRA feedback provided immediately following the DTT presentations in the instruction room and weekly 90-min video self-monitoring group sessions held outside of the instruction room. Only parents participated in the group sessions.

During the individual coaching sessions, the therapist used the TPRA to assess the mother's performance on delivering DTT instructions. Immediately upon the completion of the last trial in a session, the therapists provided specific feedback on each component of the trials to the mother, reinforced accurate presentations, and making suggestions on error presentations. Then the therapist and the mother both checked the TPRA scores for accuracy and fluency while jointly setting goals for the next TPRA observation. The individual sessions took place one time per day and four days per week with a total of 9-10 weeks.

During the group video self-monitoring sessions, the first author instructed the mothers on how to use the TPRA form, and the mothers watched their own videos on DTT presentations while using the TPRA form to evaluate their own presentations. Each group video self-monitoring session lasted approximately 90 minutes and was conducted once a week for a total of 5 weeks. After each mother conducted video self-monitoring, they shared the results and discussed some issues and questions regarding their performance and the TPRA evaluation in the group.

Maintenance. The maintenance sessions were conducted two weeks after the completion of intervention, during which the mothers remained in the instruction room implementing DTT with their own children on some of the programs but without any TPRA evaluation and feedback from the therapist or any discussion with other parents.

After the completion of intervention, a survey was conducted to assess the social validity of the intervention. The survey contained a total of five questions addressing the mothers' perception on the impact of the intervention on their performance and the children's learning, the feasibility of implementations of the intervention procedure, and the appropriateness of the intervention procedure. See Table 3 for the survey questions. A Likert scale from 1 (never) to 4 (always) scale was used to evaluate each item. The purpose of this survey is to collect information about your experience during the present study. Please check a scale ('never' through 'always') depending on your experience for each item Evaluation Scale: 1 = Never 2 = Rare 3 = Sometimes 4 = Often 5 = Always

Item Scale 1 Self-monitoring using video and TPRA improved my performance during discrete trial training instruction. 1 2 3 4 5 2 Self-monitoring using video and TPRA is an effective procedure for improvement of my performance and my child's performance during instruction. 1 2 3 4 5 3 Self-monitoring using video and TPRA is easy procedure to conduct. 1 2 3 4 5 4 I will recommend this self-monitoring using video and TPRA to other therapists or parents. 1 2 3 4 5 5 The procedure mentioned above will ultimately exert positive influence to my child's performance during instruction. 1 2 3 4 5

Total Score

Interobserver agreements (IOAs) were measured by two therapists. They watched the videos of parent DTT presentations and completed a TPRA form and a PPC separately and independently. The IOAs for accuracy in presentations were calculated by the following procedure. First, the number of DTT components which both of the therapists marked as "correct" presentations were divided by the total number of DTT components presented and multiplied with 100. The IOAs for overall evaluation on PPC were calculated by dividing lower score by higher score and multiplying by 100. See Table 4 for the mean and range of IOA scores for each mother under baseline, intervention, and maintenance conditions. The procedural fidelity was reported as number of accurate/inaccurate DTT presentations by each parent in the results section and thus not reported here. Figure 1 depicts number of correct and incorrect parent-implemented DTT per minute for both mothers across all conditions. Omon's rates of correct presentations were variable at a low level during baseline and showed an overall ascending trend with gradual increases to a slightly higher level under the intervention condition. Her rates of accurate presentation during the maintenance were at a higher level, compared to the baseline and intervention conditions. Omon had a relatively high level of inaccurate presentations during baseline but decreased to lower levels during intervention and maintenance. Minae's rates of accurate presentations were at a relatively low level, while inaccurate presentations at a relatively high level during baseline. Data during intervention had an ascending trend on accurate presentations to a relatively high level and maintained at the same level during maintenance. Rates of inaccurate presentations decreased to a low level during intervention and maintenance.

The descriptive data across all conditions and effect sizes (baseline and maintenance) for both mothers' rate of accurate DTT instruction are summarized in the left half of Table 5. The PEM indicated that the intervention was effective (100 for both mothers) and the Cohen's d indicated a large effect of intervention (2.13 for Omon and 1.69 for Minae) for both mothers. Table 5. Itae's percentage of correct responses was variable ranging between high and low levels while remained at a level between middle to high levels during intervention and maintenance. For Chai, his percentage of correct responses had a rapid descending trend during baseline, but maintained between middle and high level during intervention and maintenance. The PEM (100 for Itae and 0.71 for Chai) indicated an effective intervention, and the Cohen's d indicated a large effect (1.04) for Itae and a small effect (0.12) for Chai.

The PPC was rated 47% of the total sessions evenly distributed across conditions for both mothers. The average and range of rating scores for Omon was 18.14 (range from 14 to 23) during baseline, 26.2 (range from 23 to 31 during intervention), and 28.6 (range from 24 to 30) during maintenance. Minae's average and range rating scores on PPC was 19 (6-24), 25.8 (22-28), and 32.17 (27-34) during baseline, intervention, and maintenance, respectively. Overall, the rating scores on PPC were mostly under 20 during baseline, 20 to 30 during intervention, and increased to more than 30 during maintenance, indicating improvement in the quality of DTT presentations after intervention.

Social validity of the parent training procedure used in this study assessed via survey had an average rating of 4.2 IJSE 2021, 36(2), 78-89 on the perceived effectiveness of the intervention on mothers' performance, 4.4 on the perceived effectiveness on the children's learning, 4.3 on the perceived validity of the procedure, 4.3 on the perceived feasibility of implementations, and 4.5 on the satisfaction for the intervention.

The present study replicated the effects of 22, indicating the procedure of using TPRA with video self-feedback was effective in teaching parents to implement DTT to their children with ASD. After training, both parents implemented DTTs with a relatively high level of rate and accuracy and minimal errors. As a result, both children's average percentage of correct responses during parent-implemented DTT instruction increased to relatively high levels during intervention and maintenance, compared to baseline. Scores on PPC also indicated that the intervention improved the overall quality of parent-implemented DTT instruction. Our results are also consistent with previous literature that training parents to implement DTT increased parent implementation skills and their children learning 152011). Specially, we incorporate similar components of parent training known to be effective into our parent training program, such as instructions, modeling, practice opportunities, and immediate feedback from trainers as in 11, with the addition of video self-feedback 15 as well as TPRA for ongoing in-situ training across children's instructional programs 18.

The results of this study suggested a potential training model for center-based EIBI by training parents as therapists in Korea, where qualified professionals are limited. Using TPRA forms during training provided an objective and efficient way (e.g., 5-min observation duration) to offer specific feedback on the accuracy of implementations for each component in DTT. Video self-monitoring with TPRA also strengthened acquisition of DTT implementations by self-evaluating the accuracy. In this study, the mothers were accompanied with their children in the center all the time due to a shortage of direct therapists in Korea. However, if we estimated the training time of 5-minute TPRA observations plus 15 minute observation of therapist delivered DTT as 20 minutes per session, the individual coaching time required would be 12 and 13.3 hours for Omon and Minae. Adding the 10-hour group self-monitoring time, the total training time for each mother in this study would be 22 and 23.3 hours in a two to three-month period for a parent to reach a high level of fidelity in DTT implementations across all areas of each child's individualized intervention plan. Although our parent training seemed to require significantly more time and efforts compared to those reported in previous literature 11, the mothers in our study did not have any prior experiences or knowledge about ASD, ABA, DTT, as well as the importance of parental role in education and the benefits of early intervention. Taken this into consideration, the performance of the two mothers in our study was a big step in their achievement. They continued to serve as the therapists and received ongoing supervision for their own children after this parent training program was completed. It is also important to note that most parent training in western culture functions as a supplement for the formal training in early interventions program implemented by therapists, while our parent training is required as parents play a vital role in their children's center-based early intervention. Thus, additional time and efforts were necessary to ensure the quality of service delivery by parents.

The increased rates of accurate presentations with minimized errors in parent-implemented DTT also benefitted the children's learning reflected in increased correct responses. With minimized or eliminated errors in instructor presentations, we could also detect learning problems from data more precisely and develop relevant intervention plans to remediate the children's learning.

In this study, both parents were trained to deliver DTT in all areas of their children's instructional programs, making it difficult to assess the effects of generalization across programs. Thus, we do not know whether such a training procedure resulted in generalized DTT implementations in new programs or only remained at the trained programs.

Despite of increased average of accuracy in DTT implementation within each parent, some methodological limitations exist in this study, and the results should be interpreted with caution. First, the overlap between the intervention of the first mother and the baseline of the second mother did not show the differential effect of the intervention on the accuracy of DTT implementations. Methodologically, the second mother was introduced to the intervention prematurely in a multiple baseline across participants design. This was deemed necessary and ethical in this particular situation, where the second mother did not agree to begin intervention any later simply because of the requirement of the experimental design. Second, the accuracy of implementations had overlaps with baseline and the intervention effect was not shown until in later interventions sessions. We speculate that the rotations of between various children instructional programs during intervention may have delayed the intervention effect. However, the error rates were decreased to a very low level for the first mother immediately upon the introduction of the intervention and for the second mother after five sessions of intervention. The data indicated the total number of presentations increased over time during intervention, that the mothers presented more accurate trials while error trials remained low.

In addition, we did not evaluate the generalization effect from the training setting to their home settings. The ultimate goal of parent training is to maximize the child's independent functioning in the home and community they live. It is imperative to evaluate the generalization of acquired DTT implementation skills from structured clinical setting to the home setting or in a natural environment with embedded learning opportunities. Future research should assess the generalization effects of parent training across instructional programs and across settings.

Data on social validity indicated that the mothers perceived that this training was effective and enhance their children's learning. Both mothers indicated that the parent training was feasible and were highly satisfied with the intervention. As discussed, parents of children with ASD tend to feel shameful in Korean culture and thus many children with ASD continue to be undiagnosed and untreated 219. Our findings suggested training parents DTT implementation skills may be one of the ways to empower parents with practical skills and educate them the importance of early intervention to their children with ASD. We hope our joint efforts with parents in this study provide preliminary evidence that parents can be successfully trained to serve the capacity as their children's therapist in response to the technical and social challenges of ASD in Korea.