The purpose of this study which is follow up study of Lee and Kim(2015b)'s was to analyse error sources and estimation of reliability in peer review of forced connection method-sportscasting by applying generalizability theory. Generalizability theory quantify error sources of the data measured under certain specific situation set by the researchers. It is an analysis method that the relative influences of each error sources taking from score is determined(G-study), and the effective measurement condition future applicable is provided(D-study). Participants were 10th high school students(N=216). Data were collected from student's peer review results and analyzed using univariate and multivariate generalizability theory. Results showed that error source for video have a more significant impact than other error sources. But the result by analyzing the gender difference was that error source for the interaction of video and participants have a more significant impact than other error source in the case of girls. Peer review used in this study showed high generalizability coefficient and even when reducing the number of video or participants it can maintain the adequate reliability. But generalizability coefficient of boys was higher than girls and specific measurement conditions leading to enhanced reliability were different when analyzing by gender difference. Also, method of analysis which cannot reflect measurement conditions properly estimates the reliability excessive. Discussions were provided in term of the relative influences of each error sources, the effective measurement condition maintaining the Generalizability coefficient of a certain level, and the comparison the Generalizability coefficient with the way of estimation traditional reliability applying univariate and multivariate Generalizability theory taking from score in peer review of forced connection method-sportscasting.
Purpose The purpose of this study is conducted to analyze the objectivity of the ski technical championships hosted by Korea Ski Instructor Association (KSIA) and to identify the error sources that affect the score of the competition. Methods To this end, we used the data from the 25th(2009) to 33rd(2017) ski technical championships held by the Korea Ski Instructor Association (KSIA). The data provided by Win Excel 2010 was used to sort out the missing data, such as abandonment, according to the data processing method. The collected data were analyzed by using SPSS 22.0 to calculate the mean and standard deviation of each season, event, and judges, and the Intraclass Correlation Coefficient (ICC). In addition, by using the single facet crossed design(p*j) of the generalizability theory’s G study, the variance component estimates for the participant(p) and the judges (j) are calculated, and the influence (%). Results As a result of the research, it was confirmed that the results of all the seasons and events from the 25th to the 33th events were very consistent, with the objective of .845~.986 higher than the recognition level of .80. In addition, the results show that the relative ratio of the judges to the error of the judging score is very low as a result of the error analysis through the dispersion component estimates. Conclusion In summary, the results of the KSIA evaluation are highly evaluated objectivity and have very low impact on the judges' errors.
Purpose The purpose of this study was (1) to analyze judges’ evaluation on rhythmic gymnastics performance by applying generalizability theory and (2) to suggest recommendations to improve judges' rating. Methods Data were 34 players’ scores from Senior Part at 29th KGA President’s Cup National Rhythmic Gymnastics Championship in Korea. Difficulty and execution scores in ball, clubs, hoop and ribbon event were analyzed. Analysis models containing components of area and reputation rank were designed and multivariate generalizability theory were used for analysis. Results The G-study results showed (1) that the error source about players has more significant impact to evaluation than other error sources in analysis model containing components of only player and judge, (2) that the error source about players has more significant impact to evaluation than other error sources in analysis model adding components of area, but the error source about area has more significant impact to evaluation of clubs event than other error sources, (3) that the error source about players has more significant impact to evaluation than other error sources in analysis model adding components of reputation rank, but the error source about reputation rank has more significant impact to evaluation of hoop event than other error sources in analysis model adding components of area. The D-study results showed generalizability coefficient was stable in analysis model without components of area and reputation rank, but generalizability coefficient in analysis model containing components of area and reputation rank not stable in some event. Conclusion Recommendations for improving judging were discussed.
The purpose of this study was (1) to analyze judges’ evaluation on figure skating performance and statistical analysis and (2) to make recommendation to improve judges' performance. Data were 62 figure skaters’ scores from Senior Woman 1 Group Part at 2015 The National Figure Skating Championship in Korea. Data of presentation part in Short Program were analyzed. Presentation part consists of skating skills, transitions/linking footwork/movement, performance/execution, choreography/composition and interpretation. Nine judges evaluate skater's presentation with the score 10.0 for each factor. Generalizability theory, descriptive statistics and ANOVA were utilized. Results showed that generalizability coefficient of presentation part was over 0.9, therefore stable reliability was secured. The error source about players has more significant impact to evaluation than other error sources. Generalizability coefficient was stable when reducing the number of judges up to 2. Also, judges generally represented aspects of similar evaluation, but different aspects appeared on a few judges’ assessment in the components of presentation. Discussions were provided in terms of the reliability of the judging method for the presentation part of figure skating and the effective measurement condition.
