Research Methods

Research Methods
Paper instructions:
Does the design of this research flow from the research questions? Could there possibly be a better alternative design. What would that look like?

Did the researchers actually measure what they set out to measure? Briefly explain.

Is it possible to generalize from this sample back to the population? Briefly explain.

Were claims about the relationships between variables justified by the statistical data? Briefly explain.

Stud t Ach ‘e ement and Retent’o
ROBERT F. SCHUCK
University of Pittsburgh
tively engaged in the process in order to maximize the
probability of exhibiting the desired behavioral response.
ABSTRACT This study examined the influence of the The teaching skill directed toward eliciting this involve-
h1°°1’P°1’3tioh Of Set h1¢h1°ti°h ‘_Vhhh‘ t°3°h°”’S ihstf“°‘ ment was defined by researchers at Stanford University
fional strategies on Stlldent achlevement and retemlom and labeled set induction. Studies and related readings on
One hundred twenty ninth grade students were randomly t . d, t .t _ f 1 bl d t th
assigned to twelve groups of ten each. An in-service Se In ‘C3 6 1 15 a Power u Vfma e m_ e ermmmg e
teacher was randomly assigned to each of the student nature of the learning that Wlll occur in the classroom and
groups. To accommodate the requirements of the Solo~ that the teacher is a powerful agent in arousing or induc-
1110“ Fo“-T Gf°“P_ Designs three Sthdehhteachef 8‘°“P5 ing a pupil’s set toward learning. A study of the applica-
were “md°mly asslgned to each of t.he f°ur.req‘.med cells’ tion of set induction in a student teaching environment
One half of the teachers were given instruction in the skill d t d b W_tt k t UCLA 31 d th 1 f
of incorporation of set induction in their instructional c°n_ “C e _ y 1 Io}: 3 _ “we 6 6 re eVa”_°e 0
strategies and the remaining half served as a control group. Set lhdhchoh to P‘-‘P11 1ea1’h1hg- He reported that PhP115
Data gathered by administering pretests, posttests, and a taught by student teachers who were told their grades in
retention test were aflalfled bY 3 tWo’W3Y ahalysls of student teaching depended on the achievement gains ex-
variance. The i’esults_indicated that ‘set. induction is an hi-bited by their pupils made Significant gains in achieve_
important variable in the determination of student t .1 t t t 1 h t
achievement and retention. It was further determined that men Over pup‘ S augh In a _con ro group W 0 were no
no significant pretest-treatment interaction occurred, glveh Such 3 set (-05 level? wlttrocks 1962)-
hence, eliminating one heretofore plausible alternative In a study conducted at Stanford University, Aubertine
hYPotheSiS fe1atiVe to the results obS€fVed- (1964) reported that teachers could be trained in the skill
of set induction and that teachers who incorporated this
skill in their instructional strategies were perceived to be
more effective by their students than a control group (.05
A pproximately one decade ago the Stanford Summer level). These findings were confirmed by Schuck in studies
Micro-Teaching Program identified seven teacher conducted at Arizona State University and the University
skills hypothesized to form the core of desired teacher be- of Connecticut (.01 level; Schuck, 1969; 1970). In addi-
havior. The major focus of the research stimulated by this tion, he found that pupils taught by teachers who incor-
program centered on the definition of these behaviors, the porated set induction into their instructional strategies
development of evaluation protocols to measure teacher scored significant gains in achievement within a unit on
performance in these areas, and the establishment of respiration (.01 level) and that this finding occurred within
teacher education programs designed to foster competence all three versions of the BSCS curricula (Schuck, 1971). A
in these skills (Aubertine, 1964). related finding of importance was that a significant corre-
A review of the literature related to this area yields a lation existed between student achievement and pupil per-
dearth of studies using as the dependent variable student ceptions of effective teaching (.01 level). These findings
performance. Notable exceptions in the work of Auber- held over a series of replications of the original work em-
tine and Schuck are discussed later in this paper and pro- ploying student and teacher populations from the states
vide the foundation upon which the research reported here of Arizona and Connecticut.
is based. It is important to note, however, that the depen- This study is a logical extension of the previous work
dent variable in the current research is student achieve- reported with the following innovations: a) the teachers
merit and not teacher behavior. The fact that teachers can employed are at the in-service rather than pre-service level;
be trained in the teaching skill employed (set induction) b) the effect of the independent variable (set induction)
and that teachers so trained are judged to be more effec- on the dependent variable (student achievement) was as-
tive by their students is documented in previous studies certained over a two-unit sequence rather than the single
(Aubertine, 1964, p. 147). unit employed in previous work; c) a new dependent varia-
Several psychologists have contended that one of the ble (pupil retention of knowledge) was subjected to analy-
essential conditions in learning is that the learner be ac- sis; and d) the Solomon Four Group Design was imple-
Creativity Research Journal Copyright © 2006 by
2006, Vol. 18, No. 3, 385-390 Lawrence Erlbaum Associates, Inc.
Creative Thinking as a Predictor of Teacher Effectiveness
in Higher Education
Nitza Davidovitch
College of Judea and Samaria
Roberta M. Milgram
College of Judea and Samaria and Tel Aviv University
ABSTRACT: Creative thinking, defined as the quantity the world (Milgram, Dunn, & Price, 1993), and used
and quality of ideational fluency, was investigated as a with participants that ranged in age from 3 (Moran,
predictor of teacher eflectiveness in 58 college-level Milgram, Sawyers, & Fu, 1983) to young adult

instructors. The correlation between creative thinking (Milgram & Hong, 1994; M1121 am & L1Vn€, 2006)-
and teacher eflectiveness defined as real-life problem Taken together, the findings pfoV1d€C1 €lnp11‘lCa1
solving was r = ,64, p < _0001, The absence ofa ;~ela_ support for the reliability and construct validity of ore-
tion between creative thinking and student evaluations at1V€ thinking as defined 6a1’1i€l‘- The TACT yielded
was attributed to the fact that student evaluations did scores that were empirically distinct from intelligence
not include their opinion of their teachers’ creativity. 1351 SCof€S 1n C1111C1l‘€n, aC101€SC€ntS, and adults at th€
The findings suggest the potential benefits in sponsor- 110111131 range 01 1Q and aboVe- T1115 W35 fo1111d eV‘311
ing pre-service and in-service workshops to enhance W113“ 1113 TACT Was 810111’ 3d1111111Ste1‘€C1 (M11g1’3111
teachers’ creative thinking ability and including cre- M11g13111» 1976_)- T116113 Was» 111011“/V1313 3 51110118 161311011
m«,’v,’,«y in the ew,1,,a,,’onS Offacully between quantity and quality of ideational output, sup-
porting the conclusion that high ideational output is a
The goal of this study was to examine creative thinking P1’€°011<11t1°11 101 Cl11a111)l 1‘€SPo11SeS (11/111313111: M11g1’a111»
as a predictor of teacher effectiveness in higher educa- R°Senb1°°m’_& Rabkmr 1978)‘ An Order _effeCt was
tion. Creative thinking has been defined as a cognitive also note‘? Wlth pcfpular responses appearing _eaIher
process of Original problem solving by means of which than creative ones in the response sequence (Milgram
original products are generated (Milgram, 1989). A &Rabkm’ 1980)‘
product may be a response of any kind; an idea; a so1u- Kaufman and Baer (2005) recently presented a Sys”
tion to a problem, an actual product in am music SC} tematic and integrative surninary of the knowledge that
ence 0 1. matheméticy O 1. a Solution to a p’1.Ob1 en; that has accumulated on the issue of whether creativity is a
arises in child rearing, business, or teaching. Original gi1e_ne.ra1 or 1/i1“1’r11a111″Sf9€<9>g”1<:1processicvlgth referenceilto
is defined as unusual (i.e., statistically infrequent and t ls Issue’ glam (, .) On_g_tOO t e posmon t at
of high qua1ity_produCtiVe Valuable or Wo1 Thwhfle) general creative thinking is a critical component of cre-
Creative thinking has been operationally defined in Zusvte P etrf(.)rmfaIC1}c.:t1I:1eV€r£%om:1.1n.’tS111: pfielsefiéii a_l4X
terms of a single index of ideational fluency. A test of we me O 1 C Hess rea M y 0 6 ( 1 glam’
ideational fluency was developed that was based on the -e-e-
work Of Guilford (1950, 1956), Mednick (1962), We thank GalitMadar’for her helpful suggestions and comments on
Torrance (1962), and Wallach and Kogan (1965)_ This this a}i“t(1icle. We ssitilecially appricaclilatei her wise counsel on the re-
teS_t’ the Tel Aviv Creativity Test (TAC1? Milgram & SearCorr::i3goIi1::ncteea:Ida ::;::s:: iorartprints should be sent to
Mllgraln. 1976), has been translated Into SIX 1an- Roberta M. Milgram, College of Judea and Samaria, Ariel44837Is-
guages, used over the years in Israel and other parts of rael. E-mail: rnilgrarn@post.tau.ac.il
Creativity Research Journal 385

Yoon, J.-O. (2011). The effects of captions on deaf students’ content comprehension, cognitive load, and motivation in online learning. American Annals of tbe Deaf 156 (3), 283-289.

THE EFFECTS OF CAPTIONS ON DEAF STUDENTS’ CONTENT COMPREHENSIONJ COGNITIVE LOAD, AND MOTIVATION IN ONLINE LEARNING

HE AUTHORS examined the effects of captions on deaf students’ content comprehension, cognitive load, and motivation in online learning. The participants in the study were 62 deaf adult students who had limited reading comprehension skills and used sign language as a flrst language. Participants were randomly assigned to either the control group or the experimental group. The independent variable was the presence of captions, and the dependent variables were content comprehension, cognitive load, and motivation. The study applied a posttest-only control group design. The results of the experiment indicated a signiflcant difference (t = -2.16, p < .05) in content comprehension but no statistically significant difference in cognitive load and motivation between the two groups. These results led to suggestions for improvements in learning materials for deaf individuals.
JooNG-O YOON AND
MiJ\jEONG K I M
YOON IS A DOCTORAL STUDENT IN EDUCATIONAL TECHNOLOGY IN THE DEPARTMENT OF EDUCATION, DANKOOK UNIVERSITY, GYEONGGI-DO, SOUTH KOREA. KIM IS AN ASSISTANT PROFESSOR OF EDUCATIONAL TECHNOLOGY IN THE DEPARTMENT OF TEACHER EDUCATION, DANKOOK UNIVERSITY.

Linguistic deficiency in deaf people has been attdbuted to restricted access to information and knowledge in learning environments. Consequently, there have been efforts to support deaf people’s learning; these efforts have included the development of multimedia materials. Application of visual learning aids has been suggested as an effective way of overcoming learning difficulties caused by hearing impairment (Luckner & Humphries, 1992; Margaret & Dorothy, 2001). To be more effective, multimedia content should provide pictures, narration, and animation in combination—for example, narration and animation rather than narration alone (Mayer, 2009).

Captions
Along with visual aids such as pictures in learning materials, captions and subtides support deaf students’ learning. Text-based captions—in other words, written language—provide deaf people with access to verbal information. Many studies have reported the effects of captions on learning (e.g., M. Goldman & S. Goldman, 1988; Nugent, 1983; Smith & Shen, 1992; Stinson, Stuckless, Henderson, & Miller, 1988). M. Goldman and S. Goldman found that students who participated in classes where captions were presented on a television screen were satisfied with the classes; this satisfaction increased their interest in learning and concentration levels.

VOLUME 156, No. 3, 2011

AMERICAN ANNAI.S OF THE DEAF

EFFECTS OF CAPTIONS IN ONLINE LEARNING

Multimedia educational videos for the deaf, with captions only, have been indicated to have a positive impact on content comprehension, reading comprehension, and learning ability (Y Kim & Jeong, 2006). Furthermore, Koskinen (1986) found that captions support deaf students’ reading ability, content comprehension, word memory, and interest in learning. Royer and Cable (1976) reported finding that a combination of pictures and texts helped students understand conceptual sentences. Reading Skills But captions do not always have a positive impact, as some studies have indicated. Margaret and Dorothy (2001) found not only that reading ability is tbe most important skill required of students when captions are used in teaching, but that the effectiveness of captions is strongly related to the level of reading skills. Students with lowlevel reading skills appear to have trouble understanding captions, while students with higher reading levels indicate greater interest in learning and higher achievement as compared to their peers with lower reading levels when captions are used (Garza, 1991; National Captioning Institute, 1989; Neuman & Koskinen, 1992; Nugent, 1983). Deaf children, who have smaller vocabularies than hearing children, complain that they have difficulty learning words with multiple meanings (Paul & Quigley 1994). In a study that focused on deaf adults, Kelly (1998) found that readers with lower reading skills hardly benefited from video captions and that they had difficulty dealing with complex sentences. Moores (1997) found that deaf students have a disadvantage in the initial years of their education as compared to their hearing peers and that they lag in literacy development as they move to higher grades. This linguistic defi-

ciency influences students’ learning in other subjects as well, and their overall academic achievement is delayed relative to that of their hearing peers. Deaf students’ performance on the ninth edition of the Stanford Achievement Test in an administration of the SAT at Gallaudet University indicated low reading comprehension ability, as shown by the students’ scores on the reading comprehension portion of the test. The median reading skills of deaf students ages 8-18 years were found to be equal to those of hearing students reading at the fourth-grade level (Traxler, 2000). Sign Language and Content Comprehension However, if deaf students are taught using sign language, which is their first language, their learning of written language improves and they internalize knowledge in sign language (Erting & Pfau, 1997; Lane, 2000). It is presumed that providing sign language and captions at the same time can improve the content comprehension of deaf students as they learn. In this vein, our aim in the research for the present study was to gauge the effectiveness of media with both captions and sign language video clips in educating deaf students whose first language is sign language and whose literacy skills are limited. Cognitive Load In instructional design for multimedia learning, it is important to minimize the sources of external cognitive load so that working memory can be effectively applied to a person’s composition of schema. Most students with learning difficulties experience more problems with instructional design than with learning materials themselves (Mayer, 2009). This means that the instructional design is laid out to have more external cognitive load than

necessary, which leads to the overuse of working memory and hinders students’ concentration on full understanding of the material (H. Kim, 2002). Furthermore, the cognitive load indicates the levels of working memory load and cognitive effort. This means that if the learning content is composed only of visual materials (captions -I- sign language), the cognitive load increases (Kirschner, 2002). Motivation Students are more likely to be motivated to learn if they expect that they are going to be taught something interesting or important. The materials can foster the expectation of high interest or high importance by being shifted into an intense communication style when things that are especially important are being explained (Jere, 2010). Researchers have verified that the relevance of the learning content and learners’ interest in the subject both influence learning (Means, Jonassen, & Dwyer, 2004). When sign language and captions are used simultaneously with deaf students, these modes’ impact on motivation needs to be considered. Therefore, the present study examined the effects of captions on deaf student’ content comprehension, cognitive load, and motivation in online learning, when captions are used in conjunction with sign language. Method Participants The study participants were deaf students attending a “mechanics” class at the Vocational Training Institute of the Korean Employment Agency for the Disabled and a “universal design” class at Korean Nazarene University in 2010. They were selected according to the experimental conditions. All participants used sign language as their first language. There were 62 participants

VOLUME156, No. 3, 2011

AMERICAN ANNAI.S OF THE DEAF

altogether. Participants were not informed ofthe purpose and aims ofthe experiment until it was completed. Participants were randomly assigned to either the experimental group or the control group. Each group consisted of 20 men and 11 women. In terms of age, 60 participants were in their 20s (M = 23.0, SD = 1.92) and 2 were in their 30s (M = 30.5, SD = 0.71). There were 58 high school graduates (93.5%) and 4 college graduates (6.5%). These and other characteristics of the participants are listed in Table 1. Learning Material (Online Content) The online learning materials selected for the experiment were developed by the Korea Employment Agency for the Disabled and concerned a successful work life for people with disabilities. The content was visually composed using flash animation along with both sign language video clips and captions. The topic chosen from the online content was “successful financial life.” This topic was segmented into 20 sections, each with a running time of 40-50 minutes. The “Investment Techniques” section was chosen for user in the experiment (see Figure 1). This topic not only sparks great curiosity among deaf students but is also considered a difficult one to approach, and thus one that would minimize the possibility of prior knowledge of the test program. The goal of the lesson was to assess whether learners could understand content and apply it. In the content, sign language was basically displayed at the lower right ofthe screen, and subtitles were displayed at the lower center. The on/off control for the captions was deleted for the control group to prevent the captions from being accidentally turned on. Other text information (preparation, study, review, ordedy arrangement) on the screen

Table 1

Demographic Characteristics and Achievement Data of the Study Participants
High schooi Group (N = 62) Control Experimental (n = 31) graduates/ coiiege M(SD) graduates (years) 22.44 (1.55) 23.37 (3.02) Age Mate/ femate 20/11 20/11 Levei of hearing toss (dB) First tanguage Sion Sign TOPiK score Age (reading. (years) at writing. onset of vocabuiary) M (SD) 66.85 (3.05) 68.26 (4.42) hearing loss 0-2 0-2

mm •• WM•1
30/Í 28/3
90

Note. TOPIK, Test of Proficiency in Korean.

had been fully perceived in prior learning (see Figure 1). Experiment Design and Procedure The independent variable ofthe experiment was the presence of captions, and the dependent variables were the learning indicators (content comprehension, cognitive load, and motivation). We used a posttest-only control group design for the experiment. To minimize the differences among the test participants, the Test of Proficiency in Korean (TOPIK), developed by the Korea Institute for Curriculum and
Figure 1

Evaluation, was used. The TOPIK is a standardized test administered to foreigners who use Korean as a nonprimary language and to Koreans who dwell in foreign countries and do not use Korean as their native language. It consists of four sections (vocabulaiy, writing, reading, and listening comprehension); we omitted the listening section. The TOPIK was administered to 85 potential participants; those who scored from 60 to 80 (N = 62) were then selected before being divided into two groups. The 60-80 range was used because this was the average range of scores obtained by foreigners

Screen Display of Online Content; “Investment Techniques”

VOLUME 156, No. 3 , 2 0 1 1

AMERICAN ANNALS OE THE DEAF

EFFECTS OF CAPTIONS IN ONLINE LEARNING

who had passed the test in the preceding 5 years (2005-2009). The experiment was conducted in students’ usual study areas, to provide comfort and familiarity Further, to avoid the effects of prior knowledge, a topic at the highest level of difficulty was chosen from among the 20 sections of subject matter. Each group watched the content playing on a large screen without help from others. Most participants completed thefinalcomprehension test and the surveys hy themselves. The experiment was conducted in a computer lab and took roughly 40 minutes. The participants were allowed free choice of seating and were not informed of which condition they would experience. During the experiment, there were students who did not understand the questionnaire, and all of the participants were aided by sign language. The questionnaire was explained to all participants by means of sign language, in an effort to prevent the types of errors that occur when test takers answer questions despite the fact that they do not understand the questionnaire. In the future, to improve the credibility of the experiment, we may address problems related to comprehension by simplifying the vocabulary in the questionnaire or by using a sign languagebased questionnaire. Dependent Measures Content Comprehension A content comprehension test was conducted by means of a worksheet consisting of 10 multiple-choice questions that was handed out after the participants had viewed the “Investment Techniques” video. Because a maximum of 10 points could be awarded for each question, the total maximum score was 100 points. The questions had been written by two incumbent vocational instructors, then revised and improved by another in-

structor who was also an investment expert. Most students read and responded to the questions on their own, and professional sign language interpreters were present to provide help. Cognitive Load The main factors of cognitive load were measured by means of a scale developed by Ryu and M. Kim (2010). The scale consisted of five main factors (physical effort, mental effort, task difficulty, self-evaluation, and material design), and learners gave each item on the survey a score ranging from 1 to 7, where 1 = absolutely disagree, 2 = disagree, 3 = slightly disagree, 4 = neutral, 5 = slightly agree, 6 = agree, and 7 = completely agree. Physical effort scores were used to measure how much physical effort the learners put into solving the problems. We assumed that increased physical effort was associated with the psychological effort needed to do tasks. Mental effort scores indicated, basically, how the learners allocated cognitive resources to solve the problems. If the learners put more effort into problem solving, the imposed mental effort increased. Task difficulty scores were another indicator of how difficult a learner perceived tasks to be. Self-evaluation scores were related to the learners’ own evaluations of their learning processes and/or outcomes. The suhjective judgments were an important indication of how much cognitive load was imposed and perceived by the learners. The last factor, material design, related to how well the learning content was designed and coordinated with the learning process. These five factors were proposed as the key factors of cognitive load to measure subjectively (reliability Cronbach’s alpha = .847).

Motivation ARCS, an instructional design model developed hy Keller (1983), has 4 components: attention, relevance, confidence, and satisfaction. (The first letters of each of these components give the model its name.) Designed to improve learning motivation, ARCS emphasizes motivation as a deciding factor in academic achievement. Basing our choice on Keller’s attention, relevance, confidence, and satisfaction (i.e., ARCS) theory of motivation, we used the ARCS-based Instructional Materials Motivation Survey for the experiment (Keller, 1987). The IMMS uses a 1-5 scoring range, where 1 = absolutely disagree, 2 = disagree, 3 = neutral, 4 = agree, and 5 = completely agree. Overall scores were computed byfindingthe average score for each question. The survey included 36 items (reliability Cronbach’s alpha = .935). Data Analysis We analyzed the difference between the two groups using a t test for content comprehension and a oneway muttivariate analysis of variance (MANOVA) for cognitive load and motivation at the .05 significance level. Results Effects on Content Comprehension As shown in Table 2, the control group, which viewed learning material that was supported only by sign language video clips, scored an average of 66.13 points, and the experimental group, which had its instruction supported by both sign language and captions, scored an average of 73.23 points— 7.10 points higher than the control group. A notable statistical difference {t = -2.16, p < .05) between the two groups was drawn, showing the positive effect of the combined presence of sign language clips and captions on

VOLUME 156, No. 3, 2011

AMERICAN ANNALS OF THE DEAF

Content comprehension. Further, the experiment yielded estimates that did not rely solely on statistical data, namely the average of samples, the difference between groups, and the coefficient of correlation. The resulting eta square value was .072, which in Cohen’s (1988) terms, would be considered a medium effect size. Cohen classifies .01 as a small effect, .06 as a medium effect, and .14 as a large effect. Effect size statistics provide an indication of the magnitude of the differences between groups. Effects on Cognitive Load As Table 3 shows, no significant distinction was drawn between the two groups for any of the five factors of cognitive load. In other words, there was no statistically significant difference between the two groups for any of the listed factors. Means (with standard deviations) of the two groups’ scores are shown in Table 4. Effects on Motivation As Table 5 shows, effects on motivation were measured in four categories: attention, confidence, relevance, and satisfaction. The listed categories indicated a potential relationship but no statistically significant difference. Means (with standard deviations) of the two groups’ scores are shown in Table 6. Discussion and Conclusions Our tesults indicate a potential relationship in which providing captions in online content for deaf individuals along with sign language video clips had a positive effect on learning that exceeded that of sign language video clips alone; however, the captions did not have a significant effect on cognitive load or motivation. In regard to content comprehen-

Table 2 Results for Content Comprehension
n Score (M) SD ./f I p

Control group (sign language only) Experimental group (sign language + closed captions) *p < .05. 31 73.23 13.26 31 66.13 12.56 60 -2.16* .035

Table 3 Results of Cognitive Load Analysis Stepdown Factor PHY
Captions MEN

Wilks’s K

••
.988

F » 9 .51

df

P .765
.479

-2

F
.09 .57

df
1/6Q|| 1/59

Significance of F
.452 .982 ‘ 1

1/62
1/62

.001 .008

used
/not used

DIF
SEV

(p=.982)

.04
.43

1/62
1/62

.840
.515

.001
.007

.00
.03

1/58
1/57

.858

DES = material design.

.42

1/62

.520

.007

.04

1.56

Notes. PHY = physicaleffort. MEN – mental effort DIF = task difficuity SEV = self-evaluation. DES

sion, when the group that experienced captions and sign language video clips was compared to the group that experienced sign language video clips alone, it was clear that the group that was offered both kinds of aid experienced a positive effect. We presume that although participants had poor reading comprehension, they used the captions to complement their comprehension of the content after first gaining understanding through the sign language video clips. Our results corresponded to the findings of Nugent (1983), which indiTable 5 Results ot Motivation Analysis

Table 4 Mean Proportions ot Factors ot Cognitive Load Captions Factor PHY MEN DIF SEV DES Used M (SD) 4.33 (0.87) 4.77(1.13) 4.08 (0.90) 4.76(1.02) 4.87(1.19) Not used M (SD) 4.96(0.98) 4.04(0.85) 4.91 (0.83) 5.05(1.09)

Notes. PHY = physical effort. MEN = mental effort. DIF = task difficulty. SEV = selfevaluation. DES = material design.

Stepdown Motivation
Cittcgpry

Significance Wilkss 0.55 .937
2.21

(// 1/61
1/61

/) .462
.142

1)

F

df

of F

Captions
used

Attention Relevance

.009
.000

0.55
2.95

/not used Confidence (p=.448) 0.02 Satisfaction 0.73

1/61 1/61

.888 .788

.036 .001

0.31 0.00

1/59 1/58 1/57 1/56

.462
.092

‘H

.582 .974

VOLUME156, No. 3, 2011

AMERICAN ANNALS OF THF. DEAF

EFFECTS OF CAPTIONS IN ONLINE LEARNING
Table 6

Mean Proportions of Motivation Categories: Attention, Relevance, Confidence, and Satisfaction
Captions Motivation Used

Not used M(SD) 2.68 (.66) 2.44 (62) 2.85 (.68) 2.55 (.82)

category
Attention

M(SD)
2.57 (.48) 2.67 (.63) 2.83 (.53) 2.49 (.75)

Relevance Confidence Satisfaction

cated that deaf people understand learning content better when they are provided visual aids and captions at the same time. In regard to cognitive load, there were no indications of statistical differences between the experimental group and the control group. This finding contradicts the modality effect and the redundancy effect. The idea behind the modality effect is that the presence of two types of visual information disperses a learner’s working memory, which leads to cognitive overload. The redundancy effect is said to result when the cognitive load rises subsecjuent to several items of information being delivered in an identical form. The outcome of the present study seems to be a result of deaf people’s learning style, which, in contrast to that of hearing people, habitually relies on visual information only Moreover, the fact that the scores for self-evaluation (M = 4.76, M = 4.91) and material design (M = 4.87, M = 5.05) were high, when considered in combination with the positive feedback on the content, indicates that the participants used their cognitive ability efficiently It seems that the deaf are particularly dependent on their visual organs, given that their hearing impairment requires that they rely solely on visual information when studying (Luckner & Humphdes, 1992). Further, the high difficulty level of the

content may have raised participants’ intrinsic cognitive loads and prevented differences in their later cognitive loads. In regard to motivation, there was no significant difference between the two groups. During the experiment, three or four students even displayed signs of boredom and inattention. This can be seen as a result of a constant barrage of content that did not consider the characteristics of deaf learners, leading them to quickly lose interest. Therefore, our analysis indicates a potential relationship in which the presence of captions did not affect the learning motivation of the participants. The fact that both groups scored in the 2-3 range in all areas of motivation (i.e., below the average score of 3 points), indicates that such low scoring was not due to the presence or absence of captions or sign language video clips, but to something else. Therefore, one can look for reasons for the loss of interest or motivation by the participants by looking into caption-oriented support and the composition of the learning materials. Considering the propensity of the deaf to ignore reading because of low literacy skills, content should be specifically designed for their reading level. Then, both sign language video clips and captions should be added to complement and maximize learning by the deaf These measures would benefit deaf learners in terms of both cognitive load and motivation. Moreover, in the present experiment, mental and physical efforts related to cognitive load were higher than average, and low scores were shown for motivation. These results indicate the need for a close look at aspects of cognitive load and motivation in order to produce effective content for deaf learners. Further, sources of extraneous cognitive load such as the displayed location of sign language

clips and captions, as well as speed, color, and font size, are closely related to the understanding of content. Hence, future production of content not only will require the insertion of captions but an overall approach based on instructional technology. The present study led to a conclusion that captions provided along with sign language video clips have a positive effect on deaf learners’ understanding of the content of educational material. However, the addition of captions did not have any significant effect on cognitive load and motivation. In particular, among the substructures of cognitive load, both groups recorded higher-than-average scores for physical and mental effort, and both groups also recorded below-average values for motivation. Suggestions Based on the above conclusion, we offer two suggestions. First, captions should be provided along witb sign language video clips, and rather than translate every expression word-for-word, caption writers must represent words with words of equivalent meaning that fit the vocabulary level of the learner. Deaf people who have low literacy skills and poor text comprehension may disregard texts containing difficult vocabularies. Much as a hearing person with poor Fnglish skills tends to avoid books that contain difficult English words, deaf people with poor reading comprehension are bound to disregard the presence of complex words. Second, in the production of content for deaf people, it is common to design the content for hearing people first, then type in subtitles or insert sign language video clips in order to take credit for constructing an environment that is fdendly to people with hearing impairments. In regard to this practice, a systematic approach based

VOLUME 156, No. 3, 2011

AMERICAN ANNAI.S OF THE DFAF

on instructional technology should be considered in the process of developing learning materials responsive to deaf learners’ needs. Developing tailor-made instructional designs, evaluating the applications of the designs, and conducting further research should be emphasized to optimize learning for the deaf. Limitations of the Study The present study has two main limitations. First, because information on sign language ability was provided by the participants themselves, it may not have been entirely creditable. To address this limitation in future research, an objective measure of study participants’ sign language ability is needed. The second limitation is that prior knowledge about the content used in the experiment was not tested; such knowledge might have affected the level of comprehension.

References
Cohen. J. (1988). Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Erlbaum. Erting, L., & Pfau, J. (1997). Becoming bilingual: Facilitating English literacy development using ASL in preschool. Washington, DC: Uurent Clerc National Deaf Education Center, Gallaudet University. Garza, T. J. (1991). Evaluating the use of captioned video materials in advanced foreignlanguage learning. Foreign Language Annals, 24(i), 239-258. Goldman, M., & Goldman, S. (1988). Reading with closed-captioned TV. Journal of Reading, 31 (5), 458-461.

Jere, B. (2010). Motivating students to leam. New York, NY Routledge. Keller, J. M. (1983). Motivational design of instruction. In C. M. Reigeluth (Eà.), Instructional design theories and models: An overview of their current status (pp. 383-434). Hillsdale, NJ: Erlbaum. Keller, J. M. (1987). Instructional materials motivation survey. Tallahassee: Florida State University. Kelly, L. R (1998). Using silent motion pictures to teach complex syntax to adult deaf readets. Journal of Deaf Studies andDeaf Education, 3(5), 217-250. Kim, H. (2002). Relative effects of navigational tools on reduction of working memory load in hypertext. Korean Association for Educational Information and Broadcasting, 8(5), 189-218. Kim. Y, & Jeong, E. (2006). The effects of a caption video clip program on the written skills of students with hearing impairment./o«r nal oj Special Education: Theory and Practice, 7(1), 489-506. Kirschner, P A. (2002). Cognitive load theory. Learning and Instruction, 12, 1-10. Koskinen, P S. (1986). Using closed<aption television in the teaching of reading to deaf students. American Annals of the Deaf 131(1), 43-46. Lane, H. L. (2000). The mask of benevolence: Di.^abling the Deaf community (2nd ed.). San Diego, CA: DawnSign Press. Luckner, J. L., & Humphries, S. (1992). Picturing ideas through graphic organizers. Perspectives in Education and Deafness, 11(2), 8-22. Margaret, S. J., & Dorothy, W J. (2001). Television literacy: Comprehension of program content using closed captions for the deaf. Journal of Deaf Studies and l^eaf Education, 6(1), 43-53. Mayer, R. E. (2009). Multimedia teaming (2nd ed.). New York, NY: Cambridge University Press. Means, T. B., Jonassen. D. H., & Dwyer, F. M. (2004). Enhancing relevance: Embedded ARCS strategies vs. purpose. Educational

Technology Research and Development, 45(1), 5-17. Moores, D. F. (1997). Psycholinguistics and deafness. American Annals of the Deaf 142(5), 80-89. National Captioning Institute. (1989). Evaluating the benefits of close-captioned TV programming as instructional material for ESI. students: Final report. Vienna, VA: Author. Neuman, S. R., & Koskinen, P S. (1992). Using captioned television to improve the reading proficiency of language-minority students. Vienna, VA: National Captioning Institute. Nugent, G. C. (1983). Deaf students’ learning from captioned instruction: The relationship between the visual and captioned display Journal of Special Education, 17(2), 227-234. Paul, P, & Quigley, S. (1994). language and deafness (2nd ed.). San Diego, CA: Singular. Royer, J. M., & Cable, G. W (1976). Illustrations, analogies, and facultative transfer in prose learning. Journal of Educational Psychology, 68(2), 205-209. Ryu, J., & Kim, M. (2010, November). Sensitivity of the cognitive load measures on different task levels. Pajier pre.sented at the International Cognitive Load Theory Conference, Macau, China. Smith, E. E.. & Shen, C. W (1992). 7he effects of knowledge of resultsfeedback of captioning on listening comprehension ofEngiish as a second language in interactive videodisc systems. Bloomington, IN: Ass(KÍation for Educational Communications and Technology. (ERIC Document Reproduction Service No. ED 348026) Stin.son, M. S., Stuckle.ss, E. R.. Henderson, J., & Miller, L. (1988). Perceptions of hearing impaired college students toward real-time speech to print: RTGD and other educational support services. Volta Review, 90(7), 339-348. Ti-axler, C. B. (2000). The Stanford Achievement Test (9th ed.): National norming and performance standards for deaf and hard of hearing students. Washington, DC: Gallaudet Research Institute, Gallaudet University

VOLUME 156, No. 3,

2011

AMERICAN ANNALS OF THE DFAF

Copyright of American Annals of the Deaf is the property of American Annals of the Deaf and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder’s express written permission. However, users may print, download, or email articles for individual use.
PLACE THIS ORDER OR A SIMILAR ORDER WITH US TODAY AND GET AN AMAZING DISCOUNT 🙂