東華大學圖書館 |

Investigating Test Delivery Modes Within Video-Conferenced English Speaking Proficiency Assessment.

紀錄類型:	書目-電子資源 : Monograph/item
正題名/作者:	Investigating Test Delivery Modes Within Video-Conferenced English Speaking Proficiency Assessment./
作者:	Choi, Jin Soo.
面頁冊數:	1 online resource (253 pages)
附註:	Source: Dissertations Abstracts International, Volume: 84-03, Section: A.
Contained By:	Dissertations Abstracts International84-03A.
標題:	English as a second language. -
電子資源:	http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=29326493click for full text (PQDT)
ISBN:	9798841796428

Investigating Test Delivery Modes Within Video-Conferenced English Speaking Proficiency Assessment.
Choi, Jin Soo.

Investigating Test Delivery Modes Within Video-Conferenced English Speaking Proficiency Assessment. - 1 online resource (253 pages)

Source: Dissertations Abstracts International, Volume: 84-03, Section: A.

Thesis (Ph.D.)--Michigan State University, 2022.

Includes bibliographical references

Nonverbal behavior is essential in human interaction (Gullberg, de Bot, & Volterra, 2008; McNeill, 1992, 2005). For second language speakers, nonverbal features can be helpful for successful and efficient communication (e.g., Dahl & Ludvigsen, 2014). However, due to the complexity of nonverbal features, language testing institutions have conventionally employed an asynchronous format (e.g., audio-only mode), focusing on the test-taker's verbal features. Recently, the advance in technology, and most importantly, the coronavirus pandemic (COVID-19) outbreak in late 2019 and early 2020, have accelerated the use of video-conferencing applications in educational contexts, including speaking tests (Nakatsuhara, Inoue, Berry, & Galaczi, 2021). Accordingly, the synchronous test delivery mode (video-conferencing), which presents test-takers' visual information, is a timely and necessary approach to addressing the impact of nonverbal features on L2 oral communication. In response to these issues, I investigated the comparability of different delivery modes of speaking proficiency assessment. This study aimed to understand the dimensionality of the measured speaking construct and the impact of test-takers' visual information on rater behavior. Two datasets were used to address the research goals: first dataset (i.e., dataset 1) included scores of 110 test-takers, assessed by eight trained raters; second dataset (i.e., dataset 2) included scores of 284 prospective international teaching assistants (ITAs), assessed by 12 professional and certified raters. I collected dataset 1, and English language learning and testing program at a Midwest University provided dataset 2. I used two quantitative techniques to analyze both datasets: confirmatory factor analysis (CFA) and Multifaceted Rasch model (MFRM) analysis. For dataset 1, I qualitatively analyzed the raters' verbal reports.Dataset 1 had an asynchronous format; all test-takers' performances were recorded for rating. Eight trained raters gave scores to the audio-recorded and video 1 mode (test-taker and test giver were displayed). Three weeks later, the raters assessed video 2 mode (only test-taker's visual information was displayed). Within one month, raters participated in a one-on-one semi-structured interview. Dataset 2 stems from an operational testing context. This dataset has only scores, as I borrowed the data from the operational English testing program at the university. The scores were first given by examiners in a synchronous format (Live mode) and later by raters in an asynchronous format (Recorded mode). CFA findings indicated the multi-dimensional aspect of the underlying construct of speaking for both datasets, but the high inter-correlations showed that these are associated. Findings of MFRM revealed that raters showed leniency when rating (a) video mode over audio-only mode (dataset 1) and (b) synchronous mode over asynchronous mode (dataset 2). Findings suggest that using the video-conferenced delivery mode may be beneficial. However, the degree of usefulness across video modes differed, and how the raters utilized test-takers' nonverbal behaviors (e.g., gaze) varied. Thus, I decided that further investigation is needed to sufficiently support the use of video-conferencing applications to complement the physical face-to-face delivery mode. Overall, future research is highly recommended regarding the standardization of scoring of nonverbal features about the types of video mode, which would assist with the practical and valid application of virtual speaking tests.

Electronic reproduction.
Ann Arbor, Mich. :
ProQuest,
2023

Mode of access: World Wide Web

ISBN: 9798841796428Subjects--Topical Terms:

516208
English as a second language.
Subjects--Index Terms:

EnglishIndex Terms--Genre/Form:

542853
Electronic books.

Investigating Test Delivery Modes Within Video-Conferenced English Speaking Proficiency Assessment.
LDR:05039nmm a2200433K 4500 001 2354636
005 20230428105635.5
006 m o d
007 cr mn ---uuuuu
008 241011s2022 xx obm 000 0 eng d
020 $a 9798841796428
035 $a (MiAaPQ)AAI29326493
035 $a AAI29326493
040 $a MiAaPQ $b eng $c MiAaPQ $d NTU
100 1 $a Choi, Jin Soo. $3 3694993
245 1 0 $a Investigating Test Delivery Modes Within Video-Conferenced English Speaking Proficiency Assessment.
264 0 $c 2022
300 $a 1 online resource (253 pages)
336 $a text $b txt $2 rdacontent
337 $a computer $b c $2 rdamedia
338 $a online resource $b cr $2 rdacarrier
500 $a Source: Dissertations Abstracts International, Volume: 84-03, Section: A.
500 $a Advisor: Winke, Paula.
502 $a Thesis (Ph.D.)--Michigan State University, 2022.
504 $a Includes bibliographical references
520 $a Nonverbal behavior is essential in human interaction (Gullberg, de Bot, & Volterra, 2008; McNeill, 1992, 2005). For second language speakers, nonverbal features can be helpful for successful and efficient communication (e.g., Dahl & Ludvigsen, 2014). However, due to the complexity of nonverbal features, language testing institutions have conventionally employed an asynchronous format (e.g., audio-only mode), focusing on the test-taker's verbal features. Recently, the advance in technology, and most importantly, the coronavirus pandemic (COVID-19) outbreak in late 2019 and early 2020, have accelerated the use of video-conferencing applications in educational contexts, including speaking tests (Nakatsuhara, Inoue, Berry, & Galaczi, 2021). Accordingly, the synchronous test delivery mode (video-conferencing), which presents test-takers' visual information, is a timely and necessary approach to addressing the impact of nonverbal features on L2 oral communication. In response to these issues, I investigated the comparability of different delivery modes of speaking proficiency assessment. This study aimed to understand the dimensionality of the measured speaking construct and the impact of test-takers' visual information on rater behavior. Two datasets were used to address the research goals: first dataset (i.e., dataset 1) included scores of 110 test-takers, assessed by eight trained raters; second dataset (i.e., dataset 2) included scores of 284 prospective international teaching assistants (ITAs), assessed by 12 professional and certified raters. I collected dataset 1, and English language learning and testing program at a Midwest University provided dataset 2. I used two quantitative techniques to analyze both datasets: confirmatory factor analysis (CFA) and Multifaceted Rasch model (MFRM) analysis. For dataset 1, I qualitatively analyzed the raters' verbal reports.Dataset 1 had an asynchronous format; all test-takers' performances were recorded for rating. Eight trained raters gave scores to the audio-recorded and video 1 mode (test-taker and test giver were displayed). Three weeks later, the raters assessed video 2 mode (only test-taker's visual information was displayed). Within one month, raters participated in a one-on-one semi-structured interview. Dataset 2 stems from an operational testing context. This dataset has only scores, as I borrowed the data from the operational English testing program at the university. The scores were first given by examiners in a synchronous format (Live mode) and later by raters in an asynchronous format (Recorded mode). CFA findings indicated the multi-dimensional aspect of the underlying construct of speaking for both datasets, but the high inter-correlations showed that these are associated. Findings of MFRM revealed that raters showed leniency when rating (a) video mode over audio-only mode (dataset 1) and (b) synchronous mode over asynchronous mode (dataset 2). Findings suggest that using the video-conferenced delivery mode may be beneficial. However, the degree of usefulness across video modes differed, and how the raters utilized test-takers' nonverbal behaviors (e.g., gaze) varied. Thus, I decided that further investigation is needed to sufficiently support the use of video-conferencing applications to complement the physical face-to-face delivery mode. Overall, future research is highly recommended regarding the standardization of scoring of nonverbal features about the types of video mode, which would assist with the practical and valid application of virtual speaking tests.
533 $a Electronic reproduction. $b Ann Arbor, Mich. : $c ProQuest, $d 2023
538 $a Mode of access: World Wide Web
650 4 $a English as a second language. $3 516208
650 4 $a Educational tests & measurements. $3 3168483
650 4 $a Language. $3 643551
650 4 $a Educational technology. $3 517670
650 4 $a Educational evaluation. $3 526425
653 $a English
653 $a Language assessment
653 $a Nonverbal behavior
653 $a Speaking
653 $a Validity
653 $a Video conferencing
655 7 $a Electronic books. $2 lcsh $3 542853
690 $a 0441
690 $a 0288
690 $a 0679
690 $a 0443
690 $a 0710
710 2 $a ProQuest Information and Learning Co. $3 783688
710 2 $a Michigan State University. $b Second Language Studies - Doctor of Philosophy. $3 3171490
773 0 $t Dissertations Abstracts International $g 84-03A.
856 4 0 $u http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=29326493 $z click for full text (PQDT)