The Impact of Task effect and Examiner Conduct upon the Performance of Japanese Candidates in the Cambridge Speaking Tests

Julie Norton

University of Leicester, UK

This paper demonstrates that task effect and examiner conduct impact upon the assessment process in the Cambridge Speaking Tests. It aims to raise awareness of how specific language elicitation techniques influence the language sample produced, highlighting the implications for test standardisation and test item writing. The main finding is that the language elicited during the Picture Description task is not consistent with the expectations outlined in the assessment criteria. Data exemplifying this mismatch is presented which seeks to enhance the language awareness of test item writers, oral examiners and English language teachers. The data sample includes thirty-three candidates and consists of transcriptions of the Speaking Tests. The theoretical frameworks of conversation analysis and discourse analysis are drawn upon in the analysis of the data.

Young and Milanovic (1992: 415) found task to have the "strongest effect of any of the contextual variables considered" in their study of discourse variation in FCE Speaking Tests. Their findings reveal that task affects the quantity of talk, the number of topic initiations and the persistence of topics. The Picture Description Task was found to produce the least amount of talk of any of the tasks in their study. This is consistent with the findings of this study which considers the reliability of the data elicited during the aforementioned task.

FCE and CAE examiners are required to adhere to a script when administering the Speaking Tests. This concern for test standardisation could reinforce the institutional nature of the interview, rendering it less like conversational interaction. The trialling of rubrics by The University of Cambridge Local Examinations Syndicate suggests, however, that even minor amendments to the rubrics impact significantly upon the focus or level of the task (UCLES 1998: 19). Deviations from the rubrics by oral examiners in this study are shown to affect the complexity of the language of instructions, the nature of the task and the content of the discussion, and hence, to impact upon test standardisation. The findings of this study endorse the decision to include rubrics at particular points in the tests for purposes of standardisation.