In this article, Alderson (2000)
i. reviews the pros and cons of computer-based language tests (CBTs);
ii. explores developments in Web-based testing, citing examples such as the Educational Testing Services’ computer-based TOEFL and the large-scale diagnostic testing project, DIALANG; and
iii. outlines a research agenda for future studies
According to the author, there are technical, administrative and pedagogical advantages of using CBTs over paper-and-pencil tests. Some of these include immediate student feedback, personalized testing, increased options for test administration, the possibility of storing enormous amounts of data for research purposes, and increased test security through test item randomization.
On the flip side, notable disadvantages include the possibility of bias against computer illiterate individuals, limitations in technology which do not allow for accurate assessment of productive language skills, and limited choice of test formats (e.g. multiple choice and gap-fill) which can lead to decontextualized forms of testing.
It seems that the advantages are great enough to warrant the development and use of computer-based tests in the field of EFL; however, at the time when Alderson’s (2000) article was written, very few attempts had been made by test writers to develop value-added web-based assessments. CBT TOEFL was one of those, which, according to the author, contained a number of innovative features but offered no evidence of value-addedness in the eyes of test candidates. On the other hand, DIALANG, which was, and still is, being used for only non-certification purposes, is rather innovative. It can assess proficiency in reading, writing, listening, vocabulary and grammar in 14 different European languages.
The author concludes his article by posing a list of intriguing questions for future researchers to mull over. The ones that evoke my curiosity the most are, “What does the provision of support imply for the validity of the tests, and for the constructs that can be measured?” and “What is the value of allowing learners to have a second attempt, with or without feedback on the success of their first attempt?” For a minute, let’s just forget about the implementation of computer-based testing. In the context of Hong Kong classrooms, I would imagine that ‘self-assessment’ is, relatively speaking, an unknown practice and it is going to remain this way for the long haul that’s for certain. But if it were ever made prevalent, could you imagine the scenario where a test taker is given a second chance to answer the items which he got wrong in his or her test paper the first time round – not only this, but with teacher support thrown in as well? How would the second set of test results be used - would they be used as part of the overall grades required for entry into a secondary school or a tertiary institution. If yes, how would the candidates feel about this? If no, I wonder what proportion of students would be bothered about the results after going through self-assessment? Surely, ‘normal’ or ‘high stakes’ assessments will only ever be conducted in the traditional manner (fixed time and location, paper-pencil format, one-paper-fits-all, etc.) whereas online assessment will always be reserved for ‘low stakes’ or diagnostic purposes, which may not be considered by some candidates as being as important or essential. Would anyone care to disagree?
Things have progressed a long way since 2000. When the paper mentioned diskettes I realized how dated it was, my guess is you can find some in a museum somewhere. My guess is there is some excellent software out there that will surpass anything mentioned in the article. http://www.freerice.com/index.php for example this one is pretty cool.
ReplyDeleteYeah, it looks miles better than Hot Potatoes for a start. What I like about its multiple choice quiz is that the questions cover a somewhat broad range of subjects, so that the test taker is not only testing his or her knowledge of the English language but also his or her knowledge of different content areas. Plus it's all for a good cause!
ReplyDeleteHi there Kiat, it is a really thorough review of the article, which reminds me a lot of my own experience in the past. Actually, on line testing has many disadvantages itself. As for the "self-assessment", in my opinion, a lot of students care more about whether they have get the right answer instead of what the right answer is. From this perspective, it is actually meanless to give them second round.
ReplyDeleteOh, by the way, the students just think "self-assessment" is not necessary, I remember that once I gave a student a website, where he could find relative English execerises and games, he just said, "oh, no need, I won't do them even they are games, I think they are silly!"
ReplyDeleteHi, Kiat, I would like to write a few comments to the question 'What is the value of allowing learners to have a second attempt, with or without feedback on the success of their first attempt?' Personally, the idea works if students can have instant feedbacks and their marks that students have gained in their self-assessments could be counted for their continuous assessment (CA). As a teacher, one of the ultimate aims of our teaching is to promote independent learning where learners can learn how to learn. Though I'm nore sure how the other schools view the importance of CALL, however, it appears to be more and more important in my insitution. At our school, students are required to complete a certain amount of work on the e-learning platform in order to meet the basic requirements and thus if under two conditions, first, there are some short quizzes embedded in the program which allows students to assess their own learning and second, the school has agreed to use the students' results of these quizzes for part of their CA, I think students would be very motivated.
ReplyDeleteI took part in on-line assessment when I was a student that I found it composes several pros and cons. To me, computer-assisted test garantees test is objective and it garantees the equality of tests. Just as mentioned above, self-assessment is neglected by students that they always depend on teachers' direct assessment.
ReplyDelete