Research

The Versant Research and Development team conducts internal research as a part of ongoing validity studies and gathers substantial evidence for the validity, reliability, and practicality of the Versant tests and other applications of the technology.

The team leverages the expertise of independent international experts to conduct validation studies and responds to requests from independent researchers who conduct experiments in the field of linguistics and use Versant products as one of their measures.

Research results are published in international journals. Listed below is some of the research conducted by the Research and Development team at Pearson.

Research Papers, White Papers, and Presentations

Test Comparisons

Ordinate Corporation. (2008). Internationally recognized English language exams: How they map to the Common European Framework (CEFR) Levels. Unpublished manuscript.

Ordinate Corporation. (2007). Test comparisons: A comparison of three internationally recognized tests of spoken English. Unpublished manuscript.

Reliability Studies

Pearson, Inc. (2008). Consistency of Versant English Test scores over multiple administrators. Unpublished manuscript.

White Papers

English

Pearson, Inc. (2016). The Role of Listening Comprehension in the Versant English Test. Unpublished manuscript.

Spanish

Bernstein, J., Rosenfeld, E., Townshend, B., & Barbier, I. (2004, June). An automatically scored spoken Spanish test & its relation to OPIs. Paper presented at IAEA 2004, Pennsylvania, PA.*

Aviation

Balogh, J., Bernstein, J. & Suzuki, M. (2006, July). Automatically scored spoken language tests for air traffic controllers and pilots. Paper presented at International Aviation Training Symposium 2006, Oklahoma City.*

*Formerly part of Harcourt Assessment, Ordinate has been combined with Pearson. PhonePass SET-10 is now the Versant English Test. The Spoken Spanish Test (SST) is now the Versant Spanish Test.

Conference Presentations

Balogh, J. (2005, May). A common testing framework for computerized spoken language tests in multiple languages. Paper presented at the ALTE 2nd international conference, Berlin, Germany.*

Balogh, J., Bernstein, J., Barbier, I., & Rosenfeld, E. (2004, October). Relating automatic spoken Spanish score to the ILR scale. Paper presented at ECOLT 2004, Washington D.C. *

Benigno, V., Bonk, W., & Suzuki, M. (2016, June). Exploring the relationship between lexical diversity measures and L2 proficiency. Paper presented at Language Testing Research Colloquium (LTRC) 2016, Palermo, Italy.

Bernstein, J. (2004, April). The theory and practice of speaking assessment. Paper presented at the 38th TESOL annual convention, Long Beach, California.*

Bernstein, J., Cheng, J., Pado, U., Suzuki, M., Al-Saffar, M., Van Moere, A. (2008, November). Developing and Validating an Automated Test of Spoken MSA. Paper presented at East Coast Organization of Language Testers, Washington D.C.*

Bernstein, J., & Rosenfeld, E. (2005, June). Workable models of standard performance in English and Spanish. Paper presented at EALTA conference 2005, Voss, Norway. *

Bernstein, J., Rosenfeld, E., Townshend, B., & Barbier, I. (2004, June). An automatically scored spoken Spanish test & its relation to OPIs. Paper presented at IAEA 2004, Pennsylvania.*

Bonk, W., Oh, S., & Suzuki, M. (2018, October). Content coverage and other measures of L2 speaking and listening proficiency in integrated tasks. East Coast Association of Language Testers (ECOLT), Princeton, NK (October 12-13, 2018).

Bonk, W. & Suzuki, M. (2015, June). Use of automated writing scoring system in classroom and high-stakes contexts. Paper presented in a symposium at Language Testing Research Colloquium (LTRC) 2015, Toronto, Canada. → get slide deck from Bill

Cheng, J. & Suzuki, M. (2014, June). Beyond pronunciation and fluency: automated evaluation of prosody and accentedness. Paper presented at Language Testing Research Colloquium (LTRC) 2014, Amsterdam, The Netherlands.

Downey, R., Fond Lam, J., & Van Moere, A. (2012, April). To what extent does L2 oral reading performance in adults predict reading comprehension? Paper presented at Language Testing Research Colloquium, Princeton, New Jersey.*

Downey, R. & Van Moere, A. (2009, March). In the ear of the beholder: Dependence of comprehensibility on language background of speaker and listener. Paper presented at Language Testing Research Colloquium, Denver, Colorado.*

Downey, R., Van Moere, A., Suzuki, M., & Rubin, D. (2008, June). Developing of an automated oral reading fluency assessment. Paper presented at Language Testing Research Colloquium, Hanzhou, China.

Johnston, M., Suzuki, M., Wolfe, J., & Finn, G. (2017, June). Using an Integrated Approach to Improve the Development and Scoring of Technology-Based Speaking Tests. National Conference on Student Assessment 2017, Austin, TX.

Lam, J. & Downey, R. (2012, June) The validity of the reading comprehension test for immigrants: the Dutch case. Paper presented at European Association for Language Testing and Assessment, University of Innsbruck, Austria.*

Liu, H. Suzuki, M., & Klungtvedt, M. (2015, March). Profiling L2 learners’ pragmatic development through a computerized workplace English speaking test. Paper presented at the American Association of Applied Linguistics (AAAL) conference 2015, Toronto, Canada.

Present-Thomas, R. & Van Moere, A. (2009, November). NRS Classification Consistency of Two Spoken English Tests. Paper presented at East Coast Organization of Language Testers, Washington D.C.*

Suzuki, M. (2018, July). Re-defining the High-stakes Test of English with Automated Scoring Technology. Paper presented at The 2017 VUS TESOL conference, Ho Chi Ming, Vietnam.

Suzuki, M. (2018, August). Automated Scoring of English Language Skills to Achieve High Reliability, High Efficiency, and Standardization. Paper presented at The 27th Malaysian English Language Teaching Association International Conference, Johru Baru, Malaysia.

Suzuki, M. (2012, April). Are predictor variables for item difficulty of L2 sentence repetition consistent between English and Chinese? Paper presented at Language Testing Research Colloquium, Princeton, New Jersey.*

Suzuki, M., Balogh, J., & Bernstein, J. (2006, June). Psycholinguistics in computerized language testing. Paper presented at AAAL in Montreal, Canada. *

Suzuki, M. & Jia, Y. (2013, July). Factor structure of a spoken Chinese test: investigating five subskill scores for diagnosis. Paper presented at Language Testing Research Colloquium, Seoul, Korea.*

Van Moere, A. (2011, April). Evaluating the constructs and automated scoring performance for speaking tasks in the Versant test and PTE Academic. Paper presented at National Council on Measurement in Education, New Orleans, Louisiana.*

Van Moere, A. & Downey,R. (2010, May). Usability testing with stakeholders in the development of a formative assessment of oral reading fluency. Paper presented at European Association for Language Testing and Assessment, The Hague, the Netherlands.

Van Moere, A. Downey, R., Suzuki, M. & Klungtvedt, M. (2011, June). Assessing written English skills for business communication using time-constrained tasks. Paper presented at Language Testing Research Colloquium, Ann Arbor, Michigan.*

Van Moere, A. & Engelter, M. (2008, February). Versant™ Aviation English Test: A job-specific test of language that complies with ICAO requirements. Paper presented at the 9th Annual ATP Conference, San Antonio, TX.

Van Moere, A., Xu, X., & Klungtvedt, M. (2012, April). Elicited Imitation: What can it tell us about oral language proficiency? Paper presented at Language Testing Research Colloquium, Princeton, New Jersey.*

Xu, X., Suzuki, M. & Li, X. (2014, June). An Automated Spoken Chinese Test: Test Description, Comparison to other Tests and Classroom Application. Paper presented at presented at a trade show.*

*Formerly part of Harcourt Assessment, Ordinate has been combined with Pearson. PhonePass SET-10 is now the Versant English Test. The Spoken Spanish Test (SST) is now the Versant Spanish Test.

References for Published Papers

General

Balogh, J., Barbier, I., Bernstein, J., Suzuki, M., & Harada, Y. (2005). A Common Framework for Developing Automated Spoken Language Tests in Multiple Languages. JART Journal, 1(1), 67-79.

Balogh, J. & Bernstein, J. (2006). Workable models of standard performance in English and Spanish. In Y. Matsumoto, D.Y. Oshima, O.R. Robinson, & P. Sells (Eds.), Diversity in language: Perspective and implications (pp. 20-41). Stanford, CA: Center for the Study of Language and Information Publications.

Balogh, J., Bernstein, J., Cheng, J., Van Moere, A., Townshend, B., & Suzuki, M. (2012). Validation of automated scoring of oral reading. Educational and Psychological Measurement, 72(3) 435–452.

Bernstein, J. (2002). Constructs Broad and Narrow. In Y.N. Leung et al. (Eds.), Selected Papers from the Eleventh International Symposium on English Teaching/Fourth Pan Asia Conference (pp. 26-33). Taipei, ROC: The Crane Publishing.

Bernstein, J. (2001). Conciliation of psycholinguistic and communicative methods in spoken language testing. In Y.N. Leung et al. (Eds.), Selected Papers from the Tenth International Symposium on English Teaching (pp. 8-14). Taipei, ROC: The Crane Publishing.

Bernstein, J., Balogh, J., & Suzuki, M. (2005). Speech processing technology and spoken language tests. Proceedings of 2005 Acoustical Society of Japan Annual Conference, pp.863-866.

Bernstein, J. & Cheng, J. (2007). Logic and validation of fully automatic spoken English test. In M. Holland & F.P. Fisher (Eds.), The path of speech technologies in computer assisted language learning: From research toward practice (pp. 174-194). Florence, KY: Routledge.

Bernstein, J. & De Jong, J. H.A.L. (2001). An experiment in predicting proficiency within the Common Europe Framework Level Descriptors. In Y.N. Leung et al. (Eds.), Selected Papers from the Tenth International Symposium on English Teaching (pp. 8-14). Taipei, ROC: The Crane Publishing.

Bernstein, J., De Jong, J., Pisoni, D., & Townshend, B. (2000). Two experiments on automatic scoring of spoken language proficiency. In P. Delcloque (Ed.), Proceedings in InSTIL2000 (pp. 57-61). Dundee, Scotland: University of Abertay.

Bernstein, J., Todic, O., Townshend, B., & Wareen, E. (2000). Computer estimation of spoken language skills. In A. Ikeya & M. Kawamori (Eds.), PACLIC 14: 14th Pacific Asia Conference on Language, Information and Computation Proceedings (pp. 35-46). Chiba, Japan: PACLIC 14 Organizing Committee.

Bernstein, J., Van Moere, A. & Cheng, J. (2010). Validating automated speaking tests. Language Testing, 27(3), 355-377.

Blake, R., Wilson, N.L., Pardo-Ballestar, C., & Cetto, M. (2008). Measuring oral proficiency in distance, face-to-face, and blended classrooms. Language Learning and Technology, 12(3), pp.114-127.

Cheng, J. & Shen, J. (2011). Off-topic detection in automated speech assessment applications. Proceedings of Interspeech 2011,1597-1600.

Harada, Y. & Suzuki, M. (2005). A common framework for measuring spoken language skills of non-native speakers. In L. Anthony, S. Fujita & Y. Harada (Eds.), Proceedings of IWLeL 2004: An interactive workshop on language e-learning.

Nair, Anish. (2005). Item difficulty estimation using linguistic and acoustic predictors. Thought and Language Special Interest Group IEICE Tokyo: Technical Report of IEICE, 37-42.

Rosenfeld, E., Massaro, D., & Bernstein, J. (2003). Automatic analysis of vocal manifestations of apparent mood or affect. Proceedings of the 3rd international workshop on models and analysis of vocal emissions for biomedical applications (MAVEBA). Firenze, Italy.

Yokokawa, H., Yabuuchi, S., Suzuki, M., & Morishita, M. (2006). Vocabulary and response latencies in the spontaneous speech of Japanese EFL learners: An analysis of data from a short-term overseas study tour. Technical Report of IEICE.

Chinese Tests

Xu, X., Suzuki, M. & Cheng, J. (2012). An automated assessment of spoken Chinese: Technical definition of Hanyu standards for content and scoring development. Proceedings of the 7th Technology and Chinese Language Teaching Conference, 66-72.

Dutch Tests

De Jong, J. H.A.L., Lennig, M., Kerkhoff, A., & Poelmans, P. (2009). Development of a test of spoken Dutch for prospective immigrants. Language Assessment Quarterly, 6 (1), 41-60.

English Tests

Bernstein, J., Cheng, J., & Suzuki, M. (2011). Fluency Changes with General Progress in L2 Proficiency. Proceedings of Interspeech 2011, 31-50.

Bernstein, J., Cheng, J., & Suzuki, M. (2010). Fluency and structural complexity as predictors of L2 oral proficiency. Proceedings of Interspeech 2010, 1241-1244.

Barbier, I., & Bernstein, J. (2000/01). Design and development parameters for a rapid automatic screening tests for prospective simultaneous interpreters. Interpreting, 5(2), 221-238.

De Jong, J. & Bernstein, J. (2001). Relating PhonePass™ overall scores to the Council of Europe Framework level descriptors. In L. Mak, S. Chang, P. Foo, J. Hunter, M Keung, J. Lee, W. Mok, & N. Noakes (Eds.), Proceedings of the Conference of Technology in Language Education: Meeting the Challenges of Research and Practice (pp. 51-58). Hong Kong: Hong Kong University Press.

Downey, R., Farhady, H., Present-Thomas, R., Suzuki, M., & Van Moere, A. (2008). Evaluation of the usefulness of the Versant for English test: A response. Language Assessment Quarterly, 5, 160-167.

Todic, O. (2003). Automated spoken English tests using information technology and speech recognition. Technical Report of IEICE.

Aviation English Tests

Downey, R., Suzuki, M., & Van Moere, A. (2010). High-stakes English language assessments for aviation professionals: Justifying the use of a fully automated test of spoken language proficiency. IEEE Transactions on Professional Communication: Special Issue on Assessment in Professional Communication, 53 (1), 18-32.

Van Moere, A., Suzuki, M., Downey, R., & Cheng, J. (2009). Implementing ICAO Language Proficiency Requirements in the Versant Aviation English Test. Australian Review of Applied Linguistics, 32 (3), 27.1-27.17.

Arabic Tests

Cheng, J., Bernstein, J., Pado, U., & Suzuki, M. (2009) Automatic assessment of spoken Modern Standard Arabic. Proceedings of the Fourth Workshop on Innovative Use of NLP for Building Educational Applications. Warwickshire, England, 1-9.

Spanish Tests

Barbier, I. (2004). Design, development and evaluation of an automatically scored Spoken Spanish Test. Proceedings of CALL 2004 Conference.

Barbier, I, Rosenfeld, E., Bernstein, J. & De Jong, J. (2004). The development of the Spoken Spanish Test (SST). Language Testing Update (LTU), 34, 57-62.

Japanese Tests

Harada, Y., Suzuki, M., & Bernstein, J. (2004) Developing an Automated Test of Spoken Japanese. In H. Masuichi, T. Ohkuma, K. Ishikawa, Y. Harada, & K. Yoshimoto (Eds.), Proceedings of PACLIC 18: The 18th Pacific Asia Conference on Language, Information and Computation (pp. 291-298). Tokyo, Japan: PACLIC 18 Organizing Committee and PACLIC Steering Committee.

Harada, Y. & Suzuki, M. (2004). Developing an automated spoken Japanese test. Language Testing Update (LTU), 36, 80-84.

Harada, Y., Suzuki, M. & Bernstein, J. (2004). Developing an automated test of spoken Japanese. IEICE Technical Report, 104(503), 23-28.

Harada, Y., Suzuki, M., & Tomono, T. (2005). Development of a completely automated spoken Japanese test (title translated from Japanese). Proceedings of Forum of Information Technology 2005. Tokyo, Japan.

Suzuki, M. & Harada, Y. (2005). Using speech recognition for an automated test of spoken Japanese. Proceedings of PACLIC 19: The 19th Pacific Asia Conference of Language, Information and Computation (pp. 317-323). Taipei: Academia Sinica.

These brands trust Versant for their recruiting and placement programs.