Research
The Versant Research and Development team conducts internal research as a part of ongoing validity studies and gathers substantial evidence for the validity, reliability, and practicality of the Versant tests and other applications of the technology.
The team leverages the expertise of independent international experts to conduct validation studies and responds to requests from independent researchers who conduct experiments in the field of linguistics and use Versant products as one of their measures.
Research results are published in international journals. Here you can find some of the research conducted by the Research and Development team at Pearson and learn more about our experts.
Browse white papers, presentations and more
Ordinate Corporation. (2008). Internationally recognized English language exams: How they map to the Common European Framework (CEFR) Levels. Unpublished manuscript.
Ordinate Corporation. (2007). Test comparisons: A comparison of three internationally recognized tests of spoken English. Unpublished manuscript.
Pearson, Inc. (2008). Consistency of Versant English Test scores over multiple administrators. Unpublished manuscript.
English
Pearson, Inc. (2016). The Role of Listening Comprehension in the Versant English Test. Unpublished manuscript.
Spanish
Bernstein, J., Rosenfeld, E., Townshend, B., & Barbier, I. (2004, June). An automatically scored spoken Spanish test & its relation to OPIs. Paper presented at IAEA 2004, Pennsylvania, PA.*
Aviation
Balogh, J., Bernstein, J. & Suzuki, M. (2006, July). Automatically scored spoken language tests for air traffic controllers and pilots. Paper presented at International Aviation Training Symposium 2006, Oklahoma City.*
*Formerly part of Harcourt Assessment, Ordinate has been combined with Pearson. PhonePass SET-10 is now the Versant English Test. The Spoken Spanish Test (SST) is now the Versant Spanish Test.
Balogh, J. (2005, May). A common testing framework for computerized spoken language tests in multiple languages. Paper presented at the ALTE 2nd international conference, Berlin, Germany.*
Balogh, J., Bernstein, J., Barbier, I., & Rosenfeld, E. (2004, October). Relating automatic spoken Spanish score to the ILR scale. Paper presented at ECOLT 2004, Washington D.C. *
Benigno, V., Bonk, W., & Suzuki, M. (2016, June). Exploring the relationship between lexical diversity measures and L2 proficiency. Paper presented at Language Testing Research Colloquium (LTRC) 2016, Palermo, Italy.
Bernstein, J. (2004, April). The theory and practice of speaking assessment. Paper presented at the 38th TESOL annual convention, Long Beach, California.*
Bernstein, J., Cheng, J., Pado, U., Suzuki, M., Al-Saffar, M., Van Moere, A. (2008, November). Developing and Validating an Automated Test of Spoken MSA. Paper presented at East Coast Organization of Language Testers, Washington D.C.*
Bernstein, J., & Rosenfeld, E. (2005, June). Workable models of standard performance in English and Spanish. Paper presented at EALTA conference 2005, Voss, Norway. *
Bernstein, J., Rosenfeld, E., Townshend, B., & Barbier, I. (2004, June). An automatically scored spoken Spanish test & its relation to OPIs. Paper presented at IAEA 2004, Pennsylvania.*
Bonk, W., Oh, S., & Suzuki, M. (2018, October). Content coverage and other measures of L2 speaking and listening proficiency in integrated tasks. East Coast Association of Language Testers (ECOLT), Princeton, NK (October 12-13, 2018).
Bonk, W. & Suzuki, M. (2015, June). Use of automated writing scoring system in classroom and high-stakes contexts. Paper presented in a symposium at Language Testing Research Colloquium (LTRC) 2015, Toronto, Canada.
Cheng, J. & Suzuki, M. (2014, June). Beyond pronunciation and fluency: automated evaluation of prosody and accentedness. Paper presented at Language Testing Research Colloquium (LTRC) 2014, Amsterdam, The Netherlands.
Downey, R., Fond Lam, J., & Van Moere, A. (2012, April). To what extent does L2 oral reading performance in adults predict reading comprehension? Paper presented at Language Testing Research Colloquium, Princeton, New Jersey.*
Downey, R. & Van Moere, A. (2009, March). In the ear of the beholder: Dependence of comprehensibility on language background of speaker and listener. Paper presented at Language Testing Research Colloquium, Denver, Colorado.*
Downey, R., Van Moere, A., Suzuki, M., & Rubin, D. (2008, June). Developing of an automated oral reading fluency assessment. Paper presented at Language Testing Research Colloquium, Hanzhou, China.
Johnston, M., Suzuki, M., Wolfe, J., & Finn, G. (2017, June). Using an Integrated Approach to Improve the Development and Scoring of Technology-Based Speaking Tests. National Conference on Student Assessment 2017, Austin, TX.
Lam, J. & Downey, R. (2012, June) The validity of the reading comprehension test for immigrants: the Dutch case. Paper presented at European Association for Language Testing and Assessment, University of Innsbruck, Austria.*
Liu, H. Suzuki, M., & Klungtvedt, M. (2015, March). Profiling L2 learners’ pragmatic development through a computerized workplace English speaking test. Paper presented at the American Association of Applied Linguistics (AAAL) conference 2015, Toronto, Canada.
Present-Thomas, R. & Van Moere, A. (2009, November). NRS Classification Consistency of Two Spoken English Tests. Paper presented at East Coast Organization of Language Testers, Washington D.C.*
Suzuki, M. (2018, July). Re-defining the High-stakes Test of English with Automated Scoring Technology. Paper presented at The 2017 VUS TESOL conference, Ho Chi Ming, Vietnam.
Suzuki, M. (2018, August). Automated Scoring of English Language Skills to Achieve High Reliability, High Efficiency, and Standardization. Paper presented at The 27th Malaysian English Language Teaching Association International Conference, Johru Baru, Malaysia.
Suzuki, M. (2012, April). Are predictor variables for item difficulty of L2 sentence repetition consistent between English and Chinese? Paper presented at Language Testing Research Colloquium, Princeton, New Jersey.*
Suzuki, M., Balogh, J., & Bernstein, J. (2006, June). Psycholinguistics in computerized language testing. Paper presented at AAAL in Montreal, Canada. *
Suzuki, M. & Jia, Y. (2013, July). Factor structure of a spoken Chinese test: investigating five subskill scores for diagnosis. Paper presented at Language Testing Research Colloquium, Seoul, Korea.*
Van Moere, A. (2011, April). Evaluating the constructs and automated scoring performance for speaking tasks in the Versant test and PTE Academic. Paper presented at National Council on Measurement in Education, New Orleans, Louisiana.*
Van Moere, A. & Downey,R. (2010, May). Usability testing with stakeholders in the development of a formative assessment of oral reading fluency. Paper presented at European Association for Language Testing and Assessment, The Hague, the Netherlands.
Van Moere, A. Downey, R., Suzuki, M. & Klungtvedt, M. (2011, June). Assessing written English skills for business communication using time-constrained tasks. Paper presented at Language Testing Research Colloquium, Ann Arbor, Michigan.*
Van Moere, A. & Engelter, M. (2008, February). Versant™ Aviation English Test: A job-specific test of language that complies with ICAO requirements. Paper presented at the 9th Annual ATP Conference, San Antonio, TX.
Van Moere, A., Xu, X., & Klungtvedt, M. (2012, April). Elicited Imitation: What can it tell us about oral language proficiency? Paper presented at Language Testing Research Colloquium, Princeton, New Jersey.*
Xu, X., Suzuki, M. & Li, X. (2014, June). An Automated Spoken Chinese Test: Test Description, Comparison to other Tests and Classroom Application. Paper presented at presented at a trade show.*
*Formerly part of Harcourt Assessment, Ordinate has been combined with Pearson. PhonePass SET-10 is now the Versant English Test. The Spoken Spanish Test (SST) is now the Versant Spanish Test.
References for published papers
General
Balogh, J., Barbier, I., Bernstein, J., Suzuki, M., & Harada, Y. (2005). A Common Framework for Developing Automated Spoken Language Tests in Multiple Languages. JART Journal, 1(1), 67-79.
Balogh, J. & Bernstein, J. (2006). Workable models of standard performance in English and Spanish. In Y. Matsumoto, D.Y. Oshima, O.R. Robinson, & P. Sells (Eds.), Diversity in language: Perspective and implications (pp. 20-41). Stanford, CA: Center for the Study of Language and Information Publications.
Balogh, J., Bernstein, J., Cheng, J., Van Moere, A., Townshend, B., & Suzuki, M. (2012). Validation of automated scoring of oral reading. Educational and Psychological Measurement, 72(3) 435–452.
Bernstein, J. (2002). Constructs Broad and Narrow. In Y.N. Leung et al. (Eds.), Selected Papers from the Eleventh International Symposium on English Teaching/Fourth Pan Asia Conference (pp. 26-33). Taipei, ROC: The Crane Publishing.
Bernstein, J. (2001). Conciliation of psycholinguistic and communicative methods in spoken language testing. In Y.N. Leung et al. (Eds.), Selected Papers from the Tenth International Symposium on English Teaching (pp. 8-14). Taipei, ROC: The Crane Publishing.
Bernstein, J., Balogh, J., & Suzuki, M. (2005). Speech processing technology and spoken language tests. Proceedings of 2005 Acoustical Society of Japan Annual Conference, pp.863-866.
Bernstein, J. & Cheng, J. (2007). Logic and validation of fully automatic spoken English test. In M. Holland & F.P. Fisher (Eds.), The path of speech technologies in computer assisted language learning: From research toward practice (pp. 174-194). Florence, KY: Routledge.
Bernstein, J. & De Jong, J. H.A.L. (2001). An experiment in predicting proficiency within the Common Europe Framework Level Descriptors. In Y.N. Leung et al. (Eds.), Selected Papers from the Tenth International Symposium on English Teaching (pp. 8-14). Taipei, ROC: The Crane Publishing.
Bernstein, J., De Jong, J., Pisoni, D., & Townshend, B. (2000). Two experiments on automatic scoring of spoken language proficiency. In P. Delcloque (Ed.), Proceedings in InSTIL2000 (pp. 57-61). Dundee, Scotland: University of Abertay.
Bernstein, J., Todic, O., Townshend, B., & Wareen, E. (2000). Computer estimation of spoken language skills. In A. Ikeya & M. Kawamori (Eds.), PACLIC 14: 14th Pacific Asia Conference on Language, Information and Computation Proceedings (pp. 35-46). Chiba, Japan: PACLIC 14 Organizing Committee.
Bernstein, J., Van Moere, A. & Cheng, J. (2010). Validating automated speaking tests. Language Testing, 27(3), 355-377.
Blake, R., Wilson, N.L., Pardo-Ballestar, C., & Cetto, M. (2008). Measuring oral proficiency in distance, face-to-face, and blended classrooms. Language Learning and Technology, 12(3), pp.114-127.
Cheng, J. & Shen, J. (2011). Off-topic detection in automated speech assessment applications. Proceedings of Interspeech 2011,1597-1600.
Green, A. (2013). Relating the Versant English Placement Test to the Common European Framework of Reference. Technical report, digitally published on Versant Research. Pearson, San Jose, California.
Harada, Y. & Suzuki, M. (2005). A common framework for measuring spoken language skills of non-native speakers. In L. Anthony, S. Fujita & Y. Harada (Eds.), Proceedings of IWLeL 2004: An interactive workshop on language e-learning.
Nair, Anish. (2005). Item difficulty estimation using linguistic and acoustic predictors. Thought and Language Special Interest Group IEICE Tokyo: Technical Report of IEICE, 37-42.
Rosenfeld, E., Massaro, D., & Bernstein, J. (2003). Automatic analysis of vocal manifestations of apparent mood or affect. Proceedings of the 3rd international workshop on models and analysis of vocal emissions for biomedical applications (MAVEBA). Firenze, Italy.
Yokokawa, H., Yabuuchi, S., Suzuki, M., & Morishita, M. (2006). Vocabulary and response latencies in the spontaneous speech of Japanese EFL learners: An analysis of data from a short-term overseas study tour. Technical Report of IEICE.
Chinese Tests
Xu, X., Suzuki, M. & Cheng, J. (2012). An automated assessment of spoken Chinese: Technical definition of Hanyu standards for content and scoring development. Proceedings of the 7th Technology and Chinese Language Teaching Conference, 66-72.
Dutch Tests
De Jong, J. H.A.L., Lennig, M., Kerkhoff, A., & Poelmans, P. (2009). Development of a test of spoken Dutch for prospective immigrants. Language Assessment Quarterly, 6 (1), 41-60.
English Tests
Bernstein, J., Cheng, J., & Suzuki, M. (2011). Fluency Changes with General Progress in L2 Proficiency. Proceedings of Interspeech 2011, 31-50.
Bernstein, J., Cheng, J., & Suzuki, M. (2010). Fluency and structural complexity as predictors of L2 oral proficiency. Proceedings of Interspeech 2010, 1241-1244.
Barbier, I., & Bernstein, J. (2000/01). Design and development parameters for a rapid automatic screening tests for prospective simultaneous interpreters. Interpreting, 5(2), 221-238.
De Jong, J. & Bernstein, J. (2001). Relating PhonePass™ overall scores to the Council of Europe Framework level descriptors. In L. Mak, S. Chang, P. Foo, J. Hunter, M Keung, J. Lee, W. Mok, & N. Noakes (Eds.), Proceedings of the Conference of Technology in Language Education: Meeting the Challenges of Research and Practice (pp. 51-58). Hong Kong: Hong Kong University Press.
Downey, R., Farhady, H., Present-Thomas, R., Suzuki, M., & Van Moere, A. (2008). Evaluation of the usefulness of the Versant for English test: A response. Language Assessment Quarterly, 5, 160-167.
Todic, O. (2003). Automated spoken English tests using information technology and speech recognition. Technical Report of IEICE.
Aviation English Tests
Downey, R., Suzuki, M., & Van Moere, A. (2010). High-stakes English language assessments for aviation professionals: Justifying the use of a fully automated test of spoken language proficiency. IEEE Transactions on Professional Communication: Special Issue on Assessment in Professional Communication, 53 (1), 18-32.
Van Moere, A., Suzuki, M., Downey, R., & Cheng, J. (2009). Implementing ICAO Language Proficiency Requirements in the Versant Aviation English Test. Australian Review of Applied Linguistics, 32 (3), 27.1-27.17.
Arabic Tests
Cheng, J., Bernstein, J., Pado, U., & Suzuki, M. (2009) Automatic assessment of spoken Modern Standard Arabic. Proceedings of the Fourth Workshop on Innovative Use of NLP for Building Educational Applications. Warwickshire, England, 1-9.
Spanish Tests
Barbier, I. (2004). Design, development and evaluation of an automatically scored Spoken Spanish Test. Proceedings of CALL 2004 Conference.
Barbier, I, Rosenfeld, E., Bernstein, J. & De Jong, J. (2004). The development of the Spoken Spanish Test (SST). Language Testing Update (LTU), 34, 57-62.
Japanese Tests
Harada, Y., Suzuki, M., & Bernstein, J. (2004) Developing an Automated Test of Spoken Japanese. In H. Masuichi, T. Ohkuma, K. Ishikawa, Y. Harada, & K. Yoshimoto (Eds.), Proceedings of PACLIC 18: The 18th Pacific Asia Conference on Language, Information and Computation (pp. 291-298). Tokyo, Japan: PACLIC 18 Organizing Committee and PACLIC Steering Committee.
Harada, Y. & Suzuki, M. (2004). Developing an automated spoken Japanese test. Language Testing Update (LTU), 36, 80-84.
Harada, Y., Suzuki, M. & Bernstein, J. (2004). Developing an automated test of spoken Japanese. IEICE Technical Report, 104(503), 23-28.
Harada, Y., Suzuki, M., & Tomono, T. (2005). Development of a completely automated spoken Japanese test (title translated from Japanese). Proceedings of Forum of Information Technology 2005. Tokyo, Japan.
Suzuki, M. & Harada, Y. (2005). Using speech recognition for an automated test of spoken Japanese. Proceedings of PACLIC 19: The 19th Pacific Asia Conference of Language, Information and Computation (pp. 317-323). Taipei: Academia Sinica.
Bill grew up in Chicago, and is a graduate of Northwestern University (B.A., French & Linguistics), the University of Hawaii (M.A., Second Language Studies), and of the University of Colorado (PhD, Cognitive Science).
His work includes the study of mastery of verbal and nonverbal sequences, the psycholinguistics of word combinations and second language phonological processing, the design and psychometric analysis of second language tests, and growth modeling of various kinds of assessment data.
He taught EFL for more than ten years in Italy, Ecuador, Japan, and Brazil, and loves learning second languages himself as well. He has worked on various second language learning (L2) tests within Pearson for the past six years, including new tests for young learners (TELL, English Benchmark) and adults (Progress).
He lives in Boulder, Colorado.
Mallory has over 10 years of experience in test development and language teaching. She has an M.A. in Applied Linguistics and a certificate in Teaching English to Speakers of Other Languages (TESOL).
She has been a part of the Test Development team since 2008, working on developing automated language tests and conducting validation analyses. She has supported the Versant suite of tests as well as Pearson Test of English Academic and the Arizona English Language Learner Assessment.
Before joining Pearson, she taught English as a foreign language in the Czech Republic and Ecuador.
Saerhim is Senior Test Development Manager in the automated language assessment division at Pearson where she conducts research on automated language tests.
She received her doctorate degree in Applied Linguistics from Teachers College, Columbia University with a focus on second language assessment.
Her research interests include speech recognition in speaking assessment, test-takers' use of linguistic tools in writing assessment, pragmatics assessment, and young learners' assessment.
Before joining Pearson, Saerhim taught graduate courses in TESOL programs at New York University and City University of New York and taught in English language programs at Columbia University, University of Hawaii at Manoa, and Seoul National University.
Richard has dual Master degrees in Computer Science and Business from University of Illinois (Urbana Champaign/Chicago) and a B.S. in Electrical Engineering from Fudan University (Shanghai, China).
He has rich experience and deep understanding in acoustic modeling, automatic speech recognition, natural language processing, machine learning and algorithms.
Richard is actively involved in advancing the state of automatic speech recognition and is part of new test development, validation, research and implementation of new traits.
Prior to joining Pearson, Richard held roles as a software development manager for consumer electronics at Sony, and a lead software manager for mobile phone design at Alcatel-Bell.
Pengfei has a Ph.D. degree in Computer Science with a specialization in assistive technologies from City University of New York, Graduate Center.
As part of the Research & Development team, she works on speech recognition and automatic scoring.
Before joining Pearson, she worked on production development with speech recognition technologies at Intel Corporation.
John H.A.L. de Jong is Vice President Test Development at Pearson. He has over 30 years experience in language testing.
John graduated in General Linguistics, French and English languages from Leiden University and obtained a Ph.D. in Educational Measurement from Twente University. He holds a chair in Language Testing at the VU University Amsterdam, and has published articles and books on language assessment and educational measurement.
John has specialized in empirical scaling and the development of internationally standardized reporting scales of language proficiency. He was involved in developing the Common European Framework of Reference for Languages.
Before starting his career in language testing, John had been teaching French for seven years in secondary schools.
Dr Townshend founded Ordinate in 1996 with Dr Jared Bernstein.
He has extensive experience in signal processing, statistical modeling, computer and database systems design.
Dr Townshend has launched and managed two successful companies, and he is the inventor of the 56k modems that are now being sold by leading computing and communications companies. He was also a Consulting Professor of Electrical Engineering at Stanford University and former principal investigator at AT&T Bell Laboratories.
Jared is a recognized leader in the field of spoken language testing. He has a Ph.D. in Psycholinguistics and an M.A. in Linguistics.
He founded Ordinate with Dr Brent Townshend in 1996 and developed the patented Versant testing system.
Jared has led commercial system developments in both text-to-speech synthesis and speech recognition. He has extensive experience in speech processing and voice interactive system design at SRI International, MIT and Telesensory Systems, as well as editorial positions in linguistics and speech processing.
He is a Consulting Professor of Linguistics at Stanford University and has taught English and linguistics in the Middle East and South America.
Prof Bachman is a past president of the American Association for Applied Linguistics and of the International Language Testing Association.
He was also the first winner of the TESOL/Newbury House Award for Outstanding Research, and has won the Modern Language Association of America's Kenneth Mildenberger Award for outstanding research publication twice.
He is currently co-editor, of the journal Language Testing and of the Cambridge University Press series, Language Assessment.
Prof Brown is currently Director of the American Language Institute.
Previously, he taught at the University of Michigan and the University of Illinois. He was the President of TESOL from 1980 to 1981. And, from 1970 to 1979 he was the Editor of Language Learning.
Prof Brown has published many articles and books on second language acquisition and pedagogy, including Principles of Language Learning and Teaching.
Prof Luecht is currently Director of the Center for Educational Research and Evaluation at UNCG.
Prior to UNCG, he was Project Director for Computer-Adaptive Testing Research at the National Board of Medical Examiners (NBME).
Previously at ACT, Inc, he conducted research on automated test assembly, computer-adaptive testing simulations and IRT estimation issues. At ACT, he also worked with the National Assessment of Educational Progress (NAEP) Achievement Levels Standard Setting Project.
He is currently an Advisory Editor for the Journal of Education Measurement and the Computer Software Review Editor for Applied Psychological Measurement.
Recently Dean of the Stanford University School of Education, Dr Shavelson has also served as president of the American Educational Research Association.
Before joining Stanford, Prof Shavelson was dean of the Graduate School of Education and Professor of Statistics at the University of California, Santa Barbara from 1987 to 1993.
His recent psychometric work includes Generalizability Theory: A Primer, and research on the dependability and context representativeness of performance assessments used in education and work. His policy research includes two recent monographs on monitoring the nation's mathematics and science educatio
Prof Yang Huizhong currently is the Chairman of the National College English Testing Committee of China, in charge of a large scale standardized test with 4.5 million candidates each year.
He is a Professor of applied linguistics at Shanghai Jiao Tong University, where he was Chairman of the Department of Foreign Languages between 1985 and 1990.
For 12 years since 1987 he was President of All-China College English Teaching and Research Association. His areas of study are language testing, applied linguistics, and corpus linguistics.
Prof Yang has published many articles and books in the area, including CET Validation Study, An Introduction to Corpus Linguistics.
