Assessment through testing is a vital aspect of education. However, recent global challenges, such as the COVID-19 pandemic, have raised concerns about traditional in-person, paper-based testing. The question arises: Is it time for a paradigm shift in testing modes to embrace the growing ubiquity of Internet connectivity? Could online testing, accessible anytime and anywhere, become a more influential element in education? In reality, this transition is already underway, with universities in the U.S. and the U.K. increasingly moving towards online exams (UNIwise, 2022).
In terms of language testing and assessment, the Duolingo English Test (DET) might be a relative newcomer but is gaining traction with various international institutions such as Imperial College London and Harvard (Duolingo, n.d.). Therefore, while the authors were considering options for an easily administered proficiency test, the DET, which is an online-only test, appeared to be an interesting and viable alternative to other common tests in Japan (e.g., TOEIC).
The DET
Created by the educational technology company, Duolingo, the DET was released in 2016 and is an affordable, online, on-demand, standardised English proficiency test that can be taken from anywhere with an Internet connection, allowing test-takers to do the test in the comfort of their homes. However, to be clear, even though they share a name, the Duolingo language learning app and the DET are separate technologies.
The test is aimed at non-native-English speakers and assesses the four language skills of listening, speaking, reading, and writing in a single blended test. Compared to other proficiency tests, it is relatively short and adapts to each test-taker’s level by adjusting the difficulty of questions based on their previous answers. This ensures the AI software accurately measures competency in a rapid and efficient manner. The test structure is comprised of three sections: introduction and onboarding (5 minutes), which includes technical setup, ID input, and a rules review; the adaptive test (45 minutes); and a writing sample and speaking sample (10 minutes), totalling approximately one hour. The difficulty of the test varies throughout, and it contains numerous types of unique and randomly selected questions, such as listening to spoken words, selecting the real words, and filling in missing letters from a text. The number of questions is not predetermined as the grading engine adapts and finishes the test when it is confident of the test-taker’s score. Once the test has been submitted, the results are sent to the test-taker by email within 48 hours.
The test is assessed by AI and human proctors who check test-taker videos and keystrokes to ensure tests are completed correctly. Successful individuals will receive a computer-generated score that evaluates their proficiency in English on a scale ranging from 10 to 160, including four subscores (see Figure 1 on the previous page). The overall score is not simply an average of the subscores but rather a weighted combination. Figure 2 illustrates how the DET compares to other test scores.
Figure 2
The DET: Comparative Test Scores
Note. The overall DET score can be compared to other frameworks and test scores.
The Pros and Cons of the DET
There are many factors that make the DET captivating to takers of English proficiency tests:
- Flexibility and convenience: The DET can be taken anywhere with a stable Internet connection and a quiet room.
- Cost: 59 USD plus tax (about 8,750 JPY)
- Adaptive test: Question levels are adjusted depending on the test-taker’s answers, leading to a test time of approximately one hour (about half the time of the TOEIC test).
- Non-traditional question styles and test format: The four skills are tested in a blended manner (e.g., no extended reading, writing, or listening sections) which may suit some students better.
- Expedited results: Test scores are issued within 48 hours. Test-takers receive a holistic score and breakdown of four subscores (see Figure 1).
- Accepted at higher education establishments: The DET is used in admissions procedures of an increasing number of higher education establishments such as Stanford University in the U.S.
While the DET has many advantages over other English proficiency tests, namely its competitive price, convenience, and brevity, it also has some limitations:
- Test authentication and rules: These are the most significant issues for test-takers. Any violation of the stringent regulations risks test invalidation. Additional factors, including Internet or technology issues, can also impact success. The rules for computer-based tests without in-person invigilators differ from traditional testing, which results in some unusual requirements, such as not looking away from the screen for a prolonged period.
- ID validity: Test-takers may not have the requisite ID (a government-issued photo ID is required) so they may be unable to take the test.
- Test study: Compared to other tests, there are few specific test preparation materials and methods available. Students must improve their English across all four skills to improve their scores.
Student Impressions
A total of eight voluntary participants undertook the DET as part of a pilot study. In a survey, five participants reported that they enjoyed taking the DET, two were neutral, while one did not enjoy it. Six of the eight considered the test challenging, and six also felt that taking the test was a beneficial experience in their English language learning journey. These results are presented in Figure 3.
Figure 3
The DET: Student Impressions
Students commented that “It was [a] very strict test and hard to get verified” and “It’s too hard to get verified,” which resulted in the majority of participants needing more than one attempt to be authenticated by the DET AI and/or human proctors. However, despite these challenges, students generally had a positive impression of the test, with comments including: “Great. I assume that I can test my true practical English ability and essential understanding of language not just memorising words” and “I think it is good. The time limits make me feel pressure, and it also didn’t let me to think [about] one question a lot. It was thrilling, so I feel the time passed so quickly. I enjoyed taking the exam.”
Thus, test-takers’ impressions of the DET were positive, in terms of test style and task difficulty, but verification issues also frustrated the participants.
Recommendations
To get the most out of the DET, it is beneficial to consider the factors below, especially given that test verification can be problematic:
- Usage: This test would be extremely difficult to use in a classroom context. It is recommended for individual case-by-case instances only.
- Create an account: The DET homepage can be difficult to navigate pre-login, so it is recommended that test-takers and instructors create accounts to access vital information about the test more easily.
- Watch the summary videos: Watch the videos available to get a holistic understanding of the process of taking the DET.
- Read the rules: The DET rules are extremely rigorous and can also be unusual and unexpected for those who have not taken an online test before, so reading and understanding the rules in detail is extremely important for successful verification (e.g., test-takers can be disqualified simply for wearing their hair down, as their ears must always be visible).
- Check the environment and the technology: The test must be taken in a quiet, empty room with a reliable Internet connection. There are also various technology requirements, including a front-facing camera, microphone, and speaker.
- Do the practice test: Completing the (free) practice test is highly recommended.
Conclusion
In conclusion, the DET is gaining recognition and may play a significant role in the future of testing. From the perspective of instructors and invigilators, although its brevity, convenience, and affordability are positive aspects, there are significant practical issues that will hinder its adoption. In the context of the classroom, the DET is not a practical form of assessment, as its administration is logistically problematic—payment and ID issues, the requirement for individual test rooms and specific technology, as well as the difficulty of test validation—make it challenging for educators to facilitate multiple concurrent test-takers.
For individual higher-level test-takers who have a strong desire to study abroad, the DET may enhance applications to some institutions. However, in its current guise, and due to lagging recognition in educational institutions and workplaces in Japan, it is struggling for relevance, especially when competing with conventional testing options.
Improved technology, especially in AI proctoring, could enhance the reliability and credibility of the DET as an English proficiency test. It is likely a precursor for a future increase in online language testing and a compelling shift in educational assessment.
References
Duolingo. (n.d.). Certify your English proficiency today. Duolingo English Test. https://englishtest.duolingo.com/applicants
UNIwise. (2022, November 8). Adaptability, data and digital poverty: Digital assessment at Brunel University. Jisc. https://www.jisc.ac.uk/member-stories/adaptability-data-and-digital-poverty-digital-assessment-at-brunel-university
