Know the key components in the AI-EnglishPro report and how it can help identify the top performers
Overview
AI-EnglishPro empowers customers to assess Business English proficiency skills like Reading, Writing, Speaking and Listening in a single test. iMocha has based its EnglishPro assessment on CEFR (Common European Framework of Reference) standards to empower our customers to make data-driven hiring decisions.
Adding the AI-EnglishPro Assessment
The first step begins with adding the assessment to your account. You can request your account manager to add the test to your account. Since this is an assessment curated by language experts, with the right mix of questions for Reading, Speaking, Writing, and Listening, the editing option is disabled. Once the test has been added to your account, you can create links and invite candidates. Read How to create links and invite candidates.
Reading the Report
After the candidate completes the test, the report is generated within 2 minutes. Let's see how to read the AI-EnglishPro report.
1. Score Analysis
The Score Analysis section displays a snapshot of the candidate's performance. Each section is for 100 points, and the candidate's answer is evaluated using Artificial Intelligence and Natural Language Processing algorithms.
2. Section Score Analysis
The section analysis breaks down the scoring into two components, Communicative Skills and Enabling Skills. The bar graph indicates the percentage scored by the candidate for each skill. Each bar graph is color-coded as per the performance category.
3. Section-Skill Analysis
The section-wise skills analysis throws light on how the candidate has performed in each section for various skills. In the writing section, Grammatical mistakes indicate the errors highlighted by AI, which the human eye might miss. For example, whitespace here refers to a lack of space ( ) between two words or paragraphs or more than one space between words or paragraphs. A typographical error means that a word is misspelled due to the wrong alphabetical key being struck. For example, wrong can be typed as wrnog.
4. CEFR Level
The Common European Framework of Reference for Languages (CEFR) is a standardized grading system aiming to validate language ability. It is the weighted average and provides a composite score of the candidate’s writing abilities. The levels are- A1, A2, B1, B2, C1, C2. The CEFR levels indicate the written text falls under which category.
The CEFR level is disabled by default.
The CEFR levels will not affect the scores for AI-Speaking type questions.
We recommend keeping it disabled for questions evaluating basic English skills in email writing or essay-type answers without any word limit, which is more suitable for beginners.
And enable it for questions related to business scenarios that require email or essay-type answers, with a minimum of 100 words. The CEFR framework is well-suited for evaluating individuals in mid-level positions.
The weightage for each skill when CEFR is enabled is as follows:
| Communication Skills | Parameters | Weightage |
| Speaking | Oral Fluency | 50 |
| Speaking | Vocabulary | 50 |
| Writing | CEFR | 20 |
| Writing | Grammer | 25 |
| Writing | Vocabulary | 25 |
| Writing | Word Count | 20 |
| Writing | Readability | 10 |
The weightage for each skill when CEFR is disabled is as follows:
| Communication Skills | Parameters | Weightage |
| Speaking | Oral Fluency | 50 |
| Speaking | Vocabulary | 50 |
| Writing | Grammer | 35 |
| Writing | Vocabulary | 35 |
| Writing | Word Count | 20 |
| Writing | Readability | 10 |
5. Vocabulary categorization (based on English Vocabulary Profile)
This feature helps you to categorize the words used by the candidate (in the Writing section) in terms of the Common European Framework of Reference for Languages (CEFR) levels. It is unique in categorizing vocabulary in terms of the CEFR levels based on a huge candidate dataset. There is a total of 6 categories on which the data is displayed. The types indicate what percentage of words from each category the candidate uses in their written text.
| Council of Europe levels | Description |
| C2 Mastery |
The capacity to deal with material that is academic or cognitively demanding, and to use language to good effect at a level of performance that may in certain respects be more advanced than that of an average native speaker. |
| C1 Effective Operational Proficiency |
The ability to communicate with the emphasis on how well it is done, in terms of appropriacy, sensitivity, and the capacity to deal with unfamiliar topics. |
| B2 Vantage |
The capacity to achieve most goals and express oneself on a range of topics. |
| B1 Threshold |
The ability to express oneself in a limited way in familiar situations and to deal in a general way with non-routine information. |
| A2 Waystage |
An ability to deal with simple, straightforward information and begin to express oneself in familiar contexts. |
| A1 Breakthrough |
A basic ability to communicate and exchange information in a simple way. |
5. Readability Score
A readability score is a number that tells you how easy it will be for someone to read a particular piece of text. iMocha's readability score is based on the average length of sentences and words in your document, using a Flesch reading ease test formula. In most cases, a score of 60 or higher means that your document is easy to read for most people with at least a US eighth-grade education.
6. Linguistic Diversity
Linguistic Diversity is based on the analysis of metadiscourse markers in the text.
Also known as ‘transitions,’ these are words and phrases such as ‘firstly’ and ‘in conclusion’ that add extra information to a text.
They can:
- show how ideas in a text are connected to each other
- help the reader understand which direction the text is flowing in
- present the writer’s opinion and potentially take a stance
- express the writer’s degree of certainty
- help the writer connect with the audience
This linguistic diversity percentage is calculated as = the Total number of words defined by the meta-discourse markers/Total words used in the text * 100.
iMocha analyses the following meta-discourse markers
| Announce Goals (Frame marker) | Relational markers |
| Code glosses | Attitude markers |
| Endophorics | Emphatics (Boosters) |
| Hedges | Evidentials |
| Logical connectives | Label stages (Frame marker) |
| Person markers | Sequencing (Frame marker) |
| Topic shifts (Frame marker) |
These categories are based on the types identified by Stephen Bax, Fumiyo Nakatsuhara, and Daniel Waller in their 2019 study published in the Science Direct journal. This study was built upon the work done by renowned British linguist Prof. Ken Hyland.
The full list of words under each category are as follows:
Announce Goals (Frame marker)
|
|
|
|
|
|
|
|
|
|
|
|
Attitude markers
|
|
|
Emphatics (Boosters)
|
|
|
|
|
|
|
|
|
Topic shifts (Frame marker)
|
|
|
Evidentials
|
|
|
Sequencing (Frame marker)
|
|
|
Person markers
|
|
|
7. Context Check- Ethical Proctoring
AI-enabled Contextual Check ensures candidates' responses to the AI-enabled English Speaking and Writing questions in the tests are related, relevant, and to the point. It guarantees that English assessments perfectly fit your organizational needs, leading to a new level of integrity. It empowers you to make more refined talent decisions by focusing on specific skills. We provide ethical proctoring for cheating prevention. Our AI-based evaluation system detects when candidates try to cheat by repeating the same things or adding irrelevant information to their answers. The AI-based contextual check enhances integrity and focuses on the response relevance.
The digression percentage based on the digression and total sentences is calculated, considering the context of the answer. Our AI-based contextual check smartly detects sentences in the response that are unrelated, contain irrelevant information, or repeat the same content.
To view the Context Check Report, go to the Questions tab in the candidate's report and click get the report.
The glimpse of the iMocha AI Context Report:
For any queries or if you would like to see AI-EnglishPro in action, mail us at support@imocha.io.