Know how to set up reviewer roles in user profiles to simplify the process of manual reviewing of assessments
iMocha has added a new user role - Reviewer under user management. The reviewer role is specifically designed for teams that need to review assessments manually and score them accordingly. A Reviewer is an expert who quantifies the answers to the essay questions - coding questions, translation questions, or any questions where manual evaluation is needed.
Let's explore how the reviewer role works.
1. Adding a user as a Reviewer
Super Admin/Admin or any user with user management rights can add a Reviewer.
a. Log in to your iMocha account
b. Click Global settings -> Users -> Add Users
c. Enter the email address and full name
d. Select Reviewer roles from the listed User roles
Note: Reviewer gets default access rights to View Reports, View Questions, and Review Reports.
You can also assign a Reviewer role to an existing user by editing their access rights.
2. Enabling Manual Evaluation in Test settings
To assign reports for manual evaluation, the settings have to be enabled for the particular test.
a. Select the test
b. Click Settings -> Reviewer Settings
c. Select Yes
Note: If this setting is selected to No, reports cannot be assigned for review.
3. Assign reports to be evaluated
Once a user has been added as a Reviewer, reports can be assigned to them for evaluation. Any user with Assign Reviewer access can assign reports to a Reviewer to be evaluated.
The assignor also gets access to the Generate report after the designated Reviewers have finished reviewing their reports.
On completion of the test, the assignor will get an email to assign a reviewer.
a. Under Candidates, click Review Reports
b. Select the test and click Assign Reviewer
c. In the pop-up, you can view all the users with Reviewer access. You can select one or multiple reviewers. The reviewers will get an email notification.
d. The assigned reports will be seen in the Pending for Review tab
e. Once the Reviewer completes the evaluation, it will reflect under the Review Completed tab
4. Evaluation by Reviewer
The Reviewer will get the report for evaluation.
Note: The Reviewer will not see the candidate name; instead, it will be the candidate ID.
The Reviewer can then review the report and score the assessment accordingly.
5. Report Generation
Once all the reviewers complete their evaluation, the report is generated.
The score generated will be based on the average score given by all the reviewers. For any report, it is mandatory that at least one reviewer complete the evaluation.
The report can be viewed under Live Data or All Reports or from the Test itself.
6. Manage Reviewers
You can re-assign reports or revoke evaluation access of reviewers.
a. Click Manage Reviewers
b. From the list of reviewers, you can select reviewers to re-assign reports or revoke the access rights.
E-mails will be sent to the users informing the same.
6. How will it look for the Reviewers
a. When reports are assigned to the Reviewers, they will be informed via e-mail.
b. The reports can be seen by the Reviewers under Candidates -> Review Reports -> Pending for Review
c. Once the evaluation is complete; it will be seen under the Review Completed tab
Note: Once you click Finish Evaluation, the report cannot be modified.
In case of any queries, please write to us at support@imocha.io.
Also read,