Different types of questions are scored differently. Multiple choice questions are scored by exact matching. We display a series of choices, only one of which is correct. If you select the right choice, it’s marked correct, otherwise it's marked wrong.
Short answer questions use artificial intelligence (AI) and natural language processing (NLP) to automatically score the answer. For a question like “Type the name of the fruit you see in the picture: ____” if the picture is of an apple, any answer that contains the word “apple” will be marked correct, for example, “Apple”, “An apple”, “The apple”, “I see an apple” will all be marked right and “Mango” or “Banana” will be wrong.
For questions like “Correct the mistakes in the phrase ‘Did u went two the, store yesterday!’”, the AI will compare the answer pattern, and ‘Did you go to the store yesterday?’ will be marked correct. For more open-ended questions, such as “What is most likely to happen next?” with a picture of two cars about to collide, our AI assesses both the meaning and grammar of your answer. “There will be a car accident.” or “The two cars will crash.” or “It looks like there will be a collision.” will be marked correct, while “Two people are driving cars.” will typically be marked wrong, as it does not capture the likely event.
Note: The system independently assesses whether you have answered the meaning of the question, how many spelling and grammar errors there are, and the sophistication (word choice, word length, sentence length, etc.) of your language.
Tests may also contain ungraded questions. These are typically open-ended questions that we have reserved for manual grading, or, new questions that we are checking for calibration and feedback before using them to automatically generate a score. These questions do not count towards your score level when the test is initially graded; once the question has been reviewed by one of our team, the answer is scored and your score updated if applicable. These items will appear with a blue – instead of a check mark or x.