Practice questions in Piqosity feature a dynamically calculated difficulty rating from 1 to 9, where 1 is easiest and 9 is hardest.

  • Levels 1, 2, 3 are generally considered easy, and represent about 15% of questions
  • 4, 5, and 6 are medium difficulty, and represent about 70% of questions
  • 7, 8, and 9 are hard, and represent about 15% of questions


Initially a question's difficulty is selected by its author when it's first written.


After a question has been answered many times by actual students, Piqosity uses machine learning to dynamically set the question's difficulty level based off what percentage of students in a certain norm group are selecting the correct answer.


Piqosity calculates this difficulty level using a Z-score and dividing questions into 9 categories of difficulty, each of which represent 1/2 standard deviation.


Dynamic question difficulty offers huge benefits to students and their educators including:

  1. Ensuring that Piqosity's questions are the correct level of difficulty for the course
  2. Allowing for better personalization of question delivery to students
  3. Enabling new ways for students to practice questions including additional game modes


Here's how Piqosity dynamically calculates difficulty levels step-by-step:


  1. We continuously track what percentage of students are answering a question correctly
    • On the first attempt
    • At a certain grade level
    • Without the aid of an educator
  2. Every night at 2am Houston time, we calculate the mean and median of this accuracy by topic (i.e. ISEE Verbal Reasoning or ACT Math or Math 5 Decimals and Fractions.)
  3. We calculate the Z-score for each question by subtracting the percent answered correct for a question from the mean of the topic-level dataset and dividing by the standard deviation of the same topic-level dataset.
  4. We look at the Z-score for every question and separate questions into 9 groups, "stanines," each of which represent 1/2 standard deviation from the mean

For students viewing a question page, they will see:


  1. What percentage of students answered a question correctly
  2. A text-based difficulty level of easy, medium, hard
  3. And a numerical difficulty value from 1-9 


For educators writing or editing a question, they will additionally see:


  1. Author's manually selected difficulty
  2. Piqosity's dynamically calculated difficulty
  3. EVAD score (expected versus actual difficulty = manual difficulty - dynamic difficulty)
  4. The Z-score
  5. How many answer responses the dataset is based off
  6. The date the dataset was let reset