I’m 97% sure most CPA exam candidates are confused by the CPA exam’s psychometric scoring, either because it is supposed to be that way or they haven’t done their research. Either way, I once again got the chance to speak with John Mattar, Ed.D., Director of Psychometrics and Research and Mike Decker, Director of Operations and Development, both of the AICPA’s examinations unit. This time we focused on how the CPA exam is scored. Remember that most of this information is already available on the AICPA’s website, check out How the CPA Exam is Scored and the Psychometrics and Scoring section for more detailed, less sarcastic information than what you might find here. That being said, we appreciate John and Mike taking time to humor us anyway.

Of course, no discussion about how the exam is scored would be complete without rehashing last quarter’s somewhat tardy score release issue. John and Mike compared it to buying a new car but driving home in your old car, meaning scoring is going to be a broke down Toyota Tercel for just two more quarters but if you all can be patient, you’ll be spinning around town in that shiny new Lexus by December. “We’re doing everything we can to administer a quality exam, including communicating with candidates,” Mike told us. They also let us know that they will be using NASBA to push out timely information to candidates in the quarters ahead. See? Told you they were listening to candidates’ scoring concerns.

When talking about how the CPA exam is scored, it’s important to remember that candidates take different but equivalent exams. “It’s not possible to say what each testlet is worth because everyone is taking different exams,” said John. That being said, we did manage to get them to tell us that, contrary to popular belief, candidates are not compared to each other when they are scored. How do we know? While we still don’t know how many points each question is worth, John told us “we can say with 100% assurance if two different candidates get the same question, they will both get the same amount of points or credit for getting that question right.” This whole exam scoring thing is feeling less and less insidious by the minute, isn’t it?

For the final time: the CPA exam is not graded on a curve, nor are you compared to everyone who did better than you, nor are you compared to everyone who showed up to Prometric that day or week or month. “The way the exam is scored, candidates are compared to a fixed ability level. They are not compared to each other. If in the next window candidates maybe aren’t as well-prepared, fewer people will pass. They are being compared to a fixed level of ability,” John told us. Twice. Just to make sure we all got it. Got it? Let’s go over it one more time (from the Scoring FAQs):

The CPA Examination is NOT curved. Every candidate’s score is entirely independent of other candidates’ Examination results.

The CPA Examination is a criterion-referenced examination which means that it rests upon pre-determined standards. Every candidate’s performance is measured against established standards to determine whether the candidate has demonstrated the level of knowledge and skills that is represented by the passing score. Every candidate is judged against the same standards, and every score is an independent result.

Are we clear on that? Awesome, moving on…

Pretest questions make up fifteen MCQ in AUD/FAR and twelve in REG/BEC, one task-based simulation in FAR, AUD and REG and one written communication in BEC. The problem with gaming this system is that you can’t, since you have no way of knowing which questions are pretest and which are operational. So just guess equally on all of them as if every single one counts, mmmkay? Pretest questions will resemble operational questions since they are testing whatever is in the Content Specification Outline, meaning IFRS wasn’t tested on a pretest basis last year. What this means for candidates is that nothing outside of the CSOs will EVER be pretested. Maybe not a life-changing piece of information but really useful to know if you are prone to asking “what should I be studying?” and are familiar with the CSOs.

So what’s with the score report you get when you fail that compares you to those who scored between 75 and 80 in that window? Why not compare you to the CPA exam superstars who pulled down 99s without breaking a sweat?

“If you compared [failing candidates] to people that got all the way up to 99, it’s not as useful,” John said. “We want the basis of comparison that is going to be the most meaningful to the people who failed and that group is those that got closest to passing. If I got a 62 and the comparison group includes all the people who got above 90, that isn’t going to help me as much. Most of the people who pass are between 75 – 80.”

While we appreciate the AICPA taking the time to speak with us, we feel it’s important to point out that, at the end of the day, they are the best source of information for candidates. Answers to most of your questions can be found on their website. If you’re having trouble finding something or have a specific question related to the CPA exam, get in touch and we’ll do our best to do the Googling on your behalf.