The Cross Section of Expected CFA Exam Results

For decades, CFA candidates have attempted to understand their exam performance using CAPM.

CAPM, the CFA Assessment Passing Model states that a candidate’s performance on an exam is explained by the following:

CFA-CAPM

This Nobel prize winning model has long shaped the way academics and candidates think about probability of passing and therefore greatly informs modern exam preparation. Oddly though, the model explains only 70% of the variation in test scores among a diversified pool of exam candidates.

This year, in both July and August, we noticed one of our web pages was receiving unusually righteous traffic – the result of painful, heart-wrenching google queries. I’m talking:

cfa_results_search_1

and

cfa_results_search_2

From the potentially “try, try again” of …

cfa_results_search_3

…to the fatalistic, or voyeuristic

cfa_results_search_4

Having our own similar history of been there, searched for that along with remembering anecdotal stories of “dudes who passed without really studying” and vice versa, we began researching whether a more complete multivariate model might better explain performance drivers and help candidates plan for future exams.

We researched. We found one.

Today, I am pleased to announce the Lumi-Log Three Factor ModelTM. First, the equation:

3-factor model

The expression obviously begins with single factor CAPM. What is novel is our discovery of, and extension by two additional factors, HML and SMB, which also affect a candidate’s pass rate. Our backtesting, across both time and country, showed that the new model explained over 90% of the variation in candidate scores – a marked improvement.

HML, or High Minus Low captures, roughly, how a candidate wakes up feeling on exam day. Some candidates get a good night’s sleep and arrive at 8 AM on test day firing on all cylinders (High level of operation). Others have one of those “coffee’s not working” sort of days (Low level of operation), and feel immensely foggy, despite teetotaling for weeks and regularly taking a multivitamin.

SMB, or Small Minus Big captures, roughly, the amount of time that has elapsed between what material is actually tested and how recently the candidate reviewed it. As it is arguably impossible to be equally fresh on 3000 pages of dubious financial theory, certain candidates simply luck out and happen to get tested on what they studied recently (Small elapsed time). Others arrive only to find complex commodity carry arbitrage swaption puzzles using – what could be worse? – exponentially compounding rates of return, that they last reviewed in April (Big time ago).

In conclusion, if you start with 100,000 CFA candidates flipping coins, and only those who flip heads proceed to the next level… well, you get a 50% pass rate Einstein! But CFA pass rates have mysteriously averaged about 5% lower than this across years and levels, a result we could only duplicate in our labs using a loaded coin. Correlation does not imply causation, so the bottom line is this: If you just found out you failed, but the answer to the question “5 years from now, do I want to have CFA after my name” is still yes, then re-register and keep showing up each year. Eventually, the stars (actually, our trademarked 3 factors) will align.

And just what will you do with yourself then?

2 thoughts on “The Cross Section of Expected CFA Exam Results”

  1. Hi Lumi,
    Thinking back to the first time I read one of your posts way back in 2009, when I first enrolled in the CFA Program, I didn’t think it would take 5 years to finish this program. I finally passed L3 and I’m more relieved than happy.

    Looking back at some of your posts and some of those miserable nights studying, I’m pretty happy I stuck with it even after taking L2 three times. Luckily I was able to pass L1 and L3 on my first attempts.

    Thanks for taking the time to blog. It was a very nice break from studying…knowing that others are in the same boat, struggling to cram in more and more CFA knowledge. Now I just have to wait two more years to finally be able to put those three letters after my name.

    Good luck with your investing career and I hope you someday get those letters because you deserve them. Take care. I’ll be back here but for now, after five years, back to living a normal life.

  2. a huge congrats to you yellowman. way to stick with it! please keep in touch and let me know when you get those letters! thanks for sharing the journey and taking the time to drop a line. 🙂

Leave a Reply

Your email address will not be published.