Nate Meyvis

Don't confuse total and marginal correctness rates

Spaced repetition learners often aim at target "retention rate:" they want to review cards when the probability of a correct response is, say, .85. They then look at their overall correctness rate and compare it to .85, adjusting parameters if it's higher or lower.

I think this is a mistake. If it makes sense to think about a target retention rate, the rate is about letting a memory decay a bit, but not too much. Your target expresses a tradeoff between saving time now and preventing too much decay.

This tradeoff applies to cards you are about to forget, but some reviews are less like this. When you review a card right after you make it, or right after you miss it, you are quite likely to get it right (or at least I am). The conventional wisdom says that this not wasting time: these reviews have value for reasons other than that you're about to forget them.

So, when you're examining your results, try to compare only the relevant subset of reviews against your target retention rate. Because your overall rate is buoyed by the high-probability reviews, you'll make things too hard for yourself if you set your overall rate where you want your rate for "decaying" questions to be.

A few notes:

  1. When I analyze my results, this is a pretty large effect. I'm well over 90% on first-time and I-just-missed reviews, and these are a lot of my reviews. So this isn't just a logical point.
  2. Maybe you implicitly set your target retention rate to account for this effect. That's fine! But it's still worth being clear about what's going on.
  3. I think that if you carry out this line of reasoning, you should conclude that target retention rates are not a very good way to calibrate a spaced repetition system. But that's a separate post.

#flashcards #spaced repetition