Glossary:R-Metric: Difference between revisions

From SuperMemo 18 Help
Jump to navigation Jump to search
No edit summary
m Tools -> Toolkit
Line 1: Line 1:
;<span id="R-Metric">R-Metric</span> (Recall Metric)
;<span id="R-Metric">R-Metric</span> (Recall Metric)
:absolute measure of performance of two [[Glossary:Spaced_repetition|spaced repetition]] algorithms based on their ability to predict recall before a [[Glossary:Grade|grade]] is scored. In [[What's new in SuperMemo 17?|SuperMemo 17]], '''R-Metric''' is used solely to compare [[SuperMemo Algorithm SM-15|Algorithm SM-15]] (known from SuperMemo 16) and the new [[SuperMemo Algorithm|Algorithm SM-17]]. It is shown as percentage in '''Statistics''' and '''[[Tools menu|Tools]] : [[Tools menu#Statistics|Statistics]] : [[Analysis]] : [[Analysis#Use|Use]] : Efficiency : [[Analysis#Use_:_Efficiency_:_R-Metric|R-Metric]]'''. '''R-Metric''' is a difference between the performance of the two algorithms: <code>R-Metric=LSRM(Alg-15)-LSRM(Alg-17)</code>, where <code>LSRM</code> is the least squares predicted recall measure for a given algorithm. '''R-Metric''' greater than zero shows superiority of [[SuperMemo Algorithm|Algorithm SM-17]]. '''R-Metric''' less than zero indicates underperformance of the new algorithm. <code>LSRM</code> is a square root of the average of squared absolute differences in recall predictions: <code>abs(Recall-PredictedRecall)</code>, where <code>Recall</code> is 0 for failing [[Glossary:Grade|grades]] and <code>Recall</code> is 1 for passing grades. <code>PredictedRecall</code> is a prediction issued by the algorithm before the [[Glossary:Repetition|repetition]]. In [[SuperMemo Algorithm|Algorithm SM-17]], the prediction is a weighted average of the value taken from the [[Glossary:Recall_matrix|Recall[] matrix]], and [[Glossary:Retrievability|R (retrievability)]] computed from [[Glossary:Stability|S (stability)]] and the used [[Glossary:Interval|interval]]. The weight used is based on prior repetition cases which inform of the significance of the Recall[] matrix prediction (the prediction becomes more meaningful with more prior repetition data)
:absolute measure of performance of two [[Glossary:Spaced_repetition|spaced repetition]] algorithms based on their ability to predict recall before a [[Glossary:Grade|grade]] is scored. In [[What's new in SuperMemo 17?|SuperMemo 17]], '''R-Metric''' is used solely to compare [[SuperMemo Algorithm SM-15|Algorithm SM-15]] (known from SuperMemo 16) and the new [[SuperMemo Algorithm|Algorithm SM-17]]. It is shown as percentage in '''Statistics''' and '''[[Toolkit menu|Toolkit]] : [[Toolkit menu#Statistics|Statistics]] : [[Analysis]] : [[Analysis#Use|Use]] : Efficiency : [[Analysis#Use_:_Efficiency_:_R-Metric|R-Metric]]'''. '''R-Metric''' is a difference between the performance of the two algorithms: <code>R-Metric=LSRM(Alg-15)-LSRM(Alg-17)</code>, where <code>LSRM</code> is the least squares predicted recall measure for a given algorithm. '''R-Metric''' greater than zero shows superiority of [[SuperMemo Algorithm|Algorithm SM-17]]. '''R-Metric''' less than zero indicates underperformance of the new algorithm. <code>LSRM</code> is a square root of the average of squared absolute differences in recall predictions: <code>abs(Recall-PredictedRecall)</code>, where <code>Recall</code> is 0 for failing [[Glossary:Grade|grades]] and <code>Recall</code> is 1 for passing grades. <code>PredictedRecall</code> is a prediction issued by the algorithm before the [[Glossary:Repetition|repetition]]. In [[SuperMemo Algorithm|Algorithm SM-17]], the prediction is a weighted average of the value taken from the [[Glossary:Recall_matrix|Recall[] matrix]], and [[Glossary:Retrievability|R (retrievability)]] computed from [[Glossary:Stability|S (stability)]] and the used [[Glossary:Interval|interval]]. The weight used is based on prior repetition cases which inform of the significance of the Recall[] matrix prediction (the prediction becomes more meaningful with more prior repetition data)


[[File:Daily Algorithm SM-17 performance metric.jpg|100px|thumb|center|SuperMemo 17 performance metric]]
[[File:Daily Algorithm SM-17 performance metric.jpg|100px|thumb|center|SuperMemo 17 performance metric]]

Revision as of 06:13, 3 March 2019

R-Metric (Recall Metric)
absolute measure of performance of two spaced repetition algorithms based on their ability to predict recall before a grade is scored. In SuperMemo 17, R-Metric is used solely to compare Algorithm SM-15 (known from SuperMemo 16) and the new Algorithm SM-17. It is shown as percentage in Statistics and Toolkit : Statistics : Analysis : Use : Efficiency : R-Metric. R-Metric is a difference between the performance of the two algorithms: R-Metric=LSRM(Alg-15)-LSRM(Alg-17), where LSRM is the least squares predicted recall measure for a given algorithm. R-Metric greater than zero shows superiority of Algorithm SM-17. R-Metric less than zero indicates underperformance of the new algorithm. LSRM is a square root of the average of squared absolute differences in recall predictions: abs(Recall-PredictedRecall), where Recall is 0 for failing grades and Recall is 1 for passing grades. PredictedRecall is a prediction issued by the algorithm before the repetition. In Algorithm SM-17, the prediction is a weighted average of the value taken from the Recall[] matrix, and R (retrievability) computed from S (stability) and the used interval. The weight used is based on prior repetition cases which inform of the significance of the Recall[] matrix prediction (the prediction becomes more meaningful with more prior repetition data)
File:Daily Algorithm SM-17 performance metric.jpg
SuperMemo 17 performance metric