search for




 

Alternative representations of the net reclassification improvement for a paired sample
Journal of the Korean Data & Information Science Society 2019;30:1209-19
Published online November 30, 2019;  https://doi.org/10.7465/jkdi.2019.30.6.1209
© 2019 Korean Data and Information Science Society.

Chong Sun Hong1 · Hye Soo Shin2 · Hae Seon Jeon3

123Department of Statistics, Sungkyunkwan University
Correspondence to: Professor, Department of Statistics, Sungkyunkwan University, 25-2, Sungkyunkwan-Ro, Jongno-Gu, Seoul, 03063, Korea. E-mail: cshong@skku.edu
Received September 9, 2019; Revised October 18, 2019; Accepted October 21, 2019.
This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/3.0) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
Abstract
One of the principle methods of evaluating the performance of a predictive model is to express a confusion matrix between the predicted disease (or normal) and actual disease (or normal) populations. If there exists other method to evaluate performance, two confusion matrices corresponding to prediction models could be classified for a paired sample. Using these reclassification tables, a net reclassification improvement (NRI) was proposed as a statistic to compare the performance of the two prediction models. In this study, since NRI is expressed as both sensitivity and specificity, the relationship between NRI and some accuracy measures for estimating optimal threshold could be derived. Based on these relations, alternative test statistics are proposed to compare the performance of the two predictive models. Through empirical data and simulations for a paired sample, we can identify the characteristics of the test statistics. Therefore, we might suggest that these proposed methods are used to test the homogeneity of various accuracy measures for a paired sample.
Keywords : Accuracy, confusion, homogeneous, reclassification, threshold.