For the Kappa measure for inter-judge agreement, the P(A) is to calculate the agreement proportion of actual situation, the P(E) is to get the theoretical agreement proportion by chance. What does the Kappa equation: [P(A) - P(E)]/ [1- P(E)] mean to evaluate the performance?
And if the number of judges is larger than 2, say 5, how to average the pairwise kappas? Is that (C5,2), that is, every two of them should be calculated with kappas then get their average?
No comments:
Post a Comment