Compute test-statistic for McNemar-test on proportions as the categorical variable of discordant pairs (McNemarScore)

class cerebstats.stat_scores.mcNemarScore.McNemarScore(*args: Any, **kwargs: Any)

Compute test-statistic for McNemar’s Test of proportions for change/difference in paired data (discordant pairs).

Two DEPENDENT samples, such that they disagree. Hence, there are two types of disagreement; sample-1 agrees but not sample-2 and, sample-2 agrees but not sample-1.

Definitions

Interpretation

\(b\)

proportion of one type of disagreement

\(c\)

proportion of other type of disagreement

\(n\)

sample size of discordant pairs; \(n = b + c\)

\(p_0\)

some specified value; 0.5 for McNemar testing

test-statistic

\(b\) for one disagreement type and, \(c\) for other disagreement

Note:

  • compute() takes two argument

  • it returns proportions (discordant) in the second argument

  • therefore, for the proportion of disagreement of interest pass it as the second argument

  • to compute p-value using scipy.stats.binom_test the arguments passed for number of successes is the test-statistic, number of trials is the number of discordant pairs and null value is 0.5

Use Case:

x = McNemarScoreForMcNemarTest.compute( observation, prediction )
score = McNemarScoreForMcNemarTest(x)

Note: As part of the SciUnit framework this custom TScore should have the following methods,

  • compute() (class method)

  • sort_key() (property)

  • __str__()

classmethod compute(observation, prediction)

Argument

Value type

first argument

dictionary; sample-1 (dependent sample) with “discordant_half”

second argument

dictionary; sample-2 (dependent sample) with “discordant_half”

Note:

  • observation must have the key “raw_data” whose value is the list of numbers