Balance for positive and negative outcomes

Let $\mathrm{\Sigma }$ and $\mathcal{B}$ be nonempty finite sets, and let $t\in \{1,2\}$ index two groups. In group $t$, feature type $\sigma$ has proportion $a_{t\sigma }\ge 0$, where ${\sum }_{\sigma }{a}_{t\sigma }=1$, and total mass $N_t>0$. The positive binary outcome $Y=1$ has probability $p_{\sigma }\in [0,1]$ for feature type $\sigma$, with the same $p_{\sigma }$ in both groups. A common randomized assignment sends feature type $\sigma$ to bin $b\in \mathcal{B}$ with probability $X_{\sigma b}\ge 0$, where ${\sum }_b{X}_{\sigma b}=1$, independently of group and outcome; bin $b$ has score $v_b\in [0,1]$. Set ${\rho }_t={\sum }_{\sigma }{a}_{t\sigma }{p}_{\sigma }$ and ${\mu }_t=N_t{\rho }_t$. Assume $0<{\mu }_t<N_t$ for each group, so both outcome classes have positive mass. Define the mean score among positive outcomes and among negative outcomes in group $t$ by

$${\alpha }_t=\frac{\sum _{\sigma \in \mathrm{\Sigma }}\sum _{b\in \mathcal{B}}{a}_{t\sigma }{p}_{\sigma }{X}_{\sigma b}{v}_b}{{\rho }_t},{\beta }_t=\frac{\sum _{\sigma \in \mathrm{\Sigma }}\sum _{b\in \mathcal{B}}{a}_{t\sigma }(1-p_{\sigma })X_{\sigma b}{v}_b}{1-{\rho }_t},$$

where the finite group law is ${\mathrm{Pr}}_t(\mathrm{\Sigma }=\sigma ,Y=1,B=b)=a_{t\sigma }{p}_{\sigma }{X}_{\sigma b}$ and ${\mathrm{Pr}}_t(\mathrm{\Sigma }=\sigma ,Y=0,B=b)=a_{t\sigma }(1-p_{\sigma })X_{\sigma b}$. The assignment is balanced for positive outcomes if ${\alpha }_1={\alpha }_2$, and balanced for negative outcomes if ${\beta }_1={\beta }_2$. These are class-conditioned mean-score criteria; they do not assert equal mean scores over whole groups or equal thresholded error rates.

KMR's finite group-conditioned risk-score assignment and calibration: fixes the finite joint law, the $[0,1]$ score, and the group base rate ${\rho }_t$. The nondegeneracy condition $0<{\mu }_t<N_t$ is imposed here.

Kleinberg, Mullainathan, and Raghavan, complete 23-page paper: §1.1 defines balance for the negative and positive classes as equality of the corresponding groupwise mean assigned scores.

Applied Statistics: From Data to Decisions, full chapter: the section “The impossibility: you cannot satisfy them all” states both score-balance criteria and distinguishes them from the binary-classifier theorem.