Three distinct calibration notions in three classes

Each forecast row and each conditional-label row in Models A, B, and C has nonnegative coordinates summing to one, so all entries define simplex forecasts and conditional label distributions. In each model the three forecast vectors are distinct. Therefore full reliability would require $p_i=g_i$ at every row by [F1], but Model A row 1, Model B row 1, and Model C row 1 each have $p_i\mathrm{\ne }{g}_i$. Full reliability fails in all three models.

In Model A, for each label the groups with forecast coordinate $0.4$ have conditional frequency $0.4$, and the groups with coordinate $0.3$ have average conditional frequency $0.3$: for labels 1, 2, and 3 the latter averages are respectively $(0.2+0.4)\mathrm{/}2$, $(0.4+0.2)\mathrm{/}2$, and $(0.2+0.4)\mathrm{/}2$. Thus Model A is classwise calibrated. Its top confidence is always $0.4$, its predicted labels are $1,2,3$ in the three rows, and each corresponding conditional correctness probability is $0.4$, so it is top-confidence calibrated.

In Model B the top confidence is always $0.5$, the predicted labels are $1,2,3$, and the average conditional correctness is $(0.7+0.5+0.3)\mathrm{/}3=0.5$, so top-confidence calibration holds. Classwise calibration fails because forecast coordinate $G_1=0.5$ occurs only in row 1, where $P(Y=1\mid X=1)=0.7\mathrm{\ne }0.5$.

In Model C, for each label the forecast coordinate $0.5$ group has average conditional frequency $0.5$, and the coordinate $0$ group has frequency $0$: the three nonzero-group averages are $(0.8+0.2)\mathrm{/}2$, $(0.2+0.8)\mathrm{/}2$, and $(0.8+0.2)\mathrm{/}2$. Thus classwise calibration holds. All top confidences equal $0.5$; the tie-broken predicted labels are $1,1,2$, with conditional correctness probabilities $0.8,0.2,0.8$. Their average is $0.6\mathrm{\ne }0.5$, so top-confidence calibration fails. Steps 1.1, 1.2, and 1.3 establish the other model claims; the Model C calculation completes the pairwise distinctions.