Identical input laws can hide a large joint-labeling error

Let the input space contain one point $x_0$, and let both source and target input marginals put mass one on $x_0$. Let the source label be $0$ surely and the target label be $1$ surely. For the class $\mathcal{H}=\{h_0,h_1\}$ of the two constant binary classifiers, every classifier pair has equal source and target disagreement probabilities, so $d_{\mathcal{H}\mathrm{\Delta }\mathcal{H}}({\mu }_S,{\mu }_T)=0$. Yet ${ϵ}_S(h_0)=0$, ${ϵ}_T(h_0)=1$, and the joint-labeling error $\lambda ={\inf }_{g\in \mathcal{H}}({ϵ}_S(g)+{ϵ}_T(g))=1$. Thus the target-risk bound is sharp for $h_0$, and equal input laws alone give no target-risk guarantee when the labeling law changes.

Source and target joint laws, risks, and covariate shift: defines joint laws, conditional label kernels, input marginals, and covariate shift.

H-delta-H divergence: defines divergence using only the input marginals.

Importance weighting under covariate shift: requires the conditional label kernels to agree on the target marginal.

Target risk bound by source risk, H-delta-H divergence, and joint error: retains the joint-labeling term $\lambda$ in the target-risk bound.

Ben-David et al., Definition 2 and Theorem 2 proof, printed pp. 156–158: the joint-labeling error and the role of that term in the transfer bound.