Distribution shift and domain adaptation — Examples

Let $\mathsf{X}=\mathsf{Y}=\{0,1\}$ with the discrete sigma-algebras. Let the source joint law put mass one on $(0,0)$ and the target joint law put mass one on $(1,0)$. Both conditional label kernels assign label $0$ with probability one at each input. The input marginals are ${\mu }_S={\delta }_0$ and ${\mu }_T={\delta }_1$. Although the conditional label laws agree, there is no nonnegative measurable $w$ satisfying ${\mu }_T(A)={\int }_{A}wd{\mu }_S$ for every measurable $A$.

Source and target joint laws, risks, and covariate shift: defines the joint laws, conditional label kernels, input marginals, and the covariate-shift condition.

Importance weighting under covariate shift: requires absolute continuity of the target input marginal with respect to the source marginal.

Ben-David et al., §3, printed pp. 154–155: distinguishes shifts in input laws from changes in labeling functions.

Let $\mathsf{X}={\mathbb{N}}_{\ge 1}$ with the discrete sigma-algebra and $\mathsf{Y}=\{0,1\}$. Let the source input law have masses ${\mu }_S(\{n\})=2^{-n}$ and the target input law have masses ${\mu }_T(\{n\})=n{2}^{-(n+1)}$ for $n\ge 1$. In both domains the label is $0$ with probability one. Use the zero-one loss $\mathrm{\ell }(a,y)={\mathbf{1}}_{\{a\mathrm{\ne }y\}}$ and the constant classifier $h(n)=1$. The covariate-shift weight is $w(n)=n\mathrm{/}2$: it is unbounded, but the weighted loss is integrable and the target risk equals the importance-weighted source risk, both equal to $1$. Thus the exact risk identity holds while the bounded-weight hypothesis in any finite-sample concentration result that requires it fails.

Source and target joint laws, risks, and covariate shift: defines source and target joint laws, marginal laws, risks, and covariate shift.

Importance weighting under covariate shift: establishes the risk identity when the weighted loss is integrable.

Sugiyama, §3.1, book pp. 930–931: importance-weighted risk under covariate shift and the distinction between the exact identity and estimation methods.

Let the input space contain one point $x_0$, and let both source and target input marginals put mass one on $x_0$. Let the source label be $0$ surely and the target label be $1$ surely. For the class $\mathcal{H}=\{h_0,h_1\}$ of the two constant binary classifiers, every classifier pair has equal source and target disagreement probabilities, so $d_{\mathcal{H}\mathrm{\Delta }\mathcal{H}}({\mu }_S,{\mu }_T)=0$. Yet ${ϵ}_S(h_0)=0$, ${ϵ}_T(h_0)=1$, and the joint-labeling error $\lambda ={\inf }_{g\in \mathcal{H}}({ϵ}_S(g)+{ϵ}_T(g))=1$. Thus the target-risk bound is sharp for $h_0$, and equal input laws alone give no target-risk guarantee when the labeling law changes.

Source and target joint laws, risks, and covariate shift: defines joint laws, conditional label kernels, input marginals, and covariate shift.

H-delta-H divergence: defines divergence using only the input marginals.

Importance weighting under covariate shift: requires the conditional label kernels to agree on the target marginal.

Target risk bound by source risk, H-delta-H divergence, and joint error: retains the joint-labeling term $\lambda$ in the target-risk bound.

Ben-David et al., Definition 2 and Theorem 2 proof, printed pp. 156–158: the joint-labeling error and the role of that term in the transfer bound.