Binary logistic and multiclass softmax conditional models

For $d\ge 1$, define the sigmoid $\sigma (t)=(1+e^{-t}{)}^{-1}$. A binary logistic conditional model on $x\in {\mathbb{R}}^d$ has finite coefficients ${\beta }_0\in \mathbb{R}$, $\beta \in {\mathbb{R}}^d$ and specifies $\mathrm{Pr}(Y=1\mid x)=\sigma ({\beta }_0+x^T\beta )$ and $\mathrm{Pr}(Y=0\mid x)=1-\sigma ({\beta }_0+x^T\beta )$ for $Y\in \{0,1\}$. Its classifier predicts $1$ when ${\beta }_0+x^T\beta >0$ and predicts $0$ when ${\beta }_0+x^T\beta \le 0$. A multiclass softmax conditional model has finite ordered labels $1,\dots ,K$ with $K\ge 2$, affine logits ${\eta }_k(x)=a_k+x^T{w}_k$ for $a_k\in \mathbb{R}$ and $w_k\in {\mathbb{R}}^d$, and probabilities $\mathrm{Pr}(Y=k\mid x)=e^{{\eta }_k(x)}\mathrm{/}{\sum }_{j=1}^K{e}^{{\eta }_j(x)}$; its classifier chooses the smallest index attaining the largest logit. These models specify conditional label probabilities and impose no distributional assumption on the feature vector $X$, in particular no Gaussian feature law.

Finite linear-score classifiers with a deterministic tie rule

Hastie, Tibshirani, and Friedman, ESL, §§4.4 and 4.4.5

Stanford CS229 Spring 2022 lecture notes, §§2.1 and 3.2.3