Discrete convolutional layer with local receptive fields, channel mixing, stride, and padding

Let $D,O\subseteq {\mathbb{Z}}^2$ be input and output index sets, let $C_{\mathrm{i}\mathrm{n}},C_{\mathrm{o}\mathrm{u}\mathrm{t}}$ be positive integers, and let $x:D\to {\mathbb{R}}^{C_{\mathrm{i}\mathrm{n}}}$ assign a row vector of input channels to each input site. A boundary extension is a function $\tilde{x}:{\mathbb{Z}}^2\to {\mathbb{R}}^{C_{\mathrm{i}\mathrm{n}}}$ whose restriction to $D$ is $x$; zero padding is the extension that is zero off $D$. Choose a finite receptive-field offset set $K\subseteq {\mathbb{Z}}^2$, a positive integer stride vector $s\in {\mathbb{Z}}_{>0}^2$, an origin $a\in {\mathbb{Z}}^2$, one matrix $W_{\delta }\in {\mathbb{R}}^{C_{\mathrm{i}\mathrm{n}}\times C_{\mathrm{o}\mathrm{u}\mathrm{t}}}$ for each $\delta \in K$, and an output bias row vector $b\in {\mathbb{R}}^{C_{\mathrm{o}\mathrm{u}\mathrm{t}}}$. The discrete convolutional layer is the map $y:O\to {\mathbb{R}}^{C_{\mathrm{o}\mathrm{u}\mathrm{t}}}$ given by

$$y(p)=b+\sum _{\delta \in K}\tilde{x}(a+s\odot p+\delta )W_{\delta }(p\in O),$$

where $s\odot p$ is coordinatewise multiplication. Thus each output uses only the input sites in its receptive field, mixes input channels by matrix multiplication, and uses the same weights and bias at every output site. With unit stride on the full lattice and no boundary truncation, this formula is a discrete cross-correlation; many machine-learning software packages call it convolution. Mathematical convolution instead reverses the kernel offsets. The choices of $O$, the extension $\tilde{x}$, and the stride specify cropping, padding, and subsampling, and need not preserve a symmetry of the full lattice.

functions and declared domains/codomains

Goodfellow, Bengio, and Courville, Deep Learning, Chapter 9 §§9.1–9.5

Cohen and Welling, “Group Equivariant Convolutional Networks,” §5