$\operatorname{Sym}(X)$ is a group under composition, and it is non-abelian whenever $X$ has at least three distinct elements

Composition is associative: for $f, g, h \in \operatorname{Sym}(X)$ and $x \in X$, both $((f \circ g) \circ h)(x)$ and $(f \circ (g \circ h))(x)$ evaluate to $f(g(h(x)))$, so the two composites agree at every point and are equal.

$\mathrm{id}_X$ is a two-sided identity: for $f \in \operatorname{Sym}(X)$ and $x \in X$, $(f \circ \mathrm{id}_X)(x) = f(x)$ and $(\mathrm{id}_X \circ f)(x) = f(x)$, so $f \circ \mathrm{id}_X = f = \mathrm{id}_X \circ f$.

Every $f \in \operatorname{Sym}(X)$ is invertible in $\operatorname{Sym}(X)$: the inverse function $f^{-1}$ is again a bijection $X \to X$, hence lies in $\operatorname{Sym}(X)$, and it satisfies $f^{-1}(f(x)) = x$ and $f(f^{-1}(x)) = x$ for every $x$, that is $f^{-1} \circ f = \mathrm{id}_X = f \circ f^{-1}$.

The transposition $\tau = (a\,b)$ satisfies $\tau(a) = b$, $\tau(b) = a$ and $\tau(x) = x$ for $x \notin \{a,b\}$; the transposition $\rho = (b\,c)$ satisfies $\rho(b) = c$, $\rho(c) = b$ and $\rho(x) = x$ for $x \notin \{b,c\}$. Both are bijections of $X$, being their own inverses.

By steps 1.1 and 1.2 the pair $(\operatorname{Sym}(X), \circ)$ with the element $\mathrm{id}_X$ is a monoid; by step 1.3 every element of it is invertible; hence it is a group, and the inverse of $f$ is the inverse function $f^{-1}$.

Evaluate the two composites at $c$. Since $c \ne a$ and $c \ne b$, $\tau(c) = c$, so $(\rho \circ \tau)(c) = \rho(\tau(c)) = \rho(c) = b$. And $\rho(c) = b$, so $(\tau \circ \rho)(c) = \tau(\rho(c)) = \tau(b) = a$.

The two composites take different values at $c$, because $a \ne b$; hence $\tau \circ \rho \ne \rho \circ \tau$ and $\operatorname{Sym}(X)$ is not abelian.

$\operatorname{Sym}(X)$ is a group under composition, and it fails to be abelian as soon as $X$ has three distinct elements.