When is the automorphism group of an affine variety linear?

Let $Aut_{alg}(X)$ be the subgroup of the group of regular automorphisms $Aut(X)$ of an affine algebraic variety $X$ generated by all connected algebraic subgroups. We prove that if $dim X \ge 2$ and if $Aut_{alg}(X)$ is rich enough, $Aut_{alg}(X)$ is not linear, i.e., it cannot be embedded into $GL_n(K)$, where $K$ is an algebraically closed field of characteristic zero. Moreover, $Aut(X)$ is isomorphic to an algebraic group as an abstract group only if the connected component of $Aut(X)$ is either the algebraic torus or a direct limit of commutative unipotent groups. Finally, we prove that for an uncountable $K$ the group of birational transformations of $X$ cannot be isomorphic to the group of automorphisms of an affine variety if $X$ is endowed with a rational action of a positive-dimensional linear algebraic group.