For Markov kernels
P on a general state space
X, we introduce a new class of averaged Markov kernels
Pda(G,ν) of
P induced by a group
G that acts on
X and a probability measure
ν on
G×G. Notable special cases are the group-orbit average
P, left-average
Pla, right-average
Pra and the independent-double-average
(Pla)ra. For
π-stationary
P in which
π is invariant with respect to
G, we show that in general
Pda enjoys favorable convergence properties than
P based on metrics such as spectral gap or asymptotic variance, and within the family of
Pda the most preferable kernel is in general
(Pla)ra. We demonstrate that
Pla,Pra,(Pla)ra are comparable in terms of mixing times, which supports the use of
Pla,Pra in practice as computationally cheaper alternatives over
(Pla)ra. These averaged kernels also admit natural geometric interpretations: they emerge as unique projections of
P onto specific
G-invariant structures under the Kullback-Leibler divergence or the Hilbert-Schmidt norm and satisfy Pythagorean identities. On the other hand, in the general case if
π is not invariant with respect to
G, we propose and study a technique that we call state-dependent averaging of Markov kernels which generalizes the earlier results to this setting. As examples and applications, this averaging perspective not only allows us to recast state-of-the-art Markov chain samplers such as Hamiltonian Monte Carlo or piecewise-deterministic Markov processes as specific cases of
Pda, but also enables improvements to existing samplers such as Metropolis-Hastings, achieving rapid mixing in some toy models or when
π is the discrete uniform distribution.