In fact, the proof can be simplified and clarified by using the theory of Lefschetz classes. The correspondence $x\mapsto x_{I,J}$ on $A\times A$ is Lefschetz, as is the operator $\Lambda$ (so there is no need to appeal to Lieberman) --- see Section 5 of my paper Lefschetz classes on abelian varieties. Note that $L(A)$ is contained in the torus with $\mathbb{Q}{}% $-points $\{a\in E^{\times}\mid a\cdot\iota a\in\mathbb{Q}{}^{\times}\}$.
For the simplest proof of Clozel's theorem (simplification of an argument of Deligne) see Milne 2009b, 1.14.