Fermat's little theorem and the hidden symmetry of remainders