Self-Complementary Generalized Orbits of a Permutation Group

Abstract

AbstractA permutation group A of degree n acting on a set X has a certain number of orbits, each a subset of X. More generally, A also induces an equivalence relation on X(k) the set of all k subsets of X, and the resulting equivalence classes are called k orbits of A, or generalized orbits. A self-complementary k-orbit is one in which for every k-subset S in it, X—S is also in it. Our main results are two formulas for the number s(A) of self-complementary generalized orbits of an arbitrary permutation group A in terms of its cycle index. We show that self-complementary graphs, digraphs, and relations provide special classes of self-complementary generalized orbits.