A groupscum player is a player who belongs to a faction opposed to the town (i.e. "scum"), and which originally consisted of multiple members (i.e. part of a "group"). The usual assumption is that groupscum factions will act in a similar way to the Mafia.
Many tells work only against groupscum players. In general, members of a single-scum faction, such as a Serial Killer, are harder to catch due to the lack of interactions with the rest of their faction.