For Chaos games, assuming leavers are random (not quite true), game balance should not be affected. The effect is the same as have a game with fewer players. Even if the roles are unique, there is no practical difference between a role that was not picked and a role that suicides. It feels different, but the odds are the same. Similarly, if you held a raffle and randomly picked some losers before picking a winner, it would feel like those people were cheated, but picking WINNER, LOSER, LOSER gives the same odds as LOSER, LOSER, WINNER, or picking 3 and then picking which of those 3 is WINNER.
Quitting in Chaos games looks pretty close to random. I have seen just about every role quit, from Mayor to Werewolf. Some people hate town, some people hate neutrals, or they hate particular roles like Medium or Disguiser. The only real worry is snowballing suicides, say if one Mafia quits so another quits so… The same can happen when Town loses a few important unique roles early.
But still, you’re a jerk if you force 14 people who signed up for a 15-player game to play a 14-player game. 1 isn’t a big deal, but it feels like it is rarely 1, so those 3 jerks just forced a 12-player game on everyone else.
In the standard game, leavers are less innocuous, and therefore more problematic. Every role is random in all/any. Roles are added in a particular order in the standard game, so a planned 12-player game is not just a 15-player game with 3 random roles removed. Also the roles are almost entirely fixed, so one missing role gives a lot more information than it would in Chaos.
Playing Town of Salem, I am constantly annoyed by suicides, i.e. people quitting the game. They don’t like their role, they started without time, they anything — I’m alt-tabbed and typing this now from a game that has had 4/15 people quit so far. It adds quite a bit to the randomness when one team suddenly loses several members, which can snowball. No one cares if a neutral benign role leaves, but everyone is cheated of a decent game when one side is hamstrung. I’ve seen other perversities like a 3-arsonist game where they would have won pretty handily had not one of the arsonists quit on day 1. Even worse is when an important, unique role is AFK; I have had two games with AFK mafiosos so the Mafia just could not do anything. And then town is hamstrung when people quit after dying, given that town can have a rezzing role.
But Town of Salem is also free to play. I want there to be some disincentive to ruin the game for others, but what are you going to do, ban a F2P account that someone can re-create in a minute or two? Ooh, you wiped out their cosmetics. That is hardly a speed bump for the sort of jerk who doesn’t care about ruining others’ fun, to say nothing of the actively griefing troll. I am interested in ranked play, where those players tend to fall out as they cannot rise in the ranks, but ELO is broken and most of the roles do not appear in ranked play.
Having real costs in a game is a useful thing just because it imposes costs. If there is no cost for bad behavior, you are free to impose costs on others.
Still playing Town of Salem. I have switched to the Chaos – Any/All game because it is the only way to consistently see all the roles. The problem is that it really does mean any/all chaos, so the game might be randomly unwinnable for one team or another. Not technically unwinnable, but practically so.
If the town has less than half the population, the town loses. It is not technically impossible for the town to win, given good luck and/or incompetent opponents, but I have yet to see the town win with less than half the population. I have been in a game with only 3 people in the Town. Being mayor of a three-person town is just waiting to die. Other times, the town will get a stack of roles that cannot synergize or help find the Mafia, like 3 mediums and 3 transporters. A 2-person mafia is also likely to lose, but I have seen them pull it out occasionally. A game with 3 serial killers and/or arsonists is going to be rough for anyone except the neutrals.
The standard game uses a standard set of roles to better ensure a balanced game, but you lose out on variability and seeing all the game has to offer. You will never see about half the roles in the standard game. Chaos takes that to the other extreme. There needs to be a balance with variation but constraints on randomness. Maybe make some of these official?