In a lot of RPGs, players have skills, stats or whatever you want to call them that are well defined. Things like Medicine, Nature, Insight, etc. in D&D, or Occult, Drive, Larceny in Chronicles of Darkness. Then there are some games that are more flexible where the players are asked to fill in their own ad lib skills and stats - games like DOGS, Godbound or Chuubo's Marvelous Wish-Granting Engine, where you can pretty much write anything in as a stat - "Memory of my dead brother", "Fastest gun in the west", or "underwater basket weaving".
The problem with the latter is the fuzzy boarders each such statement creates about when a given skill should be used, when it shouldn't apply, and avoiding skills that are too broad or too narrow.
Broad vs narrow skill applications
There are RPGs out there that like to break skills down to minutia. In Burning Wheel there are a lot of skills - under Carpentry alone you have Fence Building, Ditch Digging, Carving, Carpentry, Rude Carpentry, Cooping (making barrels), Boatwright, Shipwright, Cartwright and a few more. This creates a sort of Air-Breathing Mermaid Problem, where the game basically forces the GM's hand to say "sorry, you only have Carpentry, you can't carve and decorate what you build and you can't seem to know how to make a barrel". It can also create traps if characters specialise in some skill that doesn't come up that often - "I know you're a master Cooper, but no matter how great that barrel you make will be it still won't help you sail the sea, you need Boatwright for that".
You can run into similar problems in games where players can define their own ad lib skills, but those also sway in the other direction - they can be too broadly applicable. You could have players put ranks into "underwater basket weaving" only for that to come in handy once or twice in the entire game, while someone else makes a stat called "fast" and they want to use it all the time - "I shoot him, but like, fast", "I read, fast", "I sleep, fast", etc. The boarders where such player-defined skills apply are fuzzy - you can't really tell them that because there is some other skill that also works for the situation theirs might not apply. Moreover, players will often skew the problem in their favour just so they can use their best skills - "I make a basket under water and it's so well crafted I can use it as a raft with a sail to navigate the rough sea. It's all baskets, just different shapes."
City of Mist tried addressing this problem by classifying its tags into "specific" and "broad". Most tags on a character sheet should be specific - only letting the character use them in specific, limited scenarios. One tag could be broad - applicable in wide variety of situations.
Rainbow shields and damage of the gaps
While playing Exalted / Godboundwith my group we would often reference a "rainbow shield" defence - a power to negate any kind of incoming damage, no matter whether it's physical, magical, fire, electricity, etc. A system that would allow for such widely-applicable defence to exist wouldn't be too fun to play really. Exalted 2nd edition suffered from something like that from what I heard - someone created an optimal way of playing the game called "paranoia combat" that's all about using perfect defence against the high-lethality rocket-tag and outlast your opponent.
In our first round of playing Godbound we ended up being too generous with defences, letting the players counter most attacks with most power sets - "I use Artifice to instantly build a wall around myself to deflect the incoming blow!". As we later found out, Godbound was not intended to work like that - any given Word should only counter a very narrow range of powers that are thematically linked to it. You could use Sea to counter a Fire power, but not say, Sword.
However, the game had a different problem, one which I'll call "damage of the gaps". The system did not have a specified, finite list of damage types. A number of powers did either give you things you could be immune to, or let you specify said immunity - for example, if you had the Word of Fire you had "an invincible defence against flame and smoke". But since "smoke" was ill-defined, we had an argument about whether that Word would apply against chemical fumes and poisonous gasses. Similarly, in the Ancalia expansion introduced an Incendiary Court - fire-golem-like creatures from outside of reality that while clearly having fire-based attacks nonetheless came with a note that their "flaming powers are not wholly of natural fire, and so enemies with invincible defences against fire still take half damage from them". This was probably to prevent players circumventing a big boss fight entirely, but still felt like a weird way to make a power less useful.
In a similar vein you could start splitting hairs on many things - if you are immune to mental attacks and someone uses music or mean words on you, does that count? Where does mental damage end and emotional damage begin? Is a psychic blow magic, mental or physical? Is a laser attack fire, sunlight, or neither? Can I make some very snowflake-y Word that has its own damage type that none of the statted enemies have defence against?
Because the game had no finite and well-defined list of damage types, you either have to make one yourself before the game starts, or possibly have to argue about this kind of damage of the gaps...
Conclusions
While being able to get creative with skills and powers can be fun, having things that are well-defined and concise can help everyone at the table know when something is applicable and avoid the problems of things being either too narrowly or too broadly useful.
Over the last few years my group and I have played a number of systems that had mechanics you could game to get XP or other advantages, or had some of their mechanics boil down to a solvable math problem. Both of those situations ended up detracting from the experience, either drawing more attention to themselves rather to the game being played, or just being bland mechanics.
RPG mechanics as math problems
The first category of mechanics are essentially math problems - mechanics that for any given situation have a correct solution on what to do to maximise your outcome.
This attack basically gives you a lot of extra dice to a roll, but makes the roll Difficulty higher. It's basically a move you want to use either when you need a hail mary, or the Difficulty is already so low it doesn't matter much. For other scenarios, whether or not to use this move would require running the numbers, but there is still a definitive yes or no answer to whether using it is a good move or not. Figuring it out for certain however requires some complicated math of using your AnyDice-fu.
DOGS Growth
DOGS presented a similar math problem to its players when it came to Growth.
DOGS is a system in which you have stats that take form of multiple dice of a given size - "3D4", "4D6", etc. When you undergo Growth, you get to either increase the number of dice for a given stat, or increase the size of those dice - so from "3D4" you can go into either "4D4" or "3D6". Turns out there is an optimal way of progressing through those dice to get the best result on average:
So for example, 3D6 gives you on average 0.5 higher roll than 4D4, while 5D8 is better than 7D4 by 5 whole points on average. The game doesn't explain those concepts to the players and it's simple and abstract enough that these things shouldn't matter, but for a problem-solving player it's a solvable mechanic.
Mouse Guard and optimal combat
Mouse Guard is a system with its own little combat / conflict engine that relies on picking actions (Attack, Defend, Feint and Manoeuvre) and seeing how they interact with one another. Attack lowers opponent's Disposition (HP essentially), Defend heals your Disposition, Manoeuvre is a way to get an advantage on next rolls, while a Feint is like an Attack with caveats - if played against an Attack, Feint does nothing, but if played against a Defend, Defend does nothing.
During our first game of Mouse Guard we soon learned that this setup creates a simple First Order Optimal Strategy - just always Attack. Attack vs Attack or Defend gets you closer to resolving the conflict, Attack trumps Feint, and Manoeuvre often isn't useful enough to trump dealing damage to an opponent. Attack, Attack, Attack!
FOO (First Order Optimal) Strategy
Towards the end of the game this has ended up being such a simple and optimal strategy that for our next game of Mouse Guard we had to switch the rules to give Attack a hard counter not to devolve every conflict of any type into "press A to win if you press it faster than your enemy".
Cortex and marginally useful SFXs
Recently our group picked up the modular system Cortex. We only played a few sessions of it so far, but one thing that stood out to me was how "starchy" (boring) some of its special powers were.
First of all, Cortex is another mixed-die system that puts a big emphasis on making pool of dice and manipulating your dice. So if you're Iron Man, you can have say, Eccentric Billionaire at D10, Ganius Scientist at D12 and Mk1 Iron Man Suit at D6 and roll those all together to do something.
On top of that, one module you can use in Cortex are Power SFX (special effects). Those are some extra powers your character can use that are tied to a Power Set that can alter the game a bit. So for example you can be Tony Stark with Iron Man Power Set, and one of your SFX could be "Immunity" where you spend a Power Point to negate a specific attack, simple enough.
However, a lot of those SFX are boring dice manipulators. For example - Focus lets you take two dice and turn them into one bigger die. Boost lets you shut down one power to increase the die on another die. Dangerous gives you an extra small die to roll, but changes the size of other dice. Multipower lets you use more than one dice from a given pool but they all are decreased a step or more. Versatile lets you split one die into two or more smaller dice. The list goes on.
I've ran some numbers and a lot of these powers are marginally useful. Say, turning 3D8 into 2D8+2D6 with Versatile gives you an average roll that's 0.42 higher, but gives you 0.45 smaller Effect and 0.2 more Hitches. Without going into what those are, those numbers are marginally useful. Sometimes the numbers increase marginally, sometimes they decrease, but from what I've seen it's not a big effect overall.
Unfortunately to get those numbers I had to spend a few hours programming and debugging a Cortex dice simulator. It's hard to make an informed decision as to whether a power is useful or not without a chart, and trying to play a game well that's filled with unknowable probabilities would just be the case of blind luck.
One way or another - the stat-focused SFX and similar mechanics can be one of two things - either boring because they don't change much about the roll, or having an optimal way to play it, in which case you're not engaging with the mechanics, you're solving a math problem. Either way the mechanics become irrelevant because they're either "use them always", "don't bother with them ever", or "use them under specific circumstances". Since Cortex is based on complex math with no glaringly obvious answer, I honestly can't be bothered to use these SFX.
Honourable mention - Exalted, Paranoia Combat
Honourable mention in this category should also go to Exalted. I won't elaborate much on it since this section is already getting long, but there are two things that are worth mentioning that make this epic game of sword and sandal capital H Heroes boring: Paranoia Combat and Minuscule Incremental Charms.
Paranoia Combat was a strategy from 2nd edition Exalted where the optimal way of winning the Rocket Tag combat was to turtle up and play in the most boring way possible.
Minuscule Incremental Charms were special powers you could buy with XP that would give you just small bonuses to rolls or change tiny things that were rather boring in themselves. Things like Triumph-Forged God-Body that gave you double-9s on Athletic rolls instead of double-10s, or Wyld-Forging Focus that started wyld-shaping at a higher phase. All of those were such small tweaks that they might not be worth the mental load, and weighing their effect vs XP cost would be a small math problem in itself.
Gaming mechanics for profit
Most RPG mechanics that you can game for profit I've come across were focused on being able to farm XP, or at least streamline the way you earn XP. While not a problem in itself (who cares if the party got more XP if they're having fun doing it - you're not competing with anyone), it can start to become a problem when it draws too much attention away from doing things in the game and having fun and onto "brrrr the number goes up"
Chronicles of Darkness - punch me in the face for XP
In a similar vein, the systems also let you earn XP by a number of other ways, like turning fails into botches. This can create some animosity between players when someone is invested in some scene going well, while other players are there to mess things up just to farm up some extra XP - "I failed to impress this character, I opt to botch it instead and make them hate us. Too bad they knew something about your lost sister, guess we'll never find what they knew!".
DOGS - Growth vs Consequences optimisation
Another entry for DOGS, this time about maximising the rate at which your character growths, as opposed to optimising how they grow.
DOGS is a system where you Grow when you suffer Consequences as a result of a conflict you had. To become stronger, you have to get into conflicts, get beaten up a bit, etc. However, Consequences can also have lasting effects if they are bad enough - if you roll too high on them, you may even have to step down your stats, essentially netting you zero, or potentially giving you some net negative sessions. Once again, there is a mathematically optimal way of playing:
Which is basically to get a 3D4 Consequence - it has the best chances of being a net gain. You get such small Consequences by essentially keeping non-violent in conflicts, which to an extent is a "mechanic as a metaphor" for the system.
Mouse Guard and farming Checks
Mouse Guard is a system where you grow your character by practice - aka the more you use a skill, the better you get essentially. As with any such system, the first way of farming it is by doing things all the time, which can encourage you to hog the spotlight. This can be a bit of a problem, but then there is more.
The game is broken up into two parts - the GM turn and the players' turn. During the GM's turn (which lasts about half of the session, not "a turn"...) you can earn "Checks", which you spend during the players' turn to do things and make rolls. You earn those Checks by using your Traits against yourself ("I am Small, therefore I have problems lifting this large log!"). You can use a Trait against yourself once per roll, which means the more you act and roll on GM's turn, the more Checks you can earn to act more during the players' turn.
Moreover, during a conflict you can easily earn a lot of Checks if you play in a very boring way. Essentially, during a conflict you pick actions to take - Attack, Defend, Feint and Manoeuvre. When you Defend, you essentially try to recover your HP. Since the conflict only ends when one of the party's HP goes down to zero, if you turtle up you will be rolling for a long time, letting you earn Checks for every roll. In a lot of cases you can also earn a lot more Checks during a fight under specific circumstances - breaking a tie in enemy's favour or giving an enemy more dice in a vs conflict. So if you play like a turtling asshole and have enough dice, you can in theory earn a lot of Checks.
This strategy has one counter though, Feinting makes you unable to roll Defend. You can try anticipating it though by throwing an Attack that trumps Feint into the mix to make your opponent have to Defend and recover. It's not perfect, but it can work...
While that turtle Defence is an extreme example, I have played in some sessions where a less extreme form of Check farming was involved, which later resulted in pretty neat things being accomplished during the players' turn.
Conclusions
There are a number of games out there that rely on math obscurity to give a sense of depth or agency. However, solving the game mechanic from a mathematical sense is only so fun, and once solved the complexity is replaced with an optimal way to play the game, which isn't fun. Making the math behind it harder is not making the choices more meaningful, just the decisions harder to make informed. Try pruning such mechanics from your game if possible.
Similarly, there are games that can be exploited by players to gain some disproportionate amount of XP and what have you that detract from the game by rewarding boring play.
Or in other words - if you are designing a new game system, try asking a math nerd or a game developer to break it. They might do the math and show you how balanced your system can be, and you can guide your players to playing the game well with that math as well.
Now, character deaths are outside of the scope of this blog post. While they usually also result in a net negative session, they occupy a somewhat different category of issues. Losing a character means you reset back to a starting character, losing XP means you can regress below a starting character.
DOGS and gambling with your XP
Advancement in DOGS works like this - when you are in a conflict, you spend your dice back and forth. When you are forced to spend 3 or more dice at the same time (due to being hit by a strong attack), you get Hit and take Consequence dice. The number of those dice is equal to the number of dice you had to spend at one go, and the type of dice depends on the type of conflict you are in (for example D4s for stuff like talking, D6 for chasing, D8 for beating someone up, and D10 for trying to kill someone). At the end of the conflict, you roll all of your Consequence dice. If you rolled any 1s, you experience Growth (your character improves), but if the total of the two highest dice exceeds 7, you get Long-Term Consequence (your character degrades, or worse).
If you're rolling D4s, it's generally not too bad - you are more likely to roll at least one "1" than have a pair of 4s, but as soon as you hit the D6 territory, the odds are against you! We had some players that hit both Growth and Consequences at the same time a few sessions in a row (meaning they were shifting dice from one place to another), and a session where two characters just suffered Consequences and nobody advanced one bit.
Sure, showing character's lateral growth can be interesting to an extent and somewhat thematic, but tying character progression to a random dice roll feels like a system is asking you to game it.
In nWoD, you can invest your XP into things external to your character - Merits in forms of Retainers, Contacts, etc. A 5 dot Retainer can cost you 30XP, and you can earn 1-4XP per session roughly. So that's about 8 sessions invested in one (very powerful but still mortal) person. If they happen to get into a firefight trying to protect your character and die, that's a large investment down the drain. At least with Sanctity of Merits you'd get the XP back, but that concept hasn't been invented until the CoD system.
Similarly, your character can lose their Morality due to being exposed to trauma or the supernatural. You start at Morality 7, but if you lose it and try to buy that 7th dot back, the linear XP cost will set you back 21XP for that single dot. You are out 6+ sessions worth of XP to get back where you started.
In Chronicles of Darkness, flat XP costs means you don't lose too much, and Sanctity of Merits refund you any Merits you lose. At worst you might lose about 1 session's worth of progress due to Morality loss, which isn't too bad.
Conclusions
People are pretty loss averse. The pain of losing something outweighs the joy of gaining an identical thing. Having your character suffer a setback that reverts their progress back multiple sessions can be an unpleasant experience, especially if you measure yourself against other players that didn't suffer the same adversity.