As ex-referee myself, I agree that there is no fixed rule HOWEVER, if the referee has already shown one yellow for persistent infringement, then it should only take two or three more fouls being committed before it becomes obvious that the player hasn't accepted that warning and is then shown the red. Also, if the referee has issued a yellow previously, perhaps for another reason, and then finds cause to warns the player that they are noting a number of persistent fouls now and that the next one could result in a yellow and therefore a red, the player should take note. If they don't heed that, then after the next one it is bye-bye.
The problem is context. In any high level game that attracts media attention and especially TV cameras, most fans and commentators would be perplexed if a player was penalised for a "soft" foul, then shown a yellow and red. It doesn't mean the laws of the game are in abeyance, but it does make the application that little bit more difficult.
Mind you, it was made very clear to me during my training that you never pull a first yellow unless you are willing to show a red, for exactly that reason. But then isn't that what makes the game so compelling? ;-)