An annual fee is worth paying only if the card's rewards, after the fee, beat what you'd earn on a good no-fee card. Compare the net effective rate, not the advertised rate. A $95-fee card earning 3% in your top categories might net less than a simple 2% no-fee card if most of your spending is in the base category.
The one rule that overrides everything: rewards are only real if you pay your balance in full every month. Card APRs of 20%+ dwarf any rewards rate, so carrying a balance to earn points is always a losing trade. If you sometimes carry a balance, a low-rate or no-frills card beats a rewards card every time — the interest you'd save is worth far more than the points.