My gems went down and I did not buy anything

Gems are spent in two places outside the shop. One of them is almost always the answer.

• Levelling a muscle: raising a muscle's Level spends gems as well as XP. Both are taken at the same moment.
• Joining a wager duel: your stake leaves your gems the moment you join, not when the duel ends.

Both of these need you to tap. Neither one takes gems on its own.

Learn more: What can I spend gems on?