Okay, basically imagine it like a narrated session of D&D, except instead of hiding the details like 'I cast Magic Missile', and levels and numbers, you allow those to be seen, and the characters are allowed to base some of their thinking on the underlying game rules-- indeed, it should be critical to the plot at some point that they are not simply in a fantasy world, but one with a working game system.

For example, if you can instantly restore your health/mana from a level-up, then it becomes useful and if you're close to leveling up, you might tackle a really tough fight with the notion you can kill some small fries in the middle to get back to full health/mana. Also, as a character, you might be familiar with what skills/abilities you'd gain at level-up, and have some plan for what you're going to choose from the options available.

The wider category is 'gamelit' or 'game-centric literature' which features things like Joel Rosenberg's Guardian of the Flames series, Larry Niven's Dream Park books, or Drew Hayes's NPCs, but the more specific category of LitRPG means that the game system being featured within the book is an RPG with characters having stats, skills, abilities, perks, and such, and the rules of the game are known, or knowable, to the characters.