Am I entitled to the following Monday off when a public holiday falls on a weekend?

This depends on how generous your employment contract is. Many employers will let you have the following Monday off if, say, Christmas Day falls on a weekend. However, as there is no general legal right to time off on bank holidays (though it is a common practice), there is also no legal right for you to transfer the day's leave to another day, if it isn't mentioned in your contract.

You should check your staff handbook, and discuss this with your employer, but be aware that if your contract does not specify anything to the contrary, your employer's decision on this will be final.