What’s the hardest part about serving your sentence?

Probably the hardest part is getting used to the fact that everyone you know on the outside (mostly wife and kids) are living their lives while you are locked up. You have absolutely no control over anything. It feels like you have died and are watching your loved ones live their life and you cannot participate. Your mind plays tricks with you, you lose faith in your relationship, etc.