No, you can visit any inmate as long as you are approved and that the inmate requested you to be on their list.
https://www.inmateaid.com/ask-the-inmate/would-i-get-automatically-get-denied-to-visit-another-federal-inmate-in-a-different-state#answer
Accepted Answer Date Created: February 17,2018