Being recently released from the same facility your husband is currently in puts you in one of the more unusual visitation situations, but it is not an automatic disqualification and there is a clear path to getting approved.
The standard policy at most correctional facilities restricts visitation by people with recent incarceration history, particularly at the same facility. The concern is primarily about the connections and relationships formed inside and the potential for those to create security risks during a visit. That concern exists regardless of the family relationship.
The path through is a formal petition to the warden requesting a special visitation waiver. This petition needs to be submitted in writing and should make a clear and compelling case for why the visit should be approved. Your strongest argument is the marriage itself. Married couples have a recognized legal and personal bond that carries weight with facility administration, and wardens are generally more willing to approve special visit requests for spouses than for other categories of relationships.
In your petition, be transparent about the situation. State that you were recently released from the facility, that you are legally married to the inmate, and that you are requesting approval to visit him as his spouse. Do not try to minimize or obscure your recent release because the background check will surface it anyway, and proactive honesty builds more goodwill than information discovered after the fact.
The warden's secretary is the right first point of contact. Call the facility, ask for that office, and inquire about the process for submitting a special visit petition. Given that you are married, the odds of approval are reasonable. Be patient with the timeline and be thorough in how you present your request.