In most cases, no. Federal prisons and the majority of state prison systems require visitors to be pre-approved before they can enter. The approval process typically involves the inmate submitting your name to their counselor or case manager, a background check on your end, and formal clearance before your first visit. Showing up without prior approval almost always means being turned away at the door.
County jails vary more. Some smaller county facilities allow walk-in visits without a pre-approved list, particularly for shorter stays or lower-security settings. Others apply the same approval requirements as state prisons.
The safest thing to do before making any trip is to call the facility first and ask the unit secretary or visitation coordinator exactly what their policy is. They will tell you whether approval is required, how to get on the list, and how long the process typically takes. That one phone call saves a wasted drive and a disappointing turn-away at the entrance.