But he knew he had to do it.
Home is where the heart the heart is, and Smith's had never left 135 Pinson Road off Alabama 202 since he graduated in 1987.
Smith resigned from Hueytown and was approved to become the Panthers' next coach at a Calhoun County School Board meeting Thursday afternoon.
He replaces Ricky Weems, who was the football coach and athletic director for the past three seasons, going 8-23.
His coaching duties will officially begin on July 1, but he will contact players today. The school will host a meet-and-greet at 6:30 p.m. at the school's library. His family — which includes wife Lisa Angle-Smith who also attended Wellborn, his sons Judd and Jett and daughter Leah — will be in attendance. Judd will be a freshman on the football team in the fall.
"A lot of people might look at (the move) and think that it doesn't make a lot of sense, leaving a 6A school and coming to a 3A school," Smith said. "But I'm from here. It meant a lot to me to be the head coach here and have the opportunity to make this place better."
The Panthers went a combined 2-18 the past two seasons.
Wellborn Principal Rick Carter said Smith applied just before the cut-off point, but is the right man for the job.
"This is huge," Carter said. "We hope he can replicate the success he had at Hueytown and at Ohatchee at Wellborn High School. This is a big hire for Calhoun County to get someone of Coach Smith's caliber."
Smith guided Hueytown to the playoffs in four of his six seasons at the helm. In 2007, they won 10 games, the most in school history. Last season, he led them to a 6-5 mark and a berth in the first round of the state playoffs, where they were defeated by state-finalist Hoover in the first round.
"All of my family is in Calhoun County," Smith said. "This is where I played. Wellborn has had a lot of great football players and has a great football tradition.
"It kills me to see this program doing as badly as it has been lately. I want to unite this community and get this football program back on track. I don't have anything bad to say about anyone at Hueytown. I wouldn't have left Hueytown for anywhere but Wellborn."
During Smith's playing days at Wellborn, he was chosen to to The Star's All-Calhoun County team as a linemen in his sophomore, junior and seniors seasons before heading to Jacksonville State University to plyay college ball. As a Gamecock, he was a four-year letterman and played on the 1989 and 1991 teams that competed for an NCAA Division II title.
His coaching career began as an assistant at Handley before getting another assistant gig under his former high school coach Mike Battles at Hueytown. From there, he coached in Mississippi for a year and returned to Calhoun County to coach at Ohatchee, where he won The Star's Small School Coach of the Year award three years in a row from 2000-2002.
He was the Birmingham News' West Zone Coach of the Year in 2006 and 2007. He has a career record of 76-55.