It may be, just a little. Though I fell in love with it during the summer before my senior year, when I was working for IBM in Rochester, Minnesota. So I don't think it's just an "old man" thing.
It's hard to describe the appeal for me. In part, I admire the art of storytelling that Garrison Keillor has mastered. It amazes me that he can hold an auditorium full of people absolutely speechless for twenty minutes with just the power of his voice.
It's also the ability to laugh along with stories that are about more-or-less ordinary people. I laugh because I hear that story and I can actually imagine it happening to me ... or living there and seeing it happen to people that I care about.
I suppose it depends on what you mean by "long-term setup". I've seen him tell a story that takes 20 minutes to set up one joke ... and it's well worth it.