He was the shit in the early 80's to early 90's. He did so much weird humor that no one else had done.
Then he was replaced by a bitter, jaded version of his already bitter, jaded self.
I remember him being pretty great when John McCain refused to come on his show during the 2008 election, but I've long since stopped watching non-sports television, so I can't really gauge how good or bad Letterman is on a regular basis.
Conan would be good, but I particularly enjoy Craig Ferguson whenever I see his show. They should slide him on up.
Craig's style of humor isn't really fit for the whitebread bland party that the 11:30 slot has turned into. This is the slot that Jay Leno's aggressive mediocrity dominated for over a decade, and NBC decided Conan was too edgy for.
Ferguson is awesome because he's on TV late enough that he can do basically whatever he wants. The lack of structure or network notes is what his show thrives on. Put him at 11:30 and that would all go to hell.