Despite his looks the Doctor isn't getting any younger. Even Matt Smiths Doctor said that he was an old man in a cool package. Or some such. Maybe more like, don't let the cool package fool you, I'm very old.
The thing is that with over 900 years of information in his head the poor Doctor would have a hard time recalling things instantaneously. I think Matt does a great job balancing being an ancient guy in a new young man suit.
I have been rewatching some of Tom's Doctor Who shows. I realize now that a lot of his humor zoomed at light speed past my brain pan. In places where I thought he was serious it was actually funny. I have to admit that British humor has that effect on me. Anyway, I am enjoying 'getting it' the more I watch them. Hmm, that's probably the same effect his wit has on his foes. Keep them baffled and spinning in circles. I wouldn't have stood a chance! Lol.