I've never felt much discomfort in mine either.
However, when it gets very cold, I use a sleeping mat under me. I had it from my old backpacking days, and helps keep me a bit warmer...and it is a bit more comfortable as well. They are pretty cheap, especially one like mine that rolls up. The folding mats would be what I would buy now, though. Easier to pack in and stow away.
Crow's Fork Game Calls Pro Staff