I'm surprise nobody's mentioned Knuth, though maybe that goes without saying?
As for probability and statistics, I haven't really found anything (at an advanced) level that I've been happy with. Maybe it's because my background is in statistics, so it's a perception bias (I see the flaws more easily than with other subjects), but I think that most statistics textbooks are pretty rotten.
There are really only two that I'd recommend, and only one at a high level. Gelman & Hill is a great introduction to computational statistics at a high level, while still very readable (and enjoyable!)
http://www.amazon.com/Analysis-Regression-Multilevel-Hierarc...
Other than that, the only truly stellar statistics textbook I've ever seen was the one I used in my intro class in high school. It's sad, but it's a very true comment about the current state of most statistics textbooks (that I can find).
Get the best books from Hacker News each week
Join 4,500+ subscribers and get the best books mentioned on Hacker News every Thursday.
I would highly recommend getting this for the shelf: http://www.amazon.com/Analysis-Regression-Multilevel-Hierarc...
It's one of the most readable books on data analysis I've come across and does a great job presenting both frequentist and Bayesian techniques with tons of R sample code.
There are a lot of advantages and nice things in Python, but I do think folks tend to toss out R a bit too casually. Each tool has areas they excel in. I don't even do particularly complex analysis, but have run into areas where Python is woefully lacking in fairly common (social science) models.