Reddit mentions of Fundamentals of Computational Neuroscience

This is a nice place to start. Here is another free online asset.

MATLAB for Neuroscientists will probably come in handy.

Theoretical Neuroscience is a classic.

Then we have Fundamentals of Computational Neuroscience.

I didn't find Theoretical Neuroscience particularly readable as others in the thread have said, but it is the go-to book for the classic topics in the field. I found Fundamentals of Computational Neuroscience to be a much much better book for introductions. From Computer to Brain : Foundations of Computational Neuroscience was fairly approachable. On the more cognitive side, From Neuron to Cognition via Computational Neuroscience was pretty good. If you like the nonlinear systems side, Dynamical Systems in Neuroscience: The Geometry of Excitability and Bursting was pretty tough to read but full of good content.

It really depends on what subsets of comp neuro you're most interested in. I worked mostly on the cognitive side, and I was never super satisfied with any books on comp neuro in that area. I think the field is just too young for a great summary to exist beyond the neuronal/small network level.

There is a ton of interesting mathematics that goes into other areas of neuro that wouldn't typically be included in Computational Neuroscience. Different imaging methods, for instance, have some pretty fun math involved and are very active areas of research.

Maybe this was already obvious to you, in which case apologies, but those are very broad topics. What kind of level of aggregation are you thinking of? Neural engineering sounds a bit more neural network-y, rather than large-scale human cognitive processes, which would involve measurement methods like EEG and fMRI that won't tell you much (broadly speaking) about the way networks of neurons do computations. You also have local field potential or clamping measurements, where you're looking at what specific neurons (or at least way smaller scales) are doing, which is more animal research. And there's computational modelling which is (relatively, to my knowledge) as yet hardly connected to the usual methods of measuring brain activity.

That said: I read this as an intro to neural networks, http://www.amazon.com/Fundamentals-Computational-Neuroscience-Thomas-Trappenberg/dp/0199568413 and remember liking it, but I was coming from a psych background so I don't know if it would be rigorous enough for you. For the biology / anatomy, the classic is http://www.amazon.com/Principles-Neural-Science-Edition-Kandel/dp/0071390111/ref=pd_sim_b_2?ie=UTF8&refRID=17R09KD62178HQ06E1VJ.

There's a paper by Wang (1999) with an integrate-and-fire neuron model that I implemented as a toy model that helped me get to grips with the computational side of things. I can't comment on how influential it is theoretically.