Reddit mentions of Designing BSD Rootkits: An Introduction to Kernel Hacking

I can't recommend that book enough. It will give you a great overview over the services the kernel provides, design decisions and data structures.

In addition to that, these resources might also be of interest to you:

• Stevens, TCP/IP Illustrated Vol2. This is the actual 4.4BSD Lite tcp/ip stack source code; commented and explained. http://www.amazon.de/TCP-IP-Illustrated-II-Implementation/dp/020163354X/
  
• Kong, FreeBSD Device Drivers. Kernel module and device driver programming. http://www.amazon.de/FreeBSD-Device-Drivers-Jospeh-Kong/dp/1593272049/
  
• Kong, Designing BSD Rootkits. More kernel module programming :) http://www.amazon.de/Designing-BSD-Rootkits-Introduction-Hacking/dp/1593271425/
  
• https://github.com/teachbsd/course hands on course material, understanding OS internals via DTrace and FreeBSD

I'd recommend Gentoo for learning how Linux works. The great thing there is that by building the system from the ground up, you have to configure every tiny detail, and in doing so, you get to see every tiny detail. I gained a good deal of understanding from building it first for a server, then a desktop.

As for problems, yes can be plenty, especially if you plan on installing it on something with spotty support, like, in my example, a PowerPC Macintosh G4. I think that getting over these problems is a learning experience.

P.S. If it's low level kernel functionality you want to learn, I'd suggest installing FreeBSD (or a similar pure Unix project) and going from there. The reason is that the Unix kernel is not that terrifically different from the Linux kernel, and, being a more academic system, it tends to have more interesting books written about it. For instance, I'm reading Designing BSD Rootkits as an introduction to kernel hacking, and I'm applying many of the techniques to Linux as well as FreeBSD.

In addition ktrace beats the hell out of strace, but whatever...