Reddit mentions of Pilot's Handbook of Aeronautical Knowledge: FAA-H-8083-25A (FAA Handbooks)

Every pilot needs to own the Pilots Handbook of Aeronautical Knowledge and The Airplane Flying Handbook.

This may be beyond the scope what you're looking for, but the best aviation book I have ever seen is Everything Explained for the Professional Pilot. I highly recommend it.

Find an instructor that has instructed more than 4-5 primary students. The instructor should be planning on staying with the FBO for at least the next year so you can finish the certificate with them. I would also suggest an instructor that is around your own age. This will keep you both on the same page and help build trust. Your instructor has to trust you enough to let you learn and you have to trust your instructor not to do anything reckless.

That said you should study on your own, The airplane flying handbook is your bible of practical knowledge. Also buy a copy of the Pilot's Information Manual for your aircraft (Make sure it is for your exact model of aircraft).

The Pilot's Handbook of Aeronautical Knowledge is another good resource produced by the FAA.

Private Pilot and Recreational Pilot FAA Knowledge Test 2010 gives you all the questions and answers for the ground school test. You can practice on the Sporty's Web Site for free which helps a lot.

The easiest way to save money is to study on your own, the more you study the less time your instructor has to spend with you on the ground. I wouldn't bother with flight simulators on your home computer, they are nice for learning how to scan instruments but at the beginning of your flight training they will only mess you up. That said if you decide to get an instrument rating flight sim is invaluable.

Click Here and Here for a whole bunch of useful stuff from FAA.

Worthy of specific note-

Airplane Flying Handbook (AFH) and Pilot's Handbook of Aeronautical Knowledge (PHAK, pronounced P-Hack). The PHAK and AFH will give you pretty much all the basics of how airplanes work, how airspace works, where you're allowed to fly, etc etc etc.

Aeronautical Information Manual (AIM) basically tells you how you can legally fly an airplane, but without much of the shall-be-not-less-than style legalese. Officially the FAR (Federal Aviation Regulations) is the real legal base which it stands on, but the AIM says the same thing and can be read by a human much more easily. In the AIM you will VERY often see references to "14 CFR xx.xxx", Title 14 of the Code of Federal Regulations (aka US national law) covers the FAA. This can be just as easily written (and frequently is) as FAR xx.xxx. The first part is the 'part', which covers a particular type of regulation- for example FAR Part 61 covers the certification of pilots and instructors, Part 91 covers general operating and flight rules (part 61 helps you get your license, part 91 helps you lose it). Often a particular business will be referred to by the FAR Part which governs it, for example most airlines are FAR 121 Scheduled Air Carriers. Some flight schools are Part 61 flight schools (anyone with a CFI (certified flight instructor) certification can provide flight instruction under Part 61), while some flight schools are Part 141 which covers more structured training programs with an FAA-approved curriculum, regular FAA audits, etc.

Note that if you don't mind a GIANT dead tree book, the complete AIM and several relevant parts of the FAR are available as one book for only $10

However if you just want to learn how shit works, download or buy the PHAK and AFH. They are free of legalese, are fun to read, and have lots of color illustrations so you understand what's going on. Go here and scroll down to 'frequently bought together' to get the PHAK, AFH and FAR/AIM all together for just under $40.