Reddit mentions of The Colony: A History of Early Sydney

How in depth do you want here? If you want stories that are interesting, original and accurate you should check out Grace Karsken's books. Good reading: http://www.amazon.com/Colony-History-Early-Sydney/dp/1741756375

Thats easily the best book, apart from kusaioshiri's suggestion of Leviathan. It's a cracker of a read.

On top of these good books, you could take a different route by taking one of these fantastic walking architecture tours, by passionate, practicing architects: http://www.sydneyarchitecture.org/

Or you could check out this free museum in The Rocks which is surprisingly good with lots of touchscreen interactive panels: http://www.therocks.com/sydney-Things_To_Do-The_Rocks_Discovery_Museum.htm

Mandatory mention of the consistently good Museum of Sydney run by the Historic Houses Trust. http://www.hht.net.au/museums/mos