Reddit mentions of The Collected Stories of H. P. Lovecraft: Volume One (48 Classic Horror Books in One Volume!)

After further research, it looks like the best Kindle option is this collection. Note that it's volume 1 of 3. I haven't downloaded it yet, but the reviews say it is nicely formatted for the Kindle.

If you have a Kindle or the Kindle App you can pick up almost all of his works for about four dollars.

Many of his stories will make reference to Cthulhu, especially if the story involves Innsmouth. I'd recommend just flipping through the collection and picking stories at random, they're all pretty good. I recommend At the Mountains of Madness, The Shadow Over Innsmouth, Dagon, The Colour Out of Space, The Dunwich Horror, and The Horror in the Museum.