I've had my Kindle since xmas. So far, I haven't found one downfall.
You can buy books from anywhere in any format and convert them using free software called Calibre. It takes seconds to do, and then you just upload the book to your Kindle by connecting the device to your computer.
You will also find on the Amazon Kindle forums that lots of authors (especially new ones) offer their books at discount prices - generally around the 75p mark. I have bought several of these books, and always been happy with the quality. (It's also good to be able to talk to the authors on the forum).
You can also use torrents to find books - I have downloaded several books that I already own, and can now read them on my Kindle as well as my paper copy. (Obviously you can use torrents to download books that you haven't bought a paper copy of as well - depends on how you feel about copyright uses and abuses)
If there is a book you want but you feel it is too expensive, just keep an eye on the price, as often Amazon have special offers and bring prices down on certain books for a few days - they did this over Christmas/New Year, and I got over 20 books for less than £15)
Customer service is excellent - I had an issue with the formatting of one of the books I had downloaded from the site, and emailed their customer service. Within 2 hours, Amazon phoned me, and talked me through the formating (the font was too big and wouldn't reduce enough for my liking). When they couldn't sort it (apparently it was the fault of the publisher who supplied the format) they offered me a full refund, at that time, or any time in the future if I found it too difficult to read. Can't ask for better than that. I have also heard that if you damage your Kindle, they will supply you with a replacement free of charge. Someone on the Kindle forums noted said this is a one-time replacement, they won't keep replacing it if you keep damaging it - but even then - who else would do that for you?
I'm totally sold on it. You may have noticed!