1. People don't believe in it. People are scared of the law or think that it's a pointless currency as it isn't backed by force nor commodity. But as the market for it grows, it would be like not believing in using the already-established currency. Money doesn't care if you believe in it or not. If a bunch of people do care and accept it as a medium of exchange, it effectively is one.
2. Difficulty of exchange. For now, people have a bit of a learning curve. Devices and new methods will come out to facilitate exchanges and make it easier to trade btc for products and services. It is boundless in the black markets and is growing on the markets on the web. (See silk road and even wordpress)
3. Cracking the code. There might be a flaw in the implementation of the bitcoin system that might open the system to failure. As it grows, we're more likely to find he hole(s) in the system. But on the flipside, the longer it lasts, the more confidence we have in it's robustness.
The strongest things going for bitcoins is that they're on a decentralized system and it's pretty much impossible to forge purchases or bitcoins. It is also an inflationary currency, but there is a built-in cap in the system of around 21 million btc. Mining coins is getting harder with time. As we approch 21 million coins, we'll start facing a situation where the currency is decreasing in quantity. This isn't really preferable or sustainable from an economic viewpoint, and we might see the value decrease as we are able to produce less.