The molecular chaperone Hsp90 (heat shock protein 90) is a promising target in cancer therapy. Preclinical and clinical evaluations of a variety of Hsp90 inhibitors have shown anti-tumor effect as a single agent and in combination with chemotherapy. Current Hsp90 inhibitors are categorized into several classes based on distinct modes of inhibition, including (i) blockade of ATP binding, (ii) disruption of co-chaperone/Hsp90 interactions, (iii) antagonism of client/Hsp90 associations and (iv) interference with post-translational modifications of Hsp90. The different functions of Hsp90 isoforms and the isoform selectivity of drugs need further investigation. The correlation of cell surface Hsp90 with cancer metastasis and the emerging involvement of Hsp90 inhibition in cancer stem cells have become exciting areas that could be exploited. Therefore, the aim of this review is (1) to summarize the up-to-date knowledge of mechanistic studies and clinical prospect of currently available Hsp90 inhibitors, (2) to enhance our perspectives for designing and discovering novel Hsp90 inhibitors, and (3) to provide an insight into less-understood potential of Hsp90 inhibition in cancer therapy.