Some of the more serious side effects include loss of sex drive, impotence, weakened bones (osteoporosis), and heart problems. Eventually, most patients with advanced prostate cancer stop responding to hormone therapy. Doctors call this castrate-resistant prostate cancer.