Stress can cause changes to your sex drive, problems with irregular or painful periods in women, or impotence and problems with sperm production in men. Whether you’re a man or a woman, you might also feel reduced sexual desire when you’re under too much stress.