Your state of mind can have a big impact on your ability to get aroused. Even if you're with someone who you find sexually appealing, worrying about whether you'll be able to please your partner can make it impossible for you to do just that.
One of the effects of the stress hormones is to narrow blood vessels. When less blood flows into your penis, it's more difficult to have an erection. Even guys who normally don't have any trouble getting excited might not be able to get an erection when they're overcome by sexual performance anxiety.