Sometimes our goals conflict with our habits, leading to impaired performance. Prior research shows that people are better at overcoming automatic responses and producing goal-directed responses when they are motivated by the prospect of reward. However, it is not known whether reward leads to improved performance via the inhibition of automatic responses, the facilitation of goal-directed responses, or a mixture of both. This is due in part to the fact that standard experimental paradigms used to study cognitive control rely on free response times. This allows participants to delay the initiation of their responses and avoid committing errors driven by prepotent response tendencies, making it difficult to infer how reward affects underlying cognitive processes. We addressed this limitation by using conflict tasks in which participants were forced to respond at a predetermined time. We manipulated the time available for response preparation by varying the onset of the target stimulus and we measured the participants' preparatory state at the time of the forced response. Finally, we used a probabilistic response preparation model that dissociates the preparation of habitual and goal-directed responses to infer the time required to prepare these different responses. Across two experiments, we found evidence that reward accelerated the preparation of goal-directed actions, while there was little evidence that reward further inhibited the preparation of automatic responses. This work sheds light on which cognitive control mechanisms are modulated by reward.