In uncertain environments, effective decision makers balance exploiting options that are currently preferred against exploring alternative options that may prove superior. For example, a honeybee foraging for nectar must decide whether to continue exploiting the current patch or move to a new location. When the relative reward of options changes over time, humans explore in a normatively correct fashion, exploring more often when they are uncertain about the relative value of competing options. However, rewards in these laboratory studies were objective (e.g., monetary payoff), whereas many real-world decision environments involve subjective evaluations of reward (e.g., satisfaction with food choice). In the subjective case, rather than choices following preferences, preferences may follow choices. With subjective rewards, rather than minimize uncertainty, people may seek to maximize coherency between their preferences and behavior, which would make them less likely to explore the more they exploit. We found this choice pattern over several years in 280,000 anonymized supermarket customers. In a follow-up study, as predicted by coherency maximization, customers preferred coupons to explore alternative products when they had recently started an exploitation streak. We developed a model of how preferences, represented as a weighted vector in a multidimensional space, develop in light of choices. Laboratory studies confirmed the predictions of the model, including a study of how opinions on hot-button issues, such as immigration, abortion, and free trade, develop as a consequence of making a political choice (e.g., voting).