The study examined the association between smartphone use and subjective well-being by comparing regular use with a deprivation condition. Subjective well-being is defined by cognitive and affective components. Regular smartphone use is studied in various situations and for different functions. Data were collected in a prolonged qualitative quasi-experimental deprivation study (n = 11) using diaries (n = 210 diary entries) and follow-up interviews (n = 11). Participants kept diaries for 10 days: five days during normal smartphone use and five days during deprivation. Afterwards, we compared well-being during normal use and deprivation. Results show that using the smartphone for infotainment was clearly associated with pleasant emotions, while social interaction apps caused both negative and positive emotions. However, results from the deprivation part of the study indicate that in sum, satisfaction with social relations clearly worsened when not using a smartphone. Moreover, participants had difficulty managing daily life. Taken together, non-usage seems to cause isolation and low subjective well-being.