In recent years, IoT adoption has been higher, and this causes lots of security concerns. One of the fundamental security concerns in IoT adoption is the question, “Are you who you say you are?” Thus, authentication forms the gateway for a secure communication system with IoT. So far, the human voice is one of the most natural, non-intrusive, and convenient behavioural biometric factors compared to other biometric authentication methods. Despite the non-intrusive characteristics of voice as a biometric authentication factor when accessing IoT technologies, there is a concern of a general societal trust and distrust with IoT technology and the risk of theft of users' data and imitation. This study derived a realistic trust evaluation model that incorporates privacy, reliability, security, usability, safety, and availability factors into a trust vector for a flexible measurement of trust in the user accessing IoT technologies.