Houses classification using vision transformer with shifted patch tokenization

Abstract

Deep learning has recently made great progress leading to revolutionizing image recognition, speech recognition, and natural language processing tasks that were previously challenging to make using traditional techniques. Image classification offers a lot of potential for architectural design, even though it is rarely used to uncover new techniques. It can be used to determine the client’s preferences and design a building that satisfies those preferences. The different architectural styles based on culture, region, and time are one of the main challenges for image classification in architecture. Hence, it can be challenging for untrained clients to recognize an architectural style, and sometimes some buildings are made up of various types that are difficult to classify as a single style. This paper investigates the potential of employing state-of-art cutting-edge image classification algorithms in houses classification. In addition, the paper proposes the uses of Shifted Patch Tokenization (SPT) and Locality Self-Attention (LSA) in order to enhance the performance of Vision transformer (ViT) when trained to classify house images with a small dataset, opposed to the regular ViT which requires huge dataset in order to converge. Experimentally, these techniques proved to have a positive impact on the performance of the ViT, which reached 96.85% accuracy when SPT and LSA are employed.