An In-Depth Survey of Bypassing Buffer Overflow Mitigation Techniques

Abstract

Buffer Overflow (BOF) has been a ubiquitous security vulnerability for more than three decades, potentially compromising any software application or system. This vulnerability occurs primarily when someone attempts to write more bytes of data (shellcode) than a buffer can handle. To date, this primitive attack has been used to attack many different software systems, resulting in numerous buffer overflows. The most common type of buffer overflow is the stack overflow vulnerability, through which an adversary can gain admin privileges remotely, which can then be used to execute shellcode. Numerous mitigation techniques have been developed and deployed to reduce the likelihood of BOF attacks, but attackers still manage to bypass these techniques. A variety of mitigation techniques have been proposed and implemented on the hardware, operating system, and compiler levels. These techniques include No-EXecute (NX) and Address Space Layout Randomization (ASLR). The NX bit prevents the execution of malicious code by making various portions of the address space of a process inoperable. The ASLR algorithm randomly assigns addresses to various parts of the logical address space of a process as it is loaded in memory for execution. Position Independent Executable (PIE) and ASLR provide more robust protection by randomly generating binary segments. Read-only relocation (RELRO) protects the Global Offset Table (GOT) from overwriting attacks. StackGuard protects the stack by placing the canary before the return address in order to prevent stack smashing attacks. Despite all the mitigation techniques in place, hackers continue to be successful in bypassing them, making buffer overflow a persistent vulnerability. The current work aims to describe the stack-based buffer overflow vulnerability and review in detail the mitigation techniques reported in the literature as well as how hackers attempt to bypass them.