A robust kernel-based solution to control-hijacking buffer overflow attacks

L. I.Han Chen, F. U.Hau Hsu, Cheng Hsien Huang, Chih Wen Ou, Chia Jun Lin, Szu Chi Liu

Research output: Contribution to journalArticlepeer-review

6 Scopus citations

Abstract

In this paper, we propose a robust kernel-based solution, called AURORA, to a ubiquitous security problem - control-hijacking Buffer Overflow Attacks (BOAs). AURORA utilizes either the addresses of the buffers storing input strings or signatures to detect and block control-hijacking BOA strings in the kernel, including zero-day ones. Although AURORA detects some types of BOAs through signatures, AURORA does not need to create any new signature for new attack instances after its installation because AURORA'S signatures are created based on commonality of control-hijacking BOAs. Moreover, even a process is under a BOA, AURORA allows it to continue its execution or to be terminated gracefully without the cost of process idleness or repeated process crashes. Thus, AURORA is robust to control-hijacking BOAs. AURORA does not need to modify the source code of any application programs. Furthermore, AURORA is compatible with existing operating systems and application programs; hence, AURORA could work with other protection mechanisms to provide an extra layer of protection. Our experimental results show that with less than 1% overhead and negligible false positives, AURORA can accurately block various control-hijacking BOAs.

Original languageEnglish
Pages (from-to)869-890
Number of pages22
JournalJournal of Information Science and Engineering
Volume27
Issue number3
StatePublished - May 2011

Keywords

  • AURORA
  • Buffer overflow attack
  • Control hijacking boa
  • Retum-into-libc attack
  • Stack smashing attack

Fingerprint

Dive into the research topics of 'A robust kernel-based solution to control-hijacking buffer overflow attacks'. Together they form a unique fingerprint.

Cite this