“The best way to keep a prisoner from escaping is to make sure he never knows he’s in prison.”

Fyodor Dostoyevsky

Before this project, I had what I now realise was a wildly romanticised view of hackers. I assumed they were all terrifying geniuses who setup notifications on their phone for every new CVE, built botnets out of industrial refrigerators and used headless Alpine Linux as a daily driver on their 12-year-old Thinkpad.

So, naturally, when I started exploring security engineering & general blue-teaming, I was preparing for battle.

Instead, I got… this.

Russian IPs trying the same root:root combo like it’s still 2004
Random script kiddies who type at ~20 wpm trying to rm -rf / --no-preserve-root but failing to type the command 4 times
Bots that only know how to brute-force SSH and run chmod files on a read-only system

Most “hackers” aren’t exactly Mr. Robot, their more like toddlers in a candy store, except the store is a Docker container and the candy is /etc/passwd

And aren’t toddlers let loose in a candy store fun to watch.

I decided to build a honeypot. Not one of those fake ones that echo’s everything back to you, not one that only allows you to run 4 different hard-coded commands, but one that actually runs a live Linux server, yet I still can watch everything from the safety of docker, a brutal firewall & a raspberry Pi.

Why did I do this?

I think we all know that this would’ve been a damn fun project to build, and it was. Even through the configuration of 3 different Linux environments, the nightmare of keeping my home internet secure, and the constant struggle of creating a 3-way SSH connection (not really, but who cares) and transferring files between these devices, the end result literally plastered a smile on my face that I couldn’t get rid of, even with industrial-grade equipment.

I also want to touch on the current state of the internet. Now that everything is being automated, security standards are rising and everything is becoming open source, cybersecurity is becoming a hazy battlefield that’s about 99% random noise & 1% “oh s#!&” moments. I wanted to do my tiny little part in helping cleanse the internet of some of that noise.

This was also just such a great learning opportunity. The amount of stuff I learned about networking was crazy, even if the first half of this project was me asking ChatGPT what different errors mean.

And finally, I also noticed that not many honeypots out there actually provide a linux environment for the hacker to run rampant in. Most just simulate basic shell commands, maybe create a *very* basic file system, but that’s about it. I wanted to allow the attacker to actually do things, and monitor them like a high-security prison.

And considering all of this, anyone wanting to get into cybersecurity should build their own honeypot. It’ll be one of the coolest projects you make.

Building the Bait

The first step is always, of course planning.

I say that knowing that about 10% of time is spent following the plan, and 90% is figuring out the linux command you keep forgetting, transferring files over between machines & putting out metaphorical (or, you know, maybe real) fires.

I did a little research into honeypots that people have built, but it’s literally just the same thing over and over again:

I built a python honeypot that lets you run ls!

So I made an SSH honeypot…

Watching hackers try (and fail) to run commands on a fake shell!

All of them:

Used python
Didn’t actually have a shell, just hard-coded fake commands
Ran it in an Ubuntu 22.04 Hostinger VPS
Vlogged or blogged it in a cringey way (maybe I did too, that’s for you to decide)

And the surprising thing is each of these could’ve been easily done differently, but it seems everyone followed the same tutorial, got AI to rewrite it to look like their own code, then spent more time blogging about it than actually experimenting with it.

So what did you do different?

I’m glad you asked.

Instead of Python, an extremely common, high-level interpreted language, I wanted to use a new language I’ve been learning - Go. It’s a lower level, compiled language that feels to me like “Python 2”. It’s fun, it’s incredibly performant, and it still has all the same capabilities - with the right packages of course. But it doesn’t have classes, so deal with it.

Instead of hardcoding commands like ls to just print things like:

timmys_passwords.txt

and when they run cat timmys_passwords.txt they see:

haha this is a honeypot your ip has been logged

I wanted to run an actual linux environment dedicated purely to the attacker.

Most approaches of this would require a VM - they’re secure, rugged, can be treated like crap and rebuilt, and they are notoriously hard to escape. But, I was running this on a server with 2gb of ram and a single CPU, so I don’t think it could even handle it’s own OS.

This is also part of the reason I used Go, but we’ll talk about that later.

Instead, I used docker - slightly less rugged, much more performant & lightweight, can also be treated like crap and reboot, and they are also incredibly hard to escape - but they aren’t fully secure. With the right setup though, attackers can’t do anything.

The OS of choice for docker was my beloved Alpine Linux, with the entire container coming in at under 100mb & using a maximum of 128mb of ram. Ubuntu would’ve been just too heavy to allow attackers to roam free in, plus Alpine is designed with security in mind (mostly since its attack surface is about the size of a household fly)

And, most importantly, I didn’t run it on an Ubuntu 22.04 Hostinger VPS. I ran it on an Ubuntu 22.04 Linode VPS.

Coding’s meant to be the hard bit…

The initial code for the currently horrendous honeypot, one with a fake emulated shell & uninformative logging was surprisingly easy, with just under 200 LOC. It would start a server (not as a class, something I still have to get used to) listening on a random port on localhost. Someone could connect to that port, and they’d be prompted for SSH credentials. At this point, anything worked. Once you were in, you’d be able to run ls, cat passwords.txt, and exit. That’s it.

The server itself would log every single command. I don’t mean the whole line when you press return, I mean every single keystroke. And no, i didn’t add -v twice.

A whale of a problem

I initially wanted to use the docker SDK within Go to interact with the container. It seemed easy enough, although I was a little concerned that the packages came directly from the GitHub repo, since that’s usually a sign of slightly unstable code.

When trying to run the containers (without any shell functionality yet, just boot it & let it rip) it did not seem to work. I was getting incredibly ambiguous errors about import failures when there wasn’t even any problems shown in my IDE.

Turns out that Go packages have very iffy support for Arm architecture.

What I ended up having to do was manually execute docker containers with the exec command, which felt like fixing a broken leg with a band-aid.

Keeping my home network secure

I eventually got the container working & logging all of the output, and was ready to scp everything to the freshly-bought Linode. Before I did though, I needed to set up a method of sending logs over to my Raspberry Pi at home.

Initially, I wanted to have a secondary SSH connection running at all times between the Linode & my Pi. But there were a lot of problems with that.

Firstly, if an attacker does break out of the container, they can very easily see the connection & either terminate it, or trace it to my home IP address.

Secondly, that is a lot of data.

Now, I bet these youtube gurus would’ve just said “Setup SSH keypairs!”, but even then there’s still a trace. What I eventually decided to do was pull the logs from my pi, not push them from the linode. That was the key difference I, previously, didn’t know existed.

All it took was a simple bash script and a cronjob, but you’d be surprised how tricky that was on Alpine linux. Did you know you can’t include a . in filenames in a cronjob on Alpine? I didn’t figure that out for hours!

Calm before the storm…

After setting everything up, triple-checking all the security measures & double-checking it wasn’t past 1am, I really wasn’t sure if I wanted to open that port.

Eventually I did, I tested it to make sure it worked (and made a fool of myself thinking that the hacker that just connected wasn’t me from a minute ago) and left it be, thinking I’d get a couple good bytes of the fishing hook & have a cool story to tell.

But I wasn’t prepared for the weird, wild & almost wonderful world of the hacker zoo that had just been opened.

*Part 2 is coming out soon :)

The Hacker Zoo: Letting the internet at a real server

Table of Contents