About a year ago I decided to take a moment to look back at how my network engineering has progressed throughout the years. That article game me a fresh perspective on how much I’ve grown, but more importantly it showed me how much more I still need to grow.
At the end of that article I said my next network needs to be focused on security, simplicity, and protection for my kids. A year ago I wasn’t very familiar with VLANs and even though I listed them on the network they weren’t true VLANS. When you look back at that specific network there’s one thing that comes to mind; complicated.
To be completely upfront with you I’m actually a bit amazed I was able to make this network work at all. The “VLANs” of this network were in reality either secondary routers or combinations of Raspberry Pis and Docker containers. I used UFW and IPTable firewall rules to separate critical services like my NAS away from services like Piwigo.
About nine months ago I decided to upgrade my router away from a basic commercial home user router like Netgear Orbi or Google Wi-Fi, and away from something overly complicated like my Mikrotik RB3011. I went with a custom made pfSense and I’ve never been happier.
pfSense has been fantastic and completely revolutionized my network. Almost everything I was using Raspberry Pis is available as a package in pfSense. There were even tools I had never even configured available in pfSense; things like the ability to automatically update my Cloudflare DNS records with the IP of my WAN. I’m planning on making a whole post about the things I use pfSense for but for now what’s important is that adding in pfSense opened up a while world of new opportunities for me.
A few months after adding in my pfSense I got the itch to make my network even better and started looking into proper VLANs since I had a router which supported it now (yes technically the RB3011 also supported VLANs but I couldn’t tell you how to actually activate them). I wrote another post about adding VLANs to my network. During this I got a fun change to plan out my VLANs by drawing them on a whiteboard then copying that over to draw.io
When I made that diagram I was working with what I had and trying to create Wireless VLANs by untagging the ports for their VLAN. It actually was working too, until I tried to connect two Google Wi-Fi access points to one network. The Google Wi-Fi app promptly informed me that two access points could not be added to the same network unless they were in Mesh mode which they couldn’t be without creating a double NAT since the pfSense was an edge router.
I sold my Google Wi-Fi and Netgear Orbi Wireless routers and bought some TP-Link Omada 620HD EAPs. I went with these mainly because they supported VLANs by SSID, so I could create multiple SSIDs and have each SSID pass a VLAN tag to the devices connected to it. I love my Omadas; in-fact they were so great that I went and upgraded to a TL-SG3428 as my main switch.
Now that I had a new 24 port switch and EAPs capable of handling SSID VLANs I needed to rearchitect my network. Back to the drawing board, literally.
My new VLAN wiring diagram is still very similar to my old wiring diagram but more streamlined. I’m not trying to untag VLANs for specific APs.
Around the time I was installing and setting up my Omada gear I also bought a refurbished HP Z440 server. I installed Proxmox on this server to replace the other Proxmox server I was running off my wife’s dying college laptop. With this new Proxmox server I started replacing some of my Raspberry Pis with proper VMs, or when appropriate LXC containers.
Proxmox on a proper server, with a Xenon processor and 128GB of RAM, enabled me to create resilient services. The Raspberry Pis worked well but their limited RAM and ARM based architecture meant that they weren’t universally supported. Proxmox allows me to create legitimate x64 based VMs and LXC containers.
I set out to clean up the services I used and virtualize them instead of running them on a Raspberry Pi. I removed Plex because I didn’t have enough legally owned digital copies of my movies. I replaced Piwigo with Photoprism because Piwigo is pretty basic and doesn’t handle videos well. I replaced OSSIM with an ELK stack because it worked better with pfSense. And I set up an email server so people can email me malware at firstname.lastname@example.org.
Finally Proxmox allowed me to really get my analysis lab going. Previous I was working with a combination of physical and virtual machines. My Tails machine was an old 10 year old laptop, my Kali was running on a Raspberry Pi 4, I didn’t have FLARE because I didn’t have enough RAM in my main computer to run it so I used Remnux (I still use Remnux, but FLARE is nice), and SIFT would take 10 minutes to full boot and be usable. In short, much like the rest of the network, I cobbled together what I could
The last year has seen a lot of real improvement to the network. I am proud to showcase my new, and once again, improved home network map.
The new network is using proper VLANs to separate different groups and using pfSense to firewall off the broadcast domains. I’ve separated my IoT devices off my main network, and deployed an IDS to watch them. I’m now logging all of my network traffic, IDS alerts, and DNS queries. My guest network now requires people to accept terms and conditions before they connect and Squid is monitoring their HTTP traffic for viruses.
From here I’m honestly not sure what I want to do next. I have some short term thoughts on what I want to do, but no large overarching redesigns.
- I want to move the Chromecasts over to HARROW but this would require setting up an avahi mdns reflector.
- I want to replace my TP-Link SG108 switches with TP-Link Omada Jetstream 2008 switches so I can more easily manage them in one place.
- I want to set up a DNS redirector so everything has to use pfBlockerNG instead of hardcoded DNS.
- I want to block DoH and DoT on my network unless its coming from the router itself.
- I want to set up a break and inspect SSL proxy and move squid up higher so it can monitor the whole network.
- I want to set up a Site-to-Site VPN using IPSec so I can bring a travel router with me and link my travel network and my real network together
- This is different then using an OpenVPN cert as I want full connectivity between the two networks.
That’s the list of smaller projects I have planned for the network and the next major thing I’m working on is creating a VDI solution for my kids for school. Their school provides them Chromebooks but I’m imagining they will occasionally need something more powerful, so I am setting up a Windows based VDI solution.
As always this has been a ton of fun and I’ve learned a ton about VLANs and network design. You can also see my gradual shift towards minimalism in my diagrams. I think this diagram is my cleanest yet. I’m looking forward to seeing how this network changes and evolves next year.