Today is Saturday, and I am out for an early breakfast at Panera, as usual, and I have my laptop with me to work on some AWS stuff.
My plan is to finish setting up installing Laravel PHP framework as part of a tutorial I am following for storing laravel sessions on DynamoDB. Since my Web Server is hosted on an EC2 instance on a private Subnet, I need to first VPN into my VPC to be able to get to it.
So I proceeded to connect through VPN to my OpenVPN EC2 on my public subnet. Connection is established. Now, I need to SSH to my 10.0.3.208 machine, but nothing is happening, and eventually the connection times out!
I checked my security groups, Routes, and NACLs; everything looks good, and I haven’t changed my setup from when I successfully connected the day before.
Is AWS flaky? What’s going on?
I run Wireshark to help me figure out what’s causing the connection timeout. I am trying to connect through port 22 with putty, and port 80.
I am seeing ICMP packets from a 10.128.128.128 device telling me that the destination is unreachable, and that the communication is administratively filtered.
Not Sure what this device is, but let’s do an ipconfig on the local machine:
Ok, so Panera bread, wireless router is rejecting my request for a server that it thinks is in it’s 10.0.0.0/8 network.
My VPC is on 10.0.0.0/16
My private subnet is on 10.0.3.0/24
Ok, so I guess the wireless router will not let this slide by him so my packets can travel through the VPN tunnel to my EC2 machine on my VPC.
I did some google search to see if I can make this work through some Ninja networking tricks. The threads I read unanimously agree that you need to design your VPC with a CIDR block that doesn’t conflict with your local network.
Every now and then, some network guru will tell you about all the hoops he went through to make it somehow work, but why go through the trouble, when you could design your VPC with no conflict in mind? Plus, I needed access to the local network router to try those solutions, and Panera wasn’t about to hand me the credentials to their router.
A Cisco article I read presents a solution through the use of NAT and DNS. A use case for that could be two companies merging with the same private address space, and what you would do to prevent re-addressing the local network of one of them.
Anyway, I can’t stress enough the importance of choosing the right VPC CIDR.
Avoid the 192.168.x.x since most home network routers use that RFC 1918 IP addresses.
To solve my issue, I need to go home and connect. Surely enough, as soon as I connected to my home network, I was able to VPN, and browse to my private web server and SSH to it:
Traffic through wireshark looks good. Syns are being acknowledged, and traffic is flowing nicely through the VPN adapter:
One caveat is that if you want to see network traffic for your private IP address as the destination IP before it gets encapsulated into VPN encrypted packets, you need to make sure you are listening to the VPN adapter not your wireless adapter, or your network card if you are connected with an Ethernet cable to your router. (It took me some googling to figure that out)
My VPN server is configured with this CIDR: 172.27.224.0/20, which it uses to assign IPs to connected clients.
I have tried to produce some conflict again with my VPC, and I changed my VPN server CIDR to 10.0.0.0/16:
My VPN server couldn’t route traffic correctly, I couldn’t resolve domains to browse the internet, so that’s another place to look at in case you have problems with your VPN.
Software defined networks enables services such Amazon AWS VPC, which gives us greater control over our cloud network topology.
“With great power comes great responsibility”, so choose your CIDR blocks wisely.