Follow the white rabbit

Ok, so all admins fear network problems. Routing problems and other annoyances, specially if they happened to occour when there is another problem on to solve. I’ve seen this a few times in past years, like the LAN-people are reconfiguring switches just as I reboot a Sun server from the LOM-interface and for some reason the server doesn’t come up cleanly and I can’t access that mgmt-segment where the LOM is since someone forgot my VLAN in the new switch config. Same goes if you are accessing a remote site and the WAN connection goes down or the Internet pipe drops in the middle of a NIC reconfiguration.
So what should people do about this, I’ve started using a concept I call "a rabbit" in most datacenters we have. Its can be a huge security hole if not done right. I want one machine in the network that can access all network segments, and here is the important bit; without passing any L3 switches, any routers and as few tagged VLANs as possible, simply to keep the network access as simple and straight forward as possible. This box should be your entry-point to these segments. The rabbit only works for a single location of course. WAN access without routers is most often a bit of a pain :-)

• Take a decent machine like a U5 with a QFE-card or P3 with a pack of old 100Meg NICS. Oh, and you probably need a serial port as well.
  
• Install a good secure OS on it, preferably OpenBSD. Install as little as possible. Remember this is a rabbit, not a "server".
  
• Configure pf to block all incoming traffic (on ALL interfaces) except SSH (you should probably move SSH to an obscure port like 54088 instead of the default 22).
  
• Connect a GRPS modem, can be an old Nokia phone or a proper modem like the Siemens MC35 I use and either configure PPPd dial-in to the server or simply place a simple getty on that tty. Allowing PPPd to the box is probably better since you can then forward SSH ports from all corners of the network, but in a way opens up a possible security hole (do people still use modems for hacking?).
  
• Connect ethernet cables to switches in the actual segments you want to access. Worst case, use tagged VLANS in the machine, but try to avoid it.

Having the GPRS modem hooked up gives you the possibility to access the box from anywhere in the world, great to be able to plumb NIC’s from a beach in Tahiti. :-)
One additional possibility would be to have a DSL connection to this box to better connection speed to the machine or even VPN, that’s to risky in my opinion. I probably forgot a few steps in but at least you get the picture what the rabbit is supposed to achieve.

Happy Easter.

PS. Creds to my fellow admin Wector for explaining this concept to me with his now well known matburk.