I would like to have my router on HA for keeping my network up in case of hardware failures and maintenance work. Since my router is virtualized Opnsense, when I do backups on it or do maintenance on my hypervizor, the whole network goes down. Am I violating any terms and conditions for doing the following?
I think I will end up with two public IPs and the guides say that I also need a Virtual IP for WAN as well, so in total I will have three IPs. Has anyone done this?
Configuring HA on firewalls is a complex setup with multiple options. You can configure multiple routers/firewalls (with multiple external IP’s) and have internal routing protocols determine how to get to the internet or have an interface failover on the firewall. That’s the way to go if you have multiple internet providers (which you need for true HA).
Another method is using a Active/Passive setup (which is what you are looking for, I think). Then there is 1 public IP that is shared over two firewalls. OPNSense uses CARP for that: https://docs.opnsense.org/manual/how-tos/carp.html
That is still a pretty complex setup and given your question I think you are better off just using the failover on your hypervisor. In that case you don’t need any special firewall configuration. If one hosts fails, the VM is started on the other host. If you have maintenance you can just move the VM’s to the other host (vMotion/Xenmotion).
It still might be possible that this setup wouldn’t work. Some internet providers limit the amount of MAC addresses on the WAN interface. You should probably use a static MAC address on the WAN side of the firewall just in case.
They won’t give you a 2nd IP, I’ve asked them repeatedly but they keep insisting their network can’t handle that. (while the 10Gbit customers can, they get 6 or 8 IPs)
That’s why I stick to my hardware router. The Fritz!BOX. I believe that a hardware router is much more reliable than a VM router.
When your VM server goes down completely. Due to a power problem for example. Your complete Network and Internet is down. So how do you order a new psu if your network is down and no Internet is availble.
So if I understand correctly, you are trying to set up a datacenter like configuration including 2 OPNsense instances virtualized, as a HA solution connected to 1 unmanaged switch and the switch connected to the WAN/Fiber converter.
Is there a specific reason why you want it the way you described?
If I were you I would build 2 small OPNsense boxes, one with a SPF connector and the other one with a mobile 4G/5G adapter. Configure HA solutions between those OPNsense boxes, then use 2 separate smart/managed switches to make it redundant. And of course install a couple of UPS’s, for just in case.
The virtual IP handling is done by the OPN routers/gateways.
I’m sorry for any weird typos or chaotic writing, I couldn’t sleep and it 's pretty early in the morning…
You will get 2 ips. It works this way. I have this exact setup. Both ips work and failover is seamless…
There are ways to use 1 ip, with mac spoofing and scripting your NIC to go up after a failover. There is a guide somewhere in the opnsense forum. But this is more complex and error prone.
@Alttab
Whahaha LoL, That is absolutely true. However a VM Server where many updates and other processes run (several virtual machines) Including HDD / SSD. There are simply many more components that you rely on and therefore many more possibilities that malfunctions can occur.
I dont have other vms running on that box , its just there for quorum in the cluster. It has zfs mirrored ssd disks. Where is the storage redundany in a router?
If a component dies, i just replace it. Cannot do that with a router - which has the same attributes anyway , just soldered into a single box.
