5.10.2021
In photovoltaic power plants, we typically have devices, usually industrial PCs or FotobotHW. The device runs Linux, collects data from various sources, stores them in the local database and listens on the network and waits for various control commands. Sometimes we even develop various communication protocols on such a computer. It is much easier to install a C++ compiler and develop remotely then move the piece of the power plant to the office. It is very expansive to bring a ½ MW inverter from a non-EU Ukraine to Czech Republic and there is a lot of paperwork with it. The forklift won't take the inverter to the stairs in the office and I won't pull the invertor under my desk. The computer on the power plant must therefore be reliably accessible over internet.
A large power plant is relatively well controlled environment in terms of internet connection. Usually there is a network administrator who can set a static address to DHCP and set up NAT for some ports, but this may not be the rule and most importantly, it can be a problem to get such a person and negotiate with him (remember? Ukraine).
The Internet is not simple today. That was a long time ago, when 4 billion addresses was more than anyone could ever imagine in his wildest dreams. NAT and firewalls did not exists or it was existing in academic papers only. In that time, a fixed IP address was set in the device and it was visible from anywhere. At least in theory.
Today, the situation is more complicated. There are so many computers that the IPv4 address range is not enough. Several different devices are hidden behind any public IPv4 address, their number can easily go into the hundreds or thousands. Without sophisticated cooperation with the administrator of such a network it is unthinkable to make the devices accessible. The public IP address would by ideal for any such device, but no more IPv4 addresses exist anymore. I repeat: IPv4 addresses are not available. And they won't. If we limit ourselves on IPv4, the internet just doesn't work. It's broken into millions of pieces and they cannot be glued. The solution would be IPv6, but get an IPv6 connection in the middle of the nowhere. It is not common in Eastern Europe. It is not common either in Western Europe or in Czech Republic. Today, only one important site exists without IPv6 support: github. All other web sites supports IPv6. But ISPs live in Dark Ages.
So how can be the device made reliably available right after plugging in local area networks? Let's face it: it doesn't work on IPv4. But the IPv6 protocol has enough addresses. When we take the IPv6 into account, despite the technically backward ISP in the field, we can set up a public IP address for each device or network at photovoltaic power plants.
At power plants (and elsewhere) we use the solution described below with the following features:
The whole structure is evident from the picture. The first device connects whole network (the green squares). The second and third devices are single. The gray areas delimite units that usually work within a single computer.
The addresses in the image are selected from the range 2001:db8::/32, which should be used in documentation and examples. For your purposes, you will need a prefix in the range /56 (minimum), but ideally /48 (the range used in the example).
The entire network here has a range of 2001:db8:0::/48. The range 2001:db8:0:1::/64 is used for the internal company network.
The space reserved for the devices is 2001:db8:0:1000::/52 - each address, starting with 2001:db8:0:1xxx belongs to one of the networks reserved for external devices. Because each device receives an entire /64 network, the device can be used as a router and lot of other devices can be placed behind them. 4096 networks can be addressed in the 2001:db8:0:1000::/49 range, minus one connecting network (described below). If this amount is not enough for you, it is possible to expand the number of networks for almost entire 2001:db8:0::/48 range, giving us room for more than 61 thousand devices. If you have the /56 mask from your ISP, you can address only about 250 devices.
The connecting network is indicated in yellow in the figure. The connecting network requires only one /64 range. The server address is static, the address for the device is created dynamically when an IPSec connection is established. In the example, the 2001:db8:0:1000::/64 range is used for connecting network.
The pink colored hoses represent the connection between corporate network and devices. The connection can be made using the IPv4 or IPv6 protocol Addresses are not important here. On one side, of course, will be the server's address (IPv4 or IPv6), on the other hand, the public IP address of the device - usually the only IPv4 address you get from your ISP to connect your entire network. The connection is encrypted using IPSec tunnel transport mode. What's happening inside the connection is untraceable. Only IPSec traffic can be seen externally, but the communicating addresses or contents of the communication are reliably hidden.
I covered this topic in an older article. You can read the details there. Although user computers connect slightly differently, the requirements remain in principle the same:
In an earlier article, the server was identified by a certificate. There is a good reason for this - you cannot setup the VPN without certificates in Windows. Security can be slightly higher with certificates, but the more complex server settings is a disadvantage. When connecting Linux devices, you can choose a simplier way and use password for both server and devices.
Please note an unusual trick used for strongswan settings. The left and right sides are swapped on the server. Usually the left side is reserved for the server but not in this case. The reason is simple: the device configuration part is (mostly) the same on both the server and the device. You can use CTRL-C and CTRL-V to simply copy the apropriate part of configuration.
The server must have IPv6 routing enabled in the /etc/sysctl.conf file:
net.ipv6.conf.all.forwarding = 1
Strongswan settings, /etc/ipsec.conf file:
#############################################
# Devices settings
conn device001
leftid = device001
leftsubnet = 2001:db8:0:1001::/64
also = deviceXXX
auto = add
conn device002
leftid = device002
leftsubnet = 2001:db8:0:1002::/64
also = deviceXXX
auto = add
… each device has it's own conn section
#############################################
# /etc/ipsec.conf
# common for all devices
conn deviceXXX
auto = ignore
closeaction = clear
# local side, WARNINNG - left and right side swapped
right = vpn.devices.hobrasoft.cz
rightid = vpn_devices
rightsubnet = ::/0
rightauth = psk
rightsendcert = never
# remote side, WARNINNG - left and right side swapped
left = %any
leftauth = psk
leftsourceip = 2001:db8:0:1000::/97
leftdns = 2001:db8:0:1::1 # DNS server for devices, located in the intranet here
#############################################
# common for entire ipsec server
conn %default
keyexchange = ikev2
rekey = yes
forceencaps = no
compress = no
fragmentation = yes
closeaction = restart
dpdaction = hold
dpddelay = 60s
auto = start
ike = aes256-aes128-sha512-modp4096!
esp = aes256-aes128-sha512-modp4096!
Strongswan settings, file /etc/ipsec.secrets. Change passwords to your own:
# /etc/ipsec.secrets vpn_devices : PSK "server-password" device001 : PSK "device-001-password" device001 : PSK "device-002-password" …
It is necessary to set the IPv6 address for the connecting network on the server. In the Debian distribution, the network settings is usualy located in the /etc/network/interfaces file. Only part of the file is listed here, in practice, the setup will be more complex and network interfaces will probably have more addresses:
# /etc/network/interfaces
# IPv6 connecting network settings
iface eth0 inet6 static
address 2001:db8:0:1000::1
netmask 64
It is necessary to set the assigned static IPv6 address on the device. In the Debian distribution, the network settings is usualy located in the /etc/network/interfaces file:
# /etc/network/interfaces
auto lo
iface lo inet loopback
# IPv4
iface eth0 inet dhcp
# IPv6
iface eth0 inet6 static
address 2001:db8:0:1001::1
netmask 64
The StrongSwan settings is mostly equal for all devices, only a small part is specific for each device. The specific part can be easily copied from the server:
# /etc/ipsec.conf
##########################################
# This part is specific for each device!
# The same configuration is used in the server's configuration
# Use CTRL-C CTRL-V to copy the device's part and change
# the "add" to "start"
conn device001
leftid = device001
leftsubnet = 2001:db8:0:1001::/64
also = deviceXXX
auto = start # when copying from server, change the add to start
##########################################
# Do not change. Common for all devices
conn %default
keyexchange = ikev2
closeaction = restart
dpdaction = restart
dpddelay = 30s
rekey = yes
auto = start
compress = no
fragmentation = yes
forceencaps = yes
ike = aes128-aes256-sha512-modp4096!
esp = aes128-aes256-aes128-sha512-modp4096!
conn deviceXXX
auto = ignore
# protistrana
right = vpn.devices.hobrasoft.cz
rightid = vpn_devices
rightsubnet = ::/0
rightauth = psk
# moje strana
leftsourceip = %config6
leftauth = psk
Copy the password for the specific device and the server from the server's /etc/ipsec.secrets file:
# /etc/ipsec.secrets vpn_devices : PSK "heslo-serveru" device001 : PSK "heslo-device-001"
The IPSec connection dies sometimes. Add a simple service to cron or systemd to check the connection periodically. It can restart the failed encrypted channel automatically:
#!/bin/bash
if [ "$(/usr/sbin/ipsec status | grep 'none')" = " none" ]; then
/bin/systemctl restart ipsec
fi
The server is listening on UDP port 4500. UDP is used here due to possible NAT in the way - you usually can't avoid that on IPv4 today. IPSec is encapsulated in UDP and does not change. NAT has one crazy feature - it can change the contents of packets. Modified packets IPSec would drop, the connection would not work. Death of NAT!
The request from the device usually comes to the server using the IPv4 protocol. StrongSwan creates a connection network through this connection using the range 2001:db8:0:1000::/97. The address is not fixed on the device side of the connection. You would be happy when connecting from your notebook because you had a full connection to your IPv6 internet at the moment. But the address changes every time you make a new connection.
But we always need to have the same address for the device. The device therefore has a static address set at system startup. Server-side IPSec then populates the routing tables with the path to the specific device's network when creating a connection. On the device side, IPSec sets the default gateway for IPv6 to the server address on the link network.
When the connection is inactive, the device has only one IPv6 address set:
2001:db8:0:1001::1/64
As soon as the connection is active, the address of the connecting network is added, there are two addresses at the moment:
2001:db8:0:1000::47/97 # dynamic address of the point in the connecting network 2001:db8:0:1001::1/64 # static device address set in the network configuration
The ipsec status command is a useful tool.
You should see something like this on the device when the connection is established correctly:
Security Associations (1 up, 0 connecting):
device001[11]: ESTABLISHED 2 hours ago, 10.0.3.10[device001]...123.276.123.82[vpn_devices]
device001{44}: INSTALLED, TUNNEL, reqid 1, ESP in UDP SPIs: c212bd6b_i cc854e6f_o
device001{44}: 2001:db8:0:1001::/64 === ::/0
The device is already set up to be able to mediate access to IPv6 Internet to all other devices on the local network. Other devices just don't know about it. You can set the router address (2001:db8:0:1001::1) manually in the network configuration, but automatic settins is better. The radvd program is used for this. It broadcasts information about the default router and possibly DNS (some OS does not accept DNS).
The radvd settings is very simple:
interface eth0 {
AdvSendAdvert on;
MinRtrAdvInterval 10;
MaxRtrAdvInterval 60;
AdvHomeAgentFlag off;
prefix 2001:db8:0:1001::/64 {
AdvOnLink on;
AdvAutonomous on;
AdvRouterAddr on;
};
RDNSS 2001:db8:0:1::1 {
AdvRDNSSLifetime 3600;
};
};
You should run radvd in a very well controlled environment only. With this settins, all IPv6 traffic would go through your company server. Also, the wrong radvd settins can effectivelly break the local area network operation.
No "special" network interface for encrypted data is used here. Everything is controlled through policies. Encrypted and unencrypted packets appear on the same external network interface (eth0). This can be confusing for lots of people. It the simillar IPv4 IPsec settings, the local range IP addresses would appear on the external network interface, which is usually not good. But with IPv6, no local addresses exist. The IPSec does not "create VPN" here, the IPSec only provide encryption between two points of the internet. These points can be networks or individual computers. Packets with the same content and the same addresses run between them regardless of whether they are encrypted or not. There is no reason for a virtual network interface.
We used a huge address space to connect a single device:
You can save some addresses if you have a /56 range allocated from your ISP, ie 2⁷² = 4 722 366 482 869 645 213 696 addresses. Then the balance is more favorable.
But even if you use the remaining address space to the last bit, connecting a single device will always cost you at least 2⁶⁴ + 2 = 18 446 744 073 709 551 618 addresses. It's okay and it's not a wasting. It is a IPv6's way.