Locking Down Remote Access

From Airangel Wiki
Revision as of 14:13, 26 February 2020 by SteveH (talk | contribs) (Requirements)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

Whilst the connector HTTPS service has a good level of security and Mikrotik managed-mode locks down the firewall, policy sometimes dictates that access to on-site resources should be restricted to known source IPs. It is necessary for the Airangel cloud to maintain access to these for certain elements of the system to function correctly, but at the same time the source IPs of the Airangel cloud are dynamic and frequently change depending on the load demands of the system.

To solve this issue, there are some useful tools to assist in allowing the list to be managed dynamically.

Mikrotik Address-List

An address-list has been created that automatically maintains an up-to-date list of the Airangel cloud hosts. By using this in Mikrotik firewall rules, it is possible to configure your rules and leave the platform to maintain its own access.


Juniper Event/Op Script

It is possible to configure a Juniper firewall to automatically update an address-book/address-set from our published list of IPs. This is acheived entirely through a script on the Juniper, so it can be customised and the workings are transparent to partners.

Firstly, you will need the script. In order to run this on-demand as well as on-event, copy this to both:



from jnpr.junos import Device
from jnpr.junos.utils.config import Config
from jnpr.junos.exception import *
from lxml import etree
import jcs
import requests
import sys

def main():

    dev = Device()
    dev.bind( cu=Config )

    # Get the existing config
    configText = dev.rpc.get_config(filter_xml='<security><address-book /></security>')
    secNode = configText.find("security")

    # Get the IP list from the existing AirangelCloud address-book
    addresses = []
    if secNode is not None:
        for addressBook in secNode.iter('address-book'):
            aBookName = addressBook.find('name').text
            if aBookName == 'AirangelCloud':
                # Once we have the address book, we find the IPs
                for addressNode in addressBook.iter('address'):
                    aName = addressNode.find('name')
                    aIp = addressNode.find('ip-prefix')
                    if aName is None or aIp is None:

    # Fetch the IPs from the cloud.
    print "Fetch IPs from cloud..."
    hostList = requests.get("https://cn.captive.net/gateway-api/cloudIpList", verify="/packages/mnt/junos-runtime-srx/usr/share/ui/support/Trusted_CAs.pem").json()

    inAddresses = []

    # Standardise the addresses and add /32 to single IPs.
    for address in hostList:
        txtAddress = address.encode('UTF-8')
        if txtAddress.find('/') == -1:
            txtAddress += "/32"

    # Lists for our additions and deletions
    adds = []
    deletes = []

    # Calculate adds and deletes
    for ip in inAddresses:
        if ip not in addresses:
            print "Add %s!" % ip
    for ip in addresses:
        if ip not in inAddresses:
            print "Delete %s!" % ip

    # Lock the configuration, load configuration changes, and commit
    print "Locking the configuration"
    except LockError:
        print "Error: Unable to lock configuration"

    print "Applying changes..."
    configChanges = []
    for ip in adds:
        # Remove the CIDR notation for individual IPs
        slashpos = ip.find('/32')
        if slashpos != -1:
            nameString = ip[0:slashpos]
            nameString = ip

        # Add the changes to the list
        configChanges.append("set security address-book AirangelCloud address %s %s" % (nameString, ip))
        configChanges.append("set security address-book AirangelCloud address-set AirangelCloud address %s" % (nameString))

    for ip in deletes:
        # Remove the CIDR notation for individual IPs
        slashpos = ip.find('/32')
        if slashpos != -1:
            nameString = ip[0:slashpos]
            nameString = ip

        # Add the changes to the list
        configChanges.append("delete security address-book AirangelCloud address %s %s" % (nameString, ip))
        configChanges.append("delete security address-book AirangelCloud address-set AirangelCloud address %s" % (nameString))

    print "Loading configuration changes..."

    # Join the list with \n and then apply as config.
    allChanges = "\n".join(configChanges)
        dev.cu.load(allChanges, format="set")
    except ConfigLoadError as err:
        print err
        print "Unable to load configuration changes: "
        print "Unlocking the configuration"
        except UnlockError:
            print "Error: Unable to unlock configuration"

    print "Committing the configuration..."
    except CommitError:
        print "Error: Unable to commit configuration"
        print "Unlocking the configuration"
        except UnlockError:
            print "Error: Unable to unlock configuration"

    print "Unlocking the configuration"
    except UnlockError:
         print "Error: Unable to unlock configuration"


if __name__ == "__main__":

You now need to enable Python scripting and whitelist the script by name. Apply the following configuration:

system {
    scripts {
        language python;
        op {
            file aacloudlist.py;

Due to an older Python version on the Juniper firewalls, you will receive a warning about SSL not supporting SNI. This is not a concern.

You can now test the script from the CLI with the command op aacloudlist.py

You can also automate the running of it with the following event configuration. It requires to add a user (set your own password) or you can use an existing user with config read/write privileges. Event scripts cannot be run as the root user. In this example, we create "scriptuser".

system {
    login {
        user scriptuser {
            class super-user;
            authentication {
                encrypted-password "...";
event-options {
    generate-event {
        aalistfetch time-interval 60;
    policy aalistfetch {
        events aalistfetch;
        then {
            event-script aacloudlist.py;
    event-script {
        file aacloudlist.py {
            python-script-user scriptuser;


This script will maintain an iptables 'ipset' that can be referenced in firewall rules. It is recommended to run this on a cron. Every 10 minutes would be more than sufficient.


You will need to have Python installed on the box (standard on most distros, definitely on CentOS7).

After getting this running, you need to maintain an iptable ruleset that refers to this set: e.g.

iptables -A INPUT -p tcp --dport 443 -m set --match-set aa-aws src -j ACCEPT
iptables -A INPUT -p tcp --dport 443 -j DROP

Please ensure that after a reboot, the ipset is created *before* the iptables rules are added. Otherwise the rules will refer to a non-existent set and fail to be added!


import urllib2
import json
import os

print("Fetching new IP list...")
response = urllib2.urlopen('https://cn-gateway.eu1.captivnet.com/gateway-api/cloudIpList')
jsondata = response.read()
ipdata = json.loads(jsondata)

print("Checking for existing list...")
firstRun = False
status = os.system("ipset list aa-aws")
if status:
    print("First run. We will create, rather than replace, the list.")
    firstRun = True

print("Create temporary list...")
os.system("ipset create aa-aws-new hash:net")

print("Populate temporary list from cloud...")
for ip in ipdata:
    print("\tAdd %s" % ip)
    os.system("ipset add aa-aws-new %s" % ip)

if firstRun:
    print("Installing new list as aa-aws...")
    os.system("ipset rename aa-aws-new aa-aws")
    print("Swapping new list in as aa-aws...")
    os.system("ipset swap aa-aws-new aa-aws")
    print("Destroy temporary list...")
    os.system("ipset destroy aa-aws-new")

print("All done.")


For other partners' own firewalls it is of course not possible to use the Mikrotik address-list directly. However, the list of IPs is now exposed by API in JSON format so that your own tools can retrieve this list. Due to the very volatile nature of the list, it is *strongly* recommended not to rely on manually updating the list.

The URL is: https://cn-gateway.eu1.captivnet.com/gateway-api/cloudIpList