wiki:SharedSubnetsDesign

Shared Networks Design

This page describes a design for "Shared Networks" feature and aims to fulfil the requirements defined here. Occasionally, this and related documents may refer to this feature as "Shared Subnets". We have decided to rename this feature and haven't updated all existing documents to use the new name. However, it should be noted that "Shared Subnets" is simply a synonym.

1. Intro

There are some cases when it is useful to group multiple subnets together. The most common example is when IPv4 address pool within a subnet is running out of available addresses. To avoid renumbering in the existing subnet, it is much more convenient to simply configure an additional subnet on the same link. A collection of subnets from which addresses are served to the clients connected to the same link is called a shared network. In this case, the server can allocate an address from any of the subnets within this shared network to the client connected via the given link. If the communication with the server is conducted via a relay agent, the usual solution is to form shared network from the set of subnets selected for messages received from a given relay agent.

The introduction of the shared networks allows for specifying certain parameters on the new scope, i.e. shared networks scope. That includes subnet selectors, option data, client classes and other parameters which are normally specified in the subnet scope. All subnets belonging to the particular shared network inherit those configuration parameters. If necessary, these parameters can be overridden in the subnet scope.

2. Configuration

In our design discussions we have considered several different ways in which shared networks could be represented within Kea configuration structures. The details of each proposed solution are not provided in this document of brevity. It is only worth to mention that there were two general approaches considered:

  • Nest subnets along with their full definitions within new structures, i.e. shared network structures.
  • Provide a separate structure holding shared networks with pointers to the subnets defined elsewhere.

We have decided to implement the former approach as it benefits from the hierarchical nature of JSON, is less error prone (no risk of messing up the pointers to the subnets) and, finally, it is the approach implemented in ISC DHCP, which makes it more natural for the users to migrate from old to the new DHCP implementation.

The following is the example configuration of the DHCPv4 server using shared networks:

// This is an example configuration file for DHCPv4 server in Kea.
// It demonstrates an advanced feature called shared network. Typically, for
// each physical link there is one IPv4 subnet that the server is expected
// to manage. However, in some cases there is a need to configure more subnets
// in the same physical location. The most common use case is an existing
// subnet that grew past its original assumptions and ran out of addresses,
// so the sysadmin needs to add another subnet on top of existing one.
{
    "Dhcp4": {

        // As with any other configuration, you need to tell Kea the interface
        // names, so it would listen to incoming traffic.
        "interfaces-config": {
            "interfaces": [ "ethX" ]
        },

        // You also need to tell where to store lease information.
        // memfile is the backend that is easiest to set up.
        "lease-database": {
            "type": "memfile",
            "lfc-interval": 3600
        },

        // The shared networks definition starts here. shared-networks can
        // contain a list of shared networks. There are many parameters
        // that can be specified here, so this example may be overwhelming
        // at first, but the only mandatory parameter for each shared
        // network is name. It must be unique. Typically, each shared
        // network also needs to have at least two subnets to be functional,
        // but if you really want to, you can define a degraded shared
        // network that has 1 or even 0 subnets. This may come in handy
        // when migrating between regular subnets and shared networks
        // or when debugging a problem. It is not recommended to use
        // 1 subnet per shared network, as there is extra processing
        // overhead for shared networks.
        "shared-networks": [
        {
            // Name of the shared network. It may be an arbitrary string
            // and it must be unique among all shared networks.
            "name": "frog",

            // You may specify interface name if the shared network is
            // reachable directly from the server.
            "interface": "eth1",

            // You can specify many parameters that are allowed in subnet scope
            // here. It's useful to put them here if they apply to all subnets
            // in this shared network. It's likely that the most common
            // parameter here will be option values defined with option-data.
            "match-client-id": false,
            "option-data": [ ],
            "rebind-timer": 150,

            // If all the traffic coming from that shared network is reachable
            // via relay and that relay always use the same IP address, you
            // can specify that relay address here. Since this example shows
            // a shared network reachable directly, we put 0.0.0.0 here.
            // It would be better to skip the relay scope altogether, but
            // it was left here for demonstration purposes.
            "relay": {
                "ip-address": "0.0.0.0"
            },

            // Timer values can be overridden here.
            "renew-timer": 100,
            "reservation-mode": "all",

            // This starts a list of subnets allowed in this shared network.
            // In our example, there are two subnets.
            "subnet4": [
                {
                    "id": 1,
                    "match-client-id": true,
                    "next-server": "0.0.0.0",
                    "server-hostname": "",
                    "boot-file-name": "",
                    "option-data": [ ],
                    "pools": [ ],
                    "rebind-timer": 20,

                    // You can override the value inherited from shared-network
                    // here if your relay uses different IP addresses for
                    // each subnet.
                    "relay": {
                        "ip-address": "0.0.0.0"
                    },
                    "renew-timer": 10,
                    "reservation-mode": "all",
                    "subnet": "10.0.0.0/8",
                    "valid-lifetime": 30
                },
                {
                    "id": 2,
                    "match-client-id": true,
                    "next-server": "0.0.0.0",
                    "server-hostname": "",
                    "boot-file-name": "",
                    "option-data": [ ],
                    "pools": [ ],
                    "rebind-timer": 20,
                    "renew-timer": 10,
                    "reservation-mode": "all",
                    "subnet": "192.0.2.0/24",
                    "valid-lifetime": 30
                }
            ],
            "valid-lifetime": 200
        } ], // end of shared-networks

        // It is likely that in your network you'll have a mix of regular,
        // "plain" subnets and shared networks. It is perfectly valid to mix
        // them in the same config file.
        //
        // This is regular subnet. It's not part of any shared-network.
        "subnet4": [
            {
                "pools": [ { "pool":  "192.0.3.1 - 192.0.3.200" } ],
                "subnet": "192.0.3.0/24",
                "interface": "eth0",
                "id": 3
            }
        ]

    } // end of Dhcp4
}

3. Implementation

The following sections describe the high level implementation of the Shared Networks feature.

3.1. Shared Network Class Hierarchy

The following is a class diagram including classes representing individual shared networks, subnets and all the relations between them.

The Network is the new object holding common properties for subnets and shared networks. These are the properties that can be specified in the shared network and/or subnet scope. This includes subnet selectors, lease lifetime, renew/rebind timers etc. It also provides a common function generating JSON representation of these properties. The Network4 and Network6 object derive from the Network and extend it with the parameters specific for IPv4 and IPv6 case respectively.

The Subnet4 and Subnet6 used to derive only from Subnet class. Now they use multiple inheritance to also derive from the Network4 and Network6 objects. The Subnet object implements all functions required for handling pools and other parameters which aren't allowed to be specified in the shared network scope. The Subnet4 and Subnet6 extend this functionality with IPv4 and IPv6 specific parameters which aren't defined in the shared network scope. All these child object provide their own implementations of the toElement() method to provide complete output of their configurations.

The Subnet is now extended with new functions getSharedNetwork() and setSharedNetwork() which are used to associate a subnet with a shared network they belong to. Each subnet may belong to at most one shared network to which the pointer is stored in the Subnet object. If this pointer is NULL, it means that the subnet is not associated with a shared network.

The Subnet4 and Subnet6 are now extended with the getNextSubnet() convenience functions, which can be used to iterate over all networks belonging to a shared network that the given subnet belongs to (sibling subnets). These methods return NULL when last subnet in the shared network has already been accessed, or when the subnet is not associated with a shared network.

The new SharedNetwork4 and SharedNetwork6 objects implement shared networks. Each shared network has a name. It also includes a collection of subnets associated with it. It is possible to add, delete and retrieve subnets from the shared network objects. When the subnet is added to the shared network it is necessary to update the subnet objects to point to the shared network to which it now belongs (see above). This is challenging because it causes circular dependency between the shared pointers of the Subnet and SharedNetwork4/SharedNetwork6 objects. The circular dependency between shared objects is a well known problem which can be resolved with the use of weak_ptr. In our case, the Subnet holds a weak pointer to the shared network. The shared network holds shared pointer to the subnet. Because the weak pointers do not increase reference counts on the shared network, when the shared network object is destroyed the pointers to the shared network in the associated subnets are set to NULL automatically.

When a subnet is added to a shared network, it is desired to provide a mechanism to automatically update the subnet with a pointer to this shared network. Therefore, both SharedNetwork4 and SharedNetwork6 implement AssignableNetwork interface. The implementations of sharedFromThis() return instances (shared pointers) of shared network objects from the current object. This is used in calls to Subnet::setSharedNetwork() to update the subnets with the pointer to the shared network to which the subnet is added or from which it is removed. The AssignableNetwork provide implementations of setSharedNetwork() and clearSharedNetwork() to achieve that.

setSharedNetwork is a private member of Subnet object because we don't want the shared network pointer to be freely modified within the subnet. This pointer should only be modified by the shared networks (when the subnet is added or removed from the shared network). The AssignableNetwork is a friend of Subnet class, thus allowing access to this private member function for shared network objects.

3.2. Configuration Storage

All configured subnets are stored in the same container, regardless if they belong to a shared network or not. This makes it easy to implement commands which add and remove subnets from the shared networks because there is no need to move subnets between different containers. We also get some other benefits for free, i.e. no need to modify subnet selection mechanism as it continues to use one container for searching for appropriate subnet. Finally, the existing sanity checks on new subnets continue to work for shared networks during server configuration.

However, it is also require to be able to query for all subnets within a shared network, which is not easily implementable using the container holding subnets. Therefore, we need a separate container for shared networks (with dedicated indexes). Each of the shared networks will hold instances of the subnets which belong to them. This means that instances of subnets belonging to shared networks will be duplicated. The implication is for the configuration parsers to duplicate some subnets instances and then remove them from two places when the subnet is deleted, e.g. as a result of receiving a command via REST interface.

3.3. Subnet Selection

Subnet selection mechanism remains unchanged. In the case when the subnet selector is specified at the shared network level (rather than subnet level), the subnets inherit this selector from the shared network during server configuration. Therefore, from the subnet selection perspective, the selectors are always available in the Subnet4 and Subnet6 objects. If the subnet belongs to a shared network, the subnet selected in the "subnet selection" step is usually the first subnet found within this shared network, unless client classification eliminates this subnet for a given client. The changes required for the DHCP servers are minimal. The logic which may dynamically pick a different subnet from the same shared network is made by the allocation engine, based on the following scenarios:

  • Client has host reservations in a different subnet than originally selected,
  • During the lease allocation, the allocation engine determined that there are no more available leases in a given subnet.
  • Client is providing a hint or is requesting assignment/renewal of a lease which belongs to a different subnet (with the same shared network) than originally selected.

The allocation engine has to be extended to take those scenarios into account. For example, to address the first scenario, the allocation engine has to iterate over all subnets within the shared network and retrieve host reservations for each subnet/client pair. The subnets where host reservations are found for a client are preferred over other subnets.

If the client is providing a hint, the allocation engine has to check which subnet this hint belongs to and try to allocate the requested address or another address from the same subnet.

In other cases, the server simply tries dynamic allocation from a current subnet. If allocation fails for any reason, it will try subsequent subnets.

Everytime the new candidate subnet is determined by the allocation engine, the engine updates the ClientContext4/"ClientContext6 structure to point to this new subnet. This is very important, because the DHCP server later uses this information for further packet processing. For example, DHCP options should be assigned from the same subnet from which leases have been assigned. In an IPv6 case, it must be one of the subnets from which the addresses or prefixes have been assigned. Note that the server can generally allocate multiple addresses/prefixes to the same client and in some corner cases they may come from different subnets.

3.4. Lease Selection

Lease database and the backends are not affected by this feature. We're adopting a solution where the allocation engine has to make queries for existing client's leases for each subnet belonging to a shared network. If the leases are found for a subnet, this subnet is preferred for lease allocation/renewal. If there are no leases for a client in any subnets, the allocation engine will simply iterate over the subnets in the shared network until allocation is successful, or until no more subnets are available.

3.5. Client Classification

Client classification in Kea 1.2.0 follows very simple rules:

  • If no classes are specified for a subnet, this subnet can be selected for any client.
  • If classes are specified for a subnet, only clients belonging to any of these classes are allowed.

This evaluation is performed using Pkt::allowClientClass method. The same mechanism is applied for shared networks. Since, allocation engine may now dynamically change a selected subnet based on various criteria, it is critical for the allocation engine to also verify that the subnets it is considering for use, are allowed for the client. The allocation engine will use Subnet::getNextSubnet method to iterate over available subnets. One of the variants of this function will accept a list of client classes as an argument and will omit those subnets which do not match those classes.

3.6. Options

Options can be assigned on a shared network level, subnet level and pool level. The preference of assigning options should be:

  • host level options,
  • pool level options,
  • subnet level options,
  • shared network level options
  • class level options,
  • global options

where global options are least preferred. The more preferred options override less preferred options.

4. Future Enhancements

4.1. Pools

This design assumes that the pools are subnet level resources and it is not possible to specify pools on the shared network level. In the future we will need to implement allocation strategies for single subnet which avoid iterative allocation of addresses and prefixes. In that case it is going to be natural question whether this allocation strategy should be extended to shared networks. In this case, it may be natural to create shared network level pools to make non-deterministic allocations. Currently, if there is no host reservation and the client is not providing a lease hint, the server will generally try to allocate all addresses from the first subnet before it picks next subnet for assignment.

4.2. Host reservations for shared networks

Consider a case when the client should be assign a host reservation for some other parameters than IP address, e.g. hostname, DHCP options, boot file name etc. In this case, the client should be assigned an IP address from the dynamic pool and the reserved parameters. When the client has a reservation in one of the subnets belonging to a shared network the server will find this reservation (and the subnet) and will try to allocate an address from this subnet. If the address allocation is successful, everything will work as expected, i.e. the client will be assigned reserved parameters from its host reservation and the address from the dynamic pool.

However, if there are no more addresses available for a subnet in which the client has a reservation the server will use a sibling subnet for the allocation. As a result, the server will determine that the subnet without a reservation has been used for address assignment and will not return reserved parameters to the client.

To mitigate this problem, it is possible to create multiple reservations for this client, one for each subnet within a shared network. That way, whichever subnet is picked by the allocation engine, the server will always return reserved parameters. This may however poses significant overhead for the administrator to duplicate reservations. It is also error prone, as it requires an action to create a reservation every time a new subnet is added to a shared network.

In the future we should allow specification of host reservations on the shared network level or conditionally allow for including options specified in a different subnet than used for address allocation within the same shared network.

The former solution will require changes in the hosts database schema, but is cleaner. The latter is simpler but may lead to unintended behavior when not well understood by the users.

Shared Network Commands

Proposal for shared network commands is documented here: SharedNetworkCommands.

Last modified 3 weeks ago Last modified on Oct 30, 2017, 2:21:59 PM

Attachments (1)

Download all attachments as: .zip