wiki:DhcpPoolLeaseDesign

Abstract Pool/Lease Storage

This page describes a design for abstract lease and Pool storage design. It is evolving document and its content is expected to change. This work is split into 2 phases: requirements and actual design.

This document is dedicated to the design of a data storage. Configuration aspects are outside of scope.

Introduction

ISC DHCP4 stores all lease, pool and other configuration in memory and stores it in lease and configuration files. Kea is codename for DHCPv4 and DHCPv6 modules in a BIND10 architecture. It is a complete rewrite from scratch. We decided that it would be useful to not tie our new implementation to any specific database, but rather provide an abstract API that defines interactions with such database and configuration storage. This in turn would allow us (and other interested vendors) to implement concrete database backend. Once this abstract API is defined, we are planning to write the first implementation using specific backend. It seems likely that the first backend wll be BerkeleyDB, SQLite or MySQL-based. Other back-ends are envisaged, like MySQL, Postgress, Oracle or even Cassandra.

This page is about requirements for such an API and eventually about API calls details. You are welcome to comment on it, even if you are not BIND10 developer. See related tickets for on-going discussion.

The data necessary to run a DHCP server can be split into two categories: configuration and run-time data. It was debated which parameters should be kept in configuration manager (BIND10 equivalent of configuration files) or in database backend. For each information type (pool4, pool6, lease4 etc.) its intended location is specified. The general rule for this split was that I small amounts of data (in the order of 100 entries or less) that is rarely updated got into the configuration store, the rest goes into the database.

Lease lifecycle

The proposed lease diagram looks as follows:

We do not keep free leases. They will be created when needed. (2 reasons to do this: 1) memory saving and 2) consistent behavior between v4 and v6). Lease that is assigned becomes active (i.e. it is currently used by a client). Depending on the action, an active lease can be Released (if client sent Release), Expired (if client failed to renew it in time) or Declined (if a client or a server detected that lease is used by a third party).

To avoid collecting more and more expired/released leases, a recycle-timer is implemented. After this time, expired or released lease becomes free. This is a configuration parameter. Folks you want to save memory set it to 0. Folks, who want to reassign the same lease to returning client if possible, keep this to a large value.

There should be a second timer for recovery after decline. It should be set to some reasonably large value by default.

Once a lease returns to Free state, it is deleted. That is represented by dotted blue line around Free state.

Option: We can consider keeping a fixed lease that represents a lease reserved to a host. Such fixed lease follows a different lifecycle. It can be in only 2 states: Fixed (when it is reserved for specific host and is currently not used) and Active (when said host requests a lease). After expiration/release/decline, the lease returns to Fixed state.

This lifecycle applies to both v4 and v6.

Leases are expected to be kept in the memory after they are written to DB backend. We may think about an improvement for memory-constrained environment to optionally purge memory and consult DB every time we want to pick a lease.

All timers are considered advanced features and will be implemented at a later date.

Configuration model

The following section describes relationship between specific configuration elements. The UML diagram is done according to UML. It should be read as:

There is only one configuration. Each configuration has 0 or more subnets defined. Each subnet belongs to exactly one configuration. Each subnet has 0 or more pools. Each pool belongs to exactly one subnet. Each subnet has 0 or more hosts. Each host belongs to exactly one subnet. Each pool has 0 or more leases. Each lease belongs to exactly one pool. Each lease may belong to exactly one host. Each host can have 0 or more leases.

TODO: This has to be updated if we decide to go with configuration other than proposed (proposal: host definitions are part of the subnet).

DB information

The following sections describe structures that will contain all configuration and run-time data.

LEASES4

Stored in DB. The following DHCPv4 lease information MUST be stored:

  1. address - IPv4 address;
  2. hw-address - representation of HWADDR field, usually the same as client-id;
  3. client-id - usually the same as hw address, but for some cases (e.g. Infiniband) they are different;
  4. valid-lifetime - expressed in seconds (since cltt);
  5. recycle-time - Expresses when lease can be recycled after release/expiration; see Lease lifecycle section
  6. cltt - client last communication time - required for failover and also for lease expiration;
  7. pool-id - Pool from which the lease was assigned;
  8. bool fixed - Is this lease fixed? If yes, never remove it from DB;
  9. fqdn - client hostname, possibly shared by many leases if client requests two or more addresses.
  10. fqdn_dns - key pointing to separate table with DNS update info (address, TSIG or shared key, AAAA/PTR etc.)
  11. bool fqdn_fwd - was forward update performed? Required during lease release or expiration.
  12. bool fqdn_rev - was reverse update performed? Required during lease release or expiration.
  13. options - a container option for storing extra options associated with the lease (this may eventually be moved to a separate table)
  14. comments

Required operations:

  1. leaseCreate() - adds a new lease
  2. leaseGet() - gets a lease that matches specific parameters.
  3. leaseUpdate() - updates existing lease
  4. leaseDelete() - delete

LEASES6

Stored in DB. The following DHCPv6 lease information (addresses and prefixes) MUST be stored:

  1. enum lease_type - possible values: ipv6_na, ipv6_ta, ipv6_pd;
  2. address - contains IPv6 address;
  3. hw-address - client's MAC address;
  4. client-id - DUID is up to 128 bytes long;
  5. ia-id - in DHCPv6 a lease is identified by client's DUID+ia-id;
  6. prefix-length - used only in ia_pd;
  7. preferred-lifetime - expressed in seconds (since cltt);
  8. valid-lifetime - expressed in seconds (since cltt);
  9. recycle-time - expressed in seconds (since expiration/release). Defines when a lease can be recycled after release/expiration; see Lease lifecycle section
  10. cltt - client last communication time - required for failover;
  11. pool-id - Pool from which the lease was assigned;
  12. bool fixed - Is this lease fixed? If yes, never remove it from DB.
  13. fqdn - client hostname, possibly shared by many leases if client requests two or more addresses.
  14. bool fqdn_fwd - was forward update performed? Required during lease release or expiration.
  15. bool fqdn_rev - was reverse update performed? Required during lease release or expiration.
  16. options - a container option for storing extra options associated with the lease (this may eventually be moved to a separate table)
  17. comments - just a text field

Required operations:

  1. leaseCreate() - adds a new lease
  2. leaseGet() - gets a lease that matches specific parameters.
  3. leaseUpdate() - updates existing lease
  4. leaseDelete() - delete

SUBNET4/SUBNET6

Stored in configuration manager.

  1. subnet-id (a unique value that is used to refer to this subnet instance) (we can relax the uniqueness parameter for shared subnets)
  2. network address/prefix length
  3. comment

Required operations:

  1. subnet4Create() - adds a new subnet4
  2. subnet4Get() - gets a subnet4 that matches specific parameters (e.g. that contains specific address)
  3. subnet4Update() - updates existing subnet4
  4. subnet4Delete() - delete subnet4
  1. subnet6Create() - adds a new subnet6
  2. subnet6Get() - gets a subnet6 that matches specific parameters.
  3. subnet6Update() - updates existing subnet6
  4. subnet6Delete() - deletes one instance of subnet6

Tomek: I think that is should be possible to specify parameters here (e.g. there may be several pools in one subnet that share the same parameters).

Subnet may contain zero or more pools. Subnet may contain zero or more fixed leases (host reservations).

TODO: we will have to define a shared_subnet eventually. One way we could do that is to use some form of identifier specified for each subnet. Subnets with the same identifier are shared. An example of such identifier would be an interface-id option, used in DHCPv6.

POOL4

POOL4. Stored in configuration manager. The following nformation about v4 address pools MUST be stored:

  1. pool-id (a unique value that is used to refer to this pool instance)
  2. first, last (should really be named first_address, last_address, but kept lease convention for v4/v6 consistency)
  3. t1 timer - specifies renewal time (actually 3 values: t1_min, t1_max, t1_default)
  4. t2 timer - specifies rebinding time (actually 3 values: t2_min, t2_max, t2_default)
  5. valid-lifetime - specified lifetime of a lease (actually 3 values: valid_min, valid_max, valid_default)
  6. available-leases - number of remaining available leases.
  7. used-leases - number of currently assigned leases from that pool
  8. comments

For triplet option (t1,t2, valid-lifetime) we allow either specifying just the default value (min and max equals the default then) or specifying all three. This avoids tricks in inheritance from

Useful structure: iasubopt (see includes/dhcpd.h in ISC DHCP)

Note: netmask and gateway are really options, but they are closely related to used pool, so they are kept here.

Required operations:

  1. pool4Create() - adds a new pool4
  2. pool4Get() - gets a pool4 that matches specific parameters.
  3. pool4Update() - updates existing pool4
  4. pool4Delete() - deletes one instance of pool4

POOL6

POOL6. Stored in configuration manager. The following information about v6 address and prefix pools - addresses (IA_NA), temporary addresses (IA_TA) and prefixes (IA_PD) - MUST be stored:

  1. pool-id (a unique value that is used to refer to this pool instance)
  2. pool type (IA_NA, IA_TA or IA_PD)
  3. first, last (first address, last address for IA_NA, and IA_TA; first prefix, last prefix for IA_PD)
  4. prefix-length (for IA_PD only)
  5. available-leases - number of remaining available leases.
  6. used-leases - number of currently assigned leases from that pool
  7. t1 timer - specifies renewal time (actually 3 values: t1_min, t1_max, t1_default)
  8. t2 timer - specifies rebinding time (actually 3 values: t2_min, t2_max, t2_default)
  9. preferred-lifetime - specifies preferred lifetime (actually 3 values: preferred_min, preferred_max, preferred_default)
  10. valid-lifetime - specified lifetime of a lease (actually 3 values: valid_min, valid_max, valid_default)
  11. valid-lifetime
  12. Comments

Required operations:

  1. pool6Create() - adds a new pool6
  2. pool6Get() - gets a pool6 that matches specific parameters.
  3. pool6Update() - updates existing pool6
  4. pool6Delete() - deletes one instance of pool6

Subnet and pool definitions are in configuration manager for 2 reasons. First, in a typical installation number of subnets is small. Even in the largest deployments, number of subnets rarely extends beyond 100. Second reason is more practical - when configuration changes, server is able to do sanity checks before accepting configuration.

DDNS

The DDNS component of the system is the subject of a separate set of requirements and design.

In brief, information about a DDNS update is stored in the system and is associated with the relevant lease. It is retained for the duration of the lease, this being the case even after a configuration change; it is required when attempting to clean up old leases. DNS records can be safely removed when there are no leases associated with it.

HOST

HOST. (stored in DB). The following information about fixed host reservations MUST be stored. We can't really keep this associated leases table, because sometimes a host reservation does not have a lease, e.g. when user wants to provide custom options to a given host.

  1. address - optional pointer to IPv4 lease (if there is a fixed lease for this host)
  2. address6 - optional pointer to IPv6 lease (if there is a fixed lease for this host)
  3. prefix6 - optional pointer to Prefix Delegation in DHCPv6 (if there is a fixed lease for this host)
  4. hostname -
  5. options - specify custom options that may be provided to this client
  6. some kind of unique host identification - MAC address, client-id, remote-id, subscriber-id etc.
  7. comments

Useful structure: ipv6_pool (see includes/dhcpd.h in ISC DHCP)

Required operations:

  1. hostCreate() - adds a new host
  2. hostGet() - gets a host that matches specific parameters.
  3. hostUpdate() - updates existing host
  4. hostDelete() - deletes one instance of a host

Extra Features

The following section contains list of optional extra features. They are optional in a sense that server can operate without them, but its operation may be limited in some way. For example lack of "lease consistency" feature results in inability to gently remove pools, introduce server data inconsistency, introduce server-client inconsistency or a mix of all mentioned (depending on actual implementation). Those features are likely to be implemented at a later date.

  1. Data consistency. Pool deletion. Pool cannot be deleted immediately, but is rather marked for deletion. Existing leases in such a pool are not extended. No new leases are created in such a pool. If number of leases within a pool marked for deletion decreases to zero, pool is deleted.
  2. DDNS clean up. When DDNS information changes, we sometimes may not be able to clean up properly. Depending on the nature of the change (a different server without any previous updates or just a switch to a new key), complete cleanup may be doable or not. There are 2 proposals here:

a) DDNS update info is stored in the configuration. Upon first DDNS action, an entry in DDNS table is created based on current configuration. Each lease points to a specific DDNS info entry. Each DDNS entry consists of (id or name, server address, key, algorithm) tuple. Changing any parameter creates a new entry. Old entry is removed once all on-going updates conclude and there are no leases using this entry. When configuring new DDNS info, we allow 2 types: migrate (add new entry and allow the old one to gradually expire) or replace (overwrite old configuration immediately, e.g. retire old key now). The default should be replace. b) We allow more than one DDNS info tuple to be defined (similar to primary/primary6/secondary/secondary6 in DHCP4).

  1. DDNS clean up failures. In case of DDNS removal failure, there should be a mechanism for logging it into a separate data store (perhaps simple csv will suffice). That would be useful for admins for manual removal.

Open questions

Q1: Do we want to keep separate list of fixed-leases? My proposal is no. Keep them in lease DB, but mark them as fixed. Fixed leases do not expire and are never removed from the DB. It is even ok with for most FQDN cases: if client has 2 addresses, the same name is associated with both addresses. The only problem remaining to be solved is name reservation, i.e. admin wants to register name for specific host (e.g. by specifying its MAC or DUID). There is no clear way to do this with this simplified approach. Therefore perhaps host database should be kept separated. We can make it simpler, though.

A: We decided to use separate host table for that as sometimes host reservation include other things, possibly even without the actual lease itself, like extra options (or different values for regular options).

Q2: Do we want IPv4 and IPv6 lease structures unified? (tomek: keep them separated)

A: Let's keep them separated for now. If we decide to unify them, we need to merge field lists.

Q3: Do we want to have subnets and pools split? We could do without subnets. That would make our configuration simpler. Essential part of any more complex configuration is definition of subnets (and pools within those subnets). There are 2 approaches possible:

a) we require full network topology to be described. That has benefit of server knowing complete network topology, but has drawback of making configuration more complex. An example is user complaint "but I only want to server pool X/Y on interface eth0, I don't care about any other interfaces!". This approach is used in ISC DHCP.

b) we specify only what we want to serve, eg. pool X/Y over eth0, pool Z/W over relay with relay-id ABC etc. This approach is simpler. Server does not have complete topology information, just the minimum necessary to serve clients. This approach is used in Dibbler. It has a drawback of not being able to fully support some features, like confirm. Server is supposed to answer if specified address is valid on a given link, not if it has valid leases for it. To do this, full network topology knowledge is required. But we can by-pass this problem, but adding authoritative flag, similar to DHCP4. Stephen and Tomek favor simpler approach.

A: tomek: I used to think that we may go with pools only, but now I believ we will need both subnet and pool eventually. Subnet defines that certain subnet is available on-link. That is required for 3 things:

  • in v6 decide which pool to use, when we have link address of the relay only (relay includes a global address that can be used to identify a link, where client is attached). This, by definition is not a dynamically assigned address, so it will always be outside of any pool. Therefore we need subnets.
  • when handling CONFIRM, server is supposed to answer a question "is this address valid for this link?", not "do you have binding for this lease?". To properly answer that question, server must have knowledge about full subnet on link, not only its dynamically allocated subset.
  • it is easier to handle out of pool reservations as parameters can be defined once for the whole pool, rather than

Given those two facts, we must use both subnet and pool. We may also allow simpler more (with subnets only) for folks, who does not want to use full configuration, but are happy with simpler configuration. (many users don't use confirm and they use different properties to pick pools, e.g. remote-id or subscriber-id).

Q4: Do we want to store information in human readable form? That is useful for user interaction, but is slower (require inet_pton and inet_ntop or similar). Recommendation: go with human readable content in DB, keep on-wire representation in memory.

A: Stephen pointed out that human readable data can be bad for some operations, e.g. 10.25.0.0 sorts after 10.192.0.0. We chose binary data and only convert to human readable if necessary.

Q5: We need to think about lease clean-up in DNS updates context. How much configuration changes do we allow before giving up and saying that we cannot clean up anymore? If DNS configuration changes, do we keep track of old IPs and do clean-up updates over old IPs? What if secret-key changes? Do we keep old key or use the new one? If key changes, how do we reference it? We need the same key name? What do we want to keep in the lease information? Reference to dns-address and key? Perhaps create new table with DNS update info that will keep dns-address, key, tcp/udp, ipv4/ipv6, fallback order etc.? tomek: After Stephen's suggestion, I decided to design a separate table for DNS update information.

Q6: Do we want to specify fixed values or ranges for timers like t1, t2, preferred or valid lifetimes? Dibbler-server allows specifying ranges, e.g. accept valid-lifetimes 3600 to 7200 seconds. Some clients send those values as hints. Server could use those.

A: The answer was yes.

Q7: Entries from DNS Update info may be removed when there is no more leases that used that specific entry. Do we want to do reference counting? or do we call "select count(*) from leases where dns-id=X" after every update?

A: We can read counts during start (using aforementioned select) and keep them in memory. No need for select during each DNS Update.

Q8: Which data are we going to keep in DB and which in configuration store? This answer will be answered once we get good understanding of data layouts. For now, very tentative assumption is that if some information is quantitatively small and is rather rarely updated (e.g. big deployments use over 100 pools) it goes into configuration storage. Otherwise (frequently updated lease table) it goes into DB backend.

A: Proposed layout was discussed (over mails, tickets and during meetings) and approved.

Q9: Reserved leases? Do we allow reserved/fixed leases to be within pools? If all fixed leases are outside of pool, we don't need to inspect all fixed leases before assigning a random lease from pool. On the other hand, for in-pool fixed leases, it is easier to share extra options, e.g. gateway address.

A9: We need subnets for reserved leases to work reasonably well. Otherwise we will hit one or two of the following bottlenecks. Note that in-pool and out-of-pool reservations are not mutually exclusive. We could allow zero, one of them or both.

  • If we choose to support in-pool reservations, we will need to check a global list of reservations for each new lease we create. This introduces performance degradation. We could somewhat minimize the problem by splitting global reservation into separate reservations per pool.
  • If we choose to support out-of-pool reservations, we gain in flexibility, but we loose in configuration complexity. As out of pool reservation may be completely out of the blue, each would need to have specified complete set of parameters (lifetimes, gateway, dns servers etc.) That would be highly unpleasant for administrators.

We have decided to go with pools consisting of dynamic addresses and hosts being out of pool. See "Kea PoolLeeaseDesign?: configuration proposals" started 2012-07-09.

Q10: It is tempting to say that any changes to lease and host data must be consistent with current configuration of pools. But do we want the opposite as well? What happens when user removes pool that has active leases in it? Do we keep the leases or toss them immediately? It would be better to keep them and remove them the next time client renews. (Removing a lease will trigger DDNS removal, while client is convinced that his lease is still valid and would be surprised that its FQDN stops resolving).

A: Tomek: I think we should not allow immediate removal of a lease. I propose following mechanism: once the pool is marked for removal, it is removed immediately, if the number of active leases is 0. Otherwise it is marked for removal. No new leases are allowed and existing leases are not extended. This will allow for gentle pool removal. See Data consistency in Extra features.

Q11: Do we want to formalize pool selection algorithm? Dibbler has a mechanism that we can consider. If there are two or more pools valid for a given assignment request, they are equally likely to be picked. However, use can assign weights to them, e.g. first pool is 100 and second is 200. In that case first pool has 100/(100+200)=33% changes of being picked and the second has 200/(100+200)=66%. This pool balancing is excellent way of gentle network renumbering. User defines new pool and assigns 0 weight to the old one. New leases will be assigned only from the new one and the old one will gradually stop being used when clients release/expire their leases. We can also do pool balancing based on number of remaining leases in pool. On a related note, our DhcpHooks document is silent about pool selection. We should probably add extra hook for that purpose. Added a note about it in open question in hooks design.

Yes. We need pool selection algorithm. John Brzozowski requested this feature. Reusing weighted algorithm from Dibbler is viable as an option as example usage.

Q12: How do we specify DDNS info? Is it global? Per interface? Per subnet/pool? Proposed answer: start with global. Can later add support for per subnet if needed.

Q13: Parameters hierarchy. It seems that we now have global scope with one or more subnets defined. Each subnet may have zero or more pools and zero or more fixed-leases. There are valid reasons to define parameters like T1,T2, preferred or valid lifetimes on each of those levels. Therefore we probably will need to support inheritance of some time (e.g T1 defined in global scope applies to all subnets, pools and fixed-leases, unless overwritten by more specific definition).

A: That is a configuration aspect. It will be described in a separate document. For now, it is likely that we will have following scopes:

  • global
  • subnet
  • pool (this scope is discussed)
  • host

Design/Scratchpad?

See ticket #1648 comments.

  • Are there requirements on detecting overlapping pools or other inconsistencies - that question is carried over to configuration design.
  • What reporting of pool/lease information is required?
  • How is information added to/retrieved from/updated in the underlying data source? Proposed access methods added.
  • Differences between V4 and V6 implementations
  • What sort of provisions are needed for failover? (Failover is not part of the current work; however, an initial assessment of likely requirements is needed so the design is flexible enough to meet them.)
Last modified 5 years ago Last modified on Sep 24, 2012, 10:11:22 AM

Attachments (2)

Download all attachments as: .zip