wiki:DiagnosticsRequirements

Requirements for Kea Diagnostics Features and Logging Enhancements

At the time of writing this document, the scope of work for Kea 0.9.2 included "Basic Statistics Gathering", "Logging and Diagnostics" and the "Design for Remote Management API". The requirements for "Basic Statistics Gathering" is located here. The requirements for "Remote Management API" was not available at the time of writing this document. This document presents the requirements for "Logging and Diagnostics". The separate document presenting a Design for Diagnostics and Logging Enhancements will describe the implementation details to fulfil the requirements presented herein.

Background

Kea is the implementation of the DHCP servers supporting wide range of features and will be deployed in various networks, having different requirements for the number of supported hosts, used protocols (DHCPv4, DHCPv6, DDNS), performance, types of storage for lease information (database, flat file) etc. Troubleshooting issues with the DHCP server operation is usually an involved task and often requires a good knowledge of the specifics of the particular deployment. Troubleshooting becomes even more complex when there is some additional/proprietary software attached to the server. Kea users may implement dynamically loaded libraries, a.k.a. hook libraries, which functions are called at certain stages of the DHCP servers operation to customize DHCP message processing. These hook libraries are not maintained or tested by ISC engineers, so it is critical for the server administrator and ISC staff to easilly diagnose if the source of the problem, observed by the administrator, resides in the code maintained by ISC or in the proprietary code. This allows for selecting the engineers who are the most knowledgable about the culprit code to troubleshoot the problem. It also shortens the time required to narrow down the source of the problem.

The purpose of this document is to identify the requirements for the Kea servers to facilitate troubleshooting, by providing maximum amount of diagnostic information to the user and/or ISC staff about the problem, without the need to use specialized tools, like debuggers. It also includes requirements for the tools which automate gathering the diagnostic information in the possibly most readable form.

The implementation of these requirements will not replace the advanced troubleshooting methods (debugging). It is merely meant to facilitate the initial stage of troubleshooting, which is based on analysis of server log files, its configuration, leases information, core dump etc.

Specific Requirements

The use of MUST and SHOULD keywords in the list of requirements is to differentiate the importance of these requirements. Those for which SHOULD is used can be treated as extensions. The requirements for which MUST is used formulate the core functionality.

Kea MUST provide the command line tool for gathering the full lease information from supported lease databases in the unified form.

  1. The tool MUST support gathering leases information from the MySQL database.
  2. The tool MUST support gathering leases information from the PostgreSQL database.
  3. The tool MUST output lease information to the CSV file having similar format to the lease file created by the Memfile backend. The formatting of some values MAY differ between the lease file produced by the Memfile backend and the output from the tool.
  4. The tool SHOULD parse the Kea configuration file to obtain the required information to connect to the lease database being a source of lease information.
  5. The tool MUST NOT remove any lease information when transfering this information from the database to the file. For example: it MUST NOT remove expired leases.
  6. The tool SHOULD report the number of gathered leases on exit.
  7. The tool SHOULD present the time duration between the beginning and end of its operation.
  8. The tool MUST be easily extendable to support future lease database backends.

Logging messages by the Kea servers.

  1. Kea servers MUST output log messages to indicate that control is passed to the callout. These messages will not be logged when logger is not configured to output messages at their severity.
  2. Logging message indicating that the control is passed to the callout MUST include hook name and the input parameters for the callout in the readable form. These messages will not be logged when logger is not configured to output messages at their severity.
  3. Kea servers MUST output log messages to indicate that the control has been returned by the callout.
  4. Logging message indicating that the control has been returned by the callout MUST include the hook name, the result returned by the callout and the callout execution time.
  5. Log messages MUST include process id.
  6. Log messages MUST include thread id when multiple threads are present.
  7. Kea MUST log all significant decision points during packet processing and include the data which led to this decision in the readable form. (Also see notes below).
  8. Kea MUST log when the significant logical unit of work during the packet processing has ended. (Also see notes below).
  9. Messages logged by the DHCP servers MUST contain unique client identifiers (e.g. HW address, DUID) and transaction id.
  10. Messages logged by the D2 and related to the NameChangeRequets processing MUST include both FQDN and client's IP address.
  11. DHCP servers MUST log the reason for dropping the message.
  12. DHCP servers MUST log when they find unknown message and include message type.
  13. DHCP servers MUST log when when they find unknown option and include option code.
  14. DHCP servers MUST include interface name and information about relay agents used to transmit the message to the server when logging information concerning the issues with the received DHCP messages.
  15. Different logical units of code SHOULD use different loggers to facilitate using different logging settings (e.g. severity) for the specific messages.
  16. It SHOULD be possible to use a single configuration entry in the configuration file to setup all loggers used by the server and callouts.

Documentation

  1. Developer Guide section about the Hooks development MUST include the recommendations for implementers regarding the use of logger, i.e. placement of log messages, contents of the log messages.

Other requirements

This section include requirements proposed during the review of this document, which we have decided to defer for future releases:

  1. Server MUST be able to dump all packets received and sent by the server into the binary file (packet capture). This may be served by the hook library.
  2. Server MUST be able to dump traces from all communication with different entities, e.g. D2 module, lease database.
  3. Hooks API SHOULD initiate and provide the unique logger for each loaded library.

Notes

Kea includes a number of utility libraries and some of them have dedicated loggers. For example, the libhcpsrv library contains the collection of C++ classes used by both DHCPv4 and DHCPv6 servers to perform common operations like configuration parsing, leases allocation etc. It is desired that logging messages output from these libraries contain as much information about the DHCP transaction as possible, so as they can be associated with the specific DHCP messages received by the server. When possible, the library should include the same parameters to identify the DHCP transactions as those used by the server process, e.g. the allocation engine should include DUID and transaction id. However, trying pass these parameters between all library functions may be an overkill. Thus, the requirements presented above refer mostly to logging messages output by the server program (dhcp6_srv.cc), rather than libraries.

Last modified 3 years ago Last modified on Apr 15, 2015, 10:32:58 AM