wiki:HowtoAddStatisticsItems

How to add statistics items

  1. Introduction
  2. Current model around the stats module
  3. Steps with sample codes
  4. Current problems
  5. Future plan

Introduction

  • This is a simple memo which explains how to add statistics items into each module.
  • This memo is valid in the master branch. (The buildbot failure was fixed and trac1175 was merged with the master.)

Current model around the stats module

+----------+                  cc-session
| Mudule A | cc-session           +-------------+
+----------+ ---> +-------+ <---- | stats httpd | <--- HTTP requests
+----------+      | stats |       +-------------+
| Module B | ---> +-------+  cc-session
+----------+                <---- +--------+
                                  | cmdctl |
                                  +--------+
  • Each module sends statistics data to the stats module.
  • Cmdctl and the stats httpd get the statistics data collected by stats module.
  • The stats httpd accepts HTTP requests.

Steps with sample codes

  1. Define the statistics items in the spec file.
    • Statistics items are defined in the spec file of each module. (Statistics category was newly added in trac928.)
    • This is an example spec file where per-zone counts of auth module are added.
      {
        "module_name": Auth,  /* The name of module which owns the statistics data */
        ...
        "statistics": [
          { ... },
          {                   /* An example of expression of per-zone counter */
            "item_name": "queries.per-zone",
            "item_type": "list",
            "item_optional": false,
            "item_default": [ /* An example of default. */
              {               /* (Real all zones don't have to be described in this example.) */
                "zonename": "test.example",
                "queries.tcp": 0,
                "queries.udp": 0
              }
            ],
            "item_title": "Queries per zone",
            "item_description": "Queries per zone",
            "list_item_spec": {
              "item_name": "item",
              "item_type": "map",
              "item_optional": false,
              "item_default": {
                "zonename": "test.example",
                "queries.tcp": 0,
                "queries.udp": 0
              },
              "map_item_spec": [
                {
                  "item_name": "zonename",
                  "item_type": "string",
                  "item_optional": false,
                  "item_default": "test.example"
                },
                {
                  "item_name": "queries.udp",
                  "item_type": "integer",
                  "item_optional": false,
                  "item_default": 0
                },
                {
                  "item_name": "queries.tcp",
                  "item_type": "integer",
                  "item_optional": false,
                  "item_default": 0
                }
              ]
            }
          }
        ]
      }
      
  2. Implement make the statistics data, check the format and send it to stats module
    • This is an example code in python.
      # Set owner name.
      stat_data["owner"] = "Auth"
      
      # Set statistics data. Actually, the statistics data may be given by
      # some variables in the module.
      stat_data["data"] = { "queries.per-zone": [
          { "zonename": "example.net", "queries.tcp": 12, "queries.udp": 1234 },
          { "zonename": "example.com", "queries.tcp": 34, "queries.udp": 5678 }
      ]
      
      # Verify the format of the statistics data by the
      # "validate_statistics" method given by the "module_spec" object.
      valid = self.mccs.get_module_spec().validate_statistics(True, stats_data["data"])
      
      # Send the statistics data if the format is valid
      if valid:
          cmd = isc.config.ccsession.create_command('set', stats_data)
          seq = self.cc_session.group_sendmsg(cmd, 'Stats')
          try:
              self.cc_session.group_recvmsg(False, seq)
          except isc.cc.session.SessionTimeout:
              pass
      else:
          logger.fatal(AUTH_INVALID_STATISTICS_DATA);
      
  3. Check its works on bindctl
    • Values and new definitions can be viewed by the commands in bindctl.
    • The command "show" is for viewing current values of statistics data. The command "showschema" is for viewing the definition of the statistics items.
    • This is an example image in bindctl.
      > Stats show // this command displays values of the statistics data
      {
        "Auth": {
          ...
          "queries.per-zone": [
            {
                "queries.tcp": 12,
                "queries.udp": 1234,
                "zonename": "example.com"
            },
            {
                "queries.tcp": 34,
                "queries.udp": 5678,
                "zonename": "example.net"
            }
          ]
        },
        ...
      
      > Stats showschema // this command displays same as content of the spec file
      {
        ...
        "Auth": [
          {
            "item_name": "queries.per-zone",
            "item_type": "list",
            "item_optional": false,
            "item_default": [
              {
                "zonename": "test.example",
                "queries.tcp": 0,
                "queries.udp": 0
              }
            ],
            "item_title": "Queries per zone",
            "item_description": "Queries per zone",
            "list_item_spec": {
              "item_name": "item",
              "item_type": "map",
              "item_optional": false,
              "item_default": {
                "zonename": "test.example",
                "queries.tcp": 0,
                "queries.udp": 0
              },
              "map_item_spec": [
                {
                  "item_name": "zonename",
                  "item_type": "string",
                  "item_optional": false,
                  "item_default": "test.example"
                },
                {
                  "item_name": "queries.udp",
                  "item_type": "integer",
                  "item_optional": false,
                  "item_default": 0
                },
                {
                  "item_name": "queries.tcp",
                  "item_type": "integer",
                  "item_optional": false,
                  "item_default": 0
                }
              ]
            }
          }
        ],
        ...
      }
      

Current problems

There are currently two problems on stats httpd.

  • The current stats httpd cannot properly handle a list-type item like such per-zone count. It should be fixed.
  • The current stats httpd can display only full statistics items. (It's unable to specify the item to be displayed.) This task is registered to #917.

Future plan

For easier implementation for addition of statistics, we need to add more flexible libraries for statistics and to change some existent libraries, e.g. ModuleCCSession. The following is just an idea and currently not implemented yet.

  • No need to implement sending to stats by each module, just calculate(count) the statistics data.
  • Create a statistics object from ModuleCCSession and set the data in the setter function given by the object. This function can send the specified data immediately or periodically. This option can be specified in creating the statistics object.

This is a code image in python about the above idea.

# Create ModuleCCSession with SPECFILE_LOCATION which the statistics
# items are defined in.
mccs = ModuleCCSession(SPECFILE_LOCATION, ....)
...
# Create statistics object from ModuleCCSession and also specify
# interval_timeout optionally. If the option is set, the statistics
# data is send to the stats module periodically. (This example sends
# every 60 seconds.)
stat = mccs.create_statistics(interval_timeout=60)
...
# Calculate a statistics data. e.g. count the queried counts.
some_statistics = ...
...

# Set the statistics data into the method given by the statistics
# object. Then, for example, the argument may be like a dictionary
# type having pairs of item name and item value. Then it will verify
# and send the specified data. Finally, statistics data is sent
# immediately or after interval_timeout seconds.
stat.set_statistics(some_statistics)
Last modified 6 years ago Last modified on Oct 14, 2011, 5:08:05 AM