wiki:DnsBenchmarks

Standard Benchmarks for DNS

The purpose of this wikipage is to define the standard and specific DNS benchmarks we will use for measuring BIND 10 performance improvements over time. This page lists the benchmarks and point to their graphs, or if not complete, refers to ticket numbers for tasks to be done. It also points to the procedures for generating the benchmark results on the dedicated benchmarking machine and on other systems. The baseline goals are created using BIND 9.9.0 on our dedicated test system (if has two goals, then the second is for optimum multiple cores).

Other ideas, not official, not described, or not implemented yet may be listed on the DnsBenchmarkIdeas wiki page.

Options

  • authoritative: different datasources backends, including built-in and disabling caching
  • authoritative: different amounts and sizes zones
  • authoritative: types of queries: nxdomain, nsec, nsec3, query types, sizes of responses, successful queries, etc.

Scenarios

Built-in bind.

http://git.bind10.isc.org/~jreed/bench/releases/builtin.png

  • Known as "builtin".
  • This may be used as a baseline for comparing with datasources.
  • REFUSED: query list contains 20 unique non-existent names, 5 different types (A, NS, AAAA, TXT, MX) randomized.
  • REFUSED: query list contains SOA (only)
  • Success: version.bind TXT.

Authoritative server for 1 zone (with 0 delegations and 3 records)

http://git.bind10.isc.org/~jreed/bench/releases/smallzone.png

  • Known as "smallzone"
  • Options include: sqlite3 datasource (with default caching), sqlite3 database without caching, and in-memory datasource
  • NXDOMAIN: query list contains 2 unique non-existent names, 3 different types (SOA, NS, A) randomized. Goal: 44521 qps, 73806 qps
  • SOA (only) (used as a comparison with NXDOMAIN's SOA response). Goal: 39451 qps, 72313 qps
  • Success: 2 unique existing names and 3 existing types, randomized (TODO: split this up to benchmark each individually). Goal: 41508 qps, 71060 qps

Authoritative server for 1 zone (with 312 delegations and 4206 records)

http://git.bind10.isc.org/~jreed/bench/releases/root.png

  • Known as "root"
  • Test a delegation.
  • Options include: sqlite3 datasource (with default caching), sqlite3 database without caching, and in-memory datasource
  • NXDOMAIN: query list contains 2 unique non-existent names, 3 different types (SOA, NS, A) all randomized. Goal: 61917 qps, 71846 qps
  • SOA (. only) (used as a comparison with NXDOMAIN's SOA response). Goal: 58366 qps, 62057 qps
  • Success: www.google.com A. Goal: 50519 qps

Authoritative server for 10,000 zones (of 0 delegations and 15 records)

  • Small hosting company model

Authoritative server for 100,000 zones (with 0 delegations and 15 records)

http://git.bind10.isc.org/~jreed/bench/releases/largehost.png

  • Known as "largehost"
  • Large hosting company model
  • Options include: sqlite3 datasource (with default caching), and in-memory datasource (later TODO: sqlite3 database without caching)
  • NXDOMAIN: query list contains four unique non-existent names within 3 different zones (2 not existing), 3 different types (NS, SOA, A) randomized. Goal: 61209 qps
  • SOA (only): query list contains 2 existing zone names (used as a comparison with NXDOMAIN's SOA response). Goal: 59817 qps
  • Success: query list contains 4 existing labels, 3 different types (A, SOA, NS). Goal: 60294 qps

Authoritative server for 1 zone (with 8 million delegations and ~18 million records)

http://git.bind10.isc.org/~jreed/bench/releases/largetld.png

  • Known as "largetld"
  • Large TLD model; one huge zone for delegations
  • Options include: in-memory datasource (later TODO: sqlite3 database with and without caching)
  • NXDOMAIN: query list contains three unique non-existent names, 3 different types (NS, SOA, A). Goal: 61209 qps
  • SOA (only): query list contains 1 existing zone names (used as a comparison with NXDOMAIN's SOA response). Goal: 59817 qps
  • Success: query list contains 5 existing labels, 3 different types (A, MX, TXT). Goal: 60294 qps

Setup

bind10-testing1.lab test system is a CentOS 6.5 with eight Intel Xeon CPU E5520 @ 2.27GHz (actual 2260.887 MHz). Each has 4 cpu cores. It has around 3.8GB memory. It has two Intel PRO/1000 network interfaces. The dmesg output is at http://git.bind10.isc.org/~jreed/dmesg.bind10-testing1.20140122.txt

Last modified 4 years ago Last modified on Jan 22, 2014, 1:52:39 PM