Opened 5 years ago

Closed 5 years ago

#2871 closed task (complete)

Fake work for resolution

Reported by: vorner Owned by: vorner
Priority: medium Milestone: Sprint-20130514
Component: resolver Version:
Keywords: Cc:
CVSS Scoring: Parent Tickets:
Sensitive: no Defect Severity: N/A
Sub-Project: DNS Feature Depending on Ticket:
Estimated Difficulty: 4 Add Hours to Ticket: 0
Total Hours: 6.19 Internal?: no

Description

Define set of functions and data structures that would form a fake resolution. Something like:

receive();
parseQuery();
renderQuery();
lookIntoCache();
upstreamQuery();
send();

They'd do no useful work, but would contain some workload (like calling an empty function 10 million times, or something). Try to guess the balance somehow.

The goal is to test multi threading approaches and we need some workload to test it with. This should be as close to the real resolver as we can get without actually writing the resolver.

Some detail can be found in https://lists.isc.org/pipermail/bind10-dev/2013-March/004504.html.

Maybe illustrate how to call the functions too, for example by implementing a single-process single-threaded blocking resolver by them (eg. no switching between tasks, pick up one, handle and wait for the upstream answers in blocking mode).

Subtickets

Change History (11)

comment:1 Changed 5 years ago by muks

  • Estimated Difficulty changed from 0 to 4

comment:2 Changed 5 years ago by muks

  • Milestone changed from New Tasks to Sprint-20130423

comment:3 Changed 5 years ago by vorner

  • Owner set to vorner
  • Status changed from new to accepted

comment:4 Changed 5 years ago by vorner

  • Owner changed from vorner to UnAssigned
  • Status changed from accepted to reviewing

It should be ready for review.

I defined a fake query that has several tasks on it, and it can be examined what the tasks are (so the benchmark can do locking, etc). A fake interface for „receiving“ the queries was done too.

I also included the naive approach as an example. It has something like 26 QPS with the tweak numbers in fake_query.cc.

I don't think this needs a changelog entry, as the code is not installed and for user. For the same reason, the documentation may not be polished as much and I didn't worry too much about exception safety.

comment:5 Changed 5 years ago by pselkirk

  • Owner changed from UnAssigned to vorner

I'm sorry, but where is the code to review? The only thing I find in the trac2871 branch is

* dd0c9ba [2871] Empty benchmark binary

which doesn't have a fake_query.cc.

comment:6 Changed 5 years ago by vorner

  • Owner changed from vorner to pselkirk

Are you sure? Did you fetch before checkout, to download latest snapshot?

Everything I try here indicates the upstream repository does have the complete branch, ending with

commit e4870b2ea373a314214f1ff575aa675496557b00
Author: Michal 'vorner' Vaner <michal.vaner@nic.cz>
Date:   Mon Apr 8 14:17:09 2013 +0200

    [2738] Tweak the times and iteration counts
    
    So even the naive approach terminates some day.

comment:7 Changed 5 years ago by pselkirk

Strange. My local repo is all kinds of messed up. I did a fresh clone, and I see your changes. Reviewing now.

comment:8 follow-up: Changed 5 years ago by pselkirk

  • Owner changed from pselkirk to vorner

Overall, code looks sane, but a few small things stand out:

  • Copyright date in 6 of the new files is 2009, and copyright in the existing files has not been updated to 2013.
  • The last 3 commits on this branch have messages that start with "[2738]", which seems unrelated to this ticket.
  • The existence of random factors (for cache hit, and especially for upstream query completion time) makes this less of a rigid benchmark than a statistical exercise. OTOH, over the course of 1000 iterations per run, it smooths out pretty well - I found less than 0.5% standard deviation over 12 runs.

comment:9 in reply to: ↑ 8 ; follow-up: Changed 5 years ago by vorner

  • Owner changed from vorner to pselkirk

Hello

Replying to pselkirk:

  • Copyright date in 6 of the new files is 2009, and copyright in the existing files has not been updated to 2013.

Hopefully updated. I copy-pasted the headers, which is obviously wrong from time to time. I hope the lawyers wouldn't make us add the header nobody reads anyway…

  • The last 3 commits on this branch have messages that start with "[2738]", which seems unrelated to this ticket.

I created a new branch, where this should be fixed. Look at trac2871_2.

  • The existence of random factors (for cache hit, and especially for upstream query completion time) makes this less of a rigid benchmark than a statistical exercise. OTOH, over the course of 1000 iterations per run, it smooths out pretty well - I found less than 0.5% standard deviation over 12 runs.

Does this matter for the purpose? I mean, if there are enough samples (generated queries, or whatever), it should just converge to be something almost same every time. And we are looking for large differences in the models anyway (we'll get more inaccurate by the fact we use the fake work instead of real resolution, than from the averages).

But if you have concrete idea how to solve this problem without much work, I'm not opposed.

comment:10 in reply to: ↑ 9 Changed 5 years ago by pselkirk

  • Owner changed from pselkirk to vorner

Replying to vorner:

  • The last 3 commits on this branch have messages that start with "[2738]", which seems unrelated to this ticket.

I created a new branch, where this should be fixed. Look at trac2871_2.

Ah, okay. Pulled, built, and tested.

  • The existence of random factors (for cache hit, and especially for upstream query completion time) makes this less of a rigid benchmark than a statistical exercise. OTOH, over the course of 1000 iterations per run, it smooths out pretty well - I found less than 0.5% standard deviation over 12 runs.

Does this matter for the purpose? I mean, if there are enough samples (generated queries, or whatever), it should just converge to be something almost same every time.

That was pretty much my point.

Anyway, it looks okay to merge.

comment:11 Changed 5 years ago by vorner

  • Resolution set to complete
  • Status changed from reviewing to closed
  • Total Hours changed from 0 to 6.19

Thank you, done.

Note: See TracTickets for help on using tickets.