Changes between Version 4 and Version 5 of April2013MeetingDNSFriday20130419

Apr 26, 2013, 6:40:16 AM (5 years ago)

Add meeting minutes from the notes I took


  • April2013MeetingDNSFriday20130419

    v4 v5  
     1= April 19, 2013 =
     3== JPRS zone transfer performance discussion ==
     5'''@1001 hrs'''
     7Michal: I believe the performance is OK, but rest are due to Sqlite. MySQL and PostgreSQL should solve it.
     8Jinmei: Mostly related to AXFR. May still hit limitation of sqlite3 if we use IXFR, but most notably about AXFR of large zones.
     9Shane: We could workaround limitations of sqlite.
     10Shane: If we really want to improve performance, we need MySQL and PostgreSQL.
     11Jinmei: I played with MySQL with large zones; seems to cover most of our scenarios like multiple zone transfers at same time, or AXFR while updating zones at the same time.
     12Jinmei: PostgreSQL API seems to require some more work. Non-trivial to workaround. MySQL will be relatively a trivial port of sqlite3.
     14Mark: Do we do compressed IXFRs?
     15Jinmei: We don't do that.
     16Shane: BIND9 doesn't do it right?
     17Mark: No.. pain and suffering.
     19Shane: What I'd like to do when we schedule reworking zone transfer stuff is to add MySQL as a data source.
     20Michal: There are ways to workaround it using sqlite3 itself.
     21Michal: If you put each zone in a different file, you can separate the databases [and don't hit sqlite3 bottleneck].
     23ACTION: Add ticket for lettuce test with multiple sqlite datasources.
     25Shane: We should add some zone transfer benchmarks.
     27Shane: So overall zone transfer work is well-understood. We can start this work when we are done with the shared memory work instead of doing the hooks next.
     28Shane: Depends on when DNSCo wants to get hooks first.
     30Shane: I want to improve transparency of the zone transfers in process, what are happening, how long they'll be, etc.
     31Jinmei: We should add one another data source as a kind of bugfix for scalability.
     32Jinmei: For AXFRout and loadzone into memory, we have to iterate over . Right now we iterate in a sorted way. It is difficult to do this quickly using MySQL. If we use ORDER BY, it doesn't work very well.
     33Jinmei: It works very slowly, and sometimes causes a fatal timeout.
     35Breakdown of tasks:
     37- meta
     38 + performance check of using multiple processes to do zone transfers
     40- xfrin-ng (part of zonemgr-ng)
     41 - inspection of packet (success/fail)
     43- zonemgr-ng
     44 + grand design
     45 + scheduler
     46 + soa checks
     47 + NOTIFY handling (maybe unified)
     48 + mostly completely reimplement it
     49 + admin info (xfrs in progress)
     50 + admin commands (kill xfr)
     52- xfrout-ng
     53 + revisit design of current architecture (notify out, xfr request, etc.)
     54 + benchmarks
     55 + limits on how many there are
     56 + support back to back zone transfers
     57 + ixfr + udp
     58 + persistent notify info
     59 + out of zone notify
     60 + control of notify targets
     62- mysql
     63 + research performance (RRs, sorted/grouped)
     64 + conf design
     65 + port of sqlite
     66 + document how to add data source
     67 + test setup (how to setup mysql server to run test suite?)
     68 + configure help
     69 + fix sqlite3 hardcoded bits
     70 + loadzone
     71 + dbutil (dhcp team may want to update this soon)
     72 + mysql embedded version, licensing
     77== What is missing from BIND 10? ==
     79'''@ 1129 hrs'''
     82Shane: Missing feature for sponsors - we don't have RRL. Whether it is
     83necessary or not is tricky. Didn't exist a year ago. Most TLDs didn't
     84have it in place 6 months ago.
     86Jinmei: Ported BIND 9's implementation. Wrote detailed unit tests and
     87things like that. Confirmed it worked. Not surprising because it's a
     88straightforward port of original implementation. Still missing some
     89features that exist in the original implementation, but it can be
     90easily completed. Right now, it's just my personal experiment and we
     91could just use it as a reference and do it separately, or merge it
     92later after any necessary cleanup/review/etc. I could clean it up and
     93create a few tickets mainly for reviewing. I guess it could be 5-6
     96Jeremy: Ops said they could run B10 auth in production if it had RRL.
     98Jinmei: The part we talked about for getting the number of levels from
     99the DomainTree to optimize using the origin name for statistics work
     100can also apply here.
     102Shane: Another thing is to improve our user interface.  We may
     103outsource it or do something else about it. Will know in the next few
     106Shane: signing topic
     107Michal: We may want something like on-demand signing, for things like
     108hook-generated answers.
     110Discussion about on-demand signing. Does PowerDNS sign per-query? Does
     111anyone use it? Yes and Yes.
     113Shane: I know some TLDs use BIND 9 managed zones to sign their zones
     114and use DDNS.
     116Shane: We'll have to implement command-line tools and such.
     118Michal: We'll have to add a data source example as well.
     120Mukund: We should have examples of our REST api as well.
     122Fujiwara: How about daemoninzing BIND 10?
     124Shane: We probably have to change it to run BIND 10 as a daemon by
     127Jeremy: We don't have output go to a log file by default.
     129Jinmei: One more thing to do is making sure the response is equal to the source address.
     131Shane: We should be a bit smarter about starting auth servers when we
     132have multiple cores.
     134Jinmei: We should have more fine-grained ACLs.
     136Mark: Do you have an EDNS0 client-subnet yet?
     142== Hooks discussion with DHCP team ==
     144'''@1308 hrs'''
     147Shane: We agree on the basic idea on hooks.
     148Shane: We need an option to skip processing
     150Shane: We may have a requirement about the order in which hooks are
     151called. The order in the config file sets it, but we may want it to be
     152configured on a per-hook basis.
     154Discussion on a packet context.
     156Michal presents "our" design for the hooks API.
     158  dns::Message* m;
     159  ev->get("message", m);
     161We discussed:
     162* Performance issues if marshalling is involved
     163* Explicit init() function
     164* Dynamic registering hooks
     165* Whether this design would make things more complex
     166* Examples in python, etc. were discussed
     168Shane: Is this a high priority for the DHCP team?
     169Stephen: Yes. Comcast's requirements will need it.
     171Tomek: We'd only require the C++ bits for now. We have no plans to write Python wrappers.
     172Would Comcast need python wrappers?
     174There is initial agreement that we could use this new design.
     176ACTION: Document this as a design. Mukund and Stephen will work on it.
     181== Scaling the resolver across multiple cores ==
    3 = Scaling the resolver across multiple cores =
    5185Original suggestion (years ago) not to use threads for scaling because two developers earlier on didn't want to, but not concern now.
    66246Wait for detailed analysis after research is done.
     248== Cache ==
    70 = Cache =
    72252review of doc/design/reso .... (from jinmei on shane's system)