Changes between Version 4 and Version 5 of April2013MeetingDNSFriday20130419


Ignore:
Timestamp:
Apr 26, 2013, 6:40:16 AM (5 years ago)
Author:
muks
Comment:

Add meeting minutes from the notes I took

Legend:

Unmodified
Added
Removed
Modified
  • April2013MeetingDNSFriday20130419

    v4 v5  
     1= April 19, 2013 =
     2
     3== JPRS zone transfer performance discussion ==
     4
     5'''@1001 hrs'''
     6{{{
     7Michal: I believe the performance is OK, but rest are due to Sqlite. MySQL and PostgreSQL should solve it.
     8Jinmei: Mostly related to AXFR. May still hit limitation of sqlite3 if we use IXFR, but most notably about AXFR of large zones.
     9Shane: We could workaround limitations of sqlite.
     10Shane: If we really want to improve performance, we need MySQL and PostgreSQL.
     11Jinmei: I played with MySQL with large zones; seems to cover most of our scenarios like multiple zone transfers at same time, or AXFR while updating zones at the same time.
     12Jinmei: PostgreSQL API seems to require some more work. Non-trivial to workaround. MySQL will be relatively a trivial port of sqlite3.
     13
     14Mark: Do we do compressed IXFRs?
     15Jinmei: We don't do that.
     16Shane: BIND9 doesn't do it right?
     17Mark: No.. pain and suffering.
     18
     19Shane: What I'd like to do when we schedule reworking zone transfer stuff is to add MySQL as a data source.
     20Michal: There are ways to workaround it using sqlite3 itself.
     21Michal: If you put each zone in a different file, you can separate the databases [and don't hit sqlite3 bottleneck].
     22
     23ACTION: Add ticket for lettuce test with multiple sqlite datasources.
     24
     25Shane: We should add some zone transfer benchmarks.
     26
     27Shane: So overall zone transfer work is well-understood. We can start this work when we are done with the shared memory work instead of doing the hooks next.
     28Shane: Depends on when DNSCo wants to get hooks first.
     29
     30Shane: I want to improve transparency of the zone transfers in process, what are happening, how long they'll be, etc.
     31Jinmei: We should add one another data source as a kind of bugfix for scalability.
     32Jinmei: For AXFRout and loadzone into memory, we have to iterate over . Right now we iterate in a sorted way. It is difficult to do this quickly using MySQL. If we use ORDER BY, it doesn't work very well.
     33Jinmei: It works very slowly, and sometimes causes a fatal timeout.
     34
     35Breakdown of tasks:
     36
     37- meta
     38 + performance check of using multiple processes to do zone transfers
     39
     40- xfrin-ng (part of zonemgr-ng)
     41 - inspection of packet (success/fail)
     42
     43- zonemgr-ng
     44 + grand design
     45 + scheduler
     46 + soa checks
     47 + NOTIFY handling (maybe unified)
     48 + mostly completely reimplement it
     49 + admin info (xfrs in progress)
     50 + admin commands (kill xfr)
     51
     52- xfrout-ng
     53 + revisit design of current architecture (notify out, xfr request, etc.)
     54 + benchmarks
     55 + limits on how many there are
     56 + support back to back zone transfers
     57 + ixfr + udp
     58 + persistent notify info
     59 + out of zone notify
     60 + control of notify targets
     61
     62- mysql
     63 + research performance (RRs, sorted/grouped)
     64 + conf design
     65 + port of sqlite
     66 + document how to add data source
     67 + test setup (how to setup mysql server to run test suite?)
     68 + configure help
     69 + fix sqlite3 hardcoded bits
     70 + loadzone
     71 + dbutil (dhcp team may want to update this soon)
     72 + mysql embedded version, licensing
     73}}}
     74
     75----
     76
     77== What is missing from BIND 10? ==
     78
     79'''@ 1129 hrs'''
     80
     81{{{
     82Shane: Missing feature for sponsors - we don't have RRL. Whether it is
     83necessary or not is tricky. Didn't exist a year ago. Most TLDs didn't
     84have it in place 6 months ago.
     85
     86Jinmei: Ported BIND 9's implementation. Wrote detailed unit tests and
     87things like that. Confirmed it worked. Not surprising because it's a
     88straightforward port of original implementation. Still missing some
     89features that exist in the original implementation, but it can be
     90easily completed. Right now, it's just my personal experiment and we
     91could just use it as a reference and do it separately, or merge it
     92later after any necessary cleanup/review/etc. I could clean it up and
     93create a few tickets mainly for reviewing. I guess it could be 5-6
     94tickets.
     95
     96Jeremy: Ops said they could run B10 auth in production if it had RRL.
     97
     98Jinmei: The part we talked about for getting the number of levels from
     99the DomainTree to optimize using the origin name for statistics work
     100can also apply here.
     101
     102Shane: Another thing is to improve our user interface.  We may
     103outsource it or do something else about it. Will know in the next few
     104days.
     105
     106Shane: signing topic
     107Michal: We may want something like on-demand signing, for things like
     108hook-generated answers.
     109
     110Discussion about on-demand signing. Does PowerDNS sign per-query? Does
     111anyone use it? Yes and Yes.
     112
     113Shane: I know some TLDs use BIND 9 managed zones to sign their zones
     114and use DDNS.
     115
     116Shane: We'll have to implement command-line tools and such.
     117
     118Michal: We'll have to add a data source example as well.
     119
     120Mukund: We should have examples of our REST api as well.
     121
     122Fujiwara: How about daemoninzing BIND 10?
     123
     124Shane: We probably have to change it to run BIND 10 as a daemon by
     125default.
     126
     127Jeremy: We don't have output go to a log file by default.
     128
     129Jinmei: One more thing to do is making sure the response is equal to the source address.
     130
     131Shane: We should be a bit smarter about starting auth servers when we
     132have multiple cores.
     133
     134Jinmei: We should have more fine-grained ACLs.
     135
     136Mark: Do you have an EDNS0 client-subnet yet?
     137
     138}}}
     139
     140----
     141
     142== Hooks discussion with DHCP team ==
     143
     144'''@1308 hrs'''
     145
     146{{{
     147Shane: We agree on the basic idea on hooks.
     148Shane: We need an option to skip processing
     149
     150Shane: We may have a requirement about the order in which hooks are
     151called. The order in the config file sets it, but we may want it to be
     152configured on a per-hook basis.
     153
     154Discussion on a packet context.
     155
     156Michal presents "our" design for the hooks API.
     157
     158  dns::Message* m;
     159  ev->get("message", m);
     160
     161We discussed:
     162* Performance issues if marshalling is involved
     163* Explicit init() function
     164* Dynamic registering hooks
     165* Whether this design would make things more complex
     166* Examples in python, etc. were discussed
     167
     168Shane: Is this a high priority for the DHCP team?
     169Stephen: Yes. Comcast's requirements will need it.
     170
     171Tomek: We'd only require the C++ bits for now. We have no plans to write Python wrappers.
     172Would Comcast need python wrappers?
     173
     174There is initial agreement that we could use this new design.
     175
     176ACTION: Document this as a design. Mukund and Stephen will work on it.
     177}}}
     178
     179----
     180
     181== Scaling the resolver across multiple cores ==
     182
    1183[14:37]
    2 
    3 = Scaling the resolver across multiple cores =
    4184
    5185Original suggestion (years ago) not to use threads for scaling because two developers earlier on didn't want to, but not concern now.
     
    66246Wait for detailed analysis after research is done.
    67247
     248== Cache ==
     249
    68250[15:39]
    69 
    70 = Cache =
    71251
    72252review of doc/design/reso .... (from jinmei on shane's system)