Changes between Initial Version and Version 1 of AlphaBetaProgramPlan

Sep 27, 2012, 7:53:32 PM (6 years ago)



  • AlphaBetaProgramPlan

    v1 v1  
     1'''BIND 10 Authoritative Server Alpha and Beta Program Plan'''
     3Overview : BIND 10 is in its fourth year of development, and has had little real world operational exposure. As we approach the first major production ready deliverable,  (authoritative name server due out early 2012)  the project seeks community input on features and implementation to date and going forward. To this end both an alpha and a beta test program are proposed.
     5Purpose of document:  This document describes at a high level the objectives of these programs and further test documents which will be provided during the program duration.
     8•       Real user testing of BIND 10 in real user environments
     10•       Extending knowledge about the BIND 10 project into the user community
     12•       Determining challenges in BIND 9 to 10 adoption, coexistence, and migration
     14•       Increasing adoption likelihood among existing customers
     17•       Introducing potential new customers to ISC support     
     19•       Creating BIND 10 specific collateral and webinars etc
     21•       Training support and other staff on BIND 10
     23•       Finding quotable product advocates for release promotion, case studies, etc
     25Primary BIND 10 DNS Features to be tested
     26•       IXFR zone transfers in and out
     27•       AXFR zone transfers in and out
     29•       in-memory backend for zone data
     31•       SQLite3 backend for quick startup or huge amounts of zone data
     33•       DNSSEC: serving signed zones by authoritative server
     35o       either in-memory or from DB backend
     37o       including NSEC3
     39•       DDNS (Dynamic Updates) support
     41•       Access Control (for DDNS and outbound zone transfers)
     43•       TSIG support:
     45o       Part of general ACLs (DDNS uses it)
     46o       authoritative server can verify and sign DNS messages with TSIG
     47o       Zone transfers can have signed transfers
     49Specific suggested use cases to be tested
     50•       Installation
     51•       Selection of modules
     52•       Selection of modules at runtime
     53•       Transfer zones
     54•       More as per usage patters of initial alpha participants
     56We may also get developers interested in the program, who might:
     57•       Build a new module
     58•       Add support for an additional database
     60User personas to target for Alpha:
     62•       Existing skilled BIND 9 users
     63•       BIND 10 sponsor organizations
     64•       BIND 9 vendors (OS and Appliance)
     66User personas to target for Beta:
     67•       Large ISP
     68•       Hosting company
     69•       Registrar/ccTLD
     70•       Large corporate enterprise network
     71•       Small business/home/hobbyist
     72•       Module builder
     73•       OEM
     74•       OS vendor
     75•       DNS developer
     76•       All interested participants!
     78Schedule and Milestones:
     81o       Milestone One: Draft Plan for Review: September 24th final plan Sept 27
     83o       Milestone Two: beta program page and application form page  plans and other collateral plans September 27
     85o       Milestone Three: and  BIND  alpha/beta program pages ready to accept applicants September  30
     87o       Milestone Four : Launch Alpha
     89o       Milestone Five: Select supported beta program participants by October 20th
     91o       Milestone Six: Beta Preparation Webinar October 24th
     93o       Milestone Seven: Beta Launch November 17 (beta software release)
     95•       Release schedule (all dates subject to change based on needs of the project):
     96•       Alpha  Sept. 27
     97•       Beta 1 -- Nov. 17
     98•       Beta 2 -- Jan. 10 or 17?
     99•       Release Candidate -- ~Jan. 31 (or sooner!)
     100•       Goal month for official release – February 2013
     101•       (or sooner!)
     103•       Program Execution: Alpha:
     104•       Outreach to project sponsors and other “clueful users and DNS developers” primarily     
     105•       Support through the bind10-users list   
     106•       Registration for on for tracking (but not ongoing surveys)
     107•       Product Management will reach out  to  engaged  participants     for 1:1 meetings       
     109•       Program Execution: Beta:
     110o       Weekly emails to testers with option for weekly phone calls
     111o       In person meetings as possible (local areas, Grace Hopper, NANOG, LISA, RIPE, IETF)
     112o       Ongoing schedule of web seminars, blogs, etc to drive participation, inform testers of new features going into each beta, etc.
     114Phase Four:  Postmortem, lessons learned, driving feedback into release – following each release, a postmortem within two weeks, and feedback goes into the next release cycle for consideration.
     118Three types of beta testers:
     119•       Current Support Customers – any customer who has a current support contract (open question, does this include forum members? All levels?)
     120•       Invited Supported Beta – all non-customer potential participants need to “apply” and some number will be selected (depending on how many support customers want in, up to about 50 total supported participants) to get full ISC support during the beta – there will be specific participation levels expected
     121•       General Community Beta – anyone may join , best effort support from email alias with engineers and team members on it
     123Data gathering mechanisms:
     124•       Registration, usage, and other surveys
     125•       Bug reports
     126•       Possibly “phone home“ option – opt in tool that sends isc data
     127•       isc-sysinfo
     128•       Support queues
     129•       bind10-users alias
     130•       Calls and/or face to face meetings with some participants
     133Incentives for Participants:
     134•       Participants who are invited to the “supported” program receive support benefits for duration of program
     135•       Participants who submit a registration form and two or more bugs or weekly/experience reports will receive a thank you gift
     136•       Another possible option is to use the “bounty” strategy where actual (small) cash prizes are awarded for participation – this might be good to use for substantial work like database ports of module development
     138Outreach Plan:
     140Finding test subjects:
     141•       ISC BIND, DHCP lists
     142•       Other DNS community lists:
     143•       NANOG and *NOG email campaign
     144•       Twitter, LinkedIn, Facebook campaign
     146Engagement with test subjects:
     147•       Web seminars TBD
     148•       Weekly phone chat option (up to 5 participants can sign up to discuss, if more than 5 are interested in a week, more than one session is possible)
     149•       NANOG BIND 10 BoF and 1:1s
     150•       LISA BIND 10 BoF and 1:1s
     152Potential Blog topics (over the course of alpha and beta we will blog at least weekly about…):
     153•       BIND 10 Alpha Intro Blog
     154•       BIND 10 auth architectural overview Blog
     155•       Developer Documentation (APIs, etc) blog
     156•       Logging system blog
     157•       Configuration – philosophy and how it works blog
     158•       BIND 10 beta blog
     159•       Data sources in general and specific
     160•       Modularity – why it helps with performance, scaling, security, resiliency
     161•       So you want to build a module
     162•       Hooks: what we have now, what we want to do, how you can help
     163•       Quality Assurance in BIND 10
     164•       Using statistics in BIND 10
     165•       Protocol (RFC) issues and other DNS server bugs found by meticulous BIND 10 test-driven development.
     166•       Queries per second performance blog
     167•       Resource usage blog
     168•       Efficient in-memory datastore blog
     169•       Blog about development done by developers from four organizations and different timezones and how that affects our scrum sprints
     170•       lessons learned about nsec and wildcards?
     172Program Expenses:
     173•       staff time,
     174•       incentives for participants (prizes or bounty)
     175•       travel to NANOG and LISA (already planned)
     177Program Risks:
     178•       Not enough participants
     179•       Too many participants
     180•       Software found lacking by participants
     182Documents and Resources for testers:
     183Will include but not be limited to:
     184•       Beta announcements
     185•       Beta test application
     186•       Participant guidelines
     187•       Program welcome letter
     188•       Beta release notes
     189•       Surveys
     190•       Appreciation email
     192Resources for internal support staff:
     193•       Documentation plan will outline tutorials and how-to guides for BIND 10.
     195Additional Planning documents to be provided
     197•       Documentation plan- to specify what documentation sets will be ready for beta and on what schedule
     199•       Pre-release code Test plan – to specify what standard and additional tests will be done on beta and production code to ensure it is ready
     201•       Beta Participant Test plan – to specify what scenarios in detail we want to ask beta participants to test. Lettuce system level test schemas and experience reports from alpha participants will both inform this plan.