wiki:ReleaseEngineeringNotes

This page contains generic release engineering topics (many not specific or related to BIND 10). The goal of this page is to be a catch-all for Release Engineering ideas, suggested processes, and discussion. Consider splitting this out to new wiki pages as topics get longer.

The BIND 10 release engineering processes are documented at ReleaseEngineering.

Defining Processes

  • Clearly document the release engineering processes to reduce the communications overhead.
  • Do not over-engineer processes.
  • Commit to yearly process review and tool up as necessary, instead of putting a heavyweight process in place that gets in the way.

Release Dates

  • Time to market.
  • Balancing release date to security, performance, and/or features.
  • Releases based on measurable/verifiable results?
  • Releases based on sponsored deliverables? Not during the initial "sponsorship" phase.
  • Are releases to be done on "time" schedule or "feature" or both or other? define policies
  • Meeting sponsor's expectations for release dates.
  • publish expected upcoming release dates?
  • publish expected and actual release checklist/process dates?

Release timelines and process map or outline

  • timeline for tasks
  • code review policies -- maybe different policies depending on when in schedule?

Typical Release Schedule

  • unlock tree (allow commits)
  • normal development cycle
  • "we should make a release soon" or this is a good place to start testing (or have defined release features / goals)
  • slow down, check for regressions
  • lock tree / stop commits
  • tag / branch trees (and then unlock tree)
  • developers start work on a different future release (and commits to HEAD / trunk)
  • release team developers work on the branch for the upcoming release
  • builds and tests (alpha. beta, release candidates for testing)
  • fix regressions in HEAD / trunk and then merge into the release branch
  • sometimes merging in new features too (depends on policy)
  • repeat until ready, or exhausted

See OtherReleaseExamples.

Authomated Release Tarball Checker

  • extract
  • make sure recent changed files have up-to-date copyrights
  • make sure that documents are newer than their sources
  • compare with tagged version in revision control system: any extra files? any changed files?
  • build
  • etc

Release checklist

  • webpage with detailed checklist with estimated dates and updated with actual dates
  • reminder announcement to developers with rough schedule
  • how to keep track of what features or bugs are in a release?
    • use ticket system with a new ticket specifically for the upcoming release and have it link to other tickets for features and bugs
  • code freeze begins, commits pre-approved
  • begin beta1 builds (when alpha builds?)
  • begin beta2 builds
  • release branch created -- maybe this would be earlier like after alpha builds?
  • review ticket specific for the release
  • RC1 builds
  • RC2 builds
  • Final release builds
  • announcements

Final release checklist

  • creating branch, release tags
  • increasing version (add list of all places here or tool that handles this)
  • marketing
  • creating tarballs/zips
  • testing tarballs and zips -- manual and automated tests
  • distribution of tarballs/zips/etc.

Source code control

  • different revision control systems
  • pros and cons
  • document common usages
  • but don't get into all features -- pointers to other documentation
  • migrating revision control systems
  • how is the revision control data replicated? mirrored? backed up?
  • branching, tagging, etc
  • web-based browsing
  • community access. Who has access to read commit logs? (Everyone!)
  • community commit privileges/policies. Who is given commit access? How selected?
  • Anyone who has signed over copyrights to ISC and has SSH/SVN privilege can commit to the HEAD or development branch when following the ReviewQueueProtocol?.
  • who oversees community committers? Who provides direction over community committers (and over entire project)? (Cross reference this in the wiki.)
  • locking
  • revoking privileges
  • commit emails, set up RCS to send emails on commits
  • mailing lists for commits, one has diffs too (maybe don't archive that one)
  • snapshots. When? daily?, monthly?, where?
  • integration of revision control system with bug tracking systems, etc.
  • integration with automated testing and regression systems
  • autobuilds and testing on code checkins (see other category here)
  • status webpage listing which branches are frozen and/or open (with dates and description or why)
  • policies for commit log messages?

Change History

  • Manually maintained CHANGES file versus automatically generated from source code control commit logs?
  • Machine parseable file that can be generated into CHANGES? (So can be easily searched.)

Ownership / Maintainers

  • Will any code be owned or specifically maintained by an individual or group? Where they need to okay all changes or only make code changes themselves?
  • How will code be marked as owned or maintained?
  • MAINTAINERS file in directory containing one-line notifying how each maintainer would like contributions to be made.
  • policy for when maintainer is unreachable or unreasonable?

Licensing / Copyrights

  • standard ISC copyright
  • are licenses on non-code but related documents and tools different?
  • policy on how to accept contributed code? Sign over and we copyright? Or list of acceptable licenses we may include.

Builds

Defect tracking

  • bug / ticket tracking systems
  • bug ticket tracking system (use existing?)
  • public bug ticket system versus internal. Or both?
  • automated testing and regression systems
  • automatic generation of tickets for automated tests?
  • policy for assigning bugs?

Feature tracking

  • ticket system for community sharing new code, feature ideas, etc
  • use same bug ticket system for this

Quality, reliability, and performance

  • daily code lint and mechanized code review
  • daily code security audits (like flawfinder, rats, etc.)
  • work with Coverity
  • daily code builds on various platforms (see BuildFarm)
  • daily benchmark reports: performance, memory use, code size changes, etc.
  • lint and mechanized code review on new checkins
  • builds on new checkins
  • code security audits on new checkins (like flawfinder, rats, etc.)

System integration

Version numbering, branches, classifications

  • version numbering policies
  • any new public release must have a new incremented version number (because third party packagers generate and store their own checksums)
  • publicly document timelines, support lifetimes, EOL, "minor" versions (as applicable), etc.

Stable branch

  • what is allowed to be integrated into stable branch? only bug fixes? Any new features?
  • Who makes decisions for features to pull into stable branch?
  • all code must be committed and tested to HEAD first and then merged to stable? What are policies/situations for bypassing that?
  • what about more specific stable branches with only most critical security and bug fixes?

Release candidates

If a changes were made to a release "candidate" then another release candidate (increased version like rc1 to rc2) should be done.

A "candidate" is deemed worthy of a release and no changes will be made. If changes are made (other than maybe comments or minor docs), then another release candidate must be made available to public review first.

The only difference between the final RC and the actual release should be the removal of "rc#" from the various places where the release name is there, plus one additional line in the history and changes files noting the release date. That is, truly cosmetic as far as the code is concerned.

Library versioning

  • Policies for major / minor numbering increases for API additions, removals, changes.
  • Increase minor number for API addition.
  • Increase major number for removal or change.

Distribution

Release services

Release management

Roles and Responsibilities

  • BIND 10 manager
    • Final decision for arguments, what to pull up, etc.
  • Security contacts: at least two individuals to privately accept and handle communications about security.

Communication

  • mailing lists (internal if needed and public)
  • archives for lists
  • bug / ticket tracking system (see other category)
  • publicity channels, blog, marketing, two-way communication with BIND 10 users and sponsors (define where, how often, etc)

Relationships

  • sponsors
  • paying customers (support or development)
  • end-user community
  • open source community developers
  • code and documentation contributors without commit privileges
  • management / developers
  • news media

Security

Review and Postmortem analysis

  • Outside review during the release process.
  • Analysis following the event.

Documentation

  • Reset manual page date when a new feature is added or changes; and for typo fixes. (Some projects are okay with keeping old date if the fix is just punctuation or misspelling fix.)
  • Schedule documentation reviews periodically. Maybe every 12 months we should review all the man pages, for example.
  • Code reviews should include review of documentation included.
  • New or changed features should include corresponding reviewed documentation before committing to main code.
  • Schedule community "doc-athons" for specific documentation editing and writing to engage even non-coders.
  • Document steps so others can build releases.

Outside Resources

4.3BSD (1989) http://docs.freebsd.org/44doc/papers/releng.html

Fedora http://fedoraproject.org/wiki/ReleaseEngineering/Overview

FreeBSD http://www.freebsd.org/doc/en_US.ISO8859-1/articles/releng/index.html http://www.freebsd.org/releng/

Mozilla https://wiki.mozilla.org/ReleaseEngineering https://wiki.mozilla.org/Build:Release_Automation

OpenBSD http://www.openbsd.org/papers/asiabsdcon2009-release_engineering/index.html http://2009.asiabsdcon.org/live/abc2009-PT1.html (video)

Books

  • Ship it! A Practical Guide to Successful Software Projects by Jared Richardson and William Gwaltney
  • Release It!: Design and Deploy Production-Ready Software by Michael Nygard
  • Software Release Methodology by Michael E. Bays
  • The Visible Ops Handbook: Implementing ITIL in 4 Practical and Auditable Steps by Kevin Behr, Gene Kim, and George Spafford
Last modified 8 years ago Last modified on Aug 18, 2009, 3:17:51 PM