wiki:WeeklyMinutes20111018

BIND 10 Team Call

2011-10-18

Attendees

  • Jeremy
  • Shane
  • Dima
  • Jelte
  • Likun
  • Kevin
  • jinmei
  • Aharen
  • Fujiwara
  • Stephen
  • Kambe
  • Michal

Welcome to Kevin

Place to Do Administrivia?

We want to be as open as possible, but we also don't want to spam bind10-dev with sprint-related stuff (for example). Proposal to keep this on bind10-team.

Stephen: Enough trouble doing estimates, so putting it on the -dev list will not get any more estimates back!

Michal: OTOH if someone wanted to join from the outside without previous talking, he might be more encouraged if he sees that it's there.

Jinmei: I don't have a strong opinion actually. I don't remember if I said SHOULD or SHOULD NOT or things like that. I think the decision is to basically send everything to the -dev list without worrying about noise level, unless we have a strong reason to hide the things. That is why I last.

Jeremy: I prefer -dev as well.

Stephen: If we get estimates that are at variance from ours, are we going to invite them to the sprint planning?

Jinmei: In that case we should probably recruit that guy!


Jinmei: I vaguely remember a discussion something like this, probably a similar issue, in my vague memory the conclusion was that we can say that some kind of discussion is closed to the main developers, so we do not count the number of opinions outside of the main developers. In reality I don't think we have to worry about that case. Most people will simply ignore these kinds of things.

Michal: I don't think we will have estimates before patches!

Jelte: I worry about it getting too noisy.


Shane: We could use a tag "scrum planning" to make it easier to ignore.

Jelte: Or find them later.

Conclusion: We will send to bind10-dev but put a tag on the subject.

Code freeze and sprint lag between DNS and DHCP

https://lists.isc.org/pipermail/bind10-dev/2011-October/002698.html

Stephen: DNS sprint ends on Tuesday, DHCP sprint ends on Wednesday. When we release code ideally we want to include both DNS and DHCP sprints. But what happens in the day after the DNS has finished but before the DHCP has finished. Do we include DNS changes included in master in the release code?

Stephen: Last year when we had the R-Team and the A-Team, the sprints ended on different days. It was the same situation as we have now. What did we do then?

Jinmei: I don't remember very well, but I thought at the time we didn't have the buffer for the release. We simply saw last-minute pushes from the teams.

Jelte: Solutions are

  • 1 team waits a day
  • other team merges changes to the branch

But, we tend to need a lot of last-minute changes before release anyway!


Stephen: What do we do now? Merge on release branch?

Jelte: Most get done in the master.

Stephen: What if someone commits to master a ticket from the new sprint? Does that go into the release?

Michal: Yes. It's a snapshot, and we don't promise not to include some code.

Stephen: So we don't have a problem.

Michal: I would be worried if a new ticket breaks master a lot, but that doesn't happen often.

Stephen: The code is disjoint.

Jelte: Also, we tend not to have big merges at the beginning of a cycle.

Stephen: Any work done in the few days after the sprint will also go into the release?

Jelte: We can make the non-merge window bigger. Or we can make a branch for the release itself.


Shane: I think we should not worry about fixing this unless we get complaints.

Jelte: I worry more about having stuff crammed into release!


Jeremy: I think the scrum methodology does not have 2 teams working on 1 release. Is that normal?

Michal: I think for large projects there are more teams.

Stephen: I think Michal is correct.

Jeremy: I think it would be a 3rd sprint that is coordinating. I don't mean to add more work, I just didn't think it was scrum what we are doing.


Stephen: Summary - we shouldn't worry, the two sprints will end and sometime after that Jeremy will take a branch for the release.

Jinmei: Is there a reason we have separate ending dates?

Stephen: When do you have planning for the sprint? Tuesdays is inconvenient (before or after).

Trac spam

  • delete spam accounts?
  • introduce some defense in account creation?
  • moderation?

Jinmei: I've seen these more often these couple of weeks. I wonder if we are now [more popular?].

Michal: Spam comes in waves. I see it with e-mail, and we had Trac spamming peak some time ago.


Shane: There are anti-spam plug-ins for Trac. I guess we should add one and see what happens.

Jeremy: I deleted like 25 users last week, and it's hard to know which are legitimate.

isc.dns and pydnspp

We use both as a python DNS module in an inconsistent way. I don't know why, but I think we should unify it.

Jelte: isc.dns is an alias for pydnspp, so there is no functional difference. We should probably use isc.dns.

Michal: I would like that one as well, since it is more consistent with other modules we have.


Shane: I think we can create a ticket and put it on a sprint.

Jinmei: Does not have to be the next sprint, but this kind of cleanup is better this year. At least newly developed Python programs should use isc.dns.

Difference design, possible specific topics

  • (especially for the DB backend case) whether to allow a separate storage or assume we always use some table in the same database (in this case we always use the same single transaction)
  • whether to mandate a sorted sequence when adding diffs
  • how exactly we store and retrieve the diffs in a DB table: schema, SQL, use of timestamp, use of begin-end SOA columns, etc.
  • text or binary

(especially for the DB backend case) whether to allow a separate storage or assume we always use some table in the same database (in this case we always use the same single transaction)

Jinmei: don't need intensive discussion here, since we already have a discussion on the list. My intent is to raise these explicitly, in case someone missed the discussion.

Stephen: In the first case, that was whether we use a table for differences in the same database as the zone data. Under what circumstances would we want to use separate databases?

Jelte: Existing provisioning systems, that don't have differences that we want to support.

Jelte: If we want to support that, will it have an effect on the higher levels.

Stephen: The issue here is transactions. If you have two transactions then it would be possible to add differences but not zone data (for example).

Stephen: My feeling is if you are going to use your own database, there is information you would have to supply. So I don't see what the problem would be in requiring a differences table in your provisioning system's database.

Jinmei: BTW, in the case of BIND 9, if only update cannot be stored, the name server keeps serving data and when the server restarts it merges the differences.

Stephen: We have several tables. Would we ever have a case where NSEC is on one system, but ZONES is somewhere else?

Jelte: That would be a problem that the data source layer has to solve.

Shane: Can we push this problem to the data source level?

Stephen: I think so. If we update tables it has to be a single transaction. If you want separate databases it's up to you how you write your interface.

Jinmei: So are we going to allow separate databases with restrictions?

Stephen: I think we are going to specify what the interface needs to supply. If someone wants to split across databases, then it is up to the implementor to guarantee this is done in a single transaction.

Jinmei: Does this apply to the middle-level interface.

Michal: I think it could, because if we want to spread across multiple database we cannot use our current code anyway.

Jinmei: So what are we going to do for the short-term implementation? Assume a single database?

Michal: Yes.


whether to mandate a sorted sequence when adding diffs

Stephen: I don't see any problem when doing this. It makes the code a lot easier.

Jinmei: And for the case of IXFR it does not add any more tasks. So in terms of implementation it is easier. One possible implication is how we handle dynamic updates (DDNS) since we cannot assume sorted order, so we need to sort the differences ourselves and then store them.

Michal: DDNS will have small updates anyway, so it won't be hard to sort them.

Stephen: You get updates but you don't necessarily update the SOA record do you?

Jinmei: BIND 9 always increments SOA. It's not prohibited.


Shane: Are there other DDNS implementations?

Jelte: OpenDNSSEC has option to incrementing, leaving on timestamp, or leaving "as is" and mandate that updates change it.

Jinmei: BIND 8.


Stephen: Problem comes if you are not updating the SOA with every record. Then the processing is more complicated because then we have to take account that a record may be added and subtracted.

Jelte: You must update the serial yourself if it is not changed by the update.


Jinmei: So do we store in IXFR-style ordering at the application level, or move the smartness to the lower level?

Jelte: I would say the higher should get it sorted from the lower level. Whether the lower level stores it that way does not matter.

Jinmei: Should the update program sort before adding to data source?

Jelte: Or whether the manipulation should do it itself.

Michal: I think we should leave this to the application. For IXFR we are already sorted, so this is a waste to do twice. For DDNS this is small.


Jinmei: In the case of IXFR-in you add differences as you get them from the master server, and for DDNS you sort the differences first to be like IXFR-in, then add these to the data source. If this makes sense, then it's okay to me of course.

Jinmei: How we store the differences at the lowest level depends on the SQL server or whatever.


how exactly we store and retrieve the diffs in a DB table: schema, SQL, use of timestamp, use of begin-end SOA columns, etc.

Jinmei: I showed a schema, but I am not an expert, so maybe someone can come up with a more efficient version. So if anyone has a strong suggestion please send to the bind10-dev list. I note that this level can change without affecting the upper level interface.

quick release engineering overview for last sprints' snapshot

Jeremy: If I understand correctly, we are supposed to be doing an overview after each sprint. I'll talk about last week.

Jeremy: One concern was explanation about the DHCP part of the release - missing documentation how to use it.

Stephen: At the moment we are just putting code into the repository for the next release. At the moment the server is nothing more than an echo server. The reason I suggested there is no documentation is that there is no functionality.

Jeremy: Maybe in the future, even binary names should be more specific... but that's a different topic.


Jeremy: We had dlopen() issues on some systems. This was a major regression, and there was no logging. Jinmei and Jelte never saw the issue at all, but I was seeing it every time. They helped provide workarounds for it. One minor issue was that Jinmei and Jelte were working at different times, so the work was not coordinated. Also that identified some other issues - some were fixed and some tickets were created.

Jinmei: One of the difficult points of dlopen() issue is that it does not happen with unit tests, and it does not happen with system tests somehow. We only notice it if we install the whole system and try to use it. So I only noticed it when I installed the master version very close to the release version on my personal server. We may see similar issues. The lesson is that we need some automatic way to test things in the installed environment.

Jelte: Even then it did not fail on my system!


Jelte: This does suggest that we also need to test both installed and uninstalled versions.

Jelte: Or we should run our software sooner in the cycle.

Jinmei: We should also have some automatic way to test these things.

(ran out of time)

Last modified 6 years ago Last modified on Oct 18, 2011, 8:38:58 PM