[TOOLS-DEVELOPMENT] Tools Call Notes -- 8 October 2019

[Cindy Morgan <[e-mail]>]

Hi everyone,

Below please find notes from today's Tools call.

Best regards,
Cindy

--

Tools Call Notes -- 8 October 2019 at 1:00 Eastern

0. Attendance

Glen Barney
Alissa Cooper
Ryan Cross
Roman Danyliw
Heather Flanagan
Russ Housley
Henrik Levkowetz
Karen Moreland
Cindy Morgan
Alexa Morris
Alice Russo
Robert Sparks
Portia Wenze-Danley
Greg Wood


1. Datatracker Projects

  - Expected Datatracker Releases -- Robert and Henrik
    -- https://trac.tools.ietf.org/tools/ietfdb/browser/trunk/PLAN

Robert: Nothing to highlight in the merge plan. Hopefully Henrik's email answered Alissa's question about non-ASCII in UTF-8.

Alissa: Does the item in place represent the work left to do, or not?

Henrik: There are decisions to make and work left to do in order to complete that item. If we decide that we will only permit non-ASCII author names in XML upon upload, then we're done. Or we decide otherwise, and then there is more work needed, on doing on parsing text format in order to support new features. That decision needs to be made before we take out that item.

Robert; We still have the ability when someone is uploading text for people to correct their name. Typically people just click through that page, but that is a way we could present that with the full Unicode character set; they'd just have to provide it to the Datatracker at upload time. But it wouldn't show up in the text version that gets displayed. I don't know what priority we need to put on making that decision. Is it reasonable to say, if you want to have the non-ASCII characters in your submission that you need to submit in the XML format?

Alissa: I think so, but then again I kind of wish we only had one submission format so I am maybe not the person to ask.

Robert: I think we'll work forward with that as a tentative plan and if it creates any tension in the community, we can work it out if anyone pushes back.

Russ: And if the IESG is not happy with that plan, they need to tell us.

Alissa: Roman, can you put this on our action item list so that we talk about it the next time we talk about action items?

Roman: Will so.


  - Implementation of [e-mail] -- Robert

Robert via email: Most of the work for this (other than creating the new 
  list) is in the datatracker. We plan to have this as a release ready 
  to deploy on Oct 18.

Robert: This is ready to add to a release that will happen right before the deadline for the cutover. Not foreseeing any problems here.



  - Meeting Application Improvements by IOLA -- Robert

Robert via email: IOLA began work this week. They are significantly off 
  their originally expected start date. I think it is still reasonable 
  for them to finish this project this year.

Robert: IOLA is late getting started, but they still think they'll be able to finish by the end of the year. Need to keep in mind any impact this will have on things like the automated agenda scheduler. Some of that work may depend on this work, and we shouldn't expect it until the end of the year. 


  - Datatracker maintenance by DashCare -- Robert

Robert via email: The improvements to the review tool project continues 
  to go well. The recent 6.104.0 release contained much of this work, 
  and the next several will continue to include improvements.

Russ: We're seeing notes from them in release notes, how is this going?

Robert: Exceeding well. Sasha is working through the things we've asked to be corrected at a great rate, so I'm feeling good about where this is.


  - IRSG balloting -- Robert

Robert via email: On track for completion before IETF106.

Robert: Been working with Peter on this. It is still planned to be complete before 106. We spent some time last week working through some of the harder design decisions about how to display ballots on search result rows. Now down to finishing things up and taking care of corner cases.

Russ: My understanding is we started that work based on the IESG balloting, and morphed it to fit the policies for the IRSG, but they've made improvements?

Robert: It's more generalizing what was there for the IESG, not making improvements.

Russ: Making sure it's on a common base, got it.


  - Deployment of Python3 datatracker -- Robert

Robert via email: We are going to have to separate and containerize 
  trac, which isn't likely to become python3 enabled until sometime next 
  year. We are currently planning to deploy the conversion to python 3 
  when the main ietf server is upgraded.


Robert: We are planning to deploy Python3 when Glen upgrades the main IETF servers. Maybe we can talk about that when we get to the server part of the agenda. We know we'll need to separate Trac into its own container because they won't finish their Python migration until sometime next year.

    -- Impacts on related applications

2. Community & Other Projects

  - WWW search to provide additional pointers -- Greg
    -- Give pointers to datatracker and mailarchive

Robert via email: This is on my queue to implement.

Russ: Greg, I understand we'll take links that say where else to look. Has that been done?

Robert: It's still in my queue to do.

Russ: I thought it was done on www but not the other sides?

Robert: No.

Russ: When do we plan to do that?

Robert: It might have been done with content inside the website, but if there is code that needs to be touched, it will be in the next couple of weeks.

Greg: On www there are indeed search boxes for datatracker and mailarchive, but I think there needs to be some text added to provide context. And the other step on www is creating stub pages to match up with the 10 most commonly searched terms. Those two parts of the search improvement plan are still in the works. I think Robert mentioned that doing the similar pointing for mailarchive and Datattracker are still in the queue. 


  - Host third-party subresources on mailarchive -- Ryan

Russ: This is so, right now when you go to a page it goes to another site to get the fonts; the idea is you store them locally so we don't get any cross-site leakage.

Robert: Currently prioritizing when we would do this. My read is this isn't as high priority as the registration system for the current meeting, so I think we will address this just after IETF 106.

Russ: I agree with that prioritization and think we should let the community know when this is sorted out because I know some of them care a lot about this.


  - Analytics implementation -- Greg

Roman: We launched a week ago the final approach to do analytics, but in that there is Javascript-only, and the original plan was to do something that was pixel-based, so we were going to talk about how to proceed forward. Some people liked the pixel-based way better because you can't opt-out, and you can do reduced collection.

Robert: Yes, you got the essence of it. Getting any information on the robots accessing the website on the other side of the CDN, they will definitely be in the opt-out camp. It comes down to what we are trying to accomplish with the analytics. If we have opt-in by turning Javascript on, we'll get different data than we're currently collecting, but I'm not sure it will be better data. Is it okay if we get a large section of the community that does opt out on JS, and we don't get the data on robots?

Greg: This is a bit beyond my technical depth. I am not sure I understand entirely. I do understand that people who don't have JS on will not get recorded in this system, which I think is fine. But I don't understand the complication the CDN creates, because I thought Cloudflare supports the JS approach?

Robert: Anything accessing the site through Cloudflare with JS on will get that information, which is a bit more invasive than we'd get with the pixel-based approach. But robots won't do the JS things so we won't get counters on that at all.

Alissa: I don't think counting robots was the goal of this; it was human visits. I also want to question the premise that people will turn JS off to opt out of this.

Henrik: The problem is that some people may turn JS off, but we won't know whether it's 50% or 0.5%. We will collect data and we won't know if it is representative or not. That is the issue here. If we knew how much it was, that would be different, but we have no way of knowing. I would be against having people collect JS-based data on me rather than just registering which pages I visit based on the pixel-based approach. I think the more invasive approach is not what I expected us to do.

Alissa: Part of this is not about our personal preferences, since we vetted this with the community. I think gross statistics about who browses with JS off is something we should be able to find. I think most people using the Web have JS turned on. Maybe we can do a data collection exercise to figure that out. And we can address the concerns about the representativeness of the data. Question about the pixel-based approach where we can't opt out, what data would we be able to collect? Is there a cookie attached? Is it IP data?

Henrik: It would be the data we would get today if we didn't have the CDN. 

Alissa: So the JS is more robust.

Henrik: Broader, but less robust and more invasive.

Roman: If we did a pixel-based approach, which set of that data would we get? Part of what I am reacting to is, we weren't exceedingly specific on what fields we were going to be collecting. I think we need to be clear on that, and what is the manifestation of that to whoever will be the end users of that. 

Henrik: I am not certain what you're asking for? What data fields we would get for each approach?

Roman: Yes, because I'm trying to figure out if this is robustness or data collection.

Henrik: One approach will collect a limited amount of data on all visitors. The other will collect much more data on a percentage of the visitors, and we won't know what that percentage actually is.

Roman: I've never deployed this and don't know how configureable it is, so it's about the specifics, would we collect less data than is in Table 1 (https://www.ietf.org/media/documents/www.ietf.org-AnalyticsProposal-Revised-2019-09-24.pdf)

Robert: Can it be tuned down so that it is not much more, and I think the answer is yes. Surely the JS can collect vastly more data, but I think we can tune it down to what we have in the table.

Roman: Right, this is where we got burned initially on the hypothetical. This is why we were very specific in the final proposal about what we're collecting and why.

Robert: I think the discussion quieted down when people saw they could opt out based on IP. What we would get from the pixel-based approach would be even smaller but there wouldn't be an opt-out, and I think that is the key distinction.

Roman: So I am back to what the use cases driving this are. Can we use general Internet-based sampling of who opts out of JS to scale the data that we have?

Robert: For me, reinforcing what it is we are going to do with this data once we have it is what tips this the furthest. One of these gives a limited view into the community, but is it enough to give us enough to do something different.

Alissa: The use cases are laid out in the proposal, and what we vetted with the community was the JS option where people can opt out by turning of JS. That's what we told people we would do, so we should do it.

Roman: That's where I'm at.

Greg: Looking at the general stats on JS use, I don't see that as preventing us from getting done what we need to get done.

Henrik: I don't know that I have more to say. It's a tradeoff and it's very much a matter of what we need the data for and what we plan to do with it. 

Robert: I think what I'm hearing is we move forward with the JS-only based approach and we make sure it only collects the minimal data described in the plan, deploy, and decide whether the information we get is useful to us. 

Roman: Yes, and we'll reassess whether this was valuable a year from.

Russ: Did we tell the community when we would deploy this?

Roman: We made no commitment on the timeline. We'll figure out the implementation details and then move forward. And we promised to revisit whether it was valuable a year after deployment.

Russ: Greg and Robert, do you need anything else?

Robert: I think we need to confirm that we can limit the data collection to what has been discussed.

Russ: But that's an experiment that you need to write some code to do. Can you put that into the merge plan for the website?

Robert: Yes.


3. RFC Services Projects

  - RPC use of new RFC format tools -- Heather and Henrik

Russ: What did we learn about the tools?

Heather: It's been a character-building experience. For the output, we got some compliments, and then there is the conversation with Julian about the fact that he doesn't consider it V3 complaint. The -bis document for RFC 7991 is still in progress, so he will have to live with disappointment for a while. Had a good suggestion about changing the info pages to make the HTML icon first instead of the TXT since it's now the preferred format. Still a lot of behind-the-scenes sorting things out and trying to make sure that the XML is as it should be because sometimes we have options on how things are accomplished, and Henrik has been very supported there. A certain amount of irony on the quietness of the community on the XML files versus the definition of the community. I would like more feedback on the XML stuff, but--

Russ: By the time it's in the RFC Ed Queue they could care less about the XML, they just want to know when they'll get the RFC.

Heather: I didn't say I didn't understand this, but it's taking a lot of time to get right.

Robert: One thing that came out of this is we need to talk about when and how we will link to the output of the new tools from the Datatracker when an RFC is available. Is there anything to stop us from linking to the RFC Ed version of that now?

Henrik: We need to go by the cutoff date/number. I don't think we have information to tell us whether it's published with or without XML. We can figure it out and add links in the next release, certainly.

Robert: I think there conversations about how they should affect the grammar, Julian's comments about the ToC, the RPC's experience about  having to resort to artwork to get the layout that is expected should come back to the grammar.

Heather: Right now we're talking about what the goal is for re-rendering, is the goal to do minimal re-rendering, pros and an cons. Re the ToC, I started a thread with Henrik to discuss what we want to do about that.

Alice: Big thanks to Heather and Henrik for your support as the RPC has been switching over. 

Russ: Based on the number of releases this month, it's clear Henrik was being very responsive to the RPC's needs; thanks very much.

  - XML2RFC Python3 only

Henrik: Python2 will stop getting security updates at the end of the year. Looking to turn off support for Python2 for all of our tools. The library used to generate the PDF, we are currently 10 releases behind because they stopped supporting Python2 10 releases ago. One of the fixes since then is fixing an issue where you could have an infinite loop. Alice ran into this with a PDF generation yesterday. Would like to ask if we can move onto only Python3 so we can get past this issue.

Russ: I see no problem about doing this with the RPC, but what if an author isn't ready for this, can he still run Python2 on his own machine?

Robert: No. I think for XML2RFC we just need to tell people it's a Python3 app, or else they can use the web application if needed.

Henrik: And if we do it now, I would like to jump the XML2RFC version to 3 at the same time; it's a strong signal that things are changing. 

Russ: That makes sense to me. I think that we need to tell the community that's where we're going. Does anyone thing otherwise? 

[No]

Russ: Henrik, can you draft a note to that effect, and I'll send it to the community?

Henrik: Sure. I have small things to push out for the RPC first, but I'll draft the note and we can do the version change when we move to Python3 only.


  - Deploying in-line errata on rfc-editor.org? -- Alice

Alice: We touched on this on the last call. It's' currently in a beta directory. The first step is to move it out of the beta directory, and then adding links to it throughout the site, search results, info pages, errata search results. It becomes a bigger thing if it's deployed more widely. If we elevate this to the level of an additional format RFCs are available in, it would also be listed in those places. A request to include in the RFC-specific JSON record some updates that are tangential. Right now, the files are only available for verified errata. Right now there is an errata JSON file that has that information, but getting the inline errata in places is not as simple as adding the links, but I think the links plan could be a starting place, unless it's more pressing to get it in the RFC-specific JSON files or elsewhere.

Russ: Did we get rid of the beta part?

Heather: We can do that, and I was IMing Sandy about what a better link could be. 

Russ: How about eliminating and putting it in errata?

Heather: How about eliminating it and putting in inline-errata? The only thing I won't compromise on was, the on rfc-interest someone said they didn't like having the big scary warning label, but I will not get rid of that. 


  - Security Review of RPC Tools -- Robert and Heather

Robert via email: We are in the period where we are gathering proposals. 
  The deadline is 14Oct. We have received a few clarifying questions and 
  have answers ready for the 7Oct question response deadline.

Robert: We are still gathering proposals and answering questions as they are coming in. Nothing more to report right now.

Russ: Bids are due next week?

Robert: Correct.

4. Server Infrastructure


  - Expected Website Releases -- Robert

Robert via email: I released a significant to the website behavior today 
  (affecting blog page performance). It should be in production before 
  we get to Tuesday's meeting.

  See <https://github.com/ietf-tools/wagtail_website/releases>.

  There will be several releases in the next couple of weeks that change 
  database internals without changing the public's view of the website. 
  The names of fields that editors use will be improved.

  After that, the major work items are to

  * Remove the current datatracker integration attempt

  * Migrate to Django 2 so that we can move to more recent wagtail 
    releases

  * Investigate reimplementation of the Area and Topic templates

  We'll be making smaller improvements with regular releases from now 
  on.

  We're discovering that the CSS framework used by the current system is 
  undocumented, and hard to work with. We are beginning to discuss a 
  project to rebase the site on Bootstrap4 while maintaining the look 
  and feel of the site.

Robert: I'll focus on the Matomo integration so we can schedule deploying it, and continue working through the list. One thing I want to call attention to is at some point in the future we will want to reimplement the styling basis for the website so that it uses the styles correctly. Right now it's not internally consistent and I think we'd get a lot of payback to moving to a system like bootstrap without doing any redesign of how it is supposed to look.


  - IESG discussions of DMARC -- Henrik and Alexey
    -- When do we expect the next release of OpenARC? Ever?

Henrik: I'll send another ping.

Alissa: Roman and I will see Alexey tomorrow and can ask for an update.

   - Sever upgrades

Glen: I've upgraded other servers I manage and am ready to do the updates at any time, but given the complexity of what is running on the IETF and RPC servers, I am trying to not rush blindly forward. There will be a linear approach. I've reached out to Ryan, who has a development server, and Priyanka who has a copy of the RPC server as her development server, and we'll upgrade those in the next couple of weeks and let them see what happens and what changes we need to make. Then I'll circle back with the team and we will do this as we have in the past, updating the sandbox and backup servers, and see how those go, and then propose a day and time to do the production server. It will take about 6 hours. It will be up for most of that time, but some services can be interrupted. I would be inclined to try to do this on a Sunday or Monday to give us time to recover, but in general it shouldn't be a big deal. Short answer is, we're taking the steps and we'll keep you posted.

Robert: More likely to be after 106 than before.

Glen: That's wonderful. There is no fire here. I'd like to get it done soon, but I can defer to the needs to the Tools Team and the IETF.

Russ: I hear that it sounds like to Ryan + Priyanka and the sandbox before 106, and then the other steps after.

Glen: Yes. This is not a single event, it's a series of steps, due diligence at a rate of speed we can control. The only one we have to schedule is the one on the production server.

Robert: I think we'll go with that. It may or may not make sense to move the sandbox before the meeting.

Glen: And offline we should talk about the sandbox and how use of it is increasing.

Robert: Agreed.

5. YANG Catalog

  - Putting yangcatalog.org in a Docker container -- Robert

Robert via email: As noted in the last meeting, this is prioritized 
  behind major functionality issues in the yangcatalog project, but it 
  is getting cycles. I am currently to the point where most of the 
  containers come up the way they should in my development environment , 
  but the confd and api_recovery containers are not reaching the 
  expected steady run-state. I am keeping pressure on this, but I don't 
  expect that we'll switch production to the containerized version in 
  the near future. Certainly not before IETF106.


6. Parking Lot


  - Search over www, datatracker, and mailarchive


  - Transition Datatracker to Django2


  - Transition Website to Django2



7. AOB

Robert via email: I think we should talk briefly about the state of 
  supporting markdown for minutes/agendas (and eventually other 
  documents), and how we should prioritize improving it.