Re: [hrpc] IRTF Chair review of draft-irtf-hrpc-association-13

[Tony Rutkowski <[e-mail]>]

The review and comments of the IRTF chair seem an accurate expression of 
concerns regarding a number of characteristics of the draft.  The ID is 
an interesting and effective expression of societal views and concerns 
with supporting citations by their advocates.

The attempts to tie technology features to such concerns have a long 
history - bringing back memory  of screeds in the 1970s by an earlier 
critical communications community generation against ISDN protocols and 
networks of that era that culminated in 1980 MacBride Commission 
Report.  The irony is that TCP/IP infrastructure coupled with open 
provisioning markets have actually significantly enhanced freedoms of 
association and assembly.

Clearly, the methodology used here is dubious at best: "we provide a 
brief twofold literature review addressing the philosophical and legal 
definitions of FAA and how this right has already been interpreted or 
analyzed in the digital context...[and] look at some cases...."  The 
term "digital context" is never defined and abstruse.  Ironically, the 
MacBride Commission Report is not even mentioned - even though it used 
this methodology.  Similarly, the world's largest and most widely used 
ICT infrastructure for freedom of association and assembly - the global 
mobile network infrastructure standardised and regulated by 3GPP and 
GSMA - is completely ignored.

The fundamental problem here with this research paper - like its 
precursors from previous eras - is that protocols and infrastructures 
are used largely autonomously by sets of actors for all kinds of 
purposes.  A broad array of legal obligations and enablers are imposed 
on those actors - including human rights considerations - that have an 
infinite array of implications.

The conclusion is accurate, namely "the way in which infrastructure is 
designed and implemented impacts people's ability to exercise their 
freedom of assembly and association." However, that conclusion applies 
to every technological infrastructure in existence!  There is little or 
nothing in the ID that is specific to TCP/IP protocols or actionable by 
the IETF.

--Tony Rutkowski

On 10/24/2023 9:46 AM, Mallory Knodel wrote:
> Hi all,
>
> Please send any and all responses to this review and the direction of 
> the research group document moving forward. (It’s an RG doc after all.)
>
> I wanted to bring forward one element of the review: a decision that 
> needs to be taken on whether to aim to publish this as an essay or as 
> research. Colin’s review implies there exist these two paths and they 
> would result in different documents.
>
> In my experience there are zero practical differences between any 
> paths that lead towards publication, after the RG has come to 
> consensus (which we have already done for this draft a few times so I 
> don’t predict ourselves to be a barrier). Therefore I would encourage 
> folks to weigh in on this decision based on _what should this RG try 
> to achieve with this document: essay or research?_
>
> We will discuss at length during 118 but it would be great to have a 
> head start on the discussion via the list.
>
> For my part, it has always been research. I think the academic authors 
> would agree— Niels seems to. Stephane or Gisella? Perhaps someone who 
> thinks an essay style is better could share any persuasive arguments 
> as the review doesn’t make any.[0]
>
> -Mallory
>
> [0] Colin’s review says it would be more straight forward (easier), 
> which I’ve learned is not actually the case nor something to strive 
> for, and that “it's well written and makes a clear argument,” which I 
> wouldn’t assume we would necessarily sacrifice to improve the research 
> quality.
>
> On Tue, 24 Oct 2023 at 07:24, Niels ten Oever <[e-mail]> 
> wrote:
>
>     Dear all,
>
>     I think the literature review as it was done by Prof. Dr. Stephane
>     Couture lives up to the standards of social science (which differ
>     from computer science papers which are more quantitative in their
>     approach of literature reviews). The same I think is also true for
>     the case selection.
>
>     Having worked on this paper with the RG for six years, I do not
>     feel like I want to do a full rewrite and restructuring as
>     suggested by the IRTF chair. I happily leave the lead authorship
>     to someone who wants to take it up - I will not be writing a new
>     version.
>
>     All the best,
>
>     Niels
>
>     On 24-10-2023 13:07, Colin Perkins wrote:
>     > The HRPC chairs have requested that
>     draft-irtf-hrpc-association-13
>     <https://datatracker.ietf.org/doc/draft-irtf-hrpc-association/> be
>     published as an RFC on the IRTF stream. The IRTF publication
>     process is described in RFC 5743, and comprises a review by the
>     IRSG to ensure technical and editorial quality, followed by a
>     check by the IESG to ensure the work does not conflict with IETF
>     standards activities.
>     >
>     > As IRTF Chair, I perform an initial review of all drafts
>     submitted for publication on the IRTF stream before sending them
>     for detailed review by the IRSG. This note provides my review
>     comments, for discussion.
>     >
>     > Authors, please can you also respond to this message to confirm
>     that all necessary IPR disclosures, as described on
>     https://irtf.org/policies/ipr <https://irtf.org/policies/ipr>,
>     have been made?
>     >
>     > Result:
>     >
>     >   * Not ready
>     >
>     > RFC 5743 compliance:
>     >
>     >   * The draft does not follow the guidelines in RFC 5743
>     >
>     > Comments:
>     >
>     > This is an interesting draft, but I have some concerns about the
>     way the work is presented and have identified some technical
>     issues that should be considered. I sent these comments to the
>     authors shortly after the San Francisco meeting, and Mallory and I
>     have discussed on a number of occasions. I'm now sending them to
>     the broader research group for input and discussion.
>     >
>     > Overall, the draft is presented following the structure of a
>     research paper, but the content does not follow a systematic
>     research methodology. This makes it unsuitable for publication in
>     its current form. However, restructuring the document to reflect
>     more the form of an essay or a discussion piece would be
>     straightforward and would play to the strengths of the material.
>     >
>     > *Considering presentation*, the draft is structured as a
>     research document that identifies a question to consider, conducts
>     a literature review, develops an analysis, then draws some
>     conclusions. This is reasonable structure for the draft, but I'm
>     concerned that the methodology is not sufficiently rigorous to
>     support the approach.
>     >
>     > Firstly, while a literature review is developed, it is limited
>     and highly selective, and the papers don't appear to have been
>     chosen in a systematic manner such that they are representative of
>     the literature and to avoid bias. The topic area is complex enough
>     that I wouldn't expect the survey to be exhaustive, but I would
>     expect an identified methodology for selecting papers to consider,
>     along with a discussion of potential biases in the selection
>     process and how they were mitigated when the selection of papers
>     was made. The paper lacks such discussion, and following such a
>     process would likely lead to the selection of a significantly
>     different, and likely expanded, set of papers. I also have some
>     concern that the discussion in this section adopts a less than
>     rigorous style of argumentation in places that is not well suited
>     to a research paper.
>     >
>     > Secondly, the analysis of the protocols is similarly ad-hoc.
>     There are many protocols that could be studied, but it's not clear
>     why this particular set is chosen. As with the literature survey,
>     there's no discussion of the protocol selection process and it's
>     not obvious that the protocols and systems studied were chosen in
>     a systematic way that provides a representative and unbiased sample.
>     >
>     > Given these methodological shortcomings my view is that the
>     draft is unsuitable for publication in its current form. And,
>     while it could form the basis for a strong research contribution,
>     I expect the effort needed to develop it into such would be
>     significant, and the end result would be a very different draft.
>     >
>     > With relatively minor changes, however, I believe the draft
>     would make an excellent discussion piece: it's well written and
>     makes a clear argument. Structuring it in such a manner would
>     avoid the need for the systematisation that's required in a
>     research contribution.
>     >
>     > *Regarding the technical issues*, there are a number of errors
>     and limitations in the discussion of protocols. The discussion of
>     how WebRTC has the ability to function without a central server is
>     not correct, for example, and the discussion of BGP peering and
>     transit as a peer-to-peer protocol doesn't reflect the emergence
>     of hypergiants and IXPs. The discussion of the technical issues is
>     also highly selective at times and not representative of the range
>     of protocols that exist. While the technical errors need to be
>     corrected, the extent to which the discussion must consider the
>     range of design choices depends on the chosen direction for the
>     presentation.
>     >
>     > *Detailed Comments*
>     >
>     > Abstract
>     >
>     >   *
>     >
>     >     The first sentence is ungrammatical. Perhaps remove the word
>     "whether"?
>     >
>     >   *
>     >
>     >     RFC 5743 requires that "There must be a statement in the
>     abstract identifying it as the product of the RG". This is missing.
>     >
>     >   *
>     >
>     >     “It is therefore recommended that the potential impacts of
>     Internet technologies should be assessed, reflecting
>     recommendations of various UN bodies and international norms” –
>     it's unclear who is recommended to make such an assessment, and an
>     IRTF document cannot recommend the IETF does so. Perhaps "It is
>     therefore recommended that further research be undertaken into the
>     potential impact of Internet technologies…"? Alternatively,
>     specify who the recommendation is aimed at.
>     >
>     > Introduction
>     >
>     >   * "the present document seeks to deepen the relationship
>     between these human rights and Internet architecture, protocols,
>     and standards" – seeks to deepen the relationship or seeks to
>     deepen the understanding of the relationship? The latter is within
>     scope for the IRTF; the former seems like an IETF activity.
>     >
>     > Vocabulary Used
>     >
>     >   *
>     >
>     >     “Architecture The design of a structure” – this is a very
>     generic definition. Would it be possible to relate it to network
>     architecture more specifically?
>     >
>     >   *
>     >
>     >     “Autonomous Systems are the unit of routing policy in the
>     modern world of exterior routing [RFC1930]” – I'm not sure this is
>     quite right, although BGP routing allows ASes to reflect certain
>     types of policy. Citing an RFC from 1996 as reflecting "the modern
>     world" is problematic.
>     >
>     >   *
>     >
>     >     “Decentralization Implementation or deployment of standards,
>     protocols or systems without one single point of control.” – this
>     is a very restrictive definition of decentralisation that it's not
>     clear is met by any deployed system. It may be more appropriate to
>     discuss the extent to which a system has a single point of control
>     since deployed systems have a range of designs that are neither
>     fully centralised nor fully decentralised.
>     >
>     > Research question
>     >
>     >   * The research question is open ended and doesn’t pose a clear
>     hypothesis that can be tested. This makes the later sections
>     difficult to structure as a research paper. The question is highly
>     appropriate as the basis for a discussion document or essay.
>     >
>     > Methodology
>     >
>     >   * As noted in the general comments, the methodology seems weak
>     for a research document. That the literature review is not
>     exhaustive is fine, given the size of the field, but I'd expect a
>     clearly stated methodology for identifying the literature to
>     survey that aims to find a representative sample and avoid biases.
>     Similarly, the methodology for selecting protocols to consider
>     seems arbitrary. If the draft was pitched as a discussion document
>     to highlight particular concerns, then the approach taken seems
>     reasonable. For a research document, the methodology should be
>     more rigorous.
>     >
>     > Literature survey
>     >
>     >   *
>     >
>     >     “The rights to peaceful assembly and the freedom of
>     association are defined and guaranteed in national law and
>     international treaties; however, in this document we limit
>     ourselves to international treaties. “ - this restriction is
>     significant and could be better justified.
>     >
>     >   *
>     >
>     >     The presentation, on page 12 especially but not only, does
>     not adopt the tone and style of a research contribution. If the
>     draft was presented as an essay or discussion piece that would not
>     matter. If it's intended as a research contribution, then it
>     should adopt a more rigorous, strongly justified, neutral point of
>     view and a more systematic approach.
>     >
>     >   *
>     >
>     >     Section 5.3 ("Specific questions raised from the literature
>     review") lists a set of questions to consider. While these may be
>     reasonable questions, it’s not clear how the follow from the
>     literature review. Again, as a research contribution, the draft
>     should clearly evidence why these are appropriate questions.
>     >
>     > Analysis
>     >
>     >   *
>     >
>     >     "To answer our research question regarding how Internet
>     architecture enables and/or inhibits such human rights…" – that’s
>     not the stated research question
>     >
>     >   *
>     >
>     >     This section includes a range of case studies, but doesn’t
>     clearly motivate why these particular examples were chosen, or how
>     they relate to the literature survey and the specific questions
>     identified. This is not such a concern for a discursive essay that
>     but a research contribution would need to follow a more systematic
>     approach.
>     >
>     >   *
>     >
>     >     The discussion of spam could include significantly more
>     technical depth. Many of the members of the former Anti-Spam
>     Research Group are still active in the IETF and IRTF community and
>     might be willing to provide input.
>     >
>     >   *
>     >
>     >     The inclusion of IRC as an example is surprising, given how
>     infrequently it’s used these days. There are many more widely used
>     messaging or social media services that could be used to
>     illustrate the same points.
>     >
>     >   *
>     >
>     >     The discussion of WebRTC notes the “ability to function
>     without a central server” as "a strong facilitator of freedom of
>     association and assembly", but WebRTC requires a central server to
>     operate (the media data is sometimes sent in a peer-to-peer
>     manner, but there is always a central server involved to setup the
>     communication).
>     >
>     >   *
>     >
>     >     The discussion of WebRTC notes the privacy risks of
>     collecting IP addresses, but doesn't mention that the IETF
>     considered this issue when it was identified and made
>     recommendations on how to address it that have been widely
>     implemented [RFC 8828]. Further, many of the other protocols
>     identified have similar information leaks that are not mentioned.
>     >
>     >   *
>     >
>     >     The discussion of BGP as a peer to peer protocol, in Section
>     6.3.5, reflects a model of peering and transit that is of
>     increasingly limited applicability in the Internet. Geoff Huston
>     makes this argument clearly in his "Death of Transit" talk.
>     >
>     >   *
>     >
>     >     BitTorrent is an example of a peer to peer protocol, but
>     there are other peer to peer protocols with quite different
>     properties that aren’t mentioned. In particular, many BitTorrent
>     systems use a central tracker while other designs may be more
>     decentralised.
>     >
>     >   *
>     >
>     >     The discussion of internationalisation confuses
>     internationalisation of the protocol elements and
>     internationalisation of the user visible features. Both are
>     important but combining them into a single discussion confuses
>     rather than helps. When considering internationalisation of
>     protocol elements, the document discusses the issue from a social
>     perspective but neglects that there are strong technical arguments
>     that such internationalisation hinders interoperability and
>     evolution of the network. I might agree that the social arguments
>     are often neglected by the protocol development community, and
>     that this is problematic, but a draft that highlights the social
>     issues but ignores the technical concerns is also problematic.
>     >
>     > Happy to discuss further.
>     >
>     > Regards,
>     > Colin
>     >
>     > --
>     > Colin Perkins
>     > https://csperkins.org/ <https://csperkins.org/>
>     >
>     >
>     > _______________________________________________
>     > hrpc mailing list
>     > [e-mail]
>     > https://www.irtf.org/mailman/listinfo/hrpc
>
>     -- 
>     Niels ten Oever, PhD
>     Co-Principal Investigator - critical infrastructure lab -
>     University of Amsterdam
>     Assistant Professor - Department of European Studies - University
>     of Amsterdam
>
>     W: https://criticalinfralab.net
>     W: https://nielstenoever.net
>     PGP: 4254 ECD5 D4CF F6AF 8B91 0D9F EFAD 2E49 CC90 C10C
>     _______________________________________________
>     hrpc mailing list
>     [e-mail]
>     https://www.irtf.org/mailman/listinfo/hrpc
>
>
> _______________________________________________
> hrpc mailing list
> [e-mail]
> https://www.irtf.org/mailman/listinfo/hrpc