Re: [Rfcplusplus] Conversation as metaphor

["Brian Trammell (IETF)" <[e-mail]>]

'Morning, all, pseudoacademic here (waves).

> On 11 Jul 2018, at 01:46, Brian E Carpenter <[e-mail]> wrote:
> 
> On 11/07/2018 06:15, Ted Hardie wrote:
>> Howdy,
>> 
>> On Tue, Jul 10, 2018 at 11:06 AM, Eric C Rosen <[e-mail]> wrote:
>> 
>>> Maintaining the current publication process for RG output but changing the
>>> name from "RFC" to "IRTF-Masterpiece" is not going to make any difference
>>> in how the academic community values the work.
>> 
>> That's not what others are saying.  The problem as it has been explained to
>> me is that tenure committees and similar academic assessments look at the
>> output of the individual in part by looking at where the output has
>> appeared.  When it appears in a series which looks to them to be primarily
>> engineering specifications, it is not valued as highly.  A differently
>> labeled series which was entirely composed of research output might be
>> valued differently.

It depends on the committee and the question before them, but it's not quite so cut-and-dried, at least outside the US. Engineering work is valued (as impact) when there's also "more serious" academic work next to or behind it; exclusively engineering work is generally not.

> That's highly unlikely. Although the whole business of journal impact factors
> and the value of conference publications is mainly pseudo-science,

Indeed, one of the nice things about Google Scholar (for us pseudoacademics who have been fortunate enough to be able to play in the space between Internet science and Internet engineering) is that it's good enough for these pseudoscientific purposes (i.e. people who want a number that allows them to classify a person as "a serious academic"), while being quite a bit less picky about the label on the publication or on the publications citing it.

> and the
> whole question of open-access journals (where the author's institution pays)
> versus traditional journals (where the reader's institution pays) is very
> contentious, the chance of a brand new series from the IRTF breaking into
> that game is vanishingly small. (IMNSHO, of course)
> 
> RFCs have a 50 year history and are in fact widely cited in academic
> publications. My own 2nd highest citation count** is 422 for RFC1958
> (unfortunately, most indexes have me as author, not editor). My highest
> non-RFC citation count is 153.

The case of RFCs-as-peer-reviewed-academic-work is one on which someone could almost do academic work on. (Indeed, Brian co-authored an [article on this for CCR][1], though it's an editorial note and therefore, strictly speaking in the manner of caring about the difference between Standards Track and Informational RFCs, does not count).

On the one hand, they are usually often cited, because they're often used as a citation when an academic work needs to refer to an Internet protocol, which they often do. My own most-cited non-patent in [Google Scholar][2] is an tutorial that was designed to be a more useful cite than 5101/7011 for IPFIX and than 3954 for Netflow in these contexts; number four is RFC 5103, usually cited as "this study uses bidirectional flow data [RFC5103] as input...", i.e. it's most often used as academic citation shorthand for the *problem* 5103 addresses, not the particular solution therein described.

On the other hand, protocol specifications are generally not academically rigorous, in the sense that the decisions that they document are sometimes (often?) made for accidental reasons as opposed to essential ones (i.e. tradition or politics, not a rigorous theoretical or empirical evaluation of the solution space). The criteria for evaluation of a protocol are different than those for most academic work in networking. (Whether *that's* a good thing is another question, and one that in part MAPRG was set up to provide a venue to address.)

On a, um, third hand, in academia there's a complete lack of understanding (or interest in understanding) the work that goes into an RFC. They're often thought of as equivalent to tech reports (maybe in part because that's how you cite them in bibtex), but academics I've explained the process to generally comment with surprise something like "oh, so it's kind of like a big journal article", at least in the terms of the amount of review.

> **according to Google Scholar.

The ** here is, if I recall correctly, the work of Heather, who worked to get RFCs correctly indexed in Scholar. (Thanks, Heather!!)

Back toward being on topic, what does the value of the brand mean in academia? Different things to different academics. Do I support the use of alternate stream labels and/or alternate publication venues for IRTF output? Depends on the RG and the type of output, but I don't see that a new label would automatically be a less-useful brand than RFC.

(With respect to the IETF use of alternate labels specifically and the future of the stream more generally, ISTM that the proposed experiment has started a a useful conversation, and that converging on a problem statement or statements would be a good outcome for Monday's BoF. I'm still forming my own opinions on the relative importance of the various problems with the series. "Unsuitability of RFCs as a publication stream for Internet-related research" is not a big one, though.)

[1]:http://delivery.acm.org/10.1145/1680000/1672315/p31-carpenter.pdf
[2]:https://scholar.google.ch/citations?user=vTA2dL0AAAAJ&hl=en

> 
>   Brian
> 
>> I agree with Eliot's earlier point that this is a "might", and that the
>> value ascribed to it depends not just on its output but on the work done to
>> highlight the stream's output.
>> 
>> regards,
>> 
>> Ted
>> 
>> 
>> 
>> _______________________________________________
>> Rfcplusplus mailing list
>> [e-mail]
>> https://www.ietf.org/mailman/listinfo/rfcplusplus
>> 
> 
> _______________________________________________
> Rfcplusplus mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/rfcplusplus