Re: [Mops] Roman Danyliw's Discuss on draft-ietf-mops-streaming-opcons-10: (with DISCUSS and COMMENT)

[Spencer Dawkins at IETF <[e-mail]>]

Hi, Roman,

On Fri, Aug 5, 2022 at 11:33 AM Roman Danyliw <[e-mail]> wrote:

> Hi Spencer and Jake!
>
>
>
> Thank you for the edits in -11 and your patience in the time it took me to
> review it.  It addresses my DISCUSS and primary COMMENT points, so I’ve
> cleared my ballot.  I saw a subsequent discussion/issue on saying more
> about the security properties of DRM tied to one of my unaddressed COMMENTs
> even after this document version.  I deliberated on whether more needs to
> be said.  Yes, more would be helpful, but upon reflection, I worry that
> this is a large topic to unpack and the residual benefit might not be
> sufficient.  There is a generic marker for it already.
>
>
>
> Bottom line, in my view this is ready to ship.  Again, thanks for your
> patience.
>
> Now that I've looked back at
https://github.com/ietf-wg-mops/draft-ietf-mops-streaming-opcons/issues/153,
which contains your ballot COMMENT points (as distinct from your DISCUSS
points, I'm remembering that the editors did chat about responses, but we
have not specifically addressed these comments (yet - we had two open
issues when MOPS met at IETF 114, and that was only from lack of time).
Some were on text that was changed due to other reviewer and ballot
comments, so I'm checking to see what's left.

If you're "it's ready to ship", no further action is needed on your part,
and the editors will Do The Right Thing.

So, for the editors, and the MOPS community, I'll respond here on the
mailing list, and put my proposed changes in Github for review.

DEAR FELLOW EDITORS, CHAIRS, AND WORKING GROUP MEMBERS:

I'm comparing the -10 text Roman pointed to in his comments to what's
already in the post-11 Editor's Copy in Github (for anyone following along
at home, I'm using this URL:
https://www.ietf.org/rfcdiff?url1=draft-ietf-mops-streaming-opcons-10&url2=https://github.com/ietf-wg-mops/draft-ietf-mops-streaming-opcons/raw/gh-pages/draft-ietf-mops-streaming-opcons.txt),
and proposing text where that seemed appropriate.

As a general characterization, Roman identified a LOT of uses of
inconsistent terminology. I'm suggesting these changes because, in my
experience, he was asking about things that the RFC Editor would also ask
about, except that the RFC Editor would be asking after we've forgotten
what we meant ... 🤔

Comments here, or in Github, are welcome.

My suggestion to Eric and the chairs was for us to look the draft over
between now and this coming Friday, have the editors submit the Editor's
Copy in Github as -12, and invite Eric to send -12 off to the RFC Editor as
approved.

Do The Right Thing, of course.

Best,

Spencer


----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

** Thanks for distilling a diverse ecosystem into a single document.  To
make
navigating this ecosystem easier, please provide definitions for terms like
“media provider”, “streaming media provider”, “intermediary”, “intermediate
streaming operators”, and “content provider”; and show a notional
architecture
between them.  If some of these terms or elements are interchangeable,
please
make it clear.  I’ll call out some of the places where I got confused below.

I did add selected terms to Section 1.1. I'm happy to consider others. I do
think that at least some of the confusion from -10 is lessened, because
I've worked to use the same terms consistently, rather than using multiple
synonyms in text from different authors.


** Section 3.5.  Is there are reference that can be provided to support the
assertion that better codecs (which save bandwidth) can’t offset increased
video consumption.

I see why Roman was reading this text as predicting the future, but it was
intended as a statement of past performance. I changed "could not have
offset" to "did not offset". I'm betting that trend continues, but I don't
need to say that in a WG-consensus document.


** Section 4.  Is the definition of “ultra low-latency” used here the same
as
“ultra low latency caches” in Section 3.4?

I'm not seeing "ultra low latency caches" in the draft (either -10 or the
current editor's copy). I did see that there was one occurence of "ultra
low-latency", among the occurrences of "ultra-low-latency", so I changed
that one to match, for clarity.


** Section 4.2.  What is a “premium service to the delivery of live
video”?  Is
that “premium” in the sense of my ISP and associated provisioning? Or in the
sense of my relationship with the content provider?

My understanding is that we're talking about my relationship with the
content provider, which may reflect the relationship between the content
provider and my ISP (for example, placing CDN nodes in the ISP's network).
I'm not sure there's a specific text change that would be helpful here
(we're talking about business relationships, which may vary a lot from case
to case).


** Section 4.2.  This section lists HLS, DASH and a few other enabling
technologies.  Are the technologies for ultra low-latency applicable?

(Speaking only for myself) Media providers targeting low-latency
applications are typically planning to exploit HTTP caching infrastructure,
which doesn't handle RTP-based media. So, I'd say ultra-low-latency
technologies are not applicable to low-latency applications.


** Section 4.3.

   This level
   of latency is often considered adequate for content like news or pre-
   recorded content.

What is the difference between the “pre-recorded content” described here and
the “on-demand” described in Section 4.4?

Nice catch 😉. Keeping in mind that there's not a bright line between
non-low-latency live media and on-demand media, there may not be a huge
difference, if (for example) you're using the same segments for on-demand
and for non-low-latency live media, but the media player does not adapt the
requested bitrate to measured network conditions. But I do think it's fair
to remove "or pre-recorded content" from section 4.3.


** Section 5.4

   In general this is a rapidly developing space with many
   considerations, and media streaming operators engaged in advertising
   may need to research these and other concerns to find solutions that

Who are “media streaming operators”?  Are those different than “media
providers”

No, they're not different. I'm rephrasing "media providers" as "streaming
media operators" throughout the document.


** Section 6.1.
   … we have trusted UDP-based
   applications to limit their impact on other users

Who is the “we”?  Can this “trust” please be clarified.  Same comment for
Section 6.2.

Section 6 has been significantly re-written as a result of INTDIR and
TSVART reviews - this text went away in -11.


** Section 6.1.

   Although it is
   possible to saturate a path between a DNS client and DNS server with
   DNS requests, in practice, that was rare enough that DNS included few
   mechanisms to resolve contention between DNS users and other users
   (whether they are also using DNS, or using other application
   protocols that share the same pathways).

Can this observation please be restated?  How is a DNS server to resolve
contention for non-DNS traffic?

Section 6 has been significantly re-written as a result of INTDIR and
TSVART reviews - this text went away in -11.


** Section 7.
      Media encrypted at the application layer, typically using some
      sort of Digital Rights Management (DRM) system, and typically
      remaining encrypted "at rest", when senders and receivers store
      it.

Is this proposed scheme where the media remains encrypted even when at rest
in
fact just describing object encryption/security – that is, the security
properties of the encrypted media hold and are independent of the
communication
mechanism transporting them?

Yes, and if that's not clear to a SEC AD, I should make it clearer.

I'm proposing "- Media encrypted at the application layer, typically using
some sort of Digital Rights Management (DRM) system or other object
encryption/security mechanism, and typically remaining encrypted at rest,
when senders and receivers store it."


** Section 7.

   The use of strong encryption does provide confidentiality for
   encrypted streaming media, from the sender to either an intermediary
   or the ultimate media consumer, and this does prevent Deep Packet
   Inspection by any intermediary that does not possess credentials
   allowing decryption.

What is an intermediary here?  Is it any on-path observer?

Not quite. It's on-path, with credentials allowing decryption, and often,
it's not just observing - a streaming media operator may provide an
intermediary to perform some sort of inspection or transformation. I'll
rephrase this to make that clearer.

I'm proposing "The use of strong encryption does provide confidentiality
for encrypted streaming media, from the sender to either the ultimate media
consumer, or to an intermediary that possesses credentials allowing
decryption. This does prevent Deep Packet Inspection by any on-path
intermediary that does not possess credentials allowing decryption."


** Section 7.1.

   An intermediary that can identify an
   encrypted media stream without decrypting it, may be able to
   "fingerprint" the encrypted media stream of known content, and then
   match the targeted media stream against the fingerprints of known
   content.  This protection can be lessened if a media provider is
   repeatedly encrypting the same content.

-- I had trouble following the setup.  I assumed the text was trying to
abstractly describe the methodology in [CODASPY17].  Recommend:

NEW
An on-path observer that can identify that encrypted traffic contains a
media
stream, could “fingerprint” this encrypted media steam, and then compare it
against “fingerprints” of known content.

This proposed text looks fine to me (I'll add it to the editor's copy). And
just calling out the reference doesn't add clarity - I provided the title
("Identifying HTTPS-Protected Netflix Videos in Real-Time") in this
paragraph.


-- Per the last sentence, “[t]this protection …”, what protection is being
lessened here?

I'm proposing "The protection provided by strong encryption can be further
lessened if a streaming media operator is repeatedly encrypting the same
content."


** Section 7.1

   If traffic analysis is successful at identifying encrypted content
   and associating it with specific users, this breaks privacy as
   certainly as examining decrypted traffic.

Please be more precise on the security properties being lost rather than
summarizing it has “privacy”

I'm proposing "If traffic analysis is successful at identifying encrypted
content and associating it with specific users, this tells an on-path
observer what resource is being streamed, and by who, as certainly as
examining decrypted traffic."


** Section 7.2.

   If a content provider does not actively work to avoid interception by
   intermediaries, the effect will be indistinguishable from
   "impersonation attacks", and endpoints cannot be assumed of any level
   of privacy.

Please be precise on what security properties are in play from
“impersonation
attacks” and loss of “privacy”.  In the situation described above, it seems
to
be both confidentiality (e.g., the intermediary can see the entirety of all
communication between the end-point and the media provider) and authenticity
(e.g., no party can trust the content received in fact came from the
expected
party)

I'm proposing text based on Roman's comment - "If a streaming media
operator  does not actively work to avoid interception by on-path
intermediaries, the effect will be indistinguishable from "impersonation
attacks," and endpoints cannot be assured of any level of confidentiality,
and cannot trust that the content received came from the expected sender."


** Section 7.2

      Server And Network assisted DASH [MPEG-DASH-SAND] - this
      specification introduces explicit messaging between DASH clients
      and network elements or between various network elements

Are “network elements” the same as “intermediaries?”


No, and the phrase "network elements" accurately reflects the
MPEG-DASH-SAND specification, but is very unclear in this document.

I'm proposing "- Server And Network assisted DASH {{MPEG-DASH-SAND}} - this
specification introduces explicit messaging between DASH clients and
DASH-aware network elements or among various DASH-aware network elements,
for the purpose of improving the efficiency of streaming sessions by
providing information about real-time operational characteristics of
networks, servers, proxies, caches, CDNs, as well as DASH client’s
performance and status."


** Section 7.2
   The choice of whether to involve intermediaries sometimes requires
   careful consideration.  As an example, when ABR manifests were
   commonly sent unencrypted some networks would modify manifests during
   peak hours by removing high-bitrate renditions in order to prevent
   players from choosing those renditions, thus reducing the overall
   bandwidth consumed for delivering these media streams and thereby
   improving the network load and the user experience for their
   customers.

-- Please be clear on who is deciding to involve the intermediaries. Is it
the
media provider?

It's the (checks current terminology) streaming media operator.

I proposed "A streaming media operator's choice of whether to involve
intermediaries requires careful consideration."

Note that I removed "sometimes" - in 2022, I don't think careful
consideration is a special case.


-- It isn’t clear who is operating these intermediaries – is it the media
provider or the connectivity provider of the end user?  I took to be the
latter.  Assuming that’s right, how is this a design choice by the media
providers? Did the media providers explicitly choose to use an unencrypted
protocol to support these network operators; or did the network operators
exploit this design choice without coordination?

The intermediaries are operated by - or, perhaps, better stated as
"authorized by" - streaming media operators.

In 2022, we're talking about historical practice here, but when streaming
media operators provided unencrypted media, network operators took
advantage of that to modify the media, and even the transport protocol
(e.g. "split-TCP", etc), in order to provide better performance. With
encrypted media, network operators can't insert an intermediary without
coordinating with the streaming media operator.

I think Roman's confusion was based on how unclear this text was - now that
we've clarified that streaming network operators must authorize
intermediaries when they use strong encryption, I don't think we need
additional text changes here.


** Section 7.2

Now that ubiquitous encryption typically prevents this
   kind of modification, in order to maintain the same level of network
   health and user experience across networks whose users would have
   benefitted from this intervention a media streaming operator
   sometimes needs to choose between adding intermediaries who are
   authorized to change the manifests or adding significant extra
   complexity to their service.

There is an implicit architectural assumptions being made that I’m not
following.  This text suggests that the media streaming operators are
operating
the intermediaries.

This should be clearer, because of previous text changes.


Are these intermediaries in a network controlled by the
media streaming operator?  Before, intermediaries were modifying manifests,
and
now due to encryption more intermediaries are needed?

I think Roman is correct about the architectural assumptions being implicit
(and unclear). My understanding here is that "before", access network
operators could add intermediaries to modify manifests without coordinating
with streaming media operators, and "now", the streaming media operators
must either add intermediaries that they control, or find some other way to
address peak hour traffic. There are various ways to accomplish this
("adding some other form of complexity"), but they are all more complex
than providing manifests to media players with media renditions that the
access network operators can't support, and saying "you figure it out,
ok?".

Off the top of my head, I can imagine a streaming media operator saying
"I'm going to use strong encryption, and I don't want the complexity of
adding intermediaries that I operate, so I'll ask the access network
operators to tell me when they are in peak hour traffic loads, and *I'll
remove the media reinditions myself*". That's almost certainly a bad plan,
but it would involve added complexity for the streaming media operator, and
I can't think of any other way to handle peak traffic loads in someone
else's network that *doesn't *involve added complexity for the streaming
operator.

If someone can suggest ways that don't involve added complexity for the
streaming operator, please send text!


** Section 7.3.

Unfortunately, as noted in [RFC7258], there is no way to prevent
   pervasive monitoring by an "attacker", while allowing monitoring by a
   more benign entity who "only" wants to use DPI to examine HTTP
   requests and responses in order to provide a better user experience.

s/Unfortunately//.  Keeping "unfortunately" is a value judgement.  Some
users
might be quite pleased by this development.

This is "However" in -11. I may have made that change because I remembered
reading Roman's review, but "unfortunately" has gone away.


** Section 7.3.  What is an “Intermediary streaming operator”?  How is that
different than a “media provider” or “media streaming provider”?

For reference, the text Roman is referring to is "If a modern encrypted
transport protocol is used for end-to-end media encryption, intermediary
streaming operators are unable to examine transport and application
protocol behavior."

I believe I wrote this text, and I almost certainly intended to say (so, my
proposed text is) "If a modern encrypted transport protocol is used for
end-to-end media encryption, unauthorized on-path intermediaries are unable
to examine transport and application protocol behavior."


Editorial

** Section 1.  Consider if you can simplify the first sentence, “This
document
examines …”.  It is dense.

This was substantially rewritten when synchronizing the abstract and
introduction.


** Section 1.  Typo. s/availability,but/availability, but/

Text was previously replaced.


** Section 3.2.  Typo. s/contraints/constraints/

Text was previously replaced.


** Section 3.2. s/bandwith/bandwidth/

Text was previously replaced.


** Section 3.2.  Typo. s/occuring/occurring/

Text was previously replaced.


** Section 3.4  Per the paragraph starting with “Caching and pre-loading …”,
consider if this can be simplified.  It’s exactly one sentence long with
multiple “especially …” clauses.

Yes, it is. My proposed text is "Caching and pre-loading can also reduce
exposure to peering point congestion, since less traffic crosses the
peering point exchanges if the caches are placed in peer networks. This is
especially true when the content can be pre-loaded during off-peak hours,
and if the transfer can make use of "Lower-Effort Per-Hop Behavior (LE PHB)
for Differentiated Services" {{RFC8622}}, "Low Extra Delay Background
Transport (LEDBAT)" {{RFC6817}}, or similar mechanisms."


** Section 3.4.  Editorial

   And as with other parts of the ecosystem, new technology brings new
   challenges.  For example, with the emergence of ultra-low-latency
…
Consider if the first sentence is needed.  This who paragraph seems to be
related to caches   Perhaps just start this paragraph with “With the
emergence
of …”

Agreed, and I made this change.


** Section 3.5. Consider defining “mobile data.”

I agree. I expanded this as "mobile data usage in cellular access networks".


** Section 4.1.

   This introduces new challenges relative to less-
   restricted levels of latency requirements because this latency is the
   same scale as commonly observed end-to-end network latency variation
   (for example, due to effects such as bufferbloat ([CoDel]), Wi-Fi
   error correction, or packet reordering).

This sentence doesn’t parse.

(see next comment)


** Section 4.1.

   These effects can make it
   difficult to achieve this level of latency for the general case, and
   may require tradeoffs in relatively frequent user-visible media
   artifacts.

The wording isn’t right here –- “… may require tradeoffs ...”  Does this
text
mean “... may require accepting relatively frequent user-visible media
artifacts.”

This entire paragraph wasn't easy to read, and when I fixed it, the result
made the following paragraph superfluous.

My proposed text is "Some media content providers aim to achieve this level
of latency for live media events. This introduces new challenges when
compared to the other latency categories described in this section, because
ultra-low-latency is on the same scale as commonly observed end-to-end
network latency variation, often due to bufferbloat ({{CoDel}}), Wi-Fi
error correction, or packet reordering. These effects can make it difficult
to achieve ultra-low-latency for many users, and may require accepting
relatively frequent user-visible media artifacts. However, for controlled
environments that provide mitigations against such effects,
ultra-low-latency is potentially achievable with the right provisioning and
the right media transport technologies."


** Section 5.5.3.  Typo. s/detction/detection/

Text was previously replaced.


** Section 6.1.  Per “In recent times”, please restate this as this phrase
is
not precise and is unlikely to age well.

Text was previously replaced.


** Section 6.1.  Typo. s/tansport/transport/

Text was previously replaced.


** Section 6.3.  Per “… without melting the Internet”, consider a less
colloquial expression.

My proposed text is :without causing massive congestion events for the
Internet".


** Section 6.3.  Per “As noted in [RFC8312], both the CUBIC congestion
controller and its predecessor …”, consider re-writing this sentence for
readability.  The double “and both were ...” clauses makes this text very
dense.

I agree. My proposed text is "As noted in {{RFC8312}}, both the CUBIC
congestion controller and its predecessor BIC have significantly different
behavior from Reno-style congestion controllers such as TCP NewReno
{{RFC6582}}, both were added to the Linux kernel to allow experimentation
and analysis, and both were then selected as the default TCP congestion
controllers in Linux, and both were deployed globally."


** Section 7.3.  The opening paragraph of this section appears to be a
restatement of the opening paragraph of Section 7.1.

I agree.

Because Section 7.1 is overarching, I propose starting Section 7.2 this way

"Hop-by-hop media encryption offers the benefits described in
{{gen-encrypt}} between the streaming media operator and authorized
intermediaries, among authorized intermediaries, and between authorized
intermediaries and the ultimate media consumer, but does not provide these
benefits end-to-end. The streaming media operator and ultimate media
consumer must trust the authorized intermediaries, and if these
intermediaries cannot be trusted, the benefits of encryption are lost."

and starting Section 7.3 this way:

"End-to-end media encryption offers the benefits described in
{{gen-encrypt}} from the streaming media operator to the ultimate media
consumer."