Re: [tsvwg] [saag] TSVWG WGLC: draft-ietf-tsvwg-transport-encrypt-08, -> logging

[Gorry Fairhurst <[e-mail]>]

On 13/10/2019, 22:17, Christian Huitema wrote:
> Thanks, Gorry. First, let me say that I appreciate your approach to not
> start a debate about whether encryption is good or bad, but rather to
> assume that encryption is deployed, and then study how we perform a set
> of functions.
>
> On 10/12/2019 12:48 AM, Gorry Fairhurst wrote:
>> I'm intrigued by how far we can understand opportunities for open
>> client logs to replace pcap dumps, and would like to delve a little
>> deeper into what we understand - there seem many possible actors and
>> that may be interesting to look at a few viewpoints:
>>
>> * a user of a transport protocol (e.g., understanding cost/benefit of
>> feature choices);
> I don't understand this specific use case very well. Do you mean an
> application developer, or an end user? If you do mean an application
> developer, then the features will be selected by using options exposed
> by the API of a protocol implementation. I assume that the
> implementations have a logging option, ideally providing a standard
> logging format like Qlog. This is somewhat similar to the protocol
> developer problem listed below.
Yes, and in many cases I can see these people who could have used 
wireshark to see what happens when they turn something on in the API, 
will now look to local (and possibly server) logs to find that out. If 
that's the case, the WG needs/will get the logging correct and the 
people could well be happy; or could learn how to use a wireshark plugin.
>> * a protocol developer (e.g.,  debugging transport interactions end to
>> end);
> We do have a fair bit of experience handling that issue during the
> development of QUIC. At the very beginning, each implementation had its
> own logging format. During tests, we would turn on logging at both the
> client and the server, study the traces, and rapidly find the causes of
> the interoperability problems -- and potentially add additional logging
> options to the code if we could not find the issue.
>
> As we progressed, the tests became more complex, looking for example at
> debugging issues in error recovery and congestion control. We saw the
> development of two tools:
>
> 1) Plugins for Wireshark, and an API between the plugin and protocol
> stacks. The API provides the plug-in with the decryption key, so
> Wireshark traces can include complete traces of the protocol.
>
> 2) Standardized log formats such as Qlog, and then a set of tools for
> visualizing this traces and obtain graphs similar to those common with TCP.
>
> The QUIC interop tests involve 20 different implementations, so I would
> say that developers have been quite successful using these tools.
>
> I assume that the "student" use case mentioned later in the thread is
> very similar to the developer use case.
>
>> * a service provider (e.g., trying to understand performance
>> impairments in the network);
> I think we need to split that scenario in two: debugging following a
> customer complaint, and general network monitoring.
>
> For the client case, the solution would be to gather logs on the
> client's machine. For example, using Chrome, turn on the Quic logging
> options and then send these logs to the support center. This supposes
> that the applications used by the client do have an option to turn on
> logging, similar to what Chrome does. Calling for that support in the
> draft could be useful!
Yes, I agree, if the API as well as the log were standardised it would 
be helpful - I don't think we discuss standardising the API. Would you 
like to suggest text, or should Colin and I work something out?
> For general network monitoring, tools like Netflow will provide
> statistics at the 5-tuple granularity. We have seen work reported in the
> Quic WG about using the spin bit to monitor end to end delays in that
> scenario.
The spin bit is mentioned, and thsi text could certianly be updated. I'd prefer to avoid commenting on other proposals that have yet to gain IETF consensus.

> Then there are of course tools developed in the IPPM working
> group, such as OWAMP and TWAMP. If I understand correctly these tools
> enable measurements between test points without requiring observation of
> transport headers in transit.
Yes - if I do have equipment that supports these (perhaps more likely 
with major service
providers, than in an enterprise), these could provide some 
measurements, I'm not sure of the maturity of tools and their interop - 
but that's a bigger topic.
>> * researchers/equipment developers seeking to correlate/tune transport
>> timing with network policies (e.g., interactions between congestion
>> control and link scheduling);
> This seems quite similar to the protocol developer problem.
Except it's not really... in this case, the network device/segment being 
observed is moving traffic, and observers often don't have much clue 
about the client's platform, app setups, etc. Why would they? (They also 
don't have away to turn-on debug or logging at clients) - if I look at a 
queue, I see packets. If I know these packets are using a well-known 
protocol TCP, RTP, etc, I can look to see how the CC is being impacted 
by the way that I configure my device (I could tune things if I had 
incentive), if it's IPsec-encrypted traffic, I would generally not 
bother looking more. I think some people  contributed text to an earlier 
revision in this direction, please see if that seems to be heading in a 
useful direction.
>> * exploring traffic archives (e.g., trends in what is supported or
>> determine when an anomoly appeared).
> Some traffic properties are observable on the encrypted flow. Timing of
> connection establishment, duration of sessions,  or volume of data are
> obvious to measure. The spin bit can also be observed. Fingerprinting
> techniques can be used to identify application profiles. It should even
> be possible to assess application timing. That's probably enough to
> detect macroscopic changes and flag anomalies.
I don't know. There are going to be cases where this would not be enough 
to derive useful outputs, e..g, I suspect looking at the QoE impacts of 
link layers would be challenging.
>
> Of course, you will not see which specific features the encrypted
> transport deploys, not by simply taking traffic traces. But those
> features can probably be obtained by other means, such as instrumenting
> servers.
Maybe I'm not understanding. I'd have thought this was most useful if 
you wanted to test what a specific client used. But also, perhaps it is 
not feasible for someone working on firewall or a router config within a 
company of ISP.
>> I've taken all of these viewpoints at some time using pcap to look at
>> TCP.
>>
>> Clearly when we use QUIC, there can be alternatives and there is now
>> some experience with client (and server) logs, what do you think we
>> can usefully say in addition to or instead of what the draft says
>> about logging (currently mentioned only briefly in section 6.1) ?
>>
>> The text on spin-bit could indeed be developed a little (the present
>> text predates the spin bit being incorporated as an optional part of
>> the base spec). What do you think should be said (currently at the end
>> of 6.4)?
> I would need to write a detailed review of your draft. That will take
> some time...
>
> -- Christian Huitema
Thanks for this feedback - and I'm sure we both look forward to more 
comments. It is helpful to expose issues were they can be described, but 
to avoid proposing solutions that we don't yet have operational 
experience about.

Fow what is worth as an inidvidual, I'd be keen to see "new solutions" 
collected in some future document, but this isn't the place to propose 
those.

Gorry