Re: Unit test proposal

[Kenneth McMillan <[e-mail]>]

Hi Ian,


I'm thinking of doing just QUIC for now and not HTTP/3.  So the upper interface would have operations like "listen", "connect", "open stream", "send/receive" and so on. The lower interface would have operations like "datagram send/receive", "sleep", "random bytes" and so on. I'm also thinking of an interface for cooperative multitasking. This would have two calls: "block" would be called by a thread when it is about to do an operation that might block, like taking a lock, and "unblock" would be called when the operation completes. This would give the test harness the ability to schedule threads, which would allow deterministic replay, as well as exploring thread schedules.


The interface would be defined in terms of simple C-style calls and callbacks, with basic serializable data structures and opaque pointers or ID's. This makes logging and replay possible. One option is to use an IDL (for example, gRPC) to make the interface language-independent and also generate a lot of boilerplate code automatically.


Dirkjan makes a good point that it may be difficult to capture all the functionality  of the various

implementations in different languages. On the other hand, a lot of the complexity of interfaces is for

efficiency purposes, and we are not concerned with efficiency here. We can make the interface very simple

and  the adapters as inefficient as we like, since this code is only for functional testing purposes.


Bets -- Ken











Sent from Outlook<http://aka.ms/weboutlook>


________________________________
From: Ian Swett <ianswett=[e-mail]>
Sent: Wednesday, February 27, 2019 8:21 AM
To: Dirkjan Ochtman
Cc: Kenneth McMillan; QUIC WG
Subject: Re: Unit test proposal

This would be interesting, but I'm a bit unclear on exactly what the unit test interface would be.  And are you thinking of interfacing at the QUIC transport layer, the HTTP layer, or both?

Agreed that logging and replay is extremely important in debugging.

The closest work I can think of is QUIC-Tracker, which I believe is somewhat different than what you have in mind: https://github.com/QUIC-Tracker/quic-tracker<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FQUIC-Tracker%2Fquic-tracker&data=02%7C01%7Ckenmcmil%40exchange.microsoft.com%7Ce441287ef6474c7bb7eb08d69ccfbad1%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636868813340085808&sdata=RWtaXX39RKM0YnGrLFSGHjKLTp6%2FeVUiOVRp0fNsjQ4%3D&reserved=0>

On Thu, Feb 21, 2019 at 3:22 AM Dirkjan Ochtman <[e-mail]<mailto:[e-mail]>> wrote:
On Wed, Feb 20, 2019 at 2:05 AM Kenneth McMillan <kenmcmil=[e-mail]<mailto:[e-mail]>> wrote:
Please let me know if you think this seems worthwhile, or if it's redundant with any existing work.

 It sounds useful to me. On the other hand, I'm not sure a language-independent API is the best form here? It seems that languages are sufficiently different that a single API might be hard to implement sanely/simply across different languages. Maybe it would make more sense to define a file format, similar to what's already been done for QPACK (I believe it's called QIF)? It could probably be based on how Wireshark pcaps are defined.

Regards,

Dirkjan