Re: [EToSat] Download Regression Tests

[François Michel <[e-mail]>]

Hello,

We also performed some experiments with PQUIC/Picoquic over a satellite 
link with Nicolas and Emmanuel. We analyzed the loss patterns and 
deduced some two-state Gilbert parameters from network traces for two 
different scenarios. This model is less general than the Gilbert-Elliott 
model but is allows us to represent bursty loss patterns: in the GOOD 
state, all the packets are successfully received. In the bad state, the 
packets are simply dropped.

Here are the two studied scenarios :

1) The client is connected to a wired network and its traffic towards 
the server is then routed through a satellite link.
2) The client is connected to a WiFi access point instead of a wired 
network. Its traffic towards the server is then still routed through a 
satellite link.

What differs between these two scenarios is that in 2) we can also 
encounter losses due to the WiFi communication.

We count the received packets at the receiver, knowing that the sender 
is configured to send packets with a contiguously increasing packet 
number: although it is feasible in QUIC, our Picoquic sender will never 
send packet n if it has not sent packet n-1. We provide the packet 
counts so that you can also do your own computations if needed.

Here are the packets count and estimated probability of transitions of 
the Gilbert model we deduced from our experiments :


For the scenario 1:
-------------------

from/to GOOD    BAD
GOOD    80709   1370
BAD     1370    86

This means that 80709 packets have been received while the previous 
packet was received (GOOD->GOOD), 86 packets have been lost while the 
previous packet was also lost (BAD->BAD), 1370 packets have been 
received while the previous packet was lost (BAD->GOOD) and 1370 packets 
have been lost while the previous packet was received (GOOD->BAD).


The average burst in the BAD state is quite small, but in our traces we 
still found one loss burst of 15 packets.

The estimated transitions probabilities of the Gilbert model:
P(GOOD->GOOD) = 0.9833087635083273
P(GOOD->BAD) = 0.016691236491672656
P(BAD->GOOD) = 0.9409340659340659
P(BAD->BAD) = 0.059065934065934064


For the scenario 2:
-------------------

from/to GOOD    BAD
GOOD    38700   700
BAD     699     385

P(GOOD->GOOD) = 0.9822335025380711
P(GOOD->BAD) = 0.017766497461928935
P(BAD->GOOD) = 0.6448339483394834
P(BAD->BAD) = 0.3551660516605166


Also quite isolated losses in general but we encountered more frequent 
bursts, and we encountered larger loss bursts such as one burst of size 
33 and one burst of size 30.
Keep in mind that these experiments are quite small and could be 
influenced by the period of the day during which they have been performed.


We hope this helps.


François



Le 3/12/19 à 19:24, Border, John a écrit :
> The following are the variables we play with:
> 
>   * Delay – *600 milliseconds*, 1000 milliseconds
>   * File Size – *1 MB*, 10 MB, 100 MB, *1000 MB*
>   * Packet Error Rate – *0.0%*, 0.1%, *1%*, 5%, 10%
>   * Plan Rate:
>       o Download (Internet to host)  - 10 Mbps, 25 Mbps, 100 Mbps, *250
>         Mbps*
>       o Upload (host to Internet) – *3 Mbps*, 10 Mbps, 25 Mbps
> 
> The ones in bold are the ones we focus on since all of the above result 
> in a lot of combinations.  If I had to pick just two, right now I would 
> go with:
> 
>   * 600 milliseconds, 1000 MB, 0.0%, 250 Mbps, 3 Mbps
>   * 600 milliseconds, 1000 MB, 1.0%, 250 Mbps, 3 Mbps
> 
> 600 milliseconds is the baseline delay.  Longer delays definitely happen 
> during traffic peaks.  Longer than one second delays happen at traffic 
> peak for traffic we classify as lower priority…
> 
> 1 MB tests are web page object size tests.  1 GB tests are download 
> tests.  (We would use 100 MB for upload tests but we have not really 
> done any yet.)  We really have only been focusing large file sizes since 
> we have been focusing on what we lose without TCP spoofing.
> 
> For error rates, I included 5% an 10% because they represent WiFi 
> extremes (and we did try them).  But, those values barely work at all. 
>   We mainly focus on 1%.  There are actually two variants of error 
> rate.  One is loss between the client and the satellite terminal.  (This 
> is the loss TCP spoofing helps with.)  The other is loss between the 
> satellite terminal and the satellite gateway which connects to the 
> Internet.  We focus mostly on client side loss.
> 
> We mostly have been testing with a 250 Mbps download plan rate so we can 
> see how things work without being artificially limited..  We are easily 
> able to fill this with TCP spoofing and ultimately we want QUIC to be on 
> par with that.  (I think the best we have seen Google QUIC do in recent 
> testing is ~30 Mbps but a few years ago it was only 3 Mbps so things are 
> moving in a good direction.)   But, you cannot get plan rates like that 
> outside of a lab right now.  In today’s real world, we run with 25 Mbps 
> download plans for customers.  Some providers use ~10 Mbps download plan 
> rates.  We look at the higher rates anyway because we are trying to look 
> ahead and satellite technology improves significantly over time.  We 
> envision 100 Mbps download plans someday.
> 
> We have not done much upload testing to date.  So, the upload plan rate 
> has mattered mostly for purposes of how much ACK traffic it can carry.  
> ACK reduction strategies are important for satellite but we have not 
> done much with this yet other than to observe that 30 Mbps of Google 
> QUIC download traffic fills the upload pipe.
> 
> Not sure if this is what you were looking for but…
> 
> John
> 
> 
> _______________________________________________
> EToSat mailing list
> [e-mail]
> https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Fetosat&data=02%7C01%7C%7Cc47c9ee8fadc4bc05bb908d7781e1768%7C7ab090d4fa2e4ecfbc7c4127b4d582ec%7C0%7C0%7C637109942949480777&sdata=nbhVVrQcHaOLIUvQOuW5CA8UjXQCZXkz7SFyxoreGR0%3D&reserved=0
>