Re: [6lo-fragmentation-dt] Performance report for fragment forwarding

[Rahul Jadhav <[e-mail]>]

Hello All,

Basic data with pacing is now available. We have added flat 50ms delay
before individual fragments are transmitted on the original sender.
And this has resulted in __drastic improvement in the PDR__ as compared to
experiment with no pacing. But on the negative side it has also induced
higher end to end latency. Anyways this was as expected.

Please check the data.
https://github.com/nyrahul/ietf-data/blob/master/6lo-fragfwd-perf-report.rst

Best,
Rahul


On Thu, 20 Sep 2018 at 18:30, Pascal Thubert (pthubert) <[e-mail]>
wrote:

> Hello Rahul
>
>
>
> *From:* Rahul Jadhav <[e-mail]>
> *Sent:* jeudi 20 septembre 2018 14:48
> *To:* Pascal Thubert (pthubert) <[e-mail]>
> *Cc:* Carsten Bormann <[e-mail]>; [e-mail];
> [e-mail]; [e-mail]
> *Subject:* Re: [6lo-fragmentation-dt] Performance report for fragment
> forwarding
>
>
>
> Please find my comments inline..
>
>
>
> All in all, some of the concerns mentioned in this mail thread should be
> reflected in the draft as well. A naive implementation or implementers
> without any background on mac layer may seriously goof up the performance
> if they simply go by the draft currently.
>
> *[PT>] makes sense. Note that a burst of IP packets for a same destination
> have the same issue. The problem has to do with transmitting frames at L2
> not that they are individual packets or multiple fragments at upper layer.
> If the upper layer makes 5 IP packets to avoid fragmentation of a larger
> packet in 5 fragments, and sends the 5 IP packets together, you’ll get the
> exact same result as if using fragments – unless I miss something always
> interesting to dig deeper..*
>
>
>
>
>
>
>
> 6TiSCH protects against fragments interfering with one another, but it
> does not protect against a Hop1 sending frag N+1 to Hop2 while Hop2 is
> sending frag N to Hop3. Even 6TiSCH needs to pace its traffic to be
> forwarded by a same next hop, and that’s true for all packets not just
> fragments. Do you have any form of listen before talk in your simulation?
>
>
>
> [RJ] Yes, there is carrier sensing before transmission in my simulation.
>
> *[PT>] Then other strategies are possible, like a longer CCA for the first
> sender. *
>
>
>
> If you do not delay the transmissions of intermediate fragments, your test
> certainly proves that the system cannot work satisfactorily. This much was
> predictable on paper. Say it take 10ms for a transmit + ack, and most
> transmissions succeed within 3-4 tries, it makes sense to do your testing
> injecting 50 to 100ms between fragment transmissions. Every node along the
> path must validate that fragments are thus spaced. Could you please
> relaunch your test with a rule like that?
>
>
>
> [RJ] Yes we ll try this and get the data again. Carsten also suggested the
> same before in this mail chain.
>
> *[PT>] very cool, looking forward to seeing the change. Great research!*
>
>
>
> For the rest please see below:
>
>
>
>
>
> If you could instrument the intermediate nodes, I’d like to see if for
> reason of retries a fragment may catch up with another one. This can be
> tested by making a hop in the middle very far and unreliable.
>
>
>
> [RJ] Sorry but I m not sure what you mean here.
>
>
>
> *[PT>] The idea is to make a link less reliable then others in order to
> cause retries so a virtually slower link. You’ll see that packets pile up
> there even if paced at the source. That’s what I mean by other packets
> catching up.*
>
>
>
> [RJ] Ok got it. Thanks
>
>
>
> Also, even if you pace at the source by inserting gaps, fragments may
> still pile up at the ingress of the slowest link on the way and your
> initial problem will be recreated there. This is only visible if you
> effectively have a slower hop, but is a very classical effect in window
> based networks, the full window ends up piling at ingress of the slowest
> link. You may recreate that by injecting additional traffic via a node on
> your path.
>
>
>
> [RJ] Agreed that a bottleneck link may still create a problem. But if you
> see the network in our report, there is no such bottleneck node/link ...
> the paths are sufficiently balanced.
>
>
>
> *[PT>] So the space must be inserted at the bottleneck too. In theory, a
> transmitter needs to ensure that every packet to a same next hop is
> sufficiently spaced with the next packet, unless the next hop is the
> destination, so the next hop can forward.*
>
> To avoid that you would need to pace at the source to a rate that is
> slower what the slowest hop would do if it was the source. Really hard to
> do. So ideally, each node would ensure that there is a minimum gap between
> fragments, which should be no-op everywhere along a path but at the slowest
> hop where fragments will pile up.
>
>
>
> Finally: you can limit the amount of queueing at the slowest hop if you
> use the recoverable frags draft, using windows of a few fragments for flow
> control.
>
> [RJ] Amount of queueing can be limited in our case by using L2 feedback as
> well i.e. if the fragment is lost (and the sender knows this based on L2
> feedback) then we can as well stop transmitting all subsequent fragments.
> For 802.15.4 the recovery procedure can be applied based on L2 feedback
> without any explicit ACK.
>
> But right now we arent using any of these advanced techniques.
>
>
>
> Our aim for this experiment was to check how much of an impact was it when
> fragment forwarding was used as mentioned in the VRB draft in single
> channel mode of operation.
>
>
>
> *[PT>] Without any pacing, the simulation is not so realistic and though
> interesting, conclusions there would be premature. I’d be very interested
> in seeing more tests varying the inter frame space and see how that impacts
> the delivery.*
>
>
>
> [RJ] Yes we ll try this.
>
>
>
> Bottom line: the more you test with one frequency only the more you’ll
> find it’s a bad idea : )
>
> [RJ] Yes we have realized (it the hard way) that using fragment forwarding
> in single channel mode is a very bad idea.
>
> *[PT>] Note that the same would happen with a series of packets in a row.
> I do not think that the fact these are fragments have much to do with it.*
>
>
>
> [RJ] With series of packets, do you mean per-hop reassembly? Well, per-hop
> reassembly does not suffer from this issue. But it suffers from buffering
> issue. So if the traffic is sufficiently sparse then using per-hop
> reassembly in this scenario makes more sense.
>
> *[PT>] No just different IP packet same destinations to be sent in a row,
> no fragmenting. Unless you do ECMP, they will be routed along a same path
> and have the same fate as fragments. This is why my expectation is that the
> the problem you point out is not limited to fragments.*
>
>
>
>  *[PT>] *: )
>
> *[PT>] *
>
> *Pascal*
>
>
>
> Take care,
>
>
>
> Pascal
>
>
>
> All the best;
>
>
>
> Pascal
>
>
>
> From: 6lo-fragmentation-dt <[e-mail]> On
> Behalf Of Rahul Jadhav
> Sent: mercredi 19 septembre 2018 07:59
> To: Carsten Bormann <[e-mail]>
> Cc: [e-mail]; [e-mail];
> [e-mail]
> Subject: Re: [6lo-fragmentation-dt] Performance report for fragment
> forwarding
>
>
>
> Doing pacing on origin-node only should be easy to change/hack and might
> improve the forwarding efficiency. But this may also shoot up the mean
> latency.
>
> We ll try this anways (add randomized delay between 10ms to 500ms) to
> check the impact on PDR. Thanks for this suggestion.
>
>
>
> On Wed, 19 Sep 2018 at 10:25, Carsten Bormann <[e-mail]> wrote:
>
> Hi Rahul,
>
> > Carsten also mentioned a pacing mechanism .. while it might improve
> fwding efficiency it will add to buffer requirement. Also such a scheme
> might be non-trivial to be implemented.
>
> Pacing is something the origin of the datagram does; I wasn’t suggesting
> the forwarders to add that (which indeed essentially puts the datagram into
> buffers there).
> Clearly, pacing is counterproductive when forwarders do full reassembly,
> as it increases the time the partially assembled datagram is vulnerable to
> fragments of other datagrams hijacking the buffer — you want to capture the
> channel as much as possible then (to the level that you’d want to send
> following fragments with a higher MAC priority).  Of course, as the
> datagram travels through the network, it will become more and more smeared
> over time.
>
> It would be interesting to see how your numbers change with some pacing at
> the origin; it should be easy to hack that in if you are not trying to do
> this in a general way.
>
> > Regarding keeping higher mac-retry, ... We have chosen mac-retry of 3
> after some experimentation (considering expected node densities and tx
> frequency). increasing mac-retry might not necessarily help, in fact it may
> backfire in terms of both PDR as well as mean latency. Would you still
> suggest to give it a try and what mac-retry do you think makes sense ?
>
> Yeah, yet another tunable that in a robust protocol really should be
> self-tuning.
> A MAC-layer retransmission is not useful if the layers above have already
> timed out.
> So the latency introduced by the backoff mechanism is really the
> determining factor here.
>
> Grüße, Carsten
>
>