Re: [tcpm] draft-eggert-tcpm-historicize-00

[Joe Touch <[e-mail]>]

Hi, Anumita,

Biswas, Anumita wrote:
...
> I have heard only anecdotal evidence of a very large company doing very
> large copies of data (about a Tera Byte) across a local network and then
> finding bit corruptions in the data received on the other end - clearly
> the link CRC and the TCP 1s compl summation checksum failed to catch
> those errors. 

Well, that's the point. It's not so clear that the checks are what failed. It
could easily have been a bug in the implementation, or an error in the hardware
that corrupted data after it was received correctly. I.e., no matter how good
these checks, there are other ways in which data can be corrupted.

> In my own company, I have seen a case where a very large block (1460
> bytes - a packet size) of data was replaced and gone undetected and we
> could not root cause the problem - the problem just disappeared after
> showing up for a few weeks - but in this case, it is possible that the
> corruption occurred before the data was sent by TCP itself. 

Yes, that's my concern.

> So to me, to hinge whether we support a new TCP CRC based checksum or
> not, on practical experience in the wild, is probably not a safe
> approach. Ofcourse, if we had multiple proven examples of this in the
> wild, we'd feel more motivated to do this. 

I don't follow this logic. We have no measured evidence of this ever occurring.
The evidence we have to date is that the end system software is more likely the
problem.

Everything so far points to the need for an application protocol check, i.e., a
final, total transfer checksum on each large transaction (i.e., the whole file).
That was one conclusion of the 2000 paper, FWIW.

> Also, I am more worried about packet sizes greater than 1500 bytes -
> where the Ethernet CRC checksum becomes weaker. Typical ethernet MTUs
> across the internet does not exceed 1500 bytes today - so right now, the
> problem scope is limited to Data Centers and other such local LANs where
> the end to end PMTU is greater than 1500 bytes.

That should be verifiable. Then again, if the CRC32 fails on jumboframes, why
would adding CRC16 at TCP help? other than the fact that it's a 'different' sum?
 Your argument suggests that such data centers need different *ethernet*
checksums, not different TCP ones. Then the data centers can leverage - and
reuse - that where necessary.

Joe