Re: [tcpm] draft-eggert-tcpm-historicize-00

["Biswas, Anumita" <[e-mail]>]

> -----Original Message-----
> From: Joe Touch [mailto:[e-mail]] 
> Sent: Sunday, June 27, 2010 11:51 AM
> To: Biswas, Anumita
> Cc: [e-mail]; [e-mail]; [e-mail]
> Subject: Re: [tcpm] draft-eggert-tcpm-historicize-00
> 
> 
> Hi, Anumita,
> 
> Biswas, Anumita wrote:
> ...
> > I have heard only anecdotal evidence of a very large company doing 
> > very large copies of data (about a Tera Byte) across a 
> local network 
> > and then finding bit corruptions in the data received on 
> the other end 
> > - clearly the link CRC and the TCP 1s compl summation 
> checksum failed 
> > to catch those errors.
> 
> Well, that's the point. It's not so clear that the checks are 
> what failed. It could easily have been a bug in the 
> implementation, or an error in the hardware that corrupted 
> data after it was received correctly. I.e., no matter how 
> good these checks, there are other ways in which data can be 
> corrupted.
> 
> > In my own company, I have seen a case where a very large 
> block (1460 
> > bytes - a packet size) of data was replaced and gone 
> undetected and we 
> > could not root cause the problem - the problem just 
> disappeared after 
> > showing up for a few weeks - but in this case, it is 
> possible that the 
> > corruption occurred before the data was sent by TCP itself.
> 
> Yes, that's my concern.
> 
> > So to me, to hinge whether we support a new TCP CRC based 
> checksum or 
> > not, on practical experience in the wild, is probably not a safe 
> > approach. Ofcourse, if we had multiple proven examples of 
> this in the 
> > wild, we'd feel more motivated to do this.
> 
> I don't follow this logic. We have no measured evidence of 
> this ever occurring. The evidence we have to date is that the 
> end system software is more likely the problem.
> 
> Everything so far points to the need for an application 
> protocol check, i.e., a final, total transfer checksum on 
> each large transaction (i.e., the whole file). That was one 
> conclusion of the 2000 paper, FWIW.
> 
I mentioned the advantage of being able to offload a TCP based checksum
to the NIC. If I were to extend your argument that end system software
is more likely the problem - a checksum over the whole file could also
be prone to error. 

I think we are only trying to deploy a "stronger" checksum - we are not
making claims that it will find all problems in end to end software -
and as such is not the strongest checksum.

> > Also, I am more worried about packet sizes greater than 
> 1500 bytes - 
> > where the Ethernet CRC checksum becomes weaker. Typical 
> ethernet MTUs 
> > across the internet does not exceed 1500 bytes today - so 
> right now, 
> > the problem scope is limited to Data Centers and other such 
> local LANs 
> > where the end to end PMTU is greater than 1500 bytes.
> 
> That should be verifiable. Then again, if the CRC32 fails on 
> jumboframes, why would adding CRC16 at TCP help? other than 
> the fact that it's a 'different' sum?  
The draft suggests using the Castagnoli 32-bit polynomial used in the
"CRC 32C" algorithm which generates a 32 bit checksum quantity. No
mention of CRC16.
The draft specifies using a different "space" in the TCP options area,
to transfer this checksum. It does not suggest using the 16-bit TCP
checksum space for this value. It does debate over what to do with the
16-bit summation checksum. Now, after many discussions, it is more clear
that we do need the 16-bit checksum as well to prevent middleboxes from
breaking. 

http://tools.ietf.org/html/draft-anumita-tcpm-stronger-checksum-00

Please read why the Castagnoli polynomial based CRC checksum is stronger
than the Ethernet CRC32 checksum carried by the Ethernet frame. 

Your argument suggests 
> that such data centers need different *ethernet* checksums, 
> not different TCP ones. Then the data centers can leverage - 
> and reuse - that where necessary.
The ideal way to do this would be to have Ethernet frames include the
CRC32C checksum instead of the CRC32 checksum. But changing the Ethernet
standard across the board is impossible due to backward compatibility
issues. But TCP options provides a much easier path to introducing a
stronger checksum as it does not break backward compatibility and can be
offloaded to the NIC as well. 

Data Centers are one place where Jumbo frames are deployed - but there
could be others. So I don't understand what you mean by "data centers
can leverage and reuse that where necessary".

> 
> Joe
> 
>