Re: TO COMPRESS OR NOT TO CMPRS (please reply)
Bob Monsour <rmonsour@earthlink.net> Wed, 19 February 1997 05:27 UTC
Received: (from majordom@localhost) by portal.ex.tis.com (8.8.2/8.8.2) id AAA20156 for ipsec-outgoing; Wed, 19 Feb 1997 00:27:49 -0500 (EST)
Message-Id: <3.0.32.19970218213122.00923b40@earthlink.net>
X-Sender: rmonsour@earthlink.net
X-Mailer: Windows Eudora Pro Version 3.0 (32)
Date: Tue, 18 Feb 1997 21:31:24 -0800
To: "C. Harald Koch" <chk@utcc.utoronto.ca>
From: Bob Monsour <rmonsour@earthlink.net>
Subject: Re: TO COMPRESS OR NOT TO CMPRS (please reply)
Cc: Bob Monsour <rmonsour@earthlink.net>, ipsec@tis.com
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Sender: owner-ipsec@ex.tis.com
Precedence: bulk
At 03:45 PM 2/18/97 -0500, C. Harald Koch wrote: >Does anyone (perhaps PPP people?) have statistics on whether compression at >the packet layer is actually effective? What percentage of packets on a real >network are compressible? What compression ratios do you get? What's the >extra latency of the compression? Harold, Below I have copied the appendix from draft-sabin-lzs-payload-00.txt, one of the proposed compression drafts. It contains some results from analyzing compression ratio vs. packet size with a known set of data. Note that "your're mileage may vary" as with any compression algorithm; results vary by data type. If you look at the entry for 256 byte packets, the test data shows a ratio of 1.43:1 (i.e., packet was reduced to 70% of its original size). While the data set is not "real network" data, it is a publically available dataset used commonly for analyzing compression ratios for various algorithms. The extra latency really depends on the processor you're using. We have seen results where, in a compress/encrypt/mac scenario, achieving a 2:1 compression ratio reduces the CPU cycles required for encrypt/mac functions to the point of a net benefit (note that this includes accounting for the cost of compression). I presented some numbers relating to this at the Dec wg meeting in San Jose. 8. Appendix: Compression Efficiency versus Datagram Size The following table offers some guidance on the compression efficiency that can be achieved as a function of datagram size. Each entry in the table shows the compression ratio that was achieved when the proposed transform was applied to a test file using datagrams of a specified size. The test file was the University of Calgary Text Compression Corpus [Calgary]. The length of the file prior to compression was 3,278,000 bytes. When the entire file was compressed as a single payload, a compression ratio of 2.34 resulted. Datagram size,| 64 128 256 512 1024 2048 4096 8192 16384 bytes | --------------|---------------------------------------------------- Compression |1.18 1.28 1.43 1.58 1.74 1.91 2.04 2.11 2.14 ratio | [Calgary] Text Compression Corpus, University of Calgary, available at ftp://ftp.cpsc.ucalgary.ca/pub/projects/text.compression.corpus >What's the tradeoff between compressing the data stream (above TCP) v.s. >individual packets? (The latter transmits the same number of packets with or >without compression, and some would argue that the modern internet is *not* >bandwidth limited but is limited by packet-switching rates). If you could compress the data stream (as is done in PPP), you could achieve higher compression ratios since you would be retaining the compression history across each segment of a particular "session". The dilemma, however, is that once you are above IP, you cannot predictably rely on the presence of a particular protocol. Also, the presence of encryption in IP(Sec) is one element of what is driving the need for compression. In the absence of IPSec encryption, the lower-layer PPP compression will work and provide the desired result. -Bob
- TO COMPRESS OR NOT TO CMPRS (please reply) Bob Monsour
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) Matt Thomas
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) Derek Palma
- RE: TO COMPRESS OR NOT TO CMPRS (please reply) Roy Pereira
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) Bob Monsour
- RE: TO COMPRESS OR NOT TO CMPRS (please reply) Rob Adams
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) Derrell Piper
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) Terry L. Davis, Boeing Information & Support Services, Bellevue, WA
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) Dennis Glatting
- RE: TO COMPRESS OR NOT TO CMPRS (please reply) Rob Adams
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) Michael Richardson
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) Kent Fitch
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) Daniel Harkins
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) Germano Caronni
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) Marcel Waldvogel
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) Rodney Thayer
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) Derek Palma
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) carrel
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) Matt Thomas
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) Daniel Harkins
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) Karl Fox
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) Naganand Doraswamy
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) Bob Monsour
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) C. Harald Koch
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) C. Harald Koch
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) Steven Bellovin
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) Karl Fox
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) Karl Fox
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) Angelos D. Keromytis
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) Scott Marcus
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) Matt Thomas
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) Angelos D. Keromytis
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) Dennis Glatting
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) Stephen Kent
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) Bob Monsour
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) Bob Monsour
- RE: TO COMPRESS OR NOT TO CMPRS (please reply) Bob Monsour
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) Bob Monsour
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) Bob Monsour
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) Jim Thompson
- RE: TO COMPRESS OR NOT TO CMPRS (please reply) Bob Monsour
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) Bob Monsour
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) Perry E. Metzger
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) Bob Monsour
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) C. Harald Koch
- RE: TO COMPRESS OR NOT TO CMPRS (please reply) Roy Pereira
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) Perry E. Metzger
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) EKR
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) John W. Richardson
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) Bill Sommerfeld
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) Bill Sommerfeld
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) C. Harald Koch
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) Bill Sommerfeld
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) Bob Monsour
- RE: TO COMPRESS OR NOT TO CMPRS (please reply) Rob Adams
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) Angelos D. Keromytis
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) Dennis Glatting
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) EKR
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) Rodney Thayer
- RE: TO COMPRESS OR NOT TO CMPRS (please reply) Stephen Kent
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) Stephen Kent
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) Stephen Kent
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) Dennis Glatting
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) Phil Karn
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) Phil Karn
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) Phil Karn
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) Phil Karn
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) Bob Monsour
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) Marcel Waldvogel
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) Stephen Kent
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) Perry E. Metzger
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) Perry E. Metzger
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) Phil Karn
- Re: TO COMPRESS OR NOT TO CMPRS (please reply) James Hughes