RE: [rohc] RE: [Sipping] SIGCOMP and large binary content SIP mes sages

[[e-mail]]

Mark,
 
This is exactly the concern I have regarding the SigComp performance implication. I see this as a question of *what* states to create. More precisely, a compressor should create only "useful" state(s) from the text part(s) of a message while avoiding creating "garbage" state(s) from the binary part(s). 
 
(Of course, it's possible that the binary part may also repeat across messages and thus become useful. But I wouldn't go there for simplicity.)
 
So, how can a compressor achieve that? Below are ideas I can think of. If they are correct, I can compile more "formal" text to be added to the SigComp user guide.
 
There are two steps involved. First, the compressor needs to know if current message to be compressed contain binary parts, and if yes, where they are located in the message. Second, the compressor needs to make sure it can selectively create states on the decompressor side.
 
For the first step, the best thing can happen is that the application (SIP in this case) gives hint to sigcomp compressor. This is possible, since it is a matter of local interface between application and sigcomp layer within the same endpoint. Otherwise (i.e. no help from application), the compressor has to figure it out by itself. In general, this seems to be a difficult problem to solve (if possible at all). But there are a few simple and practical approaches one can consider.
 
For example, the compressor can check the compression ratio after compressing the message. If it is close to 1, that is a hint that the current message probably contains large binary content. The compressor can choose not to create any state from current message. However, the compressor may miss a small text segment in the message that turns out being repeated later in the subsequent messages. The positive side is that at least the compressor can avoid creating "garbage states". 
 
Another factor to make a compressor's (or rather implementer's) life easier: we can assume that application messages have two parts, namely, a UTF-8 encoded part at the beginning followed by a binary part. (Note this is the case at least for SIP.) Then, the compressor can identify the UTF-8 part (by parsing) and use it to create state. This also has a good side-effect: compressor can avoid compressing the remaining binary part. 
 
Now to the second step: creating state on decompressor side. One issue I see is the limitation of decompression memory. On the decompressor side, the large binary content may overwrite text content previously stored in the ring/history buffer (part of the UDVM buffer). One solution is that the compressor should "segment" one large application message into multiple sigcomp messages, so that one of them contains only text data. This avoids the overwriting problem. (Note: I don't think this works for SigComp over UDP. But on the other hand, large SIP messages must be carried over TCP, not UDP.)
 
BR,
Zhigang
 

-----Original Message-----
From: ext Mark Watson [mailto:[e-mail]]
Sent: April 03, 2003 8:35 AM
To: 'Price, Richard'; Liu Zhigang.C (NRC/Dallas); [e-mail]; [e-mail]
Cc: [e-mail]; [e-mail]
Subject: RE: [rohc] RE: [Sipping] SIGCOMP and large binary content SIP mes sages



Richard, all, 

The issue I would like to see addressed is as follows: 

The text part of SIP/SDP is obviously highly compressible using very simple algorithms which refer back to previous parts of the message/earlier messages. In fact, subsequent messages in a SIP dialog seem to consist almost entirely of repeated text from previous messages.

So, it seems important to me that large binary content in SIP messages does not destroy state that will be useful in compressing subsequent messages.

In a wireless environment, the principle advantage of compression is reduction in the time taken to actually send the message over the wireless link (this may even be more important than the overall reduction in signalling bandwidth).

Now if the message contains large binary content, then of course we are 'out of luck' on the message transfer time stakes, but this shouldn't affect subsequent messages.

I believe we should document recommendations to SIGCOMP compressor designers to ensure this aspect is not overlooked (although it is not immediately clear to me how the compressor (being unaware of the SIP message structure) might achieve this.)

Is this something that would be appropriate to item 4 below ? 

Regards...Mark 

> -----Original Message----- 
> From: Price, Richard [ mailto:[e-mail]] 
> Sent: 26 March 2003 13:58 
> To: '[e-mail]'; [e-mail]; 
> [e-mail] 
> Cc: [e-mail]; [e-mail] 
> Subject: RE: [rohc] RE: [Sipping] SIGCOMP and large binary content SIP 
> mes sages 
> 
> 
> Hi, 
> 
> I think that the issue with SigComp and large binary 
> content SIP messages is part of a more general issue - 
> namely that before using SigComp to compress a particular 
> application we need to standardise the following: 
> 
> 1. Mechanism for SigComp discovery 
> 2. Resources offered by SigComp 
> 3. How to delimit messages 
> 4. When to save state 
> 
> Currently the SIPPING WG has provided RFCs for 1 and 2, 
> but not for 3 and 4. It would be useful to get a draft 
> out for these ASAP, as it would clarify several issues on 
> how to get SigComp working with SIP (including the issue 
> with large binary content SIP messages). 
> 
> > Unless someone objects, I'll submit it in ROHC. 
> 
> I think that any WG draft should be a product of the 
> SIPPING WG, because the proposed solution for delimiting 
> SigComp messages will need to take into account the 
> behaviour of SIP itself (or we won't be able to multiplex 
> SigComp and SIP messages on the same port). However, it 
> would be good to get the ROHC WG involved in generating 
> the draft as well! 
> 
> Regards, 
> 
> Richard 
> 
> P.S. For anyone who's interested, the mechanisms that 
> particular applications need to provide before they can 
> use SigComp are as follows... 
> 
> ------------------------------------------------------ 
> 
> 1. Mechanism for SigComp discovery 
> 
>    - How does the application know when to use SigComp? 
> 
> For SIP this issue is already sorted (RFC 3486). 
> 
> ------------------------------------------------------ 
> 
> 2. Resources offered by SigComp 
> 
>    - Does the application need to offer more than the 
>      minimum amount of resources? 
> 
>    - Does the application need any extra state (e.g. 
>      a static dictionary)? 
> 
> For SIP we've already got a static dictionary (RFC 3485). 
> However, we haven't currently discussed how much memory 
> should be provided in a SigComp implementation for SIP. 
> If we don't specify a value then SigComp defaults to the 
> minimum possible (2K), which is rather tight for some of 
> the more generously sized SIP messages! 
> 
> ------------------------------------------------------ 
> 
> 3. How to delimit messages 
> 
>    - How does the application distinguish between 
>      SigComp messages and uncompressed messages? 
> 
>    - Is the application allowed to use "continuous mode"? 
> 
> This is the issue that's been raised for large binary 
> content SIP messages. If we want to send a mix of 
> uncompressed and SigComp messages within the same TCP 
> connection, how do we delimit the end of one message and 
> the start of another? 
> 
> A related issue is how to handle very large SIP messages 
> (> 64K). SigComp is able to compress large SIP messages 
> by using "continuous mode", where the message boundaries 
> are ignored completely and the application data is just 
> transmitted as a stream. However, this mode of operation 
> raises some additional security issues, so it's up to the 
> SIPPING WG to decide whether or not to use it in SigComp 
> for SIP. 
> 
> ------------------------------------------------------ 
> 
> 4. When to save state 
> 
>    - How much security is needed in order to save state? 
> 
>    - Which messages belong to each state "compartment"? 
> 
>    - How long should the application keep state? 
> 
> This is quite an important issue that doesn't seem to 
> have been discussed in the SIPPING WG. It's up to the 
> application whether or not to let SigComp save state, 
> and for interoperability it's important to have at 
> least a general idea of how long to keep saved state, 
> when to reject state due to insufficient security etc. 
> 
> ------------------------------------------------------ 
> 
> > -----Original Message----- 
> > From: [e-mail] [ mailto:[e-mail]] 
> > Sent: Tuesday, March 25, 2003 6:00 PM 
> > To: [e-mail]; [e-mail] 
> > Cc: [e-mail]; [e-mail] 
> > Subject: RE: [rohc] RE: [Sipping] SIGCOMP and large binary 
> content SIP 
> > messages 
> > 
> > 
> > Hi Carsten and Lars-Erik, 
> > 
> > I've volunteered to submit a personal draft on this issue. From 
> > ROHC WG chairs' point of view, do you think this is the way 
> > to proceed? 
> > 
> > Of course, I understood during the IETF56 that Carsten had some 
> > doubts on whether this is indeed an issue. My email below explained 
> > the issue a bit. I don't know if it's clear. Either way, we 
> > can continue discuss this in ROHC mailing list 
> > 
> http://www1.ietf.org/mail-archive/working-groups/sipping/curre 
> nt/msg04115.html 
> > 
> > BR, 
> > Zhigang 
> > 
> > > -----Original Message----- 
> > > From: ext [e-mail] [ mailto:[e-mail]] 
> > > Sent: March 13, 2003 9:55 AM 
> > > To: [e-mail] 
> > > Cc: [e-mail]; Isomaki Markus (NRC/Helsinki); 
> > > [e-mail]; [e-mail]; [e-mail] 
> > > Subject: [rohc] RE: [Sipping] SIGCOMP and large binary content SIP 
> > > messages 
> > > 
> > > 
> > > I agree. How to handling binary content is a generic issue for any 
> > > application messages. What I have in mind is to give generic 
> > > descriptions/requirements (mainly on the SigComp receiver 
> parsing part), 
> > > then touch on SIP (perhaps RTSP too) as a particular example. 
> > > 
> > > Unless someone objects, I'll submit it in ROHC. 
> > > 
> > > Zhigang 
> > > 
> > > 
> > > > -----Original Message----- 
> > > > From: ext Gonzalo Camarillo 
> [ mailto:[e-mail]] 
> > > > Sent: March 13, 2003 5:36 AM 
> > > > To: Liu Zhigang.C (NRC/Dallas) 
> > > > Cc: [e-mail]; Isomaki Markus (NRC/Helsinki); 
> > > > [e-mail]; [e-mail] 
> > > > Subject: Re: [Sipping] SIGCOMP and large binary content 
> SIP messages 
> > > > 
> > > > 
> > > > Zhigang, 
> > > > 
> > > > you can submit that draft to ROHC. SIPPING can review 
> it afterwards. 
> > > > IMO, the competence needed to generate this draft can 
> be found more 
> > > > easily in ROHC than in SIPPING. 
> > > > 
> > > > Besides, you could use SigComp to compress RTSP 
> messages, for instance 
> > > > And RTSP is not handle by SIPPING either. 
> > > > 
> > > > Gonzalo 
> > > > 
> > > > [e-mail] wrote: 
> > > > > 
> > > > > Yes, that is also what I had in mind for SIP. But my 
> point is that 
> > > > > a SigComp receiver now needs to have some knowledge 
> about application 
> > > > > message format and do parsing to determine the end of 
> an uncompressed 
> > > > > message. This is something new (or at least 
> unspecified) in RFC 3320 
> > > > > and needs to be spelled out in a new (standard track) 
> RFC. The document 
> > > > > will be short, but I think it's needed for interoperability. 
> > > > > 
> > > > > I'll write up an I-D and submit it after IETF 56. But 
> I'd like to know 
> > > > > if folks in SIPPING think it should be done here or in ROHC. 
> > > > > 
> > > > > Zhigang 
> > > > > 
> > > > > > -----Original Message----- 
> > > > > > From: ext Adam Roach [ mailto:[e-mail]] 
> > > > > > Sent: March 11, 2003 9:03 PM 
> > > > > > To: Liu Zhigang.C (NRC/Dallas); Adam Roach; Isomaki Markus 
> > > > > > (NRC/Helsinki); [e-mail]; [e-mail] 
> > > > > > Subject: RE: [Sipping] SIGCOMP and large binary 
> content SIP messages 
> > > > > > 
> > > > > > 
> > > > > > > -----Original Message----- 
> > > > > > > From: [e-mail] 
> [ mailto:[e-mail]] 
> > > > > > > 
> > > > > > > Having binary content in SIP messages may have 
> two impacts on 
> > > > > > > SigComp: 
> > > > > > > 
> > > > > > > a) multiplexing of uncompressed messages with 
> SigComp messages on the 
> > > > > > > same port. 
> > > > > > ... 
> > > > > > > For a), it is only an issue for SigComp/TCP, not 
> SigComp/UDP (where one 
> > > > > > > SigComp message maps to one UDP packet). Adam's 
> point 1 and 2 are related 
> > > > > > > to this. As to point 1, I think the SigComp 
> receiver still needs to know 
> > > > > > > (somehow) the end of an uncompressed message, or 
> at least the end of the 
> > > > > > > binary part of each SIP messages. Otherwise, a 
> bit pattern 11111 in 
> > > > > > > the binary part will be mistaken by SigComp as 
> the beginning of a SigComp 
> > > > > > > message. As to point 2, I would add that the 
> delimiter 0xFFFF can appear 
> > > > > > > at the both the beginning and the end of a 
> SigComp message. So, the 
> > > > > > > receiver needs to have the same parsing logic 
> just mentioned for bit pattern 
> > > > > > > 11111. 
> > > > > > 
> > > > > > This isn't any more of an issue than it is without 
> SigComp. SIP 
> > > > > > messages provide their own framing, and SigComp 
> messages do the 
> > > > > > same. They can be trivially mixed on the same stream: 
> > > > > > 
> > > > > > 1. Peek at the first byte waiting in the stream. 
> > > > > > 
> > > > > > 2. If it is 0xf8 or higher, the next message in the 
> stream is 
> > > > > >    a SigComp message. The message continues until a 
> 0xFFFF is 
> > > > > >    encountered in the stream. 
> > > > > > 
> > > > > > 3. Otherwise, the next message in the stream is a 
> SIP message. 
> > > > > >    Read the stream until a CR/LF/CR/LF is encountered. Parse 
> > > > > >    the headers, and then read the number of bytes specified 
> > > > > >    in the "Content-Length" header. 
> > > > > > 
> > > > > > 4. Return to step 1. 
> > > > > > 
> > > > > > /a 
> > > > > > 
> > > > > _______________________________________________ 
> > > > > Sipping mailing list 
> https://www1.ietf.org/mailman/listinfo/sipping 
> > > > This list is for NEW development of the application of SIP 
> > > > Use [e-mail] for questions on 
> current sip 
> > > > Use [e-mail] for new developments of core SIP 
> > > 
> > > -- 
> > > Gonzalo Camarillo         Phone :  [phone] 
> > > Oy L M Ericsson Ab        Mobile:  [phone] 
> > > Telecom R&D               Fax   :  [phone] 
> > > FIN-02420 Jorvas          Email :  [e-mail] 
> > > Finland                   http://www.hut.fi/~gonzalo 
> > > 
> > _______________________________________________ 
> > Rohc mailing list 
> > [e-mail] 
> > https://www1.ietf.org/mailman/listinfo/rohc 
> > _______________________________________________ 
> > Rohc mailing list 
> > [e-mail] 
> > https://www1.ietf.org/mailman/listinfo/rohc 
> > 
> _______________________________________________ 
> Sipping mailing list  https://www1.ietf.org/mailman/listinfo/sipping 
> This list is for NEW development of the application of SIP 
> Use [e-mail] for questions on current sip 
> Use [e-mail] for new developments of core SIP 
>