MIME-Version: 1.0
References: 
 <CAHVo=Z=MwNtUtKYie2M62_mvXa26J6KfSG+DzcguDzkdER+Ocg@mail.gmail.com>
 <CANPAELv6CqDXmyR3EEWyX+XNBFVEC1URg7PPdU=1DkAATCpHZQ@mail.gmail.com>
 <CAHVo=Zmara43hi-hqk0-ZEBJ4Rz-qnNwyDfREEihLReko8FRnA@mail.gmail.com>
In-Reply-To: 
 <CAHVo=Zmara43hi-hqk0-ZEBJ4Rz-qnNwyDfREEihLReko8FRnA@mail.gmail.com>
From: Luke Curley <kixelated@gmail.com>
Date: Fri, 26 Jun 2026 11:45:09 -0700
Message-ID: 
 <CAHVo=ZkBXBGk6z-9_po8Ocfj6-OrwiB3P+8972v1KLFeMV1aww@mail.gmail.com>
To: Alan Frindell <afrind@meta.com>
Content-Type: multipart/alternative; boundary="000000000000c831b806552c8261"
Message-ID-Hash: ZFBEX2Q6MJDUPKZGPIN4JARP3CULGXZE
CC: MOQ Mailing List <moq@ietf.org>
Precedence: list
Subject: =?utf-8?q?=5BMoq=5D_Re=3A_MoQ_+_Compression?=
Archived-At: 
 <https://mailarchive.ietf.org/arch/msg/moq/lZ0ndBk_Fe96_L_sToDcWRbZiOU>

--000000000000c831b806552c8261
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Sorry hit the send button to early.

But you can imagine the headache if both of these files were large


I meant tracks.

Some companies use my libraries to publish sensor data or controls over
Starlink. We don't want a scenario where one outdated subscriber that
doesn't support compression causes 9x the data transfer from the publisher
to the relay.

Making DEFLATE mandatory to implement in moq-transport would fix a lot.


On Fri, Jun 26, 2026 at 11:35=E2=80=AFAM Luke Curley <kixelated@gmail.com> =
wrote:

> Yeah Alan, that's the problem with doing it at the application layer. I'm
> currently publishing two tracks:
>
>    - catalog.json
>    - catalog.json.z
>
> But you can imagine the headache if both of these files were large, and
> there was a 50-50 split among subscribers.
>
>
> I mocked up a transport extension that was something like:
>
> 1. SETUP indicates decompression schemes in preferred order (1 =3D deflat=
e,
> 2 =3D zstd, etc).
> 2. SUBGROUP_HEADER (or SUBSCRIBE_OK?) indicates the compression scheme.
>
> Like what Lucas said, the real problem is intermediaries. You ideally wan=
t
> a relay to proxy the data, not waste CPU on (re)compression. A relay can
> always decompress the data for subscribers that don't support a compressi=
on
> scheme, but it probably shouldn't (re)compress it.
>
>
> Suhas you run into rollout issues with that approach because it's not
> backwards compatible. You blindly have to assume that every single
> subscriber supports a compression scheme before you can turn it on.
>
>
> On Fri, Jun 26, 2026 at 11:08=E2=80=AFAM Alan Frindell <afrind@meta.com> =
wrote:
>
>> Cool work Luke.
>>
>> MOQT's model is clearly "the objects can't change for the same track
>> name" - therefore the only way we have to signal compression support now=
 is
>> it bake it into the Full Track Name (eg: SUBSCRIBE namespace--name.defla=
te
>> or something).  Otherwise, aren't you wandering into HTTP content
>> negotiation, Vary header, etc territory?
>>
>> -Alan
>>
>>

--000000000000c831b806552c8261
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Sorry hit the send button to early.<div><br></div><blockqu=
ote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px=
 solid rgb(204,204,204);padding-left:1ex"><span style=3D"background-color:t=
ransparent">But you can imagine the headache if both of these files were la=
rge</span></blockquote><div><br>I meant tracks.=C2=A0</div><div><br></div><=
div>Some companies use my libraries to publish sensor data or controls over=
 Starlink.=C2=A0We don&#39;t want a scenario where one outdated subscriber =
that doesn&#39;t support compression causes 9x the data transfer from the p=
ublisher to the relay.=C2=A0<br><br>Making <font face=3D"monospace">DEFLATE=
</font> mandatory to implement in moq-transport would fix a lot.</div><div>=
<br></div></div><br><div class=3D"gmail_quote gmail_quote_container"><div d=
ir=3D"ltr" class=3D"gmail_attr">On Fri, Jun 26, 2026 at 11:35=E2=80=AFAM Lu=
ke Curley &lt;<a href=3D"mailto:kixelated@gmail.com">kixelated@gmail.com</a=
>&gt; wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px=
 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><di=
v dir=3D"ltr"><div>Yeah Alan, that&#39;s the problem with doing it at the a=
pplication layer. I&#39;m currently publishing two tracks:<br><ul><li>catal=
og.json</li><li>catalog.json.z</li></ul>But you can imagine the headache if=
 both of these files were large, and there was a 50-50 split among subscrib=
ers.<br><br><br>I mocked up a transport extension that was something like:<=
br><span style=3D"background-color:transparent"><br>1. <font face=3D"monosp=
ace">SETUP</font> indicates </span>decompression<span style=3D"background-c=
olor:transparent">=C2=A0schemes in preferred order (1 =3D deflate, 2 =3D zs=
td, etc).</span></div><div>2. <font face=3D"monospace">SUBGROUP_HEADER</fon=
t> (or=C2=A0<font face=3D"monospace">SUBSCRIBE_OK?)</font>=C2=A0indicates t=
he compression scheme.<span style=3D"background-color:transparent"></span><=
/div><div><br>Like what Lucas said, the real problem is intermediaries. You=
 ideally want a relay to proxy the data, not waste CPU on (re)compression. =
A relay can always decompress the data for subscribers that don&#39;t suppo=
rt a compression scheme, but it probably shouldn&#39;t (re)compress it.</di=
v><div><span style=3D"background-color:transparent"><br></span></div><div><=
br></div><div>Suhas you run into rollout issues with that approach because =
it&#39;s not backwards compatible. You blindly have to assume that every si=
ngle subscriber supports a compression scheme before you can turn it on.</d=
iv><div><br></div></div><br><div class=3D"gmail_quote"><div dir=3D"ltr" cla=
ss=3D"gmail_attr">On Fri, Jun 26, 2026 at 11:08=E2=80=AFAM Alan Frindell &l=
t;<a href=3D"mailto:afrind@meta.com" target=3D"_blank">afrind@meta.com</a>&=
gt; wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0=
px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div =
dir=3D"ltr"><div class=3D"gmail_default" style=3D"font-size:large">Cool wor=
k Luke.=C2=A0=C2=A0<br><br>MOQT&#39;s model is clearly &quot;the objects ca=
n&#39;t change for the same track name&quot; - therefore the only way we ha=
ve to signal compression support now is it bake it into the Full Track Name=
 (eg: SUBSCRIBE namespace--name.deflate or something).=C2=A0 Otherwise, are=
n&#39;t you wandering into HTTP content negotiation, Vary header, etc terri=
tory?</div><br><div class=3D"gmail_default" style=3D"font-size:large">-Alan=
</div><br></div>
</blockquote></div>
</blockquote></div>

--000000000000c831b806552c8261--