Re: what is rsync, was Call for Community Feedback: Retiring IETF FTP Service

Christopher Morrow <morrowc.lists@gmail.com> Mon, 30 November 2020 15:55 UTC

Return-Path: <christopher.morrow@gmail.com>
X-Original-To: ietf@ietfa.amsl.com
Delivered-To: ietf@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DDA293A0E4B for <ietf@ietfa.amsl.com>; Mon, 30 Nov 2020 07:55:25 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.098
X-Spam-Level:
X-Spam-Status: No, score=-2.098 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id p5hqzlQC1dYY for <ietf@ietfa.amsl.com>; Mon, 30 Nov 2020 07:55:23 -0800 (PST)
Received: from mail-qv1-xf2a.google.com (mail-qv1-xf2a.google.com [IPv6:2607:f8b0:4864:20::f2a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id C9C183A0F09 for <ietf@ietf.org>; Mon, 30 Nov 2020 07:55:19 -0800 (PST)
Received: by mail-qv1-xf2a.google.com with SMTP id q7so5762891qvt.12 for <ietf@ietf.org>; Mon, 30 Nov 2020 07:55:19 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=lcbjQHDZMdaVbXWm3B7LBDZ/WgJ6vszT0YD3tg51kP0=; b=KdrF1bmymewMo1i8aPH/mjgfgJ2Tk7nGDVO/enbCk+xdzzO2/ablcaSvXLN5BsndJW tSbV9BGMD0nmZdchrah75/KlDNl5+3ErE67owNsXW4jIO/+Lmp/fRYe88LsLZJkz9NSB qvxlXYYcjteq+2tElRfY0ky0mf2AxHAxPaxi7r8Sug67y+msrAeMsbw0Wld1xoTQnmGc VzhsTBSK0H5l7JEaHDPzYFiJaI80f4tXDYSmc+uCz5EtB7hwsIC1MLEN5bKAlOr1qV0f AosWqCOMbkdmLAFQ46kWAYrorPMZWrpDdfUoqoutwaCfYb886C9s+jM4KEw5r552boxR d4eA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=lcbjQHDZMdaVbXWm3B7LBDZ/WgJ6vszT0YD3tg51kP0=; b=N8k8m2aax84OkA4qGmrmSfJykEzr8k2lyhwSa7U6bFa+zUO4GAuwwyAuXfXGFHcZ9p /c7DJ23nFRBH7aAhQaHEHs20+uxQdGrzfNsuCXUpbl5/zAkH33BPhZpY1hMO1lTFYP45 5hrlFIy0HFj3L/+EZuX/NZ06ziU2051bBrbfCl5s48Ox0jUfP1CeWqtgnLXjKhpvgIT7 lIkOwAh3RDAdxyGTN1G2SnuDwIgm118xENyTulIR5XaSab6mdbM6nT7VBBRmTpap8XvL ojgo3b5IFGBmwI/9TNeN0fFskM/1r9waBr2OgcVUQX9q4M5im6B3xFq9aCUxnRdTuiD5 wTRQ==
X-Gm-Message-State: AOAM533mteNBODUadgRMGfFHLDMU6YhYj/LB29eIXingLMoVuqN1pUso ivZ76Y1AF5YcGTxGkgF5XEWOmjRtZ3mEky3OkrhFNrsfDP0=
X-Google-Smtp-Source: ABdhPJx9GF4bxDz6bVGwXdtrRKgxZ4NLTI3tG/sbi+Vw71r5q4KqgVaQ7XkW99sFNUa69gchp5r7JastL1LdV4M1VMs=
X-Received: by 2002:ad4:4dc9:: with SMTP id cw9mr22523809qvb.22.1606751718707; Mon, 30 Nov 2020 07:55:18 -0800 (PST)
MIME-Version: 1.0
References: <20201126170826.F2FF8281C9C3@ary.qy> <c34ed9df-0d12-cab0-551e-ec146bc949b7@network-heretics.com> <BBFBE169970BD46925028069@PSB> <5FC0DCD7.4080209@btconnect.com> <CAKr6gn0sjXCzdQw=Ly0zAH=JYbmUt9hcpRQ+yoenYzgueEVbtg@mail.gmail.com>
In-Reply-To: <CAKr6gn0sjXCzdQw=Ly0zAH=JYbmUt9hcpRQ+yoenYzgueEVbtg@mail.gmail.com>
From: Christopher Morrow <morrowc.lists@gmail.com>
Date: Mon, 30 Nov 2020 10:55:03 -0500
Message-ID: <CAL9jLaZpsWehn3q1O3bhbDo9wSrN_0mFyDbNbNQ8ZDA=DZCp3Q@mail.gmail.com>
Subject: Re: what is rsync, was Call for Community Feedback: Retiring IETF FTP Service
To: George Michaelson <ggm@algebras.org>
Cc: tom petch <daedulus@btconnect.com>, John C Klensin <john-ietf@jck.com>, Keith Moore <moore@network-heretics.com>, IETF Discussion Mailing List <ietf@ietf.org>
Content-Type: text/plain; charset="UTF-8"
Archived-At: <https://mailarchive.ietf.org/arch/msg/ietf/VaQb-_q4jnIY2fcf7qi4PMb4mps>
X-BeenThere: ietf@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: IETF-Discussion <ietf.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/ietf>, <mailto:ietf-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/ietf/>
List-Post: <mailto:ietf@ietf.org>
List-Help: <mailto:ietf-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ietf>, <mailto:ietf-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 30 Nov 2020 15:55:34 -0000

On Sun, Nov 29, 2020 at 6:27 PM George Michaelson <ggm@algebras.org> wrote:
>
> In hindsight, this has been a problematic decision. rsync is highly
> exploitable for DOS and unexpected outcomes. Pragmatically it got us
> somewhere. Its a bad fit for the CDN world we live in now.

arguably, CDN doesn't matter at all here, we are only shipping around some small
number of small files per endpoint. CDN usage is really a
micro-optimization which
may be nice to have but isn't super relevant to operating the system as a whole.

what matters is a transport where there are programming libraries (in the
language(s) you choose to use for your system/tools) that report sane errors
which can be handled programmatically.

Shelling out to run your transport is ... hard to
debug/think-about-programmatically/repair. :(

Yes, rsync did and does provide a running system today.
Yes, rsync is also painful in these systems because failures are 'impossible' to
recover from (except by off/on-again)

> https://www.ietf.org/proceedings/89/slides/slides-89-sidr-6.pdf
>
> https://blog.apnic.net/2020/10/27/rpki-qa-the-trouble-with-rsync/
>
> RPKI (sidrops) is trying to define a delta protocol up into more
> normative state, and a deprecate-rsync draft is in hypothesis.
>
> -G
>
> On Fri, Nov 27, 2020 at 9:03 PM tom petch <daedulus@btconnect.com> wrote:
> >
> > On 26/11/2020 22:00, John C Klensin wrote:
> > >
> > > --On Thursday, November 26, 2020 14:11 -0500 Keith Moore
> > > <moore@network-heretics.com> wrote:
> > >
> > >> On 11/26/20 12:08 PM, John Levine wrote:
> > >>
> > >>> If you're wondering if there's an RFC for it, nope. RFC 5781
> > >>> specifies the rsync: URI but refers back
> > >>> tohttp://rsync.samba.org/  for details on rsync.
> > >>>
> > >>> I don't suppose anyone would object if we made an RFC out of
> > >>> the spec but given that the tech report, thesis, and multiple
> > >>> implementations are widely available without restriction
> > >>> (other than GPL on some of the code) it doesn't seem worth a
> > >>> lot of effort.
> > >>
> > >> I remember when IETF cared about making standards and
> > >> promoting broad interoperability.
> > >
> > > Of course, there would be another way to do this, one that would
> > > create an RFC but not a standard.  One could cobble a document
> > > together that described what rsync was all about in the abstract
> > > and introduction, include the references that have popped up in
> > > this thread (and probably including a lot of text by reference),
> > > and then hand it over to the ISE for publication of a
> > > description of a protocol that is widely used in the community
> > > for the information of the community.  If the IESG then wanted
> > > to take such a document over and classify it as standards track,
> > > I presume no one would object as long as they did not create a
> > > working group that had the goal of either hanging bags on the
> > > side of the thing or improving it enough that it was not
> > > interoperable with deployed implementations.
> > >
> > > But there is not much evidence, in this case, that anyone cares
> > > even enough to do that... and, if no one wants to invest even
> > > that level of effort, then I think we need to agree with John
> > > Levine's conclusion.
> >
> > Back in 2008, sidr chose rsync as the foundation of its protocol.  It
> > referenced rsync.samba.org.  See, for example, RFC6480.
> >
> > Tom Petch
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > >
> > >      john
> > >
> > >
> > > .
> > >
> >
>