Re: [calsify] RFC 5545/JSCalendar questions from ECMAScript TC39 "Temporal" working group

Justin Grant <justingrant.ietf.public@gmail.com> Mon, 03 August 2020 04:47 UTC

Return-Path: <justingrant.ietf.public@gmail.com>
X-Original-To: calsify@ietfa.amsl.com
Delivered-To: calsify@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4ECA33A0A3F for <calsify@ietfa.amsl.com>; Sun, 2 Aug 2020 21:47:02 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.197
X-Spam-Level:
X-Spam-Status: No, score=-0.197 tagged_above=-999 required=5 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FYUxZLmVTTUw for <calsify@ietfa.amsl.com>; Sun, 2 Aug 2020 21:46:59 -0700 (PDT)
Received: from mail-ed1-x534.google.com (mail-ed1-x534.google.com [IPv6:2a00:1450:4864:20::534]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 5F1EE3A0A4E for <calsify@ietf.org>; Sun, 2 Aug 2020 21:46:59 -0700 (PDT)
Received: by mail-ed1-x534.google.com with SMTP id i6so645704edy.5 for <calsify@ietf.org>; Sun, 02 Aug 2020 21:46:59 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=3O/tO2rX+MywN2Sv8P8UZUVt24OpyuxV2hPGI2y5RLA=; b=apmsHe8e5SI73pvpt4WgRrU5R3ZOpwHCiSTfkpAY1/T/Im8INEGOLUC20bnBjLyo/o io7X0FZif5S26q6LfLvXLQN/NsCgHQXf0XSu0bo6uuv+PQm3KuQM3ddBYpX3PLXuyFAI YqifIBhCyq6hgo7dESAiCR5grxkmkMjnXrdKQOp8CEreB+vvTUlY4K5pnFJ13LFkex2C kmFkjwoPEe4CFB+0zut+uElt+GNDA8aHzmHqHyJ9MJDpg0h9SkdcTsD2eAo6SBgY+Osa rGhYTCKBK56PDkiKkBalF3QckLwzhHwOOn+7foybSG42+GZ1/0saasKXbOyYUdWbiyt0 d/tw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=3O/tO2rX+MywN2Sv8P8UZUVt24OpyuxV2hPGI2y5RLA=; b=ocXPthsMmZTiztDT0NcJTSAOzvizwEvKj81Jnj19Q/w/ulLtON8pMbmnXxe7/KMDTX CxpJdLY853nI+5QjuV6iYWOBsMAhwex8jdf4lTOkhgFFIxupULBgmMfl1mrgH/RIEhhE 9ZEUCbZarsWOc/LcbLAETBQci3Hm9B+x12udoxoNiqbVZs/dQ3kJxs//XkqGE03NOP0r Phhvwk+5O/FQOQ3L0mk8Dmxl/kZQ6I7lx/W+5zpbOTSIBqvQZc0AUu+Y+srozPJQOe82 ioRh3zcak4ySl3WVJmxUfYDJBMcdjzHWwvbpNKjbSTrcVYJcTcNfoaPfgFosUmBCw/+R 2SpA==
X-Gm-Message-State: AOAM532yyOxYy68XJus3noeKm1zRmJoF8SGHZXW70LDoUF+Aw3w9GUrz C6c9/+w3Y3rz1RHTJ/GmS+lVwU/NLm6ETdFAuENFJJuN
X-Google-Smtp-Source: ABdhPJzo2g2hTNv3qBALuA0P8mE4gFidGjjc5tjb3ZkayY8Mt1W+kCz+v8qyFJlRXblnn777Y4dctBTHx2kp8ZNhwoA=
X-Received: by 2002:a50:d784:: with SMTP id w4mr14394490edi.34.1596430017729; Sun, 02 Aug 2020 21:46:57 -0700 (PDT)
MIME-Version: 1.0
References: <CACy7CficiNXA_tb01c0=PWQTnXPVu_e7TgWAKUVPY-ow9C=3SQ@mail.gmail.com> <5bfd4453-e9a9-4bf4-b403-1b2f19cc790a@dogfood.fastmail.com>
In-Reply-To: <5bfd4453-e9a9-4bf4-b403-1b2f19cc790a@dogfood.fastmail.com>
From: Justin Grant <justingrant.ietf.public@gmail.com>
Date: Sun, 02 Aug 2020 21:46:46 -0700
Message-ID: <CACy7CfgbPG_ugC59CZWtMwA7w4UOJR=zUkovKzvHUdeoFxnpjg@mail.gmail.com>
To: Neil Jenkins <neilj@fastmailteam.com>
Cc: calsify@ietf.org
Content-Type: multipart/alternative; boundary="0000000000001d981105abf1d414"
Archived-At: <https://mailarchive.ietf.org/arch/msg/calsify/X_dJeNs2JK08X0IwXwipgPGMots>
Subject: Re: [calsify] RFC 5545/JSCalendar questions from ECMAScript TC39 "Temporal" working group
X-BeenThere: calsify@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: <calsify.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/calsify>, <mailto:calsify-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/calsify/>
List-Post: <mailto:calsify@ietf.org>
List-Help: <mailto:calsify-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/calsify>, <mailto:calsify-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 03 Aug 2020 04:47:02 -0000

Hi Neil - Thanks for the quick reply. It's very useful to understand how
your team is thinking through these problems. Overall I agree with
everything in your reply. Here's a few notes:

Note, in general the lack of offset encoded with the time zone is a feature
> not a bug; otherwise you end up with nonsensical data should the time zone
> definitions change (what would you do with
> 2020-11-01T01:30-08:00[America/Los_Angeles] should California suddenly
> decide to join the One Glorious US Average Time Zone of -06:30?!). If the
> offset is fixed, you need to use UTC (or a fixed offset zone like
> Etc/GMT+8), not a real time zone.


Yep, we're aware of this problem but planning to handle it a little
differently. Many JS developers have limited understanding of time zones so
won't know whether to store UTC+timezone or date/time+timezone, and even
more JS won't anticipate how time zone definition changes can revise the
meaning of previously-stored values. The case we want to avoid is where
developers realize too late that their stored-long-ago data is ambiguous or
wrong. For these reasons, by default we'll persist both the offset and the
IANA time zone, e.g. 2020-11-01T01:30-08:00[America/Los_Angeles]. This
ensures that no data is lost in serialization. When developers discover the
problem (probably soon after the new time zone definition takes effect!)
they can fix it by making a small change to parsing code, instead of a
painful fixup of already-stored data. More sophisticated developers who
know exactly what they want to store can omit either the offset or the time
zone when persisting, but at least the default serialization format is not
lossy.

In the deserialization API, developers will have an option to control
whether the offset or the time zone should "win" when converting the
nominal date/time value to UTC.  The default is to throw an exception if
there's a conflict, but users can override so that the time zone or the
offset can win.

I think this one is incorrect.


Yep, you're right. Good catch.

I think I see. So the issue you have is if the end day is a daylight
> savings transition *day* and the start time is prior to the daylight
> savings transition *time*, then you have the intermediate adjustment
> which may be surprising to users. Firstly, this is actually going to be
> very rare that you hit it (so the likelihood of someone coming across it
> and actually *being *surprised is minimal, and the issue is the surprise,
> not necessarily the semantics); secondly, I think you would have to change
> the interpretation of durations (Q1) in order to do anything different,
> which seems a breaking change for minimal gain in my view.
>

Yep, agreed. This is a rare use case so it's definitely not worth a
breaking change. Our goal is to find a deterministic, least-surprising
solution for this weird case.  Ideally someone else would have already
standardized how to handle this case and we could just adopt the standard,
but so far I haven't found it.

Does JSCalendar include any guidance about how to calculate durations from
two date/time+timezone (DATE-TIME - FORM #3) values?  If not, would you be
interested in collaborating with us to define an algorithm that could be
used by Temporal and included in the JSCalendar spec? This may help with
interop.

Finally, could you explain what you mean by "*I think you would have to
change the interpretation of durations (Q1) in order to do anything
different"* ?

Cheers,
Justin


On Sun, Aug 2, 2020 at 6:35 PM Neil Jenkins <neilj@fastmailteam.com> wrote:

> On Sun, 2 Aug 2020, at 13:04, Justin Grant wrote:
>
> > Q1. [...] I'm not sure why you think the disambiguation in the middle is
> going to be problematic for users; surely they generally won't notice where
> it happens?
>
> Here's an example: what should be the result of adding P1DT12H
> to 2020-03-07T02:30-08:00[America/Los_Angeles]. My understanding of
> RFC5545's algorithm is that we'd use the following steps:
>   A) Add 1 calendar day.
>   B) Add 12 exact hours (not clock hours)
> But the after step A, the local time is not valid because 2020-03-08T02:30
> is in the middle of the hour skipped by a DST transition. My interpretation
> of RFC 5545 is that we'd interpret the invalid intermediate time by using
> the offset before the transition, so we'd end up with an intermediate value
> of 2020-03-08T03:30-07:00[America/Los_Angeles] and a final result of
> 2020-03-08T15:30-07:00[America/Los_Angeles].  Is this correct?
>
>
> Yes, that's my understanding too.
>
> Are both of these compliant with your understanding of RFC 5545?  Is only
> one correct?  Or is neither correct?
>
> One problem is that the Addition Path and the Subtraction Path can return
> different results when an endpoint is in the "repeated hour" after DST ends
> in the fall. For example, assume that X is
> 2020-10-31T01:30-07:00[America/Los_Angeles] and Y
> is 2020-11-01T01:30-08:00[America/Los_Angeles] (the "second" 1:30AM on the
> day DST ends).
>
>
> OK, so I think I see the cause of the confusion. They are both correct(!)
> BUT this is because there is a subtle difference between the Temporal
> *LocalDateTime* and RFC5545 *Date with local time and time zone reference*.
> The latter does not have a UTC reference point; it's just a wall clock time.
>
> You can add either P1D or P1DT1H to 2020-10-31T01:30 and get a wall clock
> time of 2020-11-01T01:30. Both are correct durations to calculate as the
> difference between X and Y, because Y could be two different absolute times
> (the -08:00 is not encoded in the RFC5545 model).
>
> Due to conversion rules, when converting to UTC you always interpret
> 2020-11-01T01:30[America/Los_Angeles] as -07:00, not -08:00, should you
> need an absolute time. If you want to reference the other point in time
> specifically you would need to use UTC (or I guess any time zone where it
> is unambiguous).
>
> Note, in general the lack of offset encoded with the time zone is a
> feature not a bug; otherwise you end up with nonsensical data should the
> time zone definitions change (what would you do with
> 2020-11-01T01:30-08:00[America/Los_Angeles] should California suddenly
> decide to join the One Glorious US Average Time Zone of -06:30?!). If the
> offset is fixed, you need to use UTC (or a fixed offset zone like
> Etc/GMT+8), not a real time zone.
>
> Another problem is that the logic above can deliver unexpected results
> depending on the local time of the endpoints, even if neither of those
> endpoints are close to a DST transition.  For example, using the "Addition
> Path" logic above, here's the durations that the following pairs of
> LocalDateTime values will return:
>
>    - 2020-03-07T01:30-08:00[America/Los_Angeles]
>    to 2020-03-08T13:30-07:00[America/Los_Angeles] => P1D12H
>
>
> I think this one is incorrect. Following your Addition Path algorithm:
>
>   A) Let D = Y - X => 1 day 12 hours
>   B) Truncate the time portion of D => 1 day
>   C) Let I = X+D => 2020-03-08T01:30[America/Los_Angeles] =>
> 2020-03-08T09:30:00Z
>   D) Y = 2020-03-08T20:30:00Z; I - Y => 11H
>   E) Merge N and T: P1DT11H
>
>
>    - 2020-03-07T02:30-08:00[America/Los_Angeles]
>    to 2020-03-08T14:30-07:00[America/Los_Angeles] => P1D11H (1 hour less
>    because the intermediate time 2020-03-08T02:30 is resolved
>    to 2020-03-08T03:30-07:00[America/Los_Angeles] before the exact time
>    difference is calculated)
>    - 2020-03-07T03:30-08:00[America/Los_Angeles]
>    to 2020-03-08T15:30-07:00[America/Los_Angeles] => P1D12H
>
>
> Users probably won't be surprised to see divergent results in cases where
> the endpoints are within an hour of a DST transition. But what's unexpected
> is the variation in results caused by the intermediate result landing on an
> invalid time, even though neither of the endpoints are very close to the
> transition.
>
> Does this clarify the problems we're trying to resolve with Q5?
>
>
> I think I see. So the issue you have is if the end day is a daylight
> savings transition *day* and the start time is prior to the daylight
> savings transition *time*, then you have the intermediate adjustment
> which may be surprising to users. Firstly, this is actually going to be
> very rare that you hit it (so the likelihood of someone coming across it
> and actually *being *surprised is minimal, and the issue is the surprise,
> not necessarily the semantics); secondly, I think you would have to change
> the interpretation of durations (Q1) in order to do anything different,
> which seems a breaking change for minimal gain in my view.
>
> Neil.
> _______________________________________________
> calsify mailing list
> calsify@ietf.org
> https://www.ietf.org/mailman/listinfo/calsify
>