Re: [calsify] RFC 5545/JSCalendar questions from ECMAScript TC39 "Temporal" working group

[Justin Grant <[e-mail]>]

Hi Neil - Daniel helped me get my calsify email subscription fixed, so I'm
replying on the public list.

Reading your response, I realize that I forgot to add one important piece
of context: all my questions focus on a specific type (placeholder name is
"LocalDateTime'-- see https://github.com/tc39/proposal-temporal/issues/700)
that we're adding to Temporal which represents a specific UTC instant in a
specific time zone.  This is closest to "DATE-TIME - FORM #3: DATE WITH
LOCAL TIME AND TIME ZONE REFERENCE" in RFC 5545, although there's a small
difference in that Temporal's LocalDateTime type will know its UTC value
and will derive the local date/time from the time zone and the UTC value.
This was done so that this type could represent all real-world instants
(other than leap seconds), including times during the "repeated hour" after
a Fall DST transition.

Temporal also has types that correspond to FORM #2: DATE WITH UTC TIME
(called "Absolute" in Temporal) and FORM #1: DATE WITH LOCAL TIME (called
"DateTime" in Temporal). Math with these types is straightforward because
neither of them have any awareness of time zones or DST.

The questions in https://github.com/tc39/proposal-temporal/issues/702 only
relate to the LocalDateTime type where the time zone is known and DST
transitions must be accounted for.

> Q1. [...] If I'm understanding it correctly, I do not think your proposed
alternative is acceptable because it applies the time component to the
wall-clock time, which means that if you had an event with duration PT1H
that started on a 1-hour DST-transition point it would be translated into
an absolute duration of 0 hours or 2 hours, which is clearly wrong! e.g. I
have an event that starts at 2019-03-31T02:45 in Europe/Berlin. I add a
duration of PT1H in wall clock time to get 2019-03-31T03:45 in
Europe/Berlin. However, those are the same time in UTC due to the DST
transition, so the absolute duration becomes 0!

Yep, to avoid that problem you can either add the time portion using exact
time, or you can do the time-portion math with wall-clock time and then
adjust the result for the difference in time zone offset. Either one works
to adjust for the impact of the DST transition. The advantage of the latter
approach is that if there's a DST disambiguation required, it happens at
the endpoint rather than at the intermediate value. This used to be
important because when I wrote that GitHub issue, LocalDateTime offered
user-customizable disambiguation behavior in its arithmetic functions.
Since then, we've decided to simply use the "use the earlier offset"
disambiguation method that RFC 5545 uses, so Q1 is probably a non-issue
now.  It still matters when calculating differences between LocalDateTime
values (Q4/Q5), though.

> Q1. [...] I'm not sure why you think the disambiguation in the middle is
going to be problematic for users; surely they generally won't notice where
it happens?

Here's an example: what should be the result of adding P1DT12H to
2020-03-07T02:30-08:00[America/Los_Angeles].
My understanding of RFC5545's algorithm is that we'd use the following
steps:
  A) Add 1 calendar day.
  B) Add 12 exact hours (not clock hours)
But the after step A, the local time is not valid because 2020-03-08T02:30
is in the middle of the hour skipped by a DST transition. My interpretation
of RFC 5545 is that we'd interpret the invalid intermediate time by using
the offset before the transition, so we'd end up with an intermediate value
of 2020-03-08T03:30-07:00[America/Los_Angeles] and a final result of
2020-03-08T15:30-07:00[America/Los_Angeles].  Is this correct?

> Q3. I don't think subtraction is well specified and probably wasn't
deeply considered; it is basically only used for the offset of alerts
before the start of the event and I think your proposal of subtracting
absolute components first is very reasonable for the reason you gave. We
should clarify this in the JSCalendar spec.

Sounds great. We'll plan to continue using smallest-units-first
order-of-operations for subtraction.

> Q4. [...] Subtract hours/minutes first to get the same wall clock time,
then calculate number of days difference to get a duration in
weeks/days/hours/minutes/seconds.

What should we do when the "same wall clock time" happens to be in the
middle of an hour skipped by a Spring DST transition?  See Q5 discussion
below for more details.

> Q5. I don't think I quite understand your question. You say you are
"calculating differences between UTC values" but then the "difference ends
inside a DST discontinuity" – UTC doesn't have DST, so this doesn't make
sense. Could you please clarify?

This is the inverse of Q1. In Q1 we add a Duration to a LocalDateTime and
have to deal with a DST transition in the middle. Q5 is about calculating
the Duration between two LocalDateTime values with a DST transition in the
middle. My understanding of RFC 5545 is that there are two possible ways to
calculate the Duration between two LocalDateTime values X and Y in the same
time zone. Let's call them the "Addition Path" and "Subtraction Path".

Addition Path
  A) Let D = Y - X ignoring any changes in time zone offset between X and Y
  B) Truncate the time portion of D, leaving only a date-only duration
  C) Let I = X+D. If I is an invalid time (e.g. the hour skipped when DST
starts) then resolve by using the earlier offset, which will yield a time
later than the discontinuity.
  D) Let T = the exact clock time difference between I and Y.
  E) Merge N and T into the result Duration

Subtraction Path
  A) Let D = Y - X ignoring any changes in time zone offset between X and Y
  B) Truncate the time portion of D, leaving only a date-only duration
  C) Let I = Y-D. (Hence the name "Subtraction Path"). If I is an invalid
time (e.g. the hour skipped when DST starts) then resolve by using the
earlier offset, which will yield a time later than the discontinuity.
  D) Let T = the exact clock time difference between X and I. (The other
path uses I and Y)
  E) Merge N and T into the result Duration

Are both of these compliant with your understanding of RFC 5545?  Is only
one correct?  Or is neither correct?

One problem is that the Addition Path and the Subtraction Path can return
different results when an endpoint is in the "repeated hour" after DST ends
in the fall. For example, assume that X is
2020-10-31T01:30-07:00[America/Los_Angeles] and Y
is 2020-11-01T01:30-08:00[America/Los_Angeles] (the "second" 1:30AM on the
day DST ends).  If we start from X, then we'd add one nominal day to get
2020-11-01T01:30-07:00[America/Los_Angeles] (the "first" 1:30AM), and then
add one exact hour to get to the end. So the result is P1DT1H.  But if we
start from Y, then subtracting one nominal day
from 2020-11-01T01:30-08:00[America/Los_Angeles] yields
2020-10-31T01:30-07:00[America/Los_Angeles]. There is no time remainder. So
the result is P1D.

Another problem is that the logic above can deliver unexpected results
depending on the local time of the endpoints, even if neither of those
endpoints are close to a DST transition.  For example, using the "Addition
Path" logic above, here's the durations that the following pairs of
LocalDateTime values will return:

   - 2020-03-07T01:30-08:00[America/Los_Angeles]
   to 2020-03-08T13:30-07:00[America/Los_Angeles] => P1D12H
   - 2020-03-07T02:30-08:00[America/Los_Angeles]
   to 2020-03-08T14:30-07:00[America/Los_Angeles] => P1D11H (1 hour less
   because the intermediate time 2020-03-08T02:30 is resolved
   to 2020-03-08T03:30-07:00[America/Los_Angeles] before the exact time
   difference is calculated)
   - 2020-03-07T03:30-08:00[America/Los_Angeles]
   to 2020-03-08T15:30-07:00[America/Los_Angeles] => P1D12H

Users probably won't be surprised to see divergent results in cases where
the endpoints are within an hour of a DST transition. But what's unexpected
is the variation in results caused by the intermediate result landing on an
invalid time, even though neither of the endpoints are very close to the
transition.

Does this clarify the problems we're trying to resolve with Q5?


*From: *<snip>

*Date: *Thursday, July 30, 2020 at 6:49 PM
*To: <snip>*

*Subject: *Re: RFC 5545/JSCalendar questions from ECMAScript TC39
"Temporal" working group

On Fri, 31 Jul 2020, at 05:10, Justin Grant wrote:

Hi JSCalendar team – I’m part of the TC39 working group that’s working on
the Temporal <https://github.com/tc39/proposal-temporal> proposal which
will be the future date/time API for JavaScript.  We’re planning to follow
RFC 5545 for operations that add or subtract durations from date/time
values.  But we have some questions about how to interpret the iCalendar
spec around edge cases when there’s a DST transition during the added or
subtracted duration.


[...]


Hi Justin,



I'm happy to answer some questions; your link was to the GitHub project
rather than a specific issue, but I'm presuming it's
https://github.com/tc39/proposal-temporal/issues/702 that you were
referring to.



I've copied this to the calsify (Calext WG) list
<https://www.ietf.org/mailman/listinfo/calsify>, but encourage people to
reply-all as you have had difficulty joining it.



In answer to the questions in that issue, my understanding is:



Q1. As specified in RFC5545 (and JSCalendar):

   - A duration does not have a time zone associated with it. It is just an
   abstract number of weeks/days/hours etc.
   - If you want to determine how long in absolute terms a duration with
   days/weeks is, this question only makes sense when you have a start
   date-time to apply it to.
   - A duration may be added or subtracted from a date-time. If the
   date-time does not have an associated time zone then this is very easy
   (it's just like UTC: 1 day is always exactly 24h).
   - Otherwise, as per the spec, you would add the weeks/days first to the
   calendar day/month/year. If the result is an ambiguous time due to DST, you
   resolve at this point. Then you add the absolute hours/minutes/seconds etc.
   – this cannot result in another ambiguity. There is only ambiguity mapping
   from TZ -> UTC, not UTC -> TZ.



I think your implementation in (1) seems similar to this, but I have not
looked at the source closely to confirm. I'm not sure why you think the
disambiguation in the middle is going to be problematic for users; surely
they generally won't notice where it happens? You just add the duration to
the date-time and get a result.



If I'm understanding it correctly, I do not think your proposed alternative
is acceptable because it applies the time component to the wall-clock time,
which means that if you had an event with duration PT1H that started on a
1-hour DST-transition point it would be translated into an absolute
duration of 0 hours or 2 hours, which is clearly wrong! e.g. I have an
event that starts at 2019-03-31T02:45 in Europe/Berlin. I add a duration of
PT1H in wall clock time to get 2019-03-31T03:45 in Europe/Berlin. However,
those are the same time in UTC due to the DST transition, so the absolute
duration becomes 0!



Q2. I don't think this issue arises when following the RFC5545 semantics
above.



Q3. I don't think subtraction is well specified and probably wasn't deeply
considered; it is basically only used for the offset of alerts before the
start of the event and I think your proposal of subtracting absolute
components first is very reasonable for the reason you gave. We should
clarify this in the JSCalendar spec.



Q4. This is really an implementation question. The result is clearly only
correct if you can add it to the start date and get back the end date again
with the already defined semantics. You can either:

   - Just find the difference in absolute time and represent this as a
   duration in hours/minutes/seconds.
   - Subtract hours/minutes first to get the same wall clock time, then
   calculate number of days difference to get a duration in
   weeks/days/hours/minutes/seconds.



Q5. I don't think I quite understand your question. You say you are
"calculating differences between UTC values" but then the "difference ends
inside a DST discontinuity" – UTC doesn't have DST, so this doesn't make
sense. Could you please clarify?



Cheers,

Neil.