[Tools-discuss] Re: Agenda Data Loading Slowly

[Robert Sparks <[e-mail]>]

Sure, we already have such celery tasks for several of those slow views.

Again, on a quick read, because the local cache for _this_ view is 
controlled by the decorator I first pointed to, a celery task wouldn't 
be able to invalidate that cache the way it should do. We'd need to 
tweak the way the cache is managed to allow that to happen.

The lift to serve from the edge (realized through putting the bits in a 
bucket) is separable, but still important - most of those slow views 
return large responses.

RjS

On 10/13/25 12:31 PM, Eric Rescorla wrote:
>
>
> On Mon, Oct 13, 2025 at 10:15 AM Robert Sparks <[e-mail]> 
> wrote:
>
>     First line at
>     https://github.com/ietf-tools/datatracker/blob/ed6b061cfe4279328dbb7b914f5f7f76644521f9/ietf/meeting/views.py#L2312-L2443.
>
>     It's a simple origin-side cache, time is the only invalidator.
>     More complicated invalidation would bring a new set of slow
>     calculations.
>
>     Again, reworking this to use a celery task to push the result to a
>     bucket on a similar period +/- so the browser never sees the time
>     the work takes would be one incremental step we could make.
>
> It seems like an even more incremental step would just be to use 
> celery to re-generate the top X slow view, thus keeping the cache 
> warm. This would have similar timeliness and performance consequences 
> properties but with no need to mess around with buckets.
>
> -Ekr
>
>     There's not a concrete schedule for doing that. It's highly likely
>     that we can do this between 124 and 125, but there are a lot of
>     other things in flight.
>
>     A larger refactor that would make a lot of the calculation in that
>     view, such as a data structure that makes finding the times/rooms
>     of all scheduled sessions on the official agenda in the set of all
>     sessions scheduled in different time/rooms in draft agendas or
>     those not scheduled at all,  be more efficient manner is needed -
>     what's grown incrementally from the original scheduling management
>     tooling is over-complicated at this point. _That_ level of
>     refactor is much further out.
>
>     In the meantime, if you let me know what you're trying to take
>     from the page, I might be able to point you to places that aren't
>     doing a lot of the work that endpoint does that you might not be
>     using.
>
>     RjS
>
>     On 10/13/25 11:50 AM, Eric Rescorla wrote:
>>     On Mon, Oct 13, 2025 at 9:41 AM Robert Sparks
>>     <[e-mail]> wrote:
>>
>>         Yes, the agenda is cached, and when the cache expires it is
>>         recalculated - that recalculation is very complicated (as it
>>         returns, for instance, the current versions of each
>>         (agenda/slides/drafts) for each group).
>>
>>     Can you say more about how the caching works here? Is it just a
>>     simple
>>     time-based cache or something else?
>>
>>         We could (and will) refactor the page to distribute the the
>>         work that's currently done in that one massive json load. We
>>         can also move keeping a computed version published so the
>>         browser never sees the time the computation takes, but that's
>>         not going to happen before IETF 124.
>>
>>     Is there a timeline for this work?
>>
>>     -Ekr
>>
>>         RjS
>>
>>         On 10/13/25 10:59 AM, Eric Rescorla wrote:
>>>
>>>
>>>         On Mon, Oct 13, 2025 at 8:54 AM Bob Hinden
>>>         <[e-mail]> wrote:
>>>
>>>             Ekr,
>>>
>>>             For what it’s worth, I just went to both links, only
>>>             took a few seconds for me.
>>>
>>>
>>>         Yes, presumably it's being cached somewhere. However, it's
>>>         not at all uncommon
>>>         for this to be slow, and I've seen the then-current agenda
>>>         take a long time on
>>>         a number of occasions, so I do think it's worth addressing.
>>>
>>>         -Ekr
>>>
>>>
>>>             Bob
>>>
>>>
>>>>             On Oct 13, 2025, at 8:49 AM, Eric Rescorla
>>>>             <[e-mail]> wrote:
>>>>
>>>>             Hi folks,
>>>>
>>>>             I just went to load the agenda for IETF 123
>>>>             (https://datatracker.ietf.org/meeting/123/agenda) for
>>>>             <reasons>.
>>>>
>>>>             It took 17s for the datatracker to send me the
>>>>             underlying JSON file for the agenda
>>>>             (https://datatracker.ietf.org/api/meeting/123/agenda-data)
>>>>             It seems like it's faster on repeated load, so
>>>>             presumably there is some caching going on, but 17s
>>>>             still seems pretty excessive.
>>>>
>>>>             -Ekr
>>>>
>>>>
>>>>
>>>>
>>>>             <image.png>
>>>>
>>>>             -----------------------------------------------
>>>>             Tools-discuss mailing list -- [e-mail]
>>>>             To unsubscribe send an email to
>>>>             [e-mail]
>>>>             https://mailarchive.ietf.org/arch/browse/tools-discuss/
>>>
>>>
>>>         -----------------------------------------------
>>>         Tools-discuss mailing list [e-mail]
>>>         To unsubscribe send an email [e-mail]
>>>         https://mailarchive.ietf.org/arch/browse/tools-discuss/
>>         -----------------------------------------------
>>         Tools-discuss mailing list -- [e-mail]
>>         To unsubscribe send an email to [e-mail]
>>         https://mailarchive.ietf.org/arch/browse/tools-discuss/
>>
>
> -----------------------------------------------
> Tools-discuss mailing list [e-mail]
> To unsubscribe send an email [e-mail]
> https://mailarchive.ietf.org/arch/browse/tools-discuss/