Re: [Trans] Add get_entries_max_limit attribute to get-sth request

[Al Cutter <[e-mail]>]

Hi Vaclav,

I think there might be other ways of achieving what you're after.

On Mon, 4 Feb 2019, 10:19 Václav Jirovský, <[e-mail]>
wrote:

> Hello all,
>
> I would like to propose modification Retrieve Latest Signed Tree Head section
> of RFC6962 - adding new attribute *get_entries_max_limit.*
>
> Reason for this change - 4.6 section actual version:
>
> * Logs MAY restrict the number of entries that can be retrieved per
>    "get-entries" request.  If a client requests more than the permitted
>    number of entries, the log SHALL return the maximum number of entries
>    permissible.  These entries SHALL be sequential beginning with the
>    entry specified by "start".
> *
>
>
> If you want to download all entries from CT server, you don't what number
> of entries will server return to you by request - so you have to process,
> count real number of returned entries and after that, you can do another
> request. This is not efficient, you could do these request in parallel, if
> you would have garanteed number of returned entries.
>

Perhaps you could call get-sth to learn the current tree size, then divide
the range from what you currently have to the log's actual tree size into N
subranges, start N workers each repeatedly calling get-entries always with
start=first entry they don't yet have for their subrange, and end=limit of
their allocated subrange until they have fetched all of their allocated
entries?

That seems like it might give you the parallelism you're after?



> *Proposed modification:*
>
>
> 4.3 <https://tools.ietf.org/html/rfc6962#section-4.3>.  Retrieve Latest Signed Tree Head
>
> GET https://<log server>/ct/v1/get-sth
> No inputs.
>
> Outputs:
>
>       tree_size:  The size of the tree, in entries, in decimal.
>       timestamp:  The timestamp, in decimal.
>       sha256_root_hash:  The Merkle Tree Hash of the tree, in base64.
>
> *      get_entries_max_limit: Maximum entries count provided by server get-entries method.*
>
> tree_head_signature: A TreeHeadSignature for the above data.
>
>
> 4.6 <https://tools.ietf.org/html/rfc6962#section-4.6>.  Retrieve Entries from Log
>
> GET https://<log server>/ct/v1/get-entries
>
>
> Inputs:
>       start:  0-based index of first entry to retrieve, in decimal.
>       end:  0-based index of last entry to retrieve, in decimal.
>
>
> .....
>
>
>    Logs MAY restrict the number of entries that can be retrieved per
>    "get-entries" request.  *If a client requests more than the permitted
>    number of entries ("get_entries_max_limit" output of "get-sth" request),*
>
> *   the log SHALL return the maximum number of entries
>    permissible. If a client requests less or equal than the permitted
>    number of entries ("get_entries_max_limit" output of "get-sth" request),
>    the log MUST return the maximum number of entries permissible. *
>    These entries SHALL be sequential beginning with the
>
>    entry specified by "start".
>
>
It seems like you're optimising for a very specific use case - if I just
want to inspect one entry (e.g. a monitor tells me "hey, there's a cert for
your domain at entry X) I might have to download 1000s of certs just to
fetch the one I'm interested in? Also, what happens when I just want to
request the last few entries from the tree?
(Perhaps there's a typo in your proposal?)

Just because I happen to have an STH with a given "get_entries_max_limit"
it doesn't mean that the log didn't change that limit between when the STH
was created and when I make a request so I'd have to have logic to cope
anyway - log operators may wish to adjust this limit dynamically/from time
to time in order to react to usage patterns or resource
consumption/availability etc.

Cheers,
Al.



>
> Best,
>
> Vaclav Jirovsky
>
>
> _______________________________________________
> Trans mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/trans
>