Re: [Rats] RATS Architecture

Hi Kathleen,

thanks for the review!

Please see comments in-line. All assertion are from me as an individual. 
We will document your review as an issue or issues on github and address 
it in editor sessions.

On 21.05.20 15:34, Kathleen Moriarty wrote:
> Greetings!
> 
> I took some time to review the current version of the architecture 
> document and have some comments and suggestions.  The comments are from 
> the editor's version.  Please do post early and often updates to the 
> IETF draft repository.  Version numbers do not matter, it's okay to have 
> a large number :-)
> 
> These are meant to be helpful comments to improve the draft.
> 
> Introduction:
> Could we change:
> From:
> Trust is a decision being made.
> To:
> To trust is a decision that spans beyond the technical assertions.

That reads way more applicable.

> 
> I struggle a little with the following:
> "Trustworthiness is a quality that is assessed via evidence created."
> As when one determines trustworthiness, it may span beyond evidence that 
> is "created".  Perhaps drop the word created?  I think it spans beyond 
> that as well, but may be too nit picky to expand.  The evidence could 
> have to do with something other then the system and those who manage it 
> for instance.

In principle, the scope of RATS of course encompasses all conceptional 
messages. Dropping "created" improves the text by removing an arbitrary 
constraint, I think.

> 
> Terminology:
> The following definition should be just for "appraisal policy" unless 
> there are other types of appraisal policies in context for this body of 
> work:

There are two "steering" sets of Appraisal Policies. One for Evidence 
(used by a Verifier) and one for Attestation Results (used by a Relying 
Party).

>      Appraisal Policy for Attestation Result:  A set of rules that direct
>        how a Relying Party uses the Attestation Results regarding an
>        Attester generated by the Verifiers.  Compare /security policy/ in
>        [RFC4949]
> I don't think direct is the right word.  If this is so you can make 
> judgments on relying party decisions, the policy is more of a statement 
> on the decision process than how they are directed, which could be more 
> subjective.  The verifier here reads as if they generate the policy, but 
> are verifying against a set policy.
> Proposed:
> A set of rules for a Relying Party to verify attestation results.

Hm. The Attestation Results that are "policed" by the Appraisal Policies 
here are already the output of an appraisal procedure of a Verifier. The 
Relying Party is effectively the entity that is catered by RATS, but 
still the text acknowledges here that also the Relying Party depends on 
some imperative guidance here about how to react based on a specific 
Attestation Result.

Having said that, does your comment still apply? I am bit uncertain.

> 
> The following 'definition' seems to get into role recommendations rather 
> than being a definition:
>       Attester:  An entity whose attributes must be appraised in order to
>        determine whether the entity is considered trustworthy, such as
>        when deciding whether the entity is authorized to perform some
>        operation
> Shouldn't this stick to what an attester is/does?
> Perhaps:
> attester - entity who puts forth data that may include configuration or 
> static system information, evidence, or claims.  The evidence may be 
> digitally signed.

Evidence s created by an Attesting Environment of the Attester and must 
always be temper-evident (which does actually means that it must signed 
- but if it isn't an equivalent protection provided by a secure channel 
must exist). As a first step to address "is/does": An Atterster is a 
target of evaluation, I think, and it is able to create Evidence about 
its own system characteristics.

> 
> For evidence, should this definition be more about what evidence is 
> rather than about the roles involved?  The attester may not be the same 
> component or system that attests to the evidence as well.  Evidence 
> could be measurement data, telemetry (e.g. counters on a system 
> interface - YANG or SNMP), results of a policy comparison to 
> configuration data).
>      Evidence:  A set of information about an Attester that is to be
>        appraised by a Verifier

That is an excellent question, I think. The approach to define 
conceptual messages is the result of a compromise that we are following 
at the moment.

> Perhaps:
> Evidence: Information about or originating from a 
> system/firmware/software that may include configuration data, 
> measurements, telemetry, or inferences.
> 
> Is the following definition necessary?
>      Verifier Owner:  An entity, such as an administrator, that is
>        authorized to configure Appraisal Policy for Evidence in a
>        Verifier

Only from an architectural point of view, I think. It was hard to 
motivate the use cases and the "steering factors" of Verifiers and 
Endorsers without an owner. The requirements on Appraisal Policies and 
how they effectively look like is steered by the Owners. Maybe that 
could be pulled into the definitions of Verifier and Owner itself, but 
the contributors found this layout more intuitive. Having said that, it 
is not ultimately required to delve into that.

> 
> Section 3:
> Consider changing the following sentence:
> From:
> " Each use case includes a description, and a summary of what an
>     Attester and a Relying Party refer to in the use case."
> To:
> " Each use case includes a description followed by a summary of the
>     Attester and a Relying Party roles."

I would be okay with that change - it reads more intuitively.

> 
> Section 3.1:
> Sentence 1: Consider changing from:
> "Network operators want a trustworthy report of identity and version
>     of information of the hardware and software on the machines attached
>     to their network, for purposes such as inventory, auditing, and/or
>     logging."
> To:
> "Network operators want a trustworthy report that includes identity and 
> version
>     of information of the hardware and software on the machines attached
>     to their network, for purposes such as inventory, audit, 
> anomaly detection, record maintenance and/or trending reports (logging)."

Your change provides a more detailed view on the applications 
illustrated, I think. I'd be in favor of adopting it.

> 
> In the following sentence, "health" is fine, but did you consider the 
> more commonly used, "hygiene"?

No, actually we have not considered the term "hygiene" up to this 
moment. I am curious how other feedback looks like. I admit, I am 
relatively neutral towards that change.

> 
>  From the editor's version:
> "Typically, solutions start with a specific component (called a "Root of 
> Trust") that provides device identity and protected storage for 
> measurements. These components perform a series of measurements, and 
> express this with Evidence as to the hardware and firmware/software that 
> is running."
> Perhaps better stated as:
> "Typically, solutions start with a specific component (called a "Root of 
> Trust") that provides device identity and protected storage for 
> measurements. The system components perform a series of measurements 
> that may be signed by the Root of Trust, considered as Evidence of the 
> hardware, firmware, BIOS, software, etc. that is running."
> "express this with" is confusing and possibly misleading.  This proposed 
> phrasing allows for evidence to be broader than the series of 
> measurements as it might include system or configuration information.  
> It's not the RoT that does the measurements, but they do attest to it.

I like the change to system components as it better maps to system 
characteristics, I think. Maybe "Evidence about the hardware,..."? 
Otherwise, I like the proposal to rephrase "express this with".

> 
> Would the attester in this example be the root of trust of the device 
> rather than the full system?

The "whole entity" would be the Attester, the root of trust would be a 
part of or used by the Attesting Environment of the Attester (in 
contrast to the Target Environment of the Attester that Evidence is 
created about).

> 
> Section 3.2:
> Perhaps change the wording from:
> "A device manufacturer wants to protect its intellectual property in
>     terms of the ML model it developed and that runs in the devices that
>     its customers purchased, and it wants to prevent attackers,
>     potentially including the customer themselves, from seeing the
>     details of the model."
> 
> To:
> "A device manufacturer wants to protect its intellectual property.  This 
> is primarily the ML model it developed and runs in the devices purchased 
> by its customers. The goals for the protection include preventing 
> attackers, potentially the customer themselves, from seeing the details 
> of the model."

That was a way too long sentence. Thank you for chopping that one up! I 
like your new version much better.

> 
> I'm confused by the phrasing of the rest of the section and am wondering 
> if it can be improved.
> Current:
> "This typically works by having some protected environment in the
>     device attest to some manufacturer service.  If remote attestation
>     succeeds, then the manufacturer service releases either the model, or
>     a key to decrypt a model the Attester already has in encrypted form,
>     to the requester.
> 
>     Attester:  A device desiring to run an ML model to do inferencing
> 
>     Relying Party:  A server or service holding ML models it desires to
>        protect"
> 
> I think it would help to talk about inferences in the written 
> description.  For the attester, if it's the whole system as opposed to a 
> component, then then definition makes sense.  I'm wondering if it is 
> crisper when narrowed.  RoT captures inferences of the ML modeling used 
> as evidence.
> Does the relying party hold the ML model or knowledge of expected 
> inferences?
> 
> I'm happy to propose alternative text once I have more information to 
> more fully understand the use case.

In this use case, the Relying Party holds and conducts actual 
inference/prediction/reasoning producers, at least including a sub-set 
of the corresponding ML models. I think :) That use case was not phrased 
by me.

> 
> Section 3.3:This one took a couple of reads to understand.  While it is 
> understandable, it could be more explicit to make it easier to get on 
> first read.  How about changing the following From:
> Attestation is desired to prevent leaking data to compromised devices.
> To:
> "An assessment of system hygiene is made against a set of policies to 
> verify the state of a system using attestations for the system 
> requesting data.  Attestation is desired to prevent leaking data to 
> compromised devices."

If we go with hygiene (which is only a nit'ish detail and now reading it 
here again, I am starting to like it), I like the proposed text. Use 
case text is deliberately not consistent with the terminology of the 
rest of the text, so I am okay with "verify" in contrast to "appraise" 
here, maybe "evaluate"?

> 
> Section 3.6
> Nit:
> From:
> "One significant problem is malware that holds a device hostage and does 
> not allow it to reboot to prevent updates to be applied. "
> To:
> "One significant problem is malware that holds a device hostage and does 
> not allow it to reboot to prevent updates from being applied.  Remote 
> attestation aids in the verification process that may occur locally as 
> well as it is less likely that the policy or verification processes have 
> also been compromised if hosted on a separate system."

Hm. We are still in use cases here. Jumping to the "solution" that is 
remote attestation might overload the scope of the section a bit? I 
think we might have to rephrase that. I like highlighting the "lokal" part.

> 
> The second sentence is suggested as this also occurs locally on systems 
> today, but there is value in having this via remote attestation.
> 
> I'll send the next batch when I get far enough through the document.  I 
> didn't want to hold up on these comments.
> 
> Thank you for all your work on the document!
> 

And thanks again for your time and effort!

> -- 
> 
> Best regards,
> Kathleen
> 

Viele Grüße,

Henk

> _______________________________________________
> RATS mailing list
> RATS@ietf.org
> https://www.ietf.org/mailman/listinfo/rats
> 