Re: [core] Subscribe/Notify for CoAP

["Adriano Pezzuto (apezzuto)" <[e-mail]>]

Hello,

with this flavor, the SUBSCRIBE may be replaced by a simple POST. So no needs to have an explicit SUBSCRIBE method.

If we want support a more oriented peer-to-peer transaction on CoAP, it looks to me both SUB and NOTIFY should be treated as messages.

 

Adriano

 

From: [e-mail] [mailto:[e-mail]] 
Sent: venerdì 28 maggio 2010 13.52
To: Adriano Pezzuto (apezzuto)
Cc: core; [e-mail]; [e-mail]
Subject: Re: [core] Subscribe/Notify for CoAP

 

> So strictly speaking, both NOTIFY and SUBSCRIBE are message types not  methods!

SUBSCRIBE can be just an asynchronous GET on update then I think it is a method and NOTIFY is an asynchronous response so a message.
But if we want to have selective notifications when a resource is created, updated or deleted then I think you're right NOTIFY and SUBSCRIBE are messages.

Does that make sense ?

Matthieu

 "Adriano Pezzuto (apezzuto)" <[e-mail]>





"Adriano Pezzuto (apezzuto)" <[e-mail]> 
Envoyé par : [e-mail] 

28/05/2010 12:50

 

A

 
<[e-mail]>



cc


core <[e-mail]>



Objet


Re: [core] Subscribe/Notify for CoAP

 






Hi Roberto,
I understand your point and personally agree on M2M is quite different from web page model. On the architectural RESTful style, the method information tells the server what to do with data kept in the URI information.

So strictly speaking, both NOTIFY and SUBSCRIBE are message types not  methods!

Adriano 

From: Robert Cragie [mailto:[e-mail]] 
Sent: venerdì 28 maggio 2010 11.53
To: Adriano Pezzuto (apezzuto)
Cc: Paul Duffy (paduffy); Zach Shelby; core
Subject: Re: [core] Subscribe/Notify for CoAP

I think the point here is that there are two levels we are considering.

At the lowest (foundation) level, there are the transaction messages as described below. This provides a flexible mechanism for the application with regard to synchronicity and threading. This is important for the types of devices CoAP is being aimed at.

Above that, the methods can be used according to the architectural style. So in this case, it is RESTful (COnstrained Restful Environments), which will naturally limit the number of methods and how transactions occur.

The actual architecture using a combination of the methods and messages also depends on what is required at the application layer. Consider a typical client which wants to subscribe to a resource. That client controls the feed of data but needs a component which is capable of handling (possibly buffering) the data it receives through notifications. Is this a separate server? Or would we want to consider it part of an enhanced client model which is able to process feeds of data? These are the sort of models which have led to the myriad of solutions (GENA, Webhooks, long polling, pubsubhubbub, RESTMS etc.) based around HTTP which are all essentially ingenious ways of getting around the limitations imposed by HTTP and how it is processed for anything which deviates from the classic web page access model.

I think the aim of CoAP should be clean from the word go with regard to supporting these more peer-to-peer transactions, where the client can exist on either entity and both entities can feed data to each other; typical in M2M applications.

Robert 

Robert Cragie (Pacific Gas & Electric) 

Gridmerge Ltd.
89 Greenfield Crescent,
Wakefield, WF4 4WA, UK
[phone]
[phone]
http://www.gridmerge.com <http://www.gridmerge.com/> 

On 27/05/2010 7:08 PM, Adriano Pezzuto (apezzuto) wrote: 
Hi Robert,

in my personal opinion, the option 1a) brings some sort of ambiguity to CoAP specs.

My be my understatement of new CoAP specs is not so deep, but now we have 5 methods and 3 message types: request, response and notify. Which methods are allowed with which messages types?
I suppose you have to use PUT/POST method with notify message for asynch data notification. How to make a subscribe? I suppose you would use a SUBSCRIBE method with request/response message or SUBSCRIBE with notify message? Also what about POST/DELETE methods in a notify message? They not make any sense..

I think the choice is between: option 1) -> only CRUD methods and option 1b) -> CRUD + SUB/NOTIFY methods, keeping in mind cost/benefits of both solutions.

Adriano

From: Robert Cragie [mailto:[e-mail] <mailto:[e-mail]> ] 
Sent: mercoledì 26 maggio 2010 20.09
To: Adriano Pezzuto (apezzuto)
Cc: Paul Duffy (paduffy); Zach Shelby; core
Subject: Re: [core] Subscribe/Notify for CoAP

Hi Adrian,

I would also prefer to keep the protocol in CoAP asynchronous. You can always map an asynchronous protocol to a synchronous one but, as we see in HTTP, it always ends up as a kludge to do it the other way round. The efforts which have been gone to to make HTTP quasi-asynchronous via all the schemes mentioned below and many more besides (all non-interoperable of course) is testament to how important this is for M2M communication.

So, back to Zach's list, I favor 1a) for the following reasons:

Foundation level of messages: 

1. request/response can be asynchronous or synchronous messages (as there is a transaction ID in there)
2. notify is an asynchronous message

Derived methods:

I think it makes sense to add a pub/sub model as a useful mechanism for M2M.

So, looking at it the other way round: It will be entirely possible to translate whatever is currently built on HTTP to CoAP based on the above, with all its restrictions regarding synchronous and client/server transactions. What may be harder is to translate directly is a CoAP-based application to HTTP. So I guess the question is: Do we want to be hamstrung to synchronous client/server transactions as dictated by HTTP and provide a direct mapping to HTTP, then have to come up with similar kludges for asynchronous peer-to-peer transactions as has been done in numerous ways for HTTP, or do we want to define the protocol cleanly to start with and accept that some sort of transaction relaying/conversion would have to take place at a mapping node?

Robert 

Robert Cragie (Pacific Gas & Electric) 

Gridmerge Ltd.
89 Greenfield Crescent,
Wakefield, WF4 4WA, UK
[phone]
[phone]
http://www.gridmerge.com <http://www.gridmerge.com/> 

On 26/05/2010 7:17 AM, Adriano Pezzuto (apezzuto) wrote: 
Hi,
it looks to me that CoAP should use an explicit sub/notify mechanism since this is the core of the machine-to-machine interaction model.
HTTP suffers of this lack and we have seen a plethora of solutions to give an asynch taste to it. Webhooks and websockets are only the lasts of the list.
As someone has already pointed out on this list, it is theoretically possible to describe sub/notify using only CRUD methods but it looks a little bit tricky and verbose.

Now we have a chance to build from scratch a new protocol with and I think using explicit sub/notify methods with a clear and well defined semantic is the best option. It is easily understanding from every developer and will prevent to build other fanny solutions on top of the CoAP. HTTP does not have this well defined semantic and (for hundreds of other reasons also) it is not used as wide protocol for machine-to-machine communication.

CoAP - as binary protocol - and with an explicit asynch model has a chance to be a really wide protocol for M2M communication not only for constrained environments.

my 2 cents

- adriano

-----Original Message-----
From: [e-mail] <mailto:[e-mail]>  [mailto:[e-mail] <mailto:[e-mail]> ] On Behalf Of Paul Duffy (paduffy)
Sent: mercoledì 26 maggio 2010 0.47
To: Zach Shelby
Cc: core
Subject: Re: [core] Subscribe/Notify for CoAP

On 5/25/2010 6:41 PM, Zach Shelby wrote:

Hi,

On May 26, 2010, at 12:23 AM, Charles Palmer wrote:



Hi folks

It occurs to me that CoRE should be keeping a close eye on ZigBee SE2.0 work, so that it is as easy as possible for ZigBee SE to use CoRE when ready. That suggests to me that we should align with their subscribe/notify process.



I am not sure I understand that. I mean, ZigBee SE2.0 is defining an application specific subscribe/notify mechanism for that purpose so far for HTTP. This uses standard HTTP methods and some custom payload and REST interfaces. CoAP Req/Res is already totally compatible with SE2.0 in that respect, so alignment is already OK there. Nothing stopping someone from using SE2.0 over CoAP.

Specifying a native susbcription/notify into CoAP is another matter. We can't adopt a solution specific to one application as that won't solve the problems of other applications nor general HTTP mapping at all (probably would make it worse). It seems that for the near future there will be a bunch of HTTP push mechanisms in use without any clear standard appearing - or am I wrong there?




If COAP extends HTTP semantics with new subscription methods, it will 
not be possible to easily interchange HTTP/COAP, and translation 
gateways will become more complex to implement.




Zach



Regards - Charles


-----Original Message-----
From: [e-mail] <mailto:[e-mail]>  [mailto:[e-mail] <mailto:[e-mail]> ] On Behalf Of Paul Duffy
Sent: 25 May 2010 03:48
To: Zach Shelby
Cc: core
Subject: Re: [core] Subscribe/Notify for CoAP

Recommend something like #2, primarily to avoid introducing non HTTP
method semantics, simplifying HTTP/COAP translation.gateways, etc.


On 5/24/2010 11:49 AM, Zach Shelby wrote:

(thread renamed)

We have two different paths with regard to a subscribe/notify mechanism for CoAP:

1. Use specific Subscription and Notify mechanisms for CoAP as HTTP really does not include this concept.
1a) Notify as a message and SUBSCRIBE as a method. This is currently used in coap-01.
1b) NOTIFY and SUBSCRIBE as methods. This was used in coap-00, but we had a good list discussion about this leading to a. In practice it doesn't make a big difference if notification is a message or method.

2. Use an HTTP specific framework such as Webhooks, the ZigBee SE2.0 proposal or GENA.

So far we have focused on 1 in the WG, and every now and again 2 comes up. At least I am not convinced that we need to suffer the drawbacks of HTTP here. Anyways 2 does not help our mapping to HTTP in reality very much as there is no standard way of doing this over HTTP. Thus a CoAP-HTTP proxy may end up anyways translating between multiple HTTP frameworks depending on the application. So instead of doing a CoAP Notify/Subscribe to Webhooks mapping, you will could end up having to do a CoAP Webhooks to HTTP GENA mapping.




I don't understand this last para. If CoAP sticks to the semantics of
the current HTTP methods, would this not offer a fairly straightforward
translation to/from HTTP?




>From what I have heard so far 1 still seems like a wise choice, although I need to look at Webhooks more deeply. In reality many applications specify their own way of doing a push interface using REST methods and specific payloads (ZigBee SE2.0 is such an example). That is just fine, and might be used instead of a specific CoAP notify/subscribe in that case. So 1 doesn't prevent the application using its own mechanism, it just provides a native way for doing push.

What do people think?

Zach

On May 17, 2010, at 6:44 PM, [e-mail] <mailto:[e-mail]>  wrote:




Hi,

My comments about the subscribe/notify mechanism of Zigbee IP.

Pros:
- Derived from the webhooks concept
- If used by CORE it will be easier to map to HTTP because it uses only CRUD verbs.
- The subscription message is extendable and could support advanced options (delays, increment, ...)
- Only one listening port whatever the transport binding is.

Cons:
- No interoperability without well known URIs and a well defined subscription message format (Not sure CoAP draft is the right place to specify this).
- XML/EXI is too complex to be the required format for the default subscription/notification mechanism.
- The notification should not require a subsequent GET to retrieve the content.
- Subresource subscription is redundant.

Hope this could help,
Matthieu


<graycol.gif>"Charles Palmer"<[e-mail]> <mailto:[e-mail]> 




"Charles Palmer"<[e-mail]> <mailto:[e-mail]> 
Envoyé par : [e-mail] <mailto:[e-mail]> 
15/05/2010 14:06

<ecblank.gif>
A
<ecblank.gif>
"core"<[e-mail]> <mailto:[e-mail]> 
<ecblank.gif>
cc
<ecblank.gif>
<ecblank.gif>
Objet
<ecblank.gif>
Re: [core] Selecting a WG document for CoAP
<ecblank.gif> <ecblank.gif>

Dear all

Those interested in the subscribe/notify discussion might like to look
at the draft Smart Energy Profile 2.0 Application Protocol
Specification. It is available here:
http://zigbee.org/Markets/ZigBeeSmartEnergy/ZigBeeSmartEnergy20PublicApp <http://zigbee.org/Markets/ZigBeeSmartEnergy/ZigBeeSmartEnergy20PublicApp> 
licationProfile.aspx

It manages subscribe/notify by using POST. This seems to remove the need
for SUBSCRIBE and notify.

"Imagine a host A, which exposes a resource at http{s}://A/resource and
a second host B, which wishes to learn of changes to this resource. To
facilitate a subscription/ notification mechanism, A would expose a
resource http{s}://A/sub and B would expose a resource http{s}://B/ntfy.
To subscribe to notifications regarding http{s}://A/resource, B would
send a POST to the address http{s}://A/sub/B containing the URI of the
resource of interest (http{s}://A/resource) and the URI of B's
notification resource (http{s}://B/ntfy). Following this subscription
step, should A wish to notify B of a change to the resource addressed at
http{s}://A/resource, A would send a POST to the address
http{s}://B/ntfy containing the URI of the resource changed
(http{s}://A/resource) and the URI of A's subscription resource
(http{s}://A/sub/B), should A wish to change its subscription. The host
B can then query the resource (or not) at its leisure."

Sleepy nodes are not allowed to subscribe, and must poll.

Regards - Charles Palmer

-----Original Message-----
From: [e-mail] <mailto:[e-mail]>  [mailto:[e-mail] <mailto:[e-mail]> ] On Behalf Of
Angelo P. Castellani
Sent: 14 May 2010 13:14
To: Zach Shelby
Cc: core
Subject: Re: [core] Selecting a WG document for CoAP

Zach,

thanks for the comments, but please refer to my most recent e-mail for
a more specific list of technical issues I'm pointing to.

I want to do only a little integration to what I've written there.

In my very personal opinion, to maximize adherence with the REST model
and minimize implementation effort SUBSCRIBE and NOTIFY should be
mapped to methods (as discussed many times together...).

Uniform interface principle (Fielding PhD dissertation Section 5.1.5,
"The central feature that distinguishes the REST architectural style
[...]") states that to simplify the software architecture, resource
interfaces/interfaces should be as general as possible.

I agree with this principle in this specific case, mainly because
handling a special message type for notify leads node software design
to a more complex architecture.

The reason is that this new message type requires special handling and
introduces a complexity in the software modularization.

Best,
Angelo

On Fri, May 14, 2010 at 13:06, Zach Shelby<[e-mail]> <mailto:[e-mail]>  wrote:



Hi Angelo,

On May 13, 2010, at 14:24 , Angelo P. Castellani wrote:




Dear C. Bormann, all,

before deciding for the final direction, I've the following
observations about draft-shelby-core-coap-01

While I mostly share Zach's view of the protocol approach, and
appreciate many aspects of the proposal, there are in my opinion



still



some open issues that need to be at least discussed before the



current



document can be selected.



Of course there are plenty of open issues. Remember that working group



documents still undertake as much change and improvement as the WG
wants, so by no means is coap-01 set in stone. I would expect at least
5-10 more revisions still along the way. Already several of your ideas
have been integrated into coap-01, and several more are under
consideration, so it is coming along. Patience ;-)



 

In particular, I would like to highlight the following:

a) As it is now, it is not possible to have a straightforward
translation of HTTP -> CoAP and viceversa: see
http://www.ietf.org/mail-archive/web/core/current/msg00133.html <http://www.ietf.org/mail-archive/web/core/current/msg00133.html>  (this
impacts the scalability of the web service model too)



In coap-01 the Method names are now identical to HTTP methods. The



Req/Res interaction is a direct translation. The URI hierarchy is
compatible, and the URI binary code format we are still working on
obviously. The only thing that takes some state to translate is
Subscribe/Notify. But note, Subscribe/Notify takes some state no matter
how you do it, even with HTTP-HTTP there is no clean and easy way to
handle subscriptions.



 

b) Moreover, CoAP's implementation is not as simple as it should be.
I've investigated the implementation and some design choices (as
Options) are leading to very high program complexity (ROM) on a
MSP430-based device.



This we can definitely improve, and already made several optimizations



from -00 to -01. Here I need some very concrete proposals though. Also
remember that many things are optional, for example subscribe/notify is
not required if you don't need it.



 

c) Finally when comparing HTTP message size with CoAP message size,
the resulting compression isn't as good as you may expect.

Example:
HTTP: GET /sensor/temp.xml HTTP/1.0 = 32 B
CoAP: 24 B with options parsing procedure requiring an added
implementation complexity



Right, that is not how it will work in practice. Working with a real



HTTP server that HTTP header will be more complex, and on the CoAP side
you would chose shorter URLs. The biggest improvement possible here is
from using binary coded URLs of course. We need to look at a wider range
of interactions and real HTTP headers as well to check that we are
efficient enough.



 

Addressing all these points potentially leads to major technical
modifications (especially point a) on the current draft, hence it is
appropriate in my opinion to discuss these points before making the
final decision.



I am not sure what else you have in mind for a). If we just forget



about Subscribe/Notify then you are good to go. But then you are also
not fulfilling the charter or the industry needs in that respect.



Thanks,
Zach




Best regards,

Angelo P. Castellani


On Mon, May 10, 2010 at 18:51, Carsten Bormann<[e-mail]> <mailto:[e-mail]>  wrote:



The CORE WG has a milestone to select a WG document for CoAP in



April:



http://datatracker.ietf.org/wg/core/charter/ <http://datatracker.ietf.org/wg/core/charter/> 
...
Apr 2010 Select WG document for basis of the CoAP protocol

Of the various documents that have been contributed,



draft-shelby-core-coap has significant discussion, as well as the
largest number of updates (including a previous version that was still
called -6lowapp-coap).



Today, another updated version of that draft was announced. See
http://www.ietf.org/mail-archive/web/core/current/msg00138.html <http://www.ietf.org/mail-archive/web/core/current/msg00138.html> 
for the announcement and
http://tools.ietf.org/html/draft-shelby-core-coap-01 <http://tools.ietf.org/html/draft-shelby-core-coap-01> 
for the draft itself.

However, as the authors say, there are still significant TODOs.

Are we in a state yet where we can say whether this is the right



direction for the WG to take?



If yes, is it the right direction? Should we adopt it as a WG



document?



If you don't think we can say yet, is there a set of technical



decisions you would like the authors to take with priority?



Note that once a document has become a WG document, the authors act



as editors for the working group, making (and usually fleshing out the
details of) any change that the WG decides it needs.



If you think we can still improve the draft, this is not an obstacle



to making it a WG document.



But of course we shouldn't do that if we intend to reverse its



fundamental technical direction.



In order to stay roughly in sync with our milestones, we should



reach at a decision on how to go forward this week.



Gruesse, Carsten

_______________________________________________
core mailing list
[e-mail] <mailto:[e-mail]> 
https://www.ietf.org/mailman/listinfo/core <https://www.ietf.org/mailman/listinfo/core> 




_______________________________________________
core mailing list
[e-mail] <mailto:[e-mail]> 
https://www.ietf.org/mailman/listinfo/core <https://www.ietf.org/mailman/listinfo/core> 



--
Zach Shelby, Chief Nerd, Sensinode Ltd.
http://zachshelby.org <http://zachshelby.org/>  - My blog "On the Internet of Things" <http://6lowpan.net-mybook/> 
http://6lowpan.net - My book " <http://6lowpan.net-mybook/> 6LoWPAN: The Wireless Embedded Internet"
Mobile: [phone]





_______________________________________________
core mailing list
[e-mail] <mailto:[e-mail]> 
https://www.ietf.org/mailman/listinfo/core <https://www.ietf.org/mailman/listinfo/core> 

--------------------------------
Onzo is a limited company number 06097997 registered in England& Wales. The
registered office is 6 Great Newport Street, London, WC2H 7JB, United Kingdom.

This email message may contain confidential and/or privileged information, and
is intended solely for the addressee(s). If you have received this email in
error, please notify Onzo immediately. Unauthorised copying, disclosure or
distribution of the material in this email is forbidden.
--------------------------------

_______________________________________________
core mailing list
[e-mail] <mailto:[e-mail]> 
https://www.ietf.org/mailman/listinfo/core <https://www.ietf.org/mailman/listinfo/core> 

______________________________________________________________________
This email has been scanned by the MessageLabs Email Security System.
______________________________________________________________________

_______________________________________________
core mailing list
[e-mail] <mailto:[e-mail]> 
https://www.ietf.org/mailman/listinfo/core <https://www.ietf.org/mailman/listinfo/core> 



 

_______________________________________________
core mailing list
[e-mail] <mailto:[e-mail]> 
https://www.ietf.org/mailman/listinfo/core <https://www.ietf.org/mailman/listinfo/core> 

--------------------------------
Onzo is a limited company number 06097997 registered in England& Wales. The
registered office is 6 Great Newport Street, London, WC2H 7JB, United Kingdom.

This email message may contain confidential and/or privileged information, and
is intended solely for the addressee(s). If you have received this email in
error, please notify Onzo immediately. Unauthorised copying, disclosure or
distribution of the material in this email is forbidden.
--------------------------------



 


_______________________________________________
core mailing list
[e-mail] <mailto:[e-mail]> 
https://www.ietf.org/mailman/listinfo/core <https://www.ietf.org/mailman/listinfo/core> 
_______________________________________________
core mailing list
[e-mail] <mailto:[e-mail]> 
https://www.ietf.org/mailman/listinfo/core <https://www.ietf.org/mailman/listinfo/core> 



______________________________________________________________________
This email has been scanned by the MessageLabs Email Security System.
_____________________________________________________________________________________________________________________
core mailing list
[e-mail]
https://www.ietf.org/mailman/listinfo/core