Re: [core] Subscribe/Notify for CoAP

["Adriano Pezzuto (apezzuto)" <[e-mail]>]

Hi Roberto,

I understand your point and personally agree on M2M is quite different from web page model. On the architectural RESTful style, the method information tells the server what to do with data kept in the URI information.

 

So strictly speaking, both NOTIFY and SUBSCRIBE are message types not  methods!

 

Adriano 

 

From: Robert Cragie [mailto:[e-mail]] 
Sent: venerdì 28 maggio 2010 11.53
To: Adriano Pezzuto (apezzuto)
Cc: Paul Duffy (paduffy); Zach Shelby; core
Subject: Re: [core] Subscribe/Notify for CoAP

 

I think the point here is that there are two levels we are considering.

At the lowest (foundation) level, there are the transaction messages as described below. This provides a flexible mechanism for the application with regard to synchronicity and threading. This is important for the types of devices CoAP is being aimed at.

Above that, the methods can be used according to the architectural style. So in this case, it is RESTful (COnstrained Restful Environments), which will naturally limit the number of methods and how transactions occur.

The actual architecture using a combination of the methods and messages also depends on what is required at the application layer. Consider a typical client which wants to subscribe to a resource. That client controls the feed of data but needs a component which is capable of handling (possibly buffering) the data it receives through notifications. Is this a separate server? Or would we want to consider it part of an enhanced client model which is able to process feeds of data? These are the sort of models which have led to the myriad of solutions (GENA, Webhooks, long polling, pubsubhubbub, RESTMS etc.) based around HTTP which are all essentially ingenious ways of getting around the limitations imposed by HTTP and how it is processed for anything which deviates from the classic web page access model.

I think the aim of CoAP should be clean from the word go with regard to supporting these more peer-to-peer transactions, where the client can exist on either entity and both entities can feed data to each other; typical in M2M applications.

Robert

Robert Cragie (Pacific Gas & Electric)

Gridmerge Ltd.
89 Greenfield Crescent,
Wakefield, WF4 4WA, UK
[phone]
[phone]
http://www.gridmerge.com <http://www.gridmerge.com/> 


On 27/05/2010 7:08 PM, Adriano Pezzuto (apezzuto) wrote: 

Hi Robert,

 

in my personal opinion, the option 1a) brings some sort of ambiguity to CoAP specs.

 

My be my understatement of new CoAP specs is not so deep,  but now we have 5 methods and 3 message types: request, response and notify. Which methods are allowed with which messages types?

I suppose you have to use PUT/POST method with notify message for asynch data notification. How to make a subscribe? I suppose you would use a SUBSCRIBE method with request/response message or SUBSCRIBE with notify message? Also what about POST/DELETE methods in a notify message? They not make any sense..

 

I think the choice is between: option 1) -> only CRUD methods and option 1b) -> CRUD + SUB/NOTIFY methods, keeping in mind cost/benefits of both solutions.

 

Adriano

 

From: Robert Cragie [mailto:[e-mail]] 
Sent: mercoledì 26 maggio 2010 20.09
To: Adriano Pezzuto (apezzuto)
Cc: Paul Duffy (paduffy); Zach Shelby; core
Subject: Re: [core] Subscribe/Notify for CoAP

 

Hi Adrian,

I would also prefer to keep the protocol in CoAP asynchronous. You can always map an asynchronous protocol to a synchronous one but, as we see in HTTP, it always ends up as a kludge to do it the other way round. The efforts which have been gone to to make HTTP quasi-asynchronous via all the schemes mentioned below and many more besides (all non-interoperable of course) is testament to how important this is for M2M communication.

So, back to Zach's list, I favor 1a) for the following reasons:

Foundation level of messages:

1.	request/response can be asynchronous or synchronous messages (as there is a transaction ID in there)
2.	notify is an asynchronous message

Derived methods:

I think it makes sense to add a pub/sub model as a useful mechanism for M2M.

So, looking at it the other way round: It will be entirely possible to translate whatever is currently built on HTTP to CoAP based on the above, with all its restrictions regarding synchronous and client/server transactions. What may be harder is to translate directly is a CoAP-based application to HTTP. So I guess the question is: Do we want to be hamstrung to synchronous client/server transactions as dictated by HTTP and provide a direct mapping to HTTP, then have to come up with similar kludges for asynchronous peer-to-peer transactions as has been done in numerous ways for HTTP, or do we want to define the protocol cleanly to start with and accept that some sort of transaction relaying/conversion would have to take place at a mapping node?

Robert

Robert Cragie (Pacific Gas & Electric)

Gridmerge Ltd.
89 Greenfield Crescent,
Wakefield, WF4 4WA, UK
[phone]
[phone]
http://www.gridmerge.com <http://www.gridmerge.com/> 


On 26/05/2010 7:17 AM, Adriano Pezzuto (apezzuto) wrote: 

Hi,
it looks to me that CoAP should use an explicit sub/notify mechanism since this is the core of the machine-to-machine interaction model.
HTTP suffers of this lack and we have seen a plethora of solutions to give an asynch taste to it. Webhooks and websockets are only the lasts of the list.
As someone has already pointed out on this list, it is theoretically possible to describe sub/notify using only CRUD methods but it looks a little bit tricky and verbose.
 
Now we have a chance to build from scratch a new protocol with and I think using explicit sub/notify methods with a clear and well defined semantic is the best option. It is easily understanding from every developer and will prevent to build other fanny solutions on top of the CoAP. HTTP does not have this well defined semantic and (for hundreds of other reasons also) it is not used as wide protocol for machine-to-machine communication.
 
CoAP - as binary protocol - and with an explicit asynch model has a chance to be a really wide protocol for M2M communication not only for constrained environments.
 
my 2 cents
 
- adriano
 
-----Original Message-----
From: [e-mail] [mailto:[e-mail]] On Behalf Of Paul Duffy (paduffy)
Sent: mercoledì 26 maggio 2010 0.47
To: Zach Shelby
Cc: core
Subject: Re: [core] Subscribe/Notify for CoAP
 
On 5/25/2010 6:41 PM, Zach Shelby wrote:
  

	Hi,
	 
	On May 26, 2010, at 12:23 AM, Charles Palmer wrote:
	 
	   
	    

		Hi folks
		 
		It occurs to me that CoRE should be keeping a close eye on ZigBee SE2.0 work, so that it is as easy as possible for ZigBee SE to use CoRE when ready. That suggests to me that we should align with their subscribe/notify process.
		 
		     
		      

	I am not sure I understand that. I mean, ZigBee SE2.0 is defining an application specific subscribe/notify mechanism for that purpose so far for HTTP. This uses standard HTTP methods and some custom payload and REST interfaces. CoAP Req/Res is already totally compatible with SE2.0 in that respect, so alignment is already OK there. Nothing stopping someone from using SE2.0 over CoAP.
	 
	Specifying a native susbcription/notify into CoAP is another matter. We can't adopt a solution specific to one application as that won't solve the problems of other applications nor general HTTP mapping at all (probably would make it worse). It seems that for the near future there will be a bunch of HTTP push mechanisms in use without any clear standard appearing - or am I wrong there?
	   
	    

 
 
 
If COAP extends HTTP semantics with new subscription methods, it will 
not be possible to easily interchange HTTP/COAP, and translation 
gateways will become more complex to implement.
 
 
 
  

	Zach
	 
	   
	    

		Regards - Charles
		 
		 
		-----Original Message-----
		From: [e-mail] [mailto:[e-mail]] On Behalf Of Paul Duffy
		Sent: 25 May 2010 03:48
		To: Zach Shelby
		Cc: core
		Subject: Re: [core] Subscribe/Notify for CoAP
		 
		Recommend something like #2, primarily to avoid introducing non HTTP
		method semantics, simplifying HTTP/COAP translation.gateways, etc.
		 
		 
		On 5/24/2010 11:49 AM, Zach Shelby wrote:
		     
		      

			(thread renamed)
			 
			We have two different paths with regard to a subscribe/notify mechanism for CoAP:
			 
			1. Use specific Subscription and Notify mechanisms for CoAP as HTTP really does not include this concept.
			1a) Notify as a message and SUBSCRIBE as a method. This is currently used in coap-01.
			1b) NOTIFY and SUBSCRIBE as methods. This was used in coap-00, but we had a good list discussion about this leading to a. In practice it doesn't make a big difference if notification is a message or method.
			 
			2. Use an HTTP specific framework such as Webhooks, the ZigBee SE2.0 proposal or GENA.
			 
			So far we have focused on 1 in the WG, and every now and again 2 comes up. At least I am not convinced that we need to suffer the drawbacks of HTTP here. Anyways 2 does not help our mapping to HTTP in reality very much as there is no standard way of doing this over HTTP. Thus a CoAP-HTTP proxy may end up anyways translating between multiple HTTP frameworks depending on the application. So instead of doing a CoAP Notify/Subscribe to Webhooks mapping, you will could end up having to do a CoAP Webhooks to HTTP GENA mapping.
			 
			       
			        

		 
		I don't understand this last para.  If CoAP sticks to the semantics of
		the current HTTP methods, would this not offer a fairly straightforward
		translation to/from HTTP?
		 
		 
		     
		      

				 From what I have heard so far 1 still seems like a wise choice, although I need to look at Webhooks more deeply. In reality many applications specify their own way of doing a push interface using REST methods and specific payloads (ZigBee SE2.0 is such an example). That is just fine, and might be used instead of a specific CoAP notify/subscribe in that case. So 1 doesn't prevent the application using its own mechanism, it just provides a native way for doing push.
				         
				          

			What do people think?
			 
			Zach
			 
			On May 17, 2010, at 6:44 PM, [e-mail] wrote:
			 
			 
			       
			        

				Hi,
				 
				My comments about the subscribe/notify mechanism of Zigbee IP.
				 
				Pros:
				- Derived from the webhooks concept
				- If used by CORE it will be easier to map to HTTP because it uses only CRUD verbs.
				- The subscription message is extendable and could support advanced options (delays, increment, ...)
				- Only one listening port whatever the transport binding is.
				 
				Cons:
				- No interoperability without well known URIs and a well defined subscription message format (Not sure CoAP draft is the right place to specify this).
				- XML/EXI is too complex to be the required format for the default subscription/notification mechanism.
				- The notification should not require a subsequent GET to retrieve the content.
				- Subresource subscription is redundant.
				 
				Hope this could help,
				Matthieu
				 
				 
				<graycol.gif>"Charles Palmer"<[e-mail]> <mailto:[e-mail]> 
				 
				 
				 
				 
				"Charles Palmer"<[e-mail]> <mailto:[e-mail]> 
				Envoyé par : [e-mail]
				15/05/2010 14:06
				 
				<ecblank.gif>
				A
				<ecblank.gif>
				"core"<[e-mail]> <mailto:[e-mail]> 
				<ecblank.gif>
				cc
				<ecblank.gif>
				<ecblank.gif>
				Objet
				<ecblank.gif>
				Re: [core] Selecting a WG document for CoAP
				<ecblank.gif> <ecblank.gif>
				 
				Dear all
				 
				Those interested in the subscribe/notify discussion might like to look
				at the draft Smart Energy Profile 2.0 Application Protocol
				Specification. It is available here:
				http://zigbee.org/Markets/ZigBeeSmartEnergy/ZigBeeSmartEnergy20PublicApp
				licationProfile.aspx
				 
				It manages subscribe/notify by using POST. This seems to remove the need
				for SUBSCRIBE and notify.
				 
				"Imagine a host A, which exposes a resource at http{s}://A/resource and
				a second host B, which wishes to learn of changes to this resource. To
				facilitate a subscription/ notification mechanism, A would expose a
				resource http{s}://A/sub and B would expose a resource http{s}://B/ntfy.
				To subscribe to notifications regarding http{s}://A/resource, B would
				send a POST to the address http{s}://A/sub/B containing the URI of the
				resource of interest (http{s}://A/resource) and the URI of B's
				notification resource (http{s}://B/ntfy). Following this subscription
				step, should A wish to notify B of a change to the resource addressed at
				http{s}://A/resource, A would send a POST to the address
				http{s}://B/ntfy containing the URI of the resource changed
				(http{s}://A/resource) and the URI of A's subscription resource
				(http{s}://A/sub/B), should A wish to change its subscription. The host
				B can then query the resource (or not) at its leisure."
				 
				Sleepy nodes are not allowed to subscribe, and must poll.
				 
				Regards - Charles Palmer
				 
				-----Original Message-----
				From: [e-mail] [mailto:[e-mail]] On Behalf Of
				Angelo P. Castellani
				Sent: 14 May 2010 13:14
				To: Zach Shelby
				Cc: core
				Subject: Re: [core] Selecting a WG document for CoAP
				 
				Zach,
				 
				thanks for the comments, but please refer to my most recent e-mail for
				a more specific list of technical issues I'm pointing to.
				 
				I want to do only a little integration to what I've written there.
				 
				In my very personal opinion, to maximize adherence with the REST model
				and minimize implementation effort SUBSCRIBE and NOTIFY should be
				mapped to methods (as discussed many times together...).
				 
				Uniform interface principle (Fielding PhD dissertation Section 5.1.5,
				"The central feature that distinguishes the REST architectural style
				[...]") states that to simplify the software architecture, resource
				interfaces/interfaces should be as general as possible.
				 
				I agree with this principle in this specific case, mainly because
				handling a special message type for notify leads node software design
				to a more complex architecture.
				 
				The reason is that this new message type requires special handling and
				introduces a complexity in the software modularization.
				 
				Best,
				Angelo
				 
				On Fri, May 14, 2010 at 13:06, Zach Shelby<[e-mail]> <mailto:[e-mail]>    wrote:
				 
				         
				          

					Hi Angelo,
					 
					On May 13, 2010, at 14:24 , Angelo P. Castellani wrote:
					 
					 
					           
					            

					Dear C. Bormann, all,
					 
					before deciding for the final direction, I've the following
					observations about draft-shelby-core-coap-01
					 
					While I mostly share Zach's view of the protocol approach, and
					appreciate many aspects of the proposal, there are in my opinion
					 
					             
					              

				still
				 
				         
				          

					some open issues that need to be at least discussed before the
					 
					             
					              

				current
				 
				         
				          

					document can be selected.
					 
					             
					              

					Of course there are plenty of open issues. Remember that working group
					 
					           
					            

				documents still undertake as much change and improvement as the WG
				wants, so by no means is coap-01 set in stone. I would expect at least
				5-10 more revisions still along the way.  Already several of your ideas
				have been integrated into coap-01, and several more are under
				consideration, so it is coming along. Patience ;-)
				 
				         
				          

					           
					            

					In particular, I would like to highlight the following:
					 
					a) As it is now, it is not possible to have a straightforward
					translation of HTTP ->   CoAP and viceversa: see
					http://www.ietf.org/mail-archive/web/core/current/msg00133.html (this
					impacts the scalability of the web service model too)
					 
					             
					              

					In coap-01 the Method names are now identical to HTTP methods. The
					 
					           
					            

				Req/Res interaction is a direct translation. The URI hierarchy is
				compatible, and the URI binary code format we are still working on
				obviously. The only thing that takes some state to translate is
				Subscribe/Notify. But note, Subscribe/Notify takes some state no matter
				how you do it, even with HTTP-HTTP there is no clean and easy way to
				handle subscriptions.
				 
				         
				          

					           
					            

					b) Moreover, CoAP's implementation is not as simple as it should be.
					I've investigated the implementation and some design choices (as
					Options) are leading to very high program complexity (ROM) on a
					MSP430-based device.
					 
					             
					              

					This we can definitely improve, and already made several optimizations
					 
					           
					            

				from -00 to -01. Here I need some very concrete proposals though. Also
				remember that many things are optional, for example subscribe/notify is
				not required if you don't need it.
				 
				         
				          

					           
					            

					c) Finally when comparing HTTP message size with CoAP message size,
					the resulting compression isn't as good as you may expect.
					 
					Example:
					HTTP: GET /sensor/temp.xml HTTP/1.0 = 32 B
					CoAP: 24 B with options parsing procedure requiring an added
					implementation complexity
					 
					             
					              

					Right, that is not how it will work in practice. Working with a real
					 
					           
					            

				HTTP server that HTTP header will be more complex, and on the CoAP side
				you would chose shorter URLs. The biggest improvement possible here is
				from using binary coded URLs of course. We need to look at a wider range
				of interactions and real HTTP headers as well to check that we are
				efficient enough.
				 
				         
				          

					           
					            

					Addressing all these points potentially leads to major technical
					modifications (especially point a) on the current draft, hence it is
					appropriate in my opinion to discuss these points before making the
					final decision.
					 
					             
					              

					I am not sure what else you have in mind for a). If we just forget
					 
					           
					            

				about Subscribe/Notify then you are good to go. But then you are also
				not fulfilling the charter or the industry needs in that respect.
				 
				         
				          

					Thanks,
					Zach
					 
					 
					           
					            

					Best regards,
					 
					Angelo P. Castellani
					 
					 
					On Mon, May 10, 2010 at 18:51, Carsten Bormann<[e-mail]> <mailto:[e-mail]>    wrote:
					 
					             
					              

					The CORE WG has a milestone to select a WG document for CoAP in
					 
					               
					                

				April:
				 
				         
				          

					http://datatracker.ietf.org/wg/core/charter/
					...
					Apr 2010        Select WG document for basis of the CoAP protocol
					 
					Of the various documents that have been contributed,
					 
					               
					                

				draft-shelby-core-coap has significant discussion, as well as the
				largest number of updates (including a previous version that was still
				called -6lowapp-coap).
				 
				         
				          

					Today, another updated version of that draft was announced.  See
					http://www.ietf.org/mail-archive/web/core/current/msg00138.html
					for the announcement and
					http://tools.ietf.org/html/draft-shelby-core-coap-01
					for the draft itself.
					 
					However, as the authors say, there are still significant TODOs.
					 
					Are we in a state yet where we can say whether this is the right
					 
					               
					                

				direction for the WG to take?
				 
				         
				          

					If yes, is it the right direction?  Should we adopt it as a WG
					 
					               
					                

				document?
				 
				         
				          

					If you don't think we can say yet, is there a set of technical
					 
					               
					                

				decisions you would like the authors to take with priority?
				 
				         
				          

					Note that once a document has become a WG document, the authors act
					 
					               
					                

				as editors for the working group, making (and usually fleshing out the
				details of) any change that the WG decides it needs.
				 
				         
				          

					If you think we can still improve the draft, this is not an obstacle
					 
					               
					                

				to making it a WG document.
				 
				         
				          

					But of course we shouldn't do that if we intend to reverse its
					 
					               
					                

				fundamental technical direction.
				 
				         
				          

					In order to stay roughly in sync with our milestones, we should
					 
					               
					                

				reach at a decision on how to go forward this week.
				 
				         
				          

					Gruesse, Carsten
					 
					_______________________________________________
					core mailing list
					[e-mail]
					https://www.ietf.org/mailman/listinfo/core
					 
					 
					               
					                

					_______________________________________________
					core mailing list
					[e-mail]
					https://www.ietf.org/mailman/listinfo/core
					 
					             
					              

					--
					Zach Shelby, Chief Nerd, Sensinode Ltd.
					http://zachshelby.org  - My blog "On the Internet of Things" <http://6lowpan.net-Mybook> 
					http://6lowpan.net - My book " <http://6lowpan.net-Mybook> 6LoWPAN: The Wireless Embedded Internet"
					Mobile: [phone]
					 
					 
					 
					           
					            

				_______________________________________________
				core mailing list
				[e-mail]
				https://www.ietf.org/mailman/listinfo/core
				 
				--------------------------------
				Onzo is a limited company number 06097997 registered in England&   Wales. The
				registered office is 6 Great Newport Street, London, WC2H 7JB, United Kingdom.
				 
				This email message may contain confidential and/or privileged information, and
				is intended solely for the addressee(s). If you have received this email in
				error, please notify Onzo immediately. Unauthorised copying, disclosure or
				distribution of the material in this email is forbidden.
				--------------------------------
				 
				_______________________________________________
				core mailing list
				[e-mail]
				https://www.ietf.org/mailman/listinfo/core
				 
				______________________________________________________________________
				This email has been scanned by the MessageLabs Email Security System.
				______________________________________________________________________
				 
				_______________________________________________
				core mailing list
				[e-mail]
				https://www.ietf.org/mailman/listinfo/core
				 
				         
				          

			       
			        

		_______________________________________________
		core mailing list
		[e-mail]
		https://www.ietf.org/mailman/listinfo/core
		 
		--------------------------------
		Onzo is a limited company number 06097997 registered in England&  Wales. The
		registered office is 6 Great Newport Street, London, WC2H 7JB, United Kingdom.
		 
		This email message may contain confidential and/or privileged information, and
		is intended solely for the addressee(s). If you have received this email in
		error, please notify Onzo immediately. Unauthorised copying, disclosure or
		distribution of the material in this email is forbidden.
		--------------------------------
		 
		     
		      

	   
	    

 
_______________________________________________
core mailing list
[e-mail]
https://www.ietf.org/mailman/listinfo/core
_______________________________________________
core mailing list
[e-mail]
https://www.ietf.org/mailman/listinfo/core