Re: [sop] SOP Requirements

[Vishwas Manral <[e-mail]>]

Thanks Mike.

I am just eager that we have a clear cut set of requirements and problem
statement defined, before we go ahead and write a solution.

Ashish, sure let us look at the options and see what makes sense.

On the DTCP/IP front we have both an encryption as well as a control plane
AKE, which serves the purpose for digital content.

-Vishwas

On Tue, Mar 13, 2012 at 6:15 AM, Michael Hammer <[e-mail]> wrote:

> But, if you start with an existing protocol and add methods and tweaks for
> every problem,
> you may end up with SOP again in the end.  :)
>
> Mike
>
>
> On Tue, Mar 13, 2012 at 1:45 AM, Ashish Dalela (adalela) <
> [e-mail]> wrote:
>
>> Vishwas,****
>>
>> ** **
>>
>> There are multiple problems. IKE is only key exchange. You can’t use it
>> to encrypt packets because it will break policy routing (described in my
>> email).****
>>
>> ** **
>>
>> The multicast / discovery problem can be solved by moving from TCP to
>> UDP. That alone isn’t enough because moving away from TCP means you lost
>> transaction identity. ****
>>
>> ** **
>>
>> Sometimes what seems like a solution brings some other problems, which
>> also need to be solved.****
>>
>> ** **
>>
>> If you want, we can draft up a list of enhancements needed to existing
>> protocols. I’m open to enhancing existing protocols. ****
>>
>> ** **
>>
>> Thanks, Ashish****
>>
>> ** **
>>
>> *From:* [e-mail] [mailto:[e-mail]] *On Behalf Of
>> *Vishwas Manral
>> *Sent:* Tuesday, March 13, 2012 5:43 AM
>>
>> *To:* Ashish Dalela (adalela)
>> *Cc:* [e-mail]; Michael Hammer
>> *Subject:* Re: [sop] SOP Requirements****
>>
>> ** **
>>
>> Hi Ashish,
>>
>> Yes, I was talking about UPnP/ SSDP. For hijacking prevention, we used a
>> protocol like IKE called AKE (though I am sure we could use IKE too).
>>
>> I am not trying to say the protocol you invented is wrong, but based on
>> the top level information, it looks similar to what is achievable now.
>>
>> So if the problem is multicast for discovery, can we optimize the
>> discovery part instead of doing the whole protocol itself. I think we need
>> to propose a tighter problem statement.
>>
>> Thanks,
>> Vishwas****
>>
>> On Tue, Mar 6, 2012 at 5:24 AM, Ashish Dalela (adalela) <
>> [e-mail]> wrote:****
>>
>> Hi Vishwas,****
>>
>>  ****
>>
>> I’m supposing that you are talking about SSDP (
>> http://en.wikipedia.org/wiki/Simple_Service_Discovery_Protocol)? Let me
>> know if that is correct.****
>>
>>  ****
>>
>> In any large network, multicast isn’t the right way to scale. If every
>> network element has to do a IGMP join to receive ADVERTISE then it becomes
>> a scaling issue. It also becomes a security problem where some rogue
>> element can start sending multicast ADVERTISE and hijack the orchestration
>> sessions. ****
>>
>>  ****
>>
>> Broadcast doesn’t scale either, but we can convert it to a directed
>> unicast (like DHCP for example). There are other reasons as well, such as
>> if there are multiple service specific controllers then you have to choose
>> different multicast groups for each. It seems like we should use broadcast
>> for this to be light-weight, and multicast could be an option in case of L3
>> networks, but with additional access controls.****
>>
>>  ****
>>
>> Regarding which existing protocol to extend, there are multiple options:*
>> ***
>>
>>  ****
>>
>> HTTP – has CRUD, but doesn’t support ADVERTISE, DISCOVER, REGISTER,
>> NOTIFY, SUBSCRIBE, COMMIT, CANCEL etc. So, just adding a NOTIFY, as you
>> suggest through SSDP, is still not going to be enough. Orchestration also
>> needs “identities” such as [e-mail], which HTTP doesn’t have.*
>> ***
>>
>>  ****
>>
>> XMPP – has PUBLISH, SUBSCRIBE, and identities, but not all of the above.
>> Service requests will require a “VIA” and dynamic injection of path
>> elements, especially when a request forks into multiple requests or is
>> re-directed to a different provider / location.****
>>
>>                                                                   ****
>>
>> AMQP – this is designed for messaging and again lacks many of the
>> constructs.****
>>
>>
>>    ****
>>
>> So wherever we look, the extension curve is long. It seemed like the
>> problem space is big enough to warrant a new protocol.****
>>
>>                ****
>>
>> The other issue is how easily we can implement the security for
>> orchestration. E.g. if we use IPSec end-to-end then how do hops in the
>> middle route the request differently (the nearest location, the cheapest
>> location, location with capacity, the location allowed by law, etc.). The
>> right model seems to be that we embed integrity within the protocol rather
>> than into IPSec. Privacy can be implemented separately between the edges
>> using IPSec. That security model requires another set of issues to be
>> solved in the current protocols (if we extend them).****
>>
>>  ****
>>
>> Besides security, there are other types of issues. For instance, a
>> network might use UDP internally to get broadcast but use TCP for unicast
>> externally for higher reliability. That change between TCP and UDP causes
>> loss of transaction identity, and transactions have to be built part of the
>> protocol. Likewise, with overlays, and overlay translations, the location
>> information could be easily lost. Hence, you need location in the
>> orchestration protocol. NAT may obfuscate real topology, and we lose
>> information about the actual distance between two end-points. ****
>>
>>  ****
>>
>> Given these challenges, we choose to define a protocol that can be
>> tweaked over time for orchestration specific needs without having to worry
>> about backward compatibility, and/or how this gets broken by overlay,
>> firewalls, or NAT’d networks. SIP already went over this hump and providers
>> have learnt (somewhat painfully) on how to do this in a way that works.
>> That entire learning can be leveraged for cloud.****
>>
>>                                                                   ****
>>
>> In any case, not sure if you have seen it, but there is a draft for SOP
>> that describes just what I’m talking about.****
>>
>>  ****
>>
>> http://tools.ietf.org/html/draft-dalela-sop-00****
>>
>> http://tools.ietf.org/html/draft-dalela-sop-flows-00****
>>
>>  ****
>>
>> Look forward to your comments.****
>>
>>                         ****
>>
>> Thanks, Ashish****
>>
>>  ****
>>
>> *From:* Vishwas Manral [mailto:[e-mail]]
>> *Sent:* Tuesday, March 06, 2012 2:30 AM****
>>
>>
>> *To:* Ashish Dalela (adalela)
>> *Cc:* [e-mail]; Michael Hammer
>> *Subject:* Re: [sop] SOP Requirements****
>>
>>  ****
>>
>> Hi Ashish,
>>
>> Thanks for the mail.
>>
>> So I looked at some of the reasons you mentioned you want to go in for
>> SOP instead of HTTP.
>>
>> I however have worked in the past with the DLNA stack, where we extended
>> HTTP and used protocols like SOAP to get behaviors you mention - like
>> service discovery, transaction support etc.
>>
>> If that is what we want, we should look at the DLNA stack and see how we
>> can leverage existing mechanisms for the same.
>>
>> I however think finalizing the requirements is a good start.
>>
>> Thanks,
>> Vishwas****
>>
>> On Fri, Mar 2, 2012 at 7:26 PM, Ashish Dalela (adalela) <
>> [e-mail]> wrote:****
>>
>> Hi Vishwas,****
>>
>>  ****
>>
>> It is better if you comment on the drafts because there is a section
>> dedicated to this very topic.****
>>
>>  ****
>>
>> http://tools.ietf.org/html/draft-dalela-orchestration-00#section-8 ****
>>
>>  ****
>>
>> This describes what you can’t do with web-services (assuming that’s what
>> you mean by APIs). This is **not** the shortcoming of the APIs, but that
>> of the underlying **protocol** (HTTP). So, if you changed the underlying
>> protocol to fix issues with HTTP, then the APIs would be more powerful.
>> That protocol we propose to be SOP.****
>>
>>  ****
>>
>> You are mistaking me in pitching API against protocols. I’m pitching
>> protocol (HTTP) against protocol (SOP). Unfortunately, application
>> developers abuse the term API to mean HTTP web-services, and the discussion
>> is then messed up into thinking protocol against API.****
>>
>>  ****
>>
>> Thanks, Ashish****
>>
>>  ****
>>
>> *From:* [e-mail] [mailto:[e-mail]] *On Behalf Of
>> *Vishwas Manral
>> *Sent:* Saturday, March 03, 2012 12:19 AM
>> *To:* Ashish Dalela (adalela)
>> *Cc:* [e-mail]; Michael Hammer****
>>
>>
>> *Subject:* Re: [sop] SOP Requirements****
>>
>>  ****
>>
>> Hi Ashish,
>>
>> My point was very simple.
>>
>> You had talked about cases where protocol is more flexible than an API,
>> and I was trying to help you understand that anything that can be done in
>> an East West manner (with protocols), can be done in North-South manner
>> with API's. If you say we can do something with protocols with only X
>> packets, we can do the same with just X API's too. That was my point and
>> not the fact that we have only 1 API or more.
>>
>> Also API's on which base services sit and ones which end users use could
>> be different.
>>
>> Am I missing the point altogether?
>>
>> Thanks,
>> Vishwas****
>>
>> On Wed, Feb 29, 2012 at 6:51 PM, Ashish Dalela (adalela) <
>> [e-mail]> wrote:****
>>
>> Hi Vishwas,****
>>
>>  ****
>>
>> What everyone calls API today uses a protocol – HTTP. APIs survive on the
>> interoperability provided by that protocol, and I don’t think anyone can
>> get away from that. The real question is – what is the right protocol on
>> top of which to build APIs? That’s the question SOP is raising. Once you do
>> that, then we can talk of one or many APIs.****
>>
>>  ****
>>
>> Limitations of using HTTP as the underlying protocol for any API have
>> been described in detail in the requirements draft. I would like to hear
>> your comments on that. That section describes what APIs can’t do. Some of
>> the limitations are because API is always unicast, and there are many
>> things for which you need a manycast and broadcast. Other limitations
>> because APIs are synchronous and you need to be asynchronous in some cases.
>> Yet other issues because APIs are single complete transaction, but some
>> transactions will spread over multiple such APIs. ****
>>
>>  ****
>>
>> The total amount of information in a message is unchanged whether you put
>> it inside a protocol header or the content body. Putting some things in the
>> protocol header saves you having to reinvent them in the content for every
>> type of service. In other words, a protocol saves you from increasing
>> information across various services. As an example in SIP, we put From and
>> To in the protocol header. Could we not put it in the body? Sure we could.
>> In that case we would be repeating that for voice and video and chat
>> content. ****
>>
>>  ****
>>
>> To your point, you can have a single API for doing anything. As the
>> service evolves and complexity grows, the number of parameters to that API
>> increases. And yes, you can make that backward compatible in terms of
>> implementation. But, nobody does that – especially when the interface is
>> end-user facing. If this was an acceptable design, then we would not have
>> object inheritance and there won’t be hundreds of APIs being opened up by
>> cloud providers today. You might want to suggest one API to Amazon or other
>> cloud providers. ****
>>
>>  ****
>>
>> A practical operational issue with APIs is that users don’t understand
>> all the details. A user understands a server memory and CPU, but don’t
>> understand VLAN and LUN, and zillions of other complicated things. Exposing
>> them through APIs is useless because they can’t use it. Why would a user
>> buy an expensive TV when they can’t use most of the features, because the
>> remote is too complicated? The need is to reduce complexity through
>> automation, not expose it all to the user via APIs. In other words, you
>> need a more sophisticated policy engine not a sophisticated API system. *
>> ***
>>
>>  ****
>>
>> For any problem there is a cure and there is a prevention. Building API
>> bridges is a cure to diverse APIs, it’s not a prevention. Once you
>> recognize a problem, you build a short-term cure and a long-term prevention
>> (at least ideally). Then, a single API is neither a cure nor prevention;
>> its side-effects are so severe that we might be living with the original
>> problem as well. ****
>>
>>  ****
>>
>> APIs have always existed and will continue to exist. The goal is to
>> interoperate diverse APIs without a translation bridge. That happens all
>> the time with network protocols, when one vendor’s APIs works with another
>> vendor’s APIs without a translation bridge. I think not having a bridge is
>> always better than having a bridge. Agree?****
>>
>>  ****
>>
>> Thanks, Ashish****
>>
>>  ****
>>
>>  ****
>>
>> *From:* Vishwas Manral [mailto:[e-mail]]
>> *Sent:* Wednesday, February 29, 2012 10:45 PM
>> *To:* Ashish Dalela (adalela)
>> *Cc:* Michael Hammer; [e-mail]****
>>
>>
>> *Subject:* Re: [sop] SOP Requirements****
>>
>>  ****
>>
>> Hi Ashish,****
>>
>>  ****
>>
>> As the number of services increase or the complexity in a given service
>> grows, this becomes very hard. Assume there is a service with N tunable
>> parameters. You need at least N APIs that modify these parameters
>> individually. Then permutations and combination of these parameters create
>> hundreds of more APIs. That’s just API bloat. And if you have to
>> interoperate multiple instances of these APIs through bridges, it’s just
>> inviting more complexity. Another limitation is that when APIs have
>> semantic incompatibilities, it becomes even harder to interoperate (syntax
>> incompatibility is easier).****
>>
>> Not true at all.For N parameters you can have one API. The API can be
>> made forward compatible by using simple things like unions. Having worked
>> in a software company, where we have to extend our software without
>> breaking previous API's I am well aware of the fact.
>>
>> If you give an exact example I can try to help you see how we can make an
>> extensible API for the same.
>>
>> Thanks,
>> Vishwas ****
>>
>>  ****
>>
>> From an operational standpoint, every new API introduction requires
>> software upgrades to the controllers. That eventually hinders the rate of
>> service creation.****
>>
>>  ****
>>
>> >> I know as services proliferate there could be a proliferation of
>> distict API's but the same is true of the protocol layer too.
>>  ****
>>
>> That won’t happen if we separate service-independent and
>> service-dependent pieces. An example of that is SNMP. SNMP is
>> device/service independent. MIB defines the specific service/device. If you
>> have a standard protocol to manage a device, then you just have to add a
>> new MIB to start managing it. You don’t need to upgrade all the
>> intermediate systems – hardware or software. ****
>>
>>  ****
>>
>> BTW, I’m not advocating SNMP here because SNMP has many shortcomings in
>> terms of network discovery, capability discovery, advertisements,
>> transactions, etc. But, we need to keep in mind that API proliferation is
>> inevitable as services proliferate. Protocol proliferation is not
>> inevitable. Similar separation has been done in the past in SIP/SDP,
>> HTTP/HTML, SMTP/MIME. That separation allows anyone to send any content in
>> email to anyone. Or download any web-page, or have any type of codec (voice
>> or video) use the same protocol.****
>>
>>  ****
>>
>> If you compare the success and widespread use of above mentioned
>> protocols the value of separation between service-independent and
>> service-dependent seems pretty convincing.****
>>
>>  ****
>>
>> >> Correct but the draft seems to differ.****
>>
>>  ****
>>
>> Service and instance of service are (and can be) interchangeably used. Is
>> bandwidth a service or an instance of a service? I think this is more
>> semantics.****
>>
>>  ****
>>
>> >> The requirement seems contradictory to what we agree. Similar for
>> points below.****
>>
>>  ****
>>
>> The requirement is really that services are portable across providers. I
>> think it is fair to say (as you also agree) that a user must know where
>> they are going for a service. After all, they will have to pay for the
>> service and they ought to know in advance who are they going to receive a
>> monthly check from. ****
>>
>>  ****
>>
>> Thanks, Ashish****
>>
>>  ****
>>
>>  ****
>>
>> *From:* [e-mail] [mailto:[e-mail]] *On Behalf Of
>> *Vishwas Manral
>> *Sent:* Tuesday, February 28, 2012 1:26 AM
>> *To:* Michael Hammer
>> *Cc:* [e-mail]
>> *Subject:* Re: [sop] SOP Requirements****
>>
>>  ****
>>
>> Hi Michael,
>>
>> Sounds like we agree on most of the things, though I see the draft
>> contradicting what we agree on.****
>>
>> 1. Do we really see incompatibilities in the API's soar for say IaaS? The
>> AWS API's seem to be the default standard adopted by most providers. From
>> the little I know OpenStack based API's may be the alternative way and
>> companies have built bridging layers to inter-operate between the same.**
>> **
>>
>>  ****
>>
>>  ****
>>
>> Seem?  May be?  Bridging layers?  I think you are making the case for us.
>> :)****
>>
>> I'm sure Ashish will have more to say about APIs, but I would prefer
>> there be a de jure than a default, which in the long run is likely to
>> change at the whim of a single company, and perhaps not in a direction that
>> everyone would like.****
>>
>> What I am saying is we have 2 sets of API's and there are layers used to
>> bridge the same. I know as services proliferate there could be a
>> proliferation of distict API's but the same is true of the protocol layer
>> too.
>>  ****
>>
>>  ****
>>
>> 2. Instead of the term customer/ user can we instead use the term
>> "consumer". Something like "cloud subscriber" etc could be used. All I am
>> saying is can we use standard terms here.****
>>
>>  ****
>>
>> We can settle on specific terms to use, just so long as we keep the
>> distinction between the entity (enterprise?) that provisions the software
>> in the cloud, and the user of that software, which could be an employee or
>> a user in the general public.  Using a SIP Proxy as a Service, the operator
>> of the Proxy provisions it with a CREATE, but the user is the one sending
>> INVITEs through it.  Make sense?****
>>
>> The NIST document uses terms and we should try to use similar terms as
>> far as possible.
>>  ****
>>
>>  ****
>>
>> 3. Is orchestration about creating services (from the cloud providers
>> perspective), or an instance of a service (for a particular user)? I think
>> it is the latter, but doesn't sound so from the definition.****
>>
>>  ****
>>
>> Orchestration is about the on-demand provisioning of the
>> compute/storage/network/XaaS in the cloud by the subscriber/customer.  Once
>> provisioned, the service can provide services to the intended user.  We are
>> trying to be general here.  Need to keep provisioning and operations
>> distinct.  "Service" is occurring in levels.****
>>
>> Correct but the draft seems to differ.
>>  ****
>>
>>  ****
>>
>> 4. How is Service Domain Name different from a URI? Aren't they the same?
>> ****
>>
>>  ****
>>
>> There is a distinction here between a class of services and running
>> instantiations of those services.  Either may be hierarchically named.***
>> *
>>
>> Hmm.
>>  ****
>>
>>  ****
>>
>> 5. Is Scenario -1 talking about all providers should provide the same
>> services? I guess not. I think the idea should be the same set of services
>> should be accessible from a cloud provider the same way. It however does
>> not mean that all providers need to provide the same services, as it seems
>> from the requirement.****
>>
>>  ****
>>
>> Agree.  All providers may not provide the same set of services.  ****
>>
>> But, if two providers offer the same service, it should not require a new
>> customer protocol stack to do so.****
>>
>> And users should not know that they may be going to one provider or the
>> other when using the same service.****
>>
>> The requirement seems contradictory to what we agree. Similar for points
>> below.
>>
>> Thanks,
>> Vishwas
>>  ****
>>
>>  ****
>>
>> 6. It seems for most purposes you are talking about users, but as such a
>> user in an enterprise should be unaware of where the service is coming
>> from. It is the role of the customer to actually provide clear demarcation
>> so a user is unaware of the same. Interoperability with virtual provider is
>> how companies achieve the same.****
>>
>>  ****
>>
>> Agree, and we would like that to be true for multi-provider cases as well.
>> ****
>>
>> I would go further to say that even a user not in the enterprise should
>> be unaware where the service is coming from.****
>>
>>  ****
>>
>> 7. I don't think you should mention providers should inter-operate with
>> each other. That is a business decision. I think what you mean here is that
>> providers should have a clear interoperable means should they wish to
>> inter-operate.****
>>
>>  ****
>>
>> Yes.  We want them to be able to inter-operate.  Whether they want to is
>> a business decision.****
>>
>>  ****
>>
>> 8. Is it really a requirement for the Orchestration to allow
>> inter-operation for all models? I would have thought we are focusing on the
>> IaaS alone.****
>>
>>  ****
>>
>> We don't see a reason to limit it to just IaaS.  We are looking several
>> years down the road here.****
>>
>>  ****
>>
>> 9. S-5 and S-3 sound like similar services to me. How are they different
>> - vendor versus provider?****
>>
>>  ****
>>
>> We were considering cases where multiple companies are involved in
>> providing all the capabilities needed.  One involved coordination within an
>> administrative domain, while the other involves independent administrative
>> domains.  We didn't want to limit this to single company operations.  Large
>> global providers may involve many companies.****
>>
>>  ****
>>
>> 10. I think one of the key requirements for SOP, is the ability to work
>> across only a sub-set of the base services and allow for extensible
>> services on top. There could be so many variants of the SaaS or even PaaS I
>> am not sure how you would make every service inter-operate.****
>>
>>  ****
>>
>> There needs to be several layers of standards involved.  This is an onion
>> not a single layer orange-peel.****
>>
>> Here we are trying to provide structure that allows easy extension,
>> substitution, and innovation at the more service-specific granular levels.
>> ****
>>
>>  ****
>>
>> 11. I think when a VM is moved the biggest issue is the ability to move
>> the storage along with it. All other state is minor and minimal.****
>>
>>  ****
>>
>> I would say the networking is the biggest issue, but that is my bias.  :0
>> ****
>>
>>  ****
>>
>> 12. Section 6 seems to be relevent within a cloud too and not just
>> between clouds.****
>>
>>  ****
>>
>> Agree.  Internal to a cloud and from the customer to the cloud are the
>> simple cases.  ****
>>
>> We emphasize the inter-cloud cases to test the architecture for the worst
>> cases.****
>>
>>  ****
>>
>> 13. Doesn't CDN provide the ability to separate address and ability
>> already?****
>>
>>  ****
>>
>> Probably needs more discussion.  I see content as a specific scenario.
>>  There you don't care which copy of data is accessed so long as you reach
>> it.  In other types of services, a lot more control over who accesses what
>> is needed.****
>>
>>  ****
>>
>> 14. For Service discovery. management we wrote something quite a while
>> back
>> https://datatracker.ietf.org/doc/draft-yokota-opsawg-virtnw-service-management/
>> .****
>>
>> Will take a look.  Thanks.  Mike****
>>
>>  ****
>>
>> Thanks,
>> Vishwas
>>
>> _______________________________________________
>> sop mailing list
>> [e-mail]
>> https://www.ietf.org/mailman/listinfo/sop****
>>
>>  ****
>>
>>  ****
>>
>>  ****
>>
>>  ****
>>
>>  ****
>>
>> ** **
>>
>
>