As I and others on this blog have mentioned on several occasions, the SIMPLE (or the more formal and rather awkward: “SIP for Instant Messaging and Presence Leveraging Extensions”) working group of the IETF has been responsible for defining how to do Presence and Instant Messaging applications using SIP and related protocols. The SIMPLE working group has existed for some time; in fact, it’s one of the oldest ongoing working groups in the Real-time Applications and Infrastructure (RAI) area of the IETF. I am currently a co-chair for SIMPLE.

I write to tell you that SIMPLE’s work is almost done. We are finally seeing the light at the end of this long tunnel. Of the four remaining work items, one is in the AUTH48 state. (This means that the RFC editor has presented a candidate for the final RFC version back to the authors for any last minute edits and approval.) One entered Working Group Last Call (WGLC) last week. There are only two work items that may still see controversy, and one of those is in IESG review.

These drafts are, respectively, draft-ietf-simple-msrp-acm, draft-ietf-simple-simple, draft-ietf-simple-msrp-sessmatch, and draft-ietf-simple-chat.

The first draft extends the MSRP protocol to allow the endpoints to negotiate which one will open a TCP connection to its peer. I blogged about this draft some time ago. We should see publication of the resulting RFC any day now. In fact, it’s already been assigned a number: RFC 6135. [Update: RFC 6135 was officially published on February 28.]

The second, draft-ietf-simple-simple (aka “SIMPLE made Simple”), is an informational draft that acts as a road-map and secret-decoder-ring for the various specifications produced by the SIMPLE working group. (Keep in mind, that there is no one protocol known as SIMPLE. But we still tend to use the term SIMPLE informally to refer to the resulting suite of protocols and architecture.) The fact that this draft is in WGLC means the author believes that this draft is essentially ready to be sent to the IESG for final review and publication. It’s possible that the last call review could uncover some controversial point that would require more work. But given the nature of this draft, I expect that any WGLC feedback is more likely be clarification and editorial comments.

We do know in advance, however, that draft-ietf-simple-simple may require minor editing to reflect the final disposition of the last two drafts below. This means that, regardless of its current completion state, draft-ietf-simple-simple will be the last draft to be published by SIMPLE.

Draft-ietf-simple-msrp-sessmatch describes an extension to MSRP to make it more friendly to Session Border Controllers (SBCs). The way that MSRP devices match TCP connections to message sessions means that, if an MSRP session traverses an SBC, that SBC has to re-write the To-Path and From-Path header fields in a manner similar to an MSRP Relay. Some working group participants expressed concern that this requirement could impact SBC performance. The sessmatch draft would allow supporting endpoints to work across SBCs that do not change MSRP messages en route. However, there are still ongoing discussions concerning the impact on security and interoperability.

Assuming that the sessmatch draft has not become a moot point by then, I plan to go into considerably more detail on it and the surrounding controversy in my next blog entry.

Then, finally, there’s draft-ietf-simple-chat. This draft defines how to create MSRP “chatrooms” with conference servers. There’s still some controversy over how this draft interacts with some similar work from the XCON working group.

Hopefully, we will resolve the issues around these last two drafts soon–at which time I hope to be able to entitle a blog entry as “SIMPLE Finally Done!”

My last several posts have discussed various standard optimizations to the SIP Events mechanism. Another such optimization is the “Partial Notification” mechanism described in RFC 5263.

Unlike some of the previous optimizations that I have written about, partial notification is not designed for SIP Events in general. Instead, partial notification only works with the Presence event package. The reason for that should become apparent shortly.

In a SIP Presence system that doesn’t support partial notification, each time a presentity changes state, the presence server must send the full presence document to every watcher for that presentity. Given that presence documents can get large and complex, and that a presentity could have a large number of watchers, this can consume a fair amount of bandwidth. This could be particularly troublesome in mobile data networks, where bandwidth is relatively scarce–and potentially expensive to the end user.

With partial notification, rather than sending the entire presence document to each watcher, the presence agent only sends the parts of the presence document that actually changed. RFC 5262 describes a data format for expressing a “patch” or “diff” between two different versions of a presence document. This patch mechanism is specific to PIDF, the data format used to express presence information. (This is the reason that partial notify is specific to the presence event package.)

A user agent advertises support of partial notify by putting the diff format (“application/pidf-diff+xml”) in the Accept header field of a SUBSCRIBE request. Assuming the presence server also supports the partial notify, it can initially invoke the extension by using the pidf-diff format to send the entire presence document, along with a version number. When changes occur to the presence state, the presence server sends just those changes in a new pidf-diff document, along with an incremented version number. The pidf-diff format allows the presence server to include elements to be added to or removed from the presence state, as well as elements to replace existing ones.

When the user agent receives a partial notification, it compares the version number in the notification to the last version number it received. If the new version number is less than or equal to the old number, the user agent assumes an error, and ignores the change. If the new version skips a number since the last version, the user agent assumes it missed an update, and requests a full-state refresh. If the new version is exactly one more than old version, then the user agent applies the changes to its cached version of the presence document.

Like with any optimization, the designers of the partial notification extension made some assumptions. One is that changes to a presence document are usually smaller than the document itself. This a pretty safe assumption in presence systems that generate any but the most trivial presence documents.

They also assumed that the cost of the saved bandwidth between the presence server and the watchers’ user agents would be large compared to the cost of the added complexity at the presence server. This may not always be true, given that determining the difference between two large presence documents is anything but trivial. And if the presence server supports presence rules that can result in sending different presence state to one watcher than to another, for the same presentity, then the effort required to calculate the difference between documents may be multiplied by the number of watchers. This assumption is more likely to be true in mobile data networks (and in all fairness, partial notification was designed for mobile networks), where the cost of bandwidth is usually high compared to wireline services–although this might change with the deployment of 4G mobile data networks. 

In my role as co-chair of the IETF SIMPLE working group, I’ve heard a lot of questions about the scalability of SIMPLE based presence services. This is the first in a series of posts on various subjects related to SIMPLE scalability.

One of the hard problems in scaling presence services in general is created by the very idea of contact lists. When the presence state of a given resource changes, a presence service must notify every subscriber to that resource. You can think about this in terms of relationships, and simplistically estimate the total number of relationships in the system of as the number of users times the average contact list size. As both numbers grow, you get a combinatorial explosion.

Of course, in the real world, you get a lot of variation in the number of subscribers between resources. Scalability is further impacted by “celebrity” resources that have huge numbers of subscribers.

In “base” SIMPLE, a client forms a separate SIP-events subscription for the presence state of each resource in the user’s contact list. So if your contact list has 50 entries, that’s 50 separate SIP subscription dialogs to maintain, refresh soft-state for, etc. That’s not really a problem for most modern client devices, but it can put quite a load on presence servers that need to handle that many subscriptions for each user. And if a lot of those clients share a limited bandwidth access network (such as with a mobile data network), they can put a substantial load on that network.

Resource List Servers (RLS) can help in certain circumstances. [Full Disclosure: I was a co-author for that specification] With an RLS, a SIMPLE client creates a single subscription to an entire contact list, rather than a separate subscription to each contact on the list. The RLS will then send presence notifications that contain presence state for multiple contacts at a time, along with metadata describing the current entries in the contact list. That is, an RLS subscription provides presence state for list entries, and also provides the state of the list itself.

The RLS extension seems like an obvious optimization for SIMPLE. But in reality, it only helps for some very specific situations, or in combinations with some other optimizations.

First, the RLS still has to get presence state for each resource on a list from somewhere. RFC 4662 describes the idea of a “Back-End” subscription, where the RLS subscribes to a resource on behalf of the end-user. This can be mitigated by co-locating the RLS with a presence server (PS). But practically, a contact list is likely to contain resources at many different PSs, in many different domains. So the use of RLSs doesn’t have much of an impact on the total number of subscriptions that the PSs have to manage.

At this point, many readers will think “But doesn’t a PS only need to handle one subscription to a particular resource from each RLS, instead of one from each client?” Unfortunately, this really doesn’t work. Imagine Alice and Bob both subscribe to Carol’s presence. Can we assume Carol wants to allow both Alice and Bob to subscribe? And even if she does, can we assume she wants to give the exact same information to both of them? Unless the RLS has knowledge of Carol’s presence rules, it must maintain separate back-end subscriptions for Alice and for Bob. In practice, it’s very difficult to create “shared” back-end subscriptions without creating privacy problems.

So, without some mechanism (standardized or proprietary) for an RLS to learn the presence rules for each resource it subscribes to, and a way for each PS to know it can trust the RLS to properly enforce the rules, the RLS extension really not help much with presence server subscription load. Nor does it reduce the required bandwidth between the RLS and PS.

On the other hand, it can reduce the bandwidth required over the client access network. This is extremely useful if the access network is bandwidth constrained–for example in the aforementioned mobile data network. It is less useful if access network bandwidth is plentiful, such as with broadband networks, and possibly even 4G mobile data networks (although in those cases it may still help with things like client battery life.) In fact, in a broadband network, an RLS may create new scalability problems, by creating a new bottleneck in what was previously a highly distributed subscription model.

Even with scarce bandwidth, the utility of an RLS goes down if the network traffic created by bona-fide presence changes is very large compared to the traffic created by the overhead of subscription maintenance. This can be mitigated somewhat by applying some rate-limiting optimizations. I will discuss SIMPLE presence rate-limiting in a future post.

So in summary, the RLS extension is an effective scale optimization for some very specific real-world circumstances, such as in mobile presence services. But it’s not for everyone, and it takes some additional optimizations to get the most out of it. 

The SIP Event Framework, defined in RFC 3265 , provides a way for SIP elements to learn when “something interesting” has happened somewhere else in the network.

A SIP element that wants to hear about these things SUBSCRIBEs to an event. That SUBSCRIBE will be routed to an appropriate SIP Events Server which will then send NOTIFY messages back to the subscriber whenever the “something interesting” being subscribed to happens.

This subscription model is an alternative to having elements periodically poll: asking repeatedly “did my interesting thing change yet?” For many applications, the subscription model has better traffic scaling characteristics than a polling model.

The SIP Events Framework requires that the meaning of “something interesting” be standardized in advance. Event Packages define what can be subscribed to, and what information will appear in the notifications. One of the first applications that drove the creation of the framework was Presence and there is a “presence” event package that allows watchers to subscribe to presence entities and receive new presence documents any time that presence changes.

There are around a dozen other event packages defined, including:

• the “refer” package used primarily to learn whether a transfer attempt has succeeded or not
• the “message-summary” package which allows someone to learn they have new messages at a service like a voice-mail server
• the “dialog” package which lets applications learn when endpoints initiate, accept, and terminate sessions. This package is used to realize features like automated-callback and directed call pickup.
• the “kpml” or Key Pad Markup Language package which lets an application learn when keys on an endpoint are pressed and released

The current set of standardized packages can be found at the sip-events namespace registry page at IANA.

Subscriptions are soft-state. They have to be periodically refreshed or they expire. Either the subscriber or the notifier can end the subscription at any time. The framework requires that the subscriber be authorized before actual event information will be sent (allowing you to control who sees your presence for example).

The framework allows the notifier to be any element in the network. The “presence” package, for example, supports both the model of a concentrated, centralized Presence Server that services all the subscriptions for a service, and a more peer-to-peer model where each endpoint accepts subscriptions and sends notifications on its own.

An initial SUBSCRIBE request might be forked to several event servers, and the framework allows all of them to accept the subscription. Individual packages are required to define how to combine the information received from multiple notifiers if the package allows this to happen.

The framework has proven to be a powerful tool for realizing new applications and providing new services. In future FAQs, we’ll look more deeply into some of these packages, the way the framework works, and how the challenges authentication presents are managed.

SIP is robust in the Internet environment.

SIP still works in predictable ways when packets are lost, when routes fail, and when endpoints fall off the network.

SIP achieves some of this robustness through a design principle to fail to a known-safe condition. The idea is that any change to the system has to be maintained through periodic refreshment. If the authority for a change isn’t available to maintain the change, then it’s not around to correct the change when new conditions cause it to be wrong.

For example, a presence user could post a presence document that says “I’m driving”. Somewhere along that drive, the user goes into a parking garage, losing connection to the network. The user leaves his car and makes it into a movie theater without reconnecting. In a less-robust system, that user’s presence will be wrong for the next hour or so. A SIP based presence system will notice, after some time, that the user is no longer maintaining the presence document and will fall back to a known safe default, probably “I don’t know anything about this user’s presence right now.”

This SIP-based presence user’s published presence is “soft-state”. It is information (state) the user put into the system that will go away if the user doesn’t maintain it. Stated another way, the information will expire unless it is refreshed.

By contrast, the position of a typical simple light-switch is “hard-state”. If you flip it up, it will stay up, possibly forever. It will only change back to down when you (or some other user) explicitly comes back to manipulate it.

The soft-state concept is reused in many parts of the SIP protocol. Registrations expire. Subscriptions expire. Presence documents (or any other published SIP Event documents) expire. The time it takes to expire is negotiated between the element providing the state, usually an endpoint, and the element keeping the state, such as a registrar or presence server. That negotiation goes like this:

Handset: “Would you please send requests for me to this Contact for the next hour?”

Registrar: “Yes, I’ll hold onto this Contact for you, but I’ll only keep it for 5 minutes.”

When making that request, the handset’s user is also saying “If something unexpected happens, I’m willing for the state I’ve handed you to be wrong for up to an hour”. The server is replying “I’m only willing to let things be wrong for 5 minutes”.

This points out why the Registrar above cannot say “Yes, I’ll hold onto this Contact for you, and to save you some trouble, I’ll keep it active for a month”. The handset user is only willing for things to be wrong for an hour. Half a month from now, that handset might not even exist.

Once soft-state is accepted, it must be refreshed or it will be deleted. That means that before the negotiated time passes, the element placing the state into the system must return and say “please hold onto this state a little longer”. In SIP, this refresh also re-negotiates the next expiration time.

In the figure below, Phil subscribes to Robert, asking for an hour. Robert grants the subscription, but limits the expiration of the subscription (the soft-state) to 5 minutes. Four minutes into the subscription, Phil refreshes it, asking for another hour, but again getting another 5 minutes. This time Phil doesn’t refresh the subscription, letting it expire. Note that the total subscription time was 9 minutes, not 10. Phil’s second 5 minutes started at the point of negotiation, not when the first 5 ran out.