Those crazy Rock Hounds

I got a great response from Deborah Volk at Identigral regarding my little thought exercise on dynamic opt-in/opt-out mailing lists. In this case, a mailing list for Geology majors, as well as anyone else interested in the occasional spelunking field trip.

Her approach is perfectly valid (and truthfully, makes more sense than what I was trying to do...). Basically, her suggestion is to use the various provisioning mechanisms of an identity management package, like OIM, to maintain the membership of a particular mailing list or group. People could be automatically provisioned into a group at time of account creation, or be event-based, such as someone switching majors. By providing other workflows, such as opt-in or opt-out, users could also add or remove themselves from that static group ad hoc.

What I was trying to accomplish was to put the logic of list membership into the list definition itself. Meaning, if I wanted to send out this week's Geologic Times newsletter, the group membership would be dynamically determined as soon as I hit the "Send" button. Anyone, at that point in time, who was either a Geology major, or had opted in to the list, would then be sent the email.

In summary...

The IdM-centric approach:
IdM workflows provision users into a static group for mailing list membership. The triggers for adding users into this group could be event-driven, such as at time of account creation, or manual, such as an end-user opting in or out of the list. The 'dynamic' part of the list is handled by the IdM software.

The mailing list logic approach:
Using advanced LDAP filters, create a mailing list that would dynamically determine membership at the point in time an email was sent to the list. This would most likely be driven off of attributes or roles assigned to the user objects in a directory store, such as Sun Directory Server or Active Directory. There is no 'group' per se -- it is the LDAP query filter that determines list membership.

Like I said earlier, Deborah's approach makes much more sense, if you have the IdM workflow engine already. However, true dynamic opt-in/opt-out lists are still possible without an IdM solution, but would be more difficult to create and maintain.

Another use case for good IdM workflows?

While I'm sure UC San Diego will learn all sorts of valuable lessons from this situation, what it should teach everyone else is the importance of establishing proper approval chains for workflows (such as sending out acceptance letters), and a strong business case for some sort of distribution list management tool...

http://www.nbcsandiego.com/news/local/Youre-Out-Youre-In-No-Youre-Out.html?yhp=1

ILM2 delayed until Q1 2010

This news certainly comes as a surprise, considering our Microsoft sales team apparently wasn't even aware of the delay. Architecting a solution around software that might ship sometime in the next 12 months doesn't seem like a wise decision to me though...

Jackson's Identity Management & Active Directory Reality Tour Travelblog: Microsoft's ILM"2" delay hurts

Directory Self-service applications

These are some of the vendors that provide products that enable end-users to update their own records in a directory server (most, if not all, of the products below are primarily AD focused though). This could include the ability to do password resets without calls to the help desk, but I was looking at their capability for managing opt-in/opt-out of distribution lists.

(Note: I haven't used any of these products, although several do offer live demos from their web sites)

Imamani:
Their Smart DL product allows end-users to create dynamic distribution lists based on AD attributes. I don't know whether they also provide an opt-in/opt-out capability though.

Namescape:
There are several different versions of their rDirectory product. However, according to their comparison chart, both the Professional and Enterprise editions offer a 'Group Self-Subscription' feature. The Enterprise edition looks to be a full-fledged IdM solution, with User Provisioning capabilites,

ManageEngine:
Their AD Self-Service Plus product looks to primarily be aimed at allowing end-users to perform password resets without calling in to a help desk, but according to the product page, one of the listed features is 'Update Personal AD Info'. I'm not sure if this could be extended to provide some sort of group/mailing list subscription capability though.

Securitay:
Like the Imanami product, the Group Management Portal allows end-users to create and manage their own distribution lists in AD/Exchange. Although from what I could tell from the live demo site, it seems to be only managed groups (meaning, members are manually added vs. dynamically updated based on user attributes)

And I would remiss if I didn't mention Microsoft's new ILM"2" product (which is only a release candidate now, but should available soon). This latest version of their identity management offering does have some pretty nice group management capabilities, along with the ability to delegate dynamic list creation to end-users. Of course, the primary focus of ILM"2" is user provisioning and the synchronization of identity data across many different target systems. But the rich end-user self-service interface is a nice differentiator compared to many other full-service IdM stacks out there today.

Combining dynamic and static groups

After my previous post, I've been trying to figure out some solutions to the scenario I presented (a dynamic email group with opt-in/opt-out capabilities). I got some good suggestions from Matt Flynn and will have a subsequent post with some commercial tools that provide end-user self-service for things like distribution list management in AD or LDAP.

My bigger concern, though, was trying to prove out my little quasi-code example in Sun Directory Server. While the software supports dynamic groups, defined with an LDAP search URL, it seems to be client-specific as to whether those dynamic URLs in the group definition are actually followed (meaning, the users matching the URL filter are returned to the client). I was unable to get a command-line ldapsearch query to successfully return all the members of a dynamic group (suggestions, anyone?). However, by using a combination of managed and filtered roles, instead of groups, it was quite easy to accomplish this.

First, you can define a managed (static) role and manually assign users in. This could be your "opt-in" group -- those users who would like to receive an email newsletter, for example, but would not normally be included automatically based on degree or major or what have you. With a self-service tool, users could add themselves into this managed role as a sort of 'subscribe' functionality.

Secondly, if needed, you could assign an attribute for the opt-out capability. Unfortunately, you can't use the 'nsRole' calculated attribute as part of the filter in the definition of another role. But there are certainly ways around this limitation (extend the schema to include an attribute called 'optout' with the value of the list name, or just use an existing attribute like 'memberof' with a value of your choice). You also can't use the 'isMemberOf' attribute (which is used to return all the static group memberships for a particular user), because, again, this is a calculated attribute.

For reference purposes, here is the filtered role definition I used for testing my example:
nsRoleFilter: (&(sn=Black)(!(memberof=optout_dllist)))

(All users with a last name of "Black", EXCEPT for those who have a memberof attribute set to "optout_dllist")

Once the filtered role is defined, you can then create a nested role with the managed (static) role you created for 'opt-ins', along with the filtered role with users matching a particular criteria (and optionally, excluding those who may have opted out) After the nested role has been defined, you can simply query a given user for the 'nsrole' attribute to see which roles they belong to, or search directly against the 'nsrole=nested_role_dn' to show all the members of the role. Note that you must explicitly ask for the nsrole attribute in an ldapsearch command -- just doing a normal search against a user's attributes will not show their role memberships.

And to save you some of the same headaches that I went through, since a role is defined as a subentry, it is not returned in a normal ldapsearch command. You must specifically ask for them using a search filter like :
"(&(objectclass=nsManagedRoleDefinition)(objectclass=ldapSubEntry))"

For additional information about Roles in SUN DSEE 6.3, the documentation is here.

Dynamic distribution lists with opt-out capabilities?

After posing this question to several vendors in our IdM evaluation meetings, I actually think that there are no current solutions out there that will allow you to create dynamic distribution lists with opt-in/opt-out capabilities. I'm not entirely sure why one might want to do this, but being at a university, I can do things simply for academics' sake, right?

Here's the scenario:

• A school wants to offer a newsletter to all of its Geology majors. Maybe it will provide information about guest lecturers or other events. This DL could be pretty easily set up in Exchange 2003 (or 2007) using a query-based distribution group (ie: an AD attribute matching "studentMajor=Geology"). As new students join the program, they will automatically receive the newsletter because their 'studentMajor' attribute will match the filter criteria, without any manual intervention from list owners or AD administrators.
  
• However, maybe there are students with other majors who have an interest in geology, and would like to receive the weekly email newsletter. Or conversely, a Geology major who wants to reduce his inbox clutter and wishes to opt out.

I realize that you could construct a complex filter for the mailing list, which would take into account these other two situations (an opt-in, and an opt-out). So now your filter string for the DL would look something like this (in quasi-code...):
(((studentMajor=Geology)(|optInDL=Geology))(&!optOutDL=Geology))

(roughly translated: your major is Geology, or you've opted in to the DL, AND you have not opted out of the list)

However, I haven't really found anything out there to do something like this. There are some products from Imanami called SmartDL and WebDir, though I'm not sure they're a fit either. SmartDL allows for the creation of query-based distribution groups (just like in Exchange...), and WebDir offers end-users some control over their AD attributes. I guess a combination of these two products would (somewhat inelegantly) accomplish the above scenario.

Any other ideas out there?

Long overdue update

Despite the lack of posts over the past few weeks, the project has been making good progress. Here are some of the things that we've been working on:

Wrap-up of initial round of interviews
After meeting with all the schools and departments around the University, Identropy took the information, analyzed it, and boiled it down into a nice presentation for all the participants. This initial analysis helped us to decide on an initial scope for the pilot phase of the project -- something manageable, but that will still provide tangible and meaningful benefits to the University. We chose to focus first on one school and its various user populations, including students, faculty and staff.

An in-depth business process analysis
In an effort to better understand the current business process for on-boarding students in the Warner School, we had several days of meetings with the various departments within the school. These meetings were very useful, had great participation (from the Registrar, Admissions department, Financial Aid office, and especially Dave Garcia, their IT director), and allowed us to really delve into the processes in place today, as well as the target systems involved. Doing this upfront work also brought to light some places in the processes that could be improved, both from an efficiency standpoint, as well as to prevent some problems with creating bad or duplicated identity data.

Vendor evaluations are underway
We've had several good vendor demos already, with a few more later on this week. Once these have been completed, we'll look at the feedback from the various technical folks from the University who have been in attendance, weigh the strengths and weaknesses of the different solutions, and select the one with the best fit for our environment and business processes.

After having made it this far, there are a few early lessons that we've learned that might help another institution starting a similar project:

• For a University especially (or any highly decentralized organization), it is good to keep the different parts of the organization involved and engaged as the project progresses. Even though the initial phase will focus on one school, the lessons learned there will be applied to other schools and departments, and it's good to keep the other organizations apprised of the project's status.

• Limit your vendor analysis to a handful of ones you want to really focus on. But don't necessarily include only those vendors that may already be existing partners of your institution. It's good to include current vendors, since the integration time (and ultimately, cost) might be reduced, but by bringing in a few new faces, you might be surprised at the number of different approaches to the same problems.

• When evaluating vendors, we have tried to keep a level playing field by providing each vendor with the same set of use cases, and then evaluating them with the same weighting system. Identropy really helped to facilitate this process for the University of Rochester, by creating that initial set of use cases based upon our business process analysis. As your project begins to get ready to evaluate product offerings, use your BPM analysis results to create simplified, but representative, use cases that the vendors can demonstrate for you.

A balancing act

We had some great sessions last week! Throughout the 10+ meetings, we had excellent turnout and participation from the many different schools and departments around the University. A ton of good process information was collected, and now Identropy has the (unenviable) task of analyzing it all and presenting a distilled version back to the team.

One common theme that sort of emerged for me after mentally digesting last week is the sense that any identity management project is really a series of balancing acts. On many different fronts, decisions need to be made that will impact large numbers of users, while still accomplishing the overriding goal of the project. For example,

Business Processes:
Current processes are most likely deficient in some way (inefficient, or insecure) but introducing new processes or procedures requires the buy-in of many different departments. It's important to balance their needs when implementing changes -- not to cause too much disruption to their daily routines and deliverables, while still making meaningful improvements to the processes as required by the project.

Technology:
Specifically, keeping current technology versus introducing new products. There may be times when existing products may be sufficient to achieve the goals laid out by the IdM project, but more often than not, you're going to have gaps in the stack of software installed today. The bigger question is whether to replace current systems with different vendors. It's a tough call -- on the one hand, you have already invested time and energy into learning your current product, and have some institutional knowledge around it. On the other hand, it may not be the right solution for the ultimate end-state, and the project would be better served with a new or different product, which of course introduces some learning curve issues for admins, on top of all the other process changes going on simultaneously.

Project Timeline/Deliverables:
As I discussed in a previous post, determining the scope of the initial phase of your IdM project is key to the overall success of it. Doing too much at once will likely cause burnout, and potential failure, but making too little concrete progress will cause problems with project sponsors and the leadership team who want to see that the (increasingly tight) IT budget is being well-spent.

A long week ahead

Today we kicked off the week-long information gathering sessions with Identropy. We spent the day filling in Ash and Byron about the various meetings that have been scheduled, and a little overview of the technical architecture, as well as I could do, at least...

We should have a very informative week ahead of us -- the meetings have been scheduled with many of the key people around the University, including business and technical people from many of the colleges and departments, as they pertain to identity management. The response to the meeting invitations has been good so far -- hopefully the sub-zero temperatures later in the week don't discourage people from attending!

These first sessions will help to provide a general overview of many of the processes and systems for the vast majority of user populations at the UofR. Once this first round of meetings is complete, we'll have some more "deep-dive" sessions to really drill down into the details of whatever subset of users/processes is decided to be phase 1 of the project.

I'll post again at the end of the week, with some more thoughts about the interview sessions with the data and system owners.