The Coeus application allows for the specification of what it calls "Maps" which define a set of sequential stops at which requests are generated. Depending on whether maps are being used in routing or notification, they may produce either approval requests (at which point, all approvals must be satisfied before proceeding to the next stop in the map) or FYI notifications (which are dispatched all at once).
At each stop in the map, a request or notification can be sent to an individual or a "role" (in coeus terms). A typical Coeus Map setup might look like the following:
In the above example, the first stop will request an "approve" action from "Lucile Piazza", with an "alternate" approver of "Jennifer Flach". After either Lucile or Jennifer have approved the document, it will then proceed to the second stop where a request will be sent to "Stephen Dowdy".
In order to satisfy Kuali Coeus requirements, we must be able to provide the same level of functionality with KEW.
Functional Design and Requirements
Creating a "Map-like" Construct in Kuali Enterprise Workflow
KEW does not presently have a concept which provides the same level of functionality as Coeus Maps. Certainly, workflow processes can be defined using "nodes" on a document type which could model such a sequence of "stops", however one of the requirements for maps is that it is possible for end-users to set these up easily via a user interface and maintain them. Changing a document type process definition in KEW is intended to be an infrequent and somewhat "heavy-weight" event, typically requiring someone with a strong understanding of the effect it will have on the overall process (and often requiring appropriate change management procedures). More frequent changes are meant to be handled by external routing configuration functionality like KEW rules and KIM roles (and in Rice 2.0+, the Kuali Rule Management System).
To that end, and in order to satisfy the Kuali Coeus requirements for functional equivalence with the original Coeus application, it makes sense to create a new construct in KEW which will approximate Coeus map functionality.
One important aspect of this is to come up appropriate terminology for this new KEW feature. The term "Map" is slightly nondescript and tends to be overloaded (at least in the technical space). Other suggestions which have been presented include:
- Request Chain
- Simple Sequential Workflow Definition
- Serial Approval Workflow Definition (SAWD)
- Approve Queue
- Stop List
- Serial Workflow Interface Model (SWIM)
- Serial Workflow Approval Model (SWAM)
- Serial Approval Workflow Definition (SAWD)
- Serial Approval Workflow (SAW)
- Serial Workflow Approval Definition (SWAD)
- Serial Approval Definition (SAD)
- Serial Approval Model (SAM)
- KEW Editor Lite (KEWEL or KEWL)
- KEW Designer (KEWD)
- Request Flow
Considering that this functionality will be used to define "snippets" of workflow definitions that will be composed as part of larger processes, terms like KEW "Editor" or "Designer" would apply to the larger project being taken on later in Rice 2.x to create a full KEW editor (for editing all aspects of a workflow process definition). This map-like functionality would function more as a "tool" which forms part of a larger set of workflow editor capabilities.
Additionally, our initial approach is toward providing a "sequential" solution, but there's no reason the model couldn't be extended in the future to allow for "parallel" approval capabilities (though a kind of parallelism is required for the Map functionality, but can only occur "within" a sequential stop). For these reasons, using terms like "sequential", "serial", "list", "chain", etc. appear to be limiting ourselves to sequential processes and might result in confusion moving forward. Futhermore, using terms like "approval" are also a bit of a misnomer since these map processes can be used to send notifications as well, not just approvals.
A term like "PeopleFlow" or "PersonFlow" may be the most logical as they are succinct and describe the fact that we are building a flow of requests and/or notifications that involve people (either directly or through their roles).
Update: The group has decided to go with the term "PeopleFlow" for the conceptual name and "PeopleFlow Editor" for the user interface tool that can be used to create and maintain them.
Baseline Requirements for Functional Equivalence
It is important to note that "functional equivalence" does not mean that we need to provide and present everything in the exact same fashion as it is in Coeus. Rather, we must ensure that the same outcomes can be accomplished in a similar manner and satisfies the requirements of the original. We are, therefore, not required to implement the Maps functionality in perfect parity with the original Coeus implementation, but it should provide a similar or greater level of functionality and capability. In order to accomplish this goal, we must first understand what the requirements are.
The following is a list of general requirements that have to be satisfied in order to achieve functional equivalence with Coeus Maps:
- In situations where a map is used to route approval requests, a document that is routed to a map will start at the first sequential stop, wait for any outstanding approvals, and then route to the second stop, etc. until it reaches the end.
- Note that 1 or more maps can also be initiated simultaneously and in parallel if the overarching workflow process contains parallelism. See the routing process section below for how this would work in the case of the KC unit hierarchy.
- In situations where a map is used to dispatch notifications for a document, all stops in the map will be executed at once without waiting for action to be taken between stops.
- You can specify any number of primary and alternate approvers at each stop.
- If more than one "primary" approvers is specified, then all primary approvers (or their alternates) must take action.
- Each primary approver has it's own (optional) set of alternate approvers at a specific stop. In otherwords, alternates are a associated with a specific approver, not the stop.
- Alternate approvers function as delegates for their primary approvers and, in the general case, the primary approver will be the one to take action but the alternate is there as backup.
- The relationship between the Primary approver and the alternates is like a leader and his or her group. In the real world it may be the unit head and an assistant unit head. The unit head is the primary but is assistant can sign for him or her.
- In the case where a map is used for notifications, all primary and alternate approvers will receive the notification.
- If a map is modified, while a document is already routed to that map (and sitting at some "stop" in the map), the document will not be affected. The configuration of the map as it existed at the time the document landed on that map will be used.
- It is legal for the workflow engine to resolve to more than one map, in which case they will be processed in the order in which they are resolved (i.e. Map 1 -> Map 2 -> Map 3 -> etc.) This could specifically happen in cases where more than one Map is returned from evaluation for a particular unit within the KRMS engine.
- Once a Map has "completed", future Maps will be evaluated according to their latest state (not necessarily the state they were in when the first map was triggered). This means that a future maps configuration can be changed while a document is "waiting" in an earlier map and these changes will be reflected when the document reaches the future maps.
- Each "approver" within a Map can be either a KIM principal, KIM role, or KIM group.
- In the case of KIM groups, only one individual from the group needs to take action (this is equivalent to how groups function currently in KEW according to a "first approve" request policy)
- In the case of KIM roles, there must be some way to indicate any relevant qualifiers that might need to be used when qualifying the role (for example, proposal number in the case of Principal Investigator and IRB Administrator which is qualified by unit number)
- However, certain roles don't require qualifiers so qualifiers should therefore be optional when specifying a role.
- Maps must be able to have additional data attributes associated with them.
- Most notably (in the case of Kuali Coeus) the unit number.
- Though the attributes supported by a map would depend on it's "type" (see later in this document)
Additional Requirements related to implementation in KEW
Because of the nature of the Kuali Rice framework and it's various components, we must impose additional requirements on the implementation in order to allow for the functionality to be reusable outside of just the Kuali Coeus application:
- Maps must be able to be used by applications other then Kuali Coeus. To this end, a typing system must be introduced to identify types that are utilized by different applications. This will inform which additional data attributes need to be collected when the Map is constructed (in the case of Kuali Coeus maps, this would be the unit number).
- The functional implication here is that, when creating a new map, the author must first select the type of map they want to create.
- Note however that URLS to the map editor could be fashioned such that they "pre-select" some particular type. Links of these kind could simply be added to something like the Kuali Coeus portal to provide a more "app-focussed" user experience.
- For the sake of completeness, maps should also support generating requests to KIM Groups in addition to principals and roles.
- When a map is used for routing, it will generate approve requests. When it it used for notification, it will generate FYI requests.
- It is not necessary to support "returning" to a previous sequential stop inside of a Map (either as part of the workflow process or via initiation of a "return to previous" command issued against the document).\
The Kuali Coeus Routing Process
In order to achieve functional equivalence with Coeus, Kuali Coeus requires that it be possible to set up a routing process which "walks up" the unit hierarchy. At each unit within the hierarchy, it will execute a set of rules which will produce 0..n "maps", generating the appropriate approval requests for each in sequence.
The specific units that will be included on a particular transaction will likely be different for each transaction. It is therefore not possible to define a "static" process definition for a document which utilizes this unit-hierarchy based routing model. Instead, in order to accomplish this in KEW, KC can take advantage of a "Dynamic Node" along with rules (defined in KRMS) that will execute at each unit as the document climbs the hierarchy. To understand how exactly this works, there are a few important pieces of terminology to understand:
- Node - a node represents a step in a workflow process. They generally represent some action that takes place in the workflow process such as "route approval requests to the fiscal officers for all accounts on the document" or "send an email".
- Dynamic Node - a dynamic node is a component which executes a sub-process that is generated at runtime. They allow for the generation of a dynamic route path based on document data. In otherwords, a node that generates other nodes.
- Sub Process - a sub-process is simply a sequence of nodes and node transitions
- Action Request - a request that is generated to a principal which is requesting some action from them (i.e. approve, fyi, etc.)
- Request Activiation - the process by which requests at a particular node are "activated" so that they enter the action list of the appropriate parties. The process of activating requests can form a non-node-based workflow depending on which "activiation type" is used.
- Activation Type - defines the strategy that is used when activating action requests at a particular node in the workflow process. As of Rice 1.x, only Sequential and Parallel is supported.
- During parallel activation, all requests are activated and enter users' actions lists at the same time.
- During sequential activation, only one request is activated at a time based on the request with the highest priority (which is represented by the smallest priority number).
Here is a diagram of the what the statically defined KEW process would look like:Gliffy
Not very interesting. This is because most of the work is happening inside of that dynamic node which is generating a route path on the fly in order to traverse the unit hierarchy.
Now, consider the following "units" and their reporting hierarchies (up to the root of the unit hiearchy, IU-UNIV):
- BL-BUS -> BL-BL -> IU-UNIV
- BL-CSCI -> BL-INFO -> BL-BL -> IU-UNIV
- UA-VPIT -> UA-UA -> IU-UNIV
Imagine a Kuali Coeus document is submitted which has the BL-BUS, BL-CSCI, and UA-VPIT units represented on it. In this case, the document's workflow process should begin at each of these units and then execute in parallel, walking up the hierarchy as it transitions from node to node, pausing to ensure that all outstanding branches coming into a specific unit in the hierarchy are completed before walking up to the next node in the hierarchy. This behavior is accomplished by the "Hiearchy Routing Node" which is a dynamic node implementation that is provided out-of-the-box with KEW and can be plugged into various hierarchy implementations.
If we "zoom in" on the dynamic routing process for the example given above, the resulting dynamically generated workflow process would look like the following:Gliffy
At each of these dynamically generated nodes, one or more action requests can be generated. This is the typical Kuali Enteprise Workflow model. In the case of Kuali Coeus, the requests that are generated at each of these routing nodes could be driven by a "map" definition.
For example, one might define two maps attached to the UA-VPIT unit which specified the following:
- Sequential Stop 1:
- Primary approver: ewestfal, Alternates: none
- Sequential Stop 2:
- Primary approver: gilesp, Alternates: dseibert, cneisen
- Primary approver: ryetter, Alternates: gmcgrego
- Sequential Stop 1:
- Primary approver: khensley, Alternates: none
- Primary approver: lshultz, Alternates: jthomas, lsymms, jcoltrin
It's possible that these maps could be produced and supplied to the workflow engine to trigger routing in various ways, but in terms of Kuali Coeus requirements these will be generated as part of KRMS execution. Imagine that an agenda executes which produces Map 1 and Map 2 (in that order). If we zoom in even further on just the UA-VPIT node in the dynamic unit hierarchy routing process, we can see what the request-based workflow which is triggered by a Map would look like:Gliffy
All of the requests above would get generated at the same time, but each would be assigned a numeric "priority". The workflow engine would then activate the requests at the highest priority first, wait for all of those to be satisfied, and then activates any requests at the next priority level. This activation strategy enforces the sequence of stops that are declared by the map. In terms of the workflow engine, request "activation" is the process by which action list entries are materialized from an action request. Requests that exist at the same priority level are activated in parallel. This activation strategy does not fit into either the classic parallel or sequential activation types supported by the current KEW implementation. So a new activation strategy must be created which activates requests at the same priority in parallel, and those at different priorities in sequence. For lack of a better term, let's refer to this as a "Priority-Parallel" activation strategy.
Note that, in the diagram above, the arrows between requests represent "secondary" delegation of authority to the alternates that are defined on the map. The concept of secondary delegation is one which is already supported by KEW.
So far, we have discussed maps primarily in terms of Kuali Enterprise Workflow. And, in fact, KEW will be the place where maps are maintained and stored. However, there still needs to be support for "triggering" the execution of maps at various nodes in the routing process (as shown in our previous examples). For the purposes of Kuali Coeus, the driving requirement is that it needs to be possible to do this via business rules. The KRMS module is being built in Rice 2.0 largely to satisfy this requirement. However, the link between rules and the resulting execution of maps also needs to be implemented as part of this effort.
Recall from the design of KRMS that a business rule is effectively a condition with a set of actions. If the condition evaluates to "true", the actions get executed. To implement KRMS integration with maps, a special action type will be created that allows for execution of map to be linked as the action that is executed when a rule succeeds. From the perspective of the rule author, the process will look like the following:
- Create any maps that are needed for execution within the rules engine. This setup process will happen via the maps user interface which will not be part of KRMS but rather KEW.
- Note: in the future as we build out more of a comprehensive workflow "designer" it might make sense to make it so that maps can be created on the fly as rules are being authored rather than having to be done as part of a separate process.
- Go into the KRMS UI and select the context to edit, bringing up the agenda and rules editor.
- Create the rule that will be used to drive routing. When creating the action, choose the appropriate "action type" which allows for specifying map execution as the outcome of successful rule execution.
- Use the Map Lookup screen to search for and select the map(s) to be executed as part of the rule action.
- Add the rule to the agenda (if necessary) and submit the changes.
User Interface for Map Maintenance
- needs to have inquiry, lookup, and maint doc
- needs to have a way to select the "type" of map being created
Below is a placeholder mockup - will refresh with finalized design shortly! The "Other Required Type Attributes" and "Other Required People Attributes" only appear if the identified namespace-type or people-type require other attributes. These other attributes could vary across applications and institutions.
Sections are expandable / collapsible. This design assumes that "People" added to the flow (Principals, Groups, Roles) are primary. Alternates are defined in line with their primary (in subcollection UI). This is yet to be added to the mockup. (Note: We also need to think through whether it is important that users be able to change a person from a primary to an alternate or vice versa - without deleting and re-adding them.)
This section is draft material and will be updated shortly!
Technical Design and Implementation
There are effectively two ways this could be done in KEW today.
- By defining sub-processes that include a set of individual nodes that represent each approver and stop within the flow.
- By executing the PeopleFlow at a single node, but using request activation strategies to "activate" the requests in the proper order.
Must of the discussion in the functional requirements sections above alludes to the fact that option 2 is the simplest approach and, therefore, recommended approach.
Since the creation of new nodes is intended to be a rare activity, KEW is not currently optimized for large numbers of dynamically generated nodes and, to introduce such, would make a given routing process significantly more complex.. So the second approach mentioned is the simpler and less time consuming implementation to undertake. Though, as a result of this, it will require a few new database tables in order to store the PeopleFlow configuration(s) since we cannot simply leverage the existing schema used to store route nodes.
The database model for this work should be fairly straightforward, though it will be complicated by the need to implement a typing system for PeopleFlows (in order to allow us to determine which flows require which additional attributes of data). We will effectively need the following:
The PeopleFlow table will store the identifying information about the flow, including:
- PeopleFlow ID
- Namespace Code
- PeopleFlow Type ID
Stores generic key-value pair attributes that can be associated with a PeopleFlow.
Model this similar to the attribute tables found in the KRMS Data Model.
Defines the various "members" of the flow as well as the priority at which they are defined in the flow. This priority is used to define the "order" of the people in the flow.
This table should include the following pieces of data:
- PeopleFlow Member ID
- PeopleFlow ID
- Member Type Code - see member type codes used in KIM; principal, group, or role)
- Member ID - principal id, group id, or role id
- Priority - this is effectively the "sequential stop" number
- Delegated from ID - optional...if this member represents an "alternate/delegate" then this will be the flow member id which is delegating to this member
- note that this could have been done with a separate table called KREW_PPL_FLW_DLGN_T or something along those lines, but this approach may be simple enough to model
Table(s): KREW_TYP_T, KREW_TYP_ATTR_T, KREW_TYP_ATTR_DEFN_T
KEW does not currently have a "typing" mechanism along the same lines as KIM and KRMS have. In order to allow for custom attributes and types to be implemented for PeopleFlows, it will be useful to have this and model it the same way we are modelling it elsewhere. For this reason, we should create something similar to what KIM and KRMS have.
See tables KRMS_TYP_T, KRMS_TYP_ATTR_T, and KRMS_TYP_ATTR_DEFN_T in the KRMS Data Model.
Since the rules engine and workflow engine can execute within the client application (depending on application integration option being used), there needs to be a remotely accessible SOAP service which can be used to retrieve and lookup the definition of a PeopleFlow so that KRMS or KEW can utilize it.
The main thing that needs to be done here is designing a JAXB annotated "dto" representation for the PeopleFlow. There is no need (at least not initially) to provide operations on the service for updating these flows, it just needs to support read only operations.
This service should follow all the standard and best practices for SOAP based services that can be safely evolved across versions of Rice (see the work that has been done on version compatibility for Rice 2.0).
A PeopleFlow "type" service (similar to the type service in KIM and KRMS) will need to be created in order to support the specification of custom PeopleFlow types which can define the following:
- The attributes that the PeopleFlow type supports
- this will be called by the PeopleFlow Editor.
- A method that allows for the flow to restrict the roles that can be selected when building the flow
- this will be called by the editor as well
- A method which takes a reference to the role being resolved and a reference to the workflow "Document" and "DocumentContent" classes so that the code can resolve qualifiers as needed from that information. It should return a Map of qualifiers that are passed to kim to "resolve" the KIM role membership (see RoleRouteModule and the QualifierResolver class in KEW for for a similar set of concepts, may make sense to reuse some of this stuff where it makes sense).
- this will be invoked at the time the PeopleFlow is evaluated to determine who to send requests to
KrmsTypeRepositoryService for an example.
Integration of KEW Process Definitions with KRMS
In order to allow someone to define a route path on a Document Type which integrates with KRMS, we need to add support to the
RequestsNode so that it can execute the KRMS engine in order to generate action requests.
The configuration for the
RequestsNode in order to support this should look something like the following:
executorName attribute would need to point to the name of a KEW extension that resolves to an instance which implements the
executorClass attribute would need to point to a fully qualified class name of a class which implements the
RulesEngineExecutor interface will be defined as follows:
EngineResults should contain information on which PeopleFlows to execute. The specification for how these are presented via the
EngineResults will be as follows:
- A value will be stored in the
EngineResultsobject with a key of "peopleFlowsSelected".
- The value of this attribute will be a comma-separated list of
ActionRequestTypecode and PeopleFlow ID pairs in a format similar to the following
The process by which the KEW routing integration works would be similar to the following:
- When the node executes, it acquires a reference to the KRMS engine and invoke the
EngineExecutor.execute(...)method passing the current
RouteContextfor the workflow engine and the reference to the KRMS
- Once the engine executes, the
EngineResultsare returned to the node implementation.
- The node looks in the engine results and pulls out the list of PeopleFlow definitions that were generated by execution of the rules engine.
- NOTE: if the engine results does not already have a general purpose Map on it to store custom results, that needs to be added
- The node then processes the list of flows and validates and generates action requests for the first PeopleFlow in the list.
PeopleFlowServicewhich we mentioned earlier will be used to retrieve the flow information
- validates and saves the actions requests in the initialized status and lets the RequestActiviationNode (it's super class probably) handle things from there.
- if a flow has roles on it, will need to invoke the appropriate method on
PeopleFlowTypeServicein order to pass the qualifiers and resolve the role
- These action requests (when generated) need to be set at the appropriate priority to model the desired behavior of the flow.
- In the case of alternate approvers, the appropriate action request "graph" should be returned (to model delegation, an action request can have child requests). Secondary delegation should be used as the delegation type.
- If there is more than one PeopleFLow id returned, it should save the ids of future PeopleFLows to execute on the "route node instance state".
- This is because the requests for a given flow should not be generated until the point at which the flow is first reached (see requirements in the baseline requirements section above for details on how multiple PeopleFlows should be handled)
- Once the requests for the first PeopleFlow have been generated (in the 'I' or initialized status), the node implementation will let it's super class (
RequestActivationNodeprobably) handle activating the requests).
- Once all requests have been satisfied from the current PeopleFlow, the node will check if there are any other pending flows to execute (which should be stored in the node instance state, as mentioned earlier). If so it generates requests for the next flow and repeats the process.
- If this is being used for notification instead of approval, all requests should be created and activated at the same time.
- Note that it may be possible to just implement a
RouteModulehere which returns a list of action requests, that is what the KIM routing integration does with the
- Note that it may be possible to just implement a
In order to allow for the PeopleFlow action request activation to proceed as required, we need to introduce a new request activation strategy. Currently, the RequestActivationNode only supports the following activation types:
Sequential - activates the single request with the smallest priority number first. If there are multiple initialized requests with the same priority, it will activate based on the order in the list returned from the route module. If an approve request, the next priority request is not activated until the current activated request is satisfied by an action from the requested recipients.
Parallel - all requests returned from the route module are activated at the same time, regardless of priority.
What we need is an activation type which activates all requests at the same priority in parallel but at different priorities in sequence (starting with the smallest priority number and working up from there). We will call this activation type "Priority-Parallel Activation".
Note that this type of activation should be the default on nodes which integrate with KRMS (i.e. they shouldn't have to explicitly set this activation type when they configure the node on their document type processes).
Integration of KRMS with PeopleFlows
On the KRMS side, in order to integrate with we will use the concept of custom "Actions" that KRMS supports. Essentially, if a rule execution succeeds, then it will execute all actions that are associated with that rule. So an action type can be provided with KRMS which allows for the rule author to point the action of the rule at a PeopleFlow.
The execution of this action will work as follows:
- A class will be created which implements the
org.kuali.rice.krms.framework.type.ActionTypeService. This can be called something like
- Two methods will need to be added to the
ActionTypeServiceinterface which looks something like the following:
LazyActionclass will then need to be modified to invoke the appropriate methods on the ActionTypeService.
- Two methods will need to be added to the
- The id of the PeopleFlow to execute should be stored on the attributes for the
ActionDefinitionand so can be retrieved from there.
- Matching PeopleFlow ids should be added to the
- Note, it's possible that more than one action could be configured on a rule or more then one rule executed such that multiple PeopleFlows could be selected. To facilitate some object that stores an ordered list of all the PeopleFlow ids probably be established in the engine results to store this. This list could get populated with flows from multiple actions (so should check if the list in the results exists first, create it if not, and then add to it)
- Once the PeopleFlow ids are in the engine results, the KRMS action implementation's work is complete.
Work Breakdown and Estimates
KEW Backend Work
- Create PeopleFlow database table
- Create KEW type tables
- Create and implement
- export service to the KSB
- create class to help with exporting custom PeopleFlowTypeService
- export service to the KSB
- Create a new KRMS node implementation (extends from
RequestActivationNodewhich invokes KRMS engine
- implement support for the
- for Roles, integrate with calls to the appropriate
PeopleFlowTypeServiceto resolve role membership
- implement custom action type
- general work toward creating this node
- implement support for the
- Add support to Document Type XML schema to specify the krms integrating route node
- Write KEW integration tests to test the KEW integration with the KRMS PeopleFlows
KRMS Backend Work
- Add support to EngineResults to support a general-purpose Hash Map of results
- Create PeopleFlowActionTypeService which places PeopleFlow ids into the engine results
- Create PeopleFlowAction that PeopleFlowActionType loads
- create sql to insert type service into the appropriate KRMS tables
- publish custom action type service to the KSB
User Interface Work
- On the KEW side, create the PeopleFlow maintenance document, lookup, and inquiry
- Still need to come up with a general mock for this
- On the KRMS side, will need to add equivalent of
getAttributeDefinitionMapin KIM to the
- talk to Version Compatibility team about this first as they have been working on a new format for this "remotable" piece of UI configuration
- Implement above method on PeopleFlowActionTypeService so that it returns a single field for the PeopleFlow id with a lookup to the PeopleFlow lookup