The goals for this work are fairly well-stated in KULRICE-4816 but I will include them here as well:
- SOAP-based interface to the service registry
- Replace serialized form of service definition with XML/JAXB-based form
- Remove "dead" indicator from the registry, move the "dead service" state to the client app so it's not in the registry.
- Add "version" information into the registry.
- API to get version for a service.
- Implement a model by which applications automatically have a version and their services are published under that version.
- Client application configures a registry "url" in order to configure it. Build in backend support for "loading" the service that points to the registry.
Issues with Rice 1.0.x KSB
In order to implement the enhancements required for this work item, some refactoring had to be done on the Kuali Service Bus first. In Rice 1.0.x, the service bus contains three main components:
- RemoteResourceServiceLocator - handles proxies to all "remote" services that it has read from the service registry
- RemotedServiceRegistry - handles all services that a client application has published to the registry
- ServiceRegistry - reads and writes directly from the service registry tables in the database
On the surface this breakdown makes sense, however there are some issues with it:
- The functionality of RemoteResourceServiceLocator and RemotedServiceRegistry is somewhat tangled.
- The RemoteResourceServiceLocator actually calls into the RemotedServiceRegistry for various operations, but in certain situations it's required to go explicitly to the RemotedServiceRegistry.
- The ServiceRegistry is accessible only via direct database connections.
- These classes share a common object called ServiceDefinition which is used both during service publishing and when reading services from the registry. However, these are two slightly different situations that need to be considered.
In order to get around some of these issues, for 2.0 we chose to do a re-implementation of the KSB apis in order to address some of these issues.
KSB Architecture Redesign and Reimplementation
The KSB refactoring for version compatibility focuses on the design and implementation of the following components:
- Service Registry
- Service Bus
- Service Export Manager
The Service Registry contains information about all services which have registered with the service bus from all instances and applications that are "connected" to the bus. The
ServiceRegistry is a remotable implementation of the registry (available as a SOAP endpoint) which allows for registering, removing, and querying on information about these service endpoints. It is important in Rice 2.0 that this service is accessible remotely because this allows client applications to connect to it through a remote interface instead of through direct database connections as was done in Rice 1.0.x. The registry is intended to be deployed and available as a separate service which all clients who are joined up to the "bus" can connect into.
In general, Java client applications will not interact directly with the
ServiceRegistry, instead they will leverage the
ServiceBus api (see below) which manages a client's state in respect to the central registry. Non-java clients could take advantage of the SOAP apis provided for the service registry if they would like to publish their own services or query registry state in order to locate services.
Because the registry is now accessed remotely, a discovery process has to be put into place that allows for client applications to "connect" to it (see Service Registry Discovery below). This allows for the registry to be "bootstrapped" into existence from the perspective of the client. The registry itself is a service, but obviously it is not possible to use the registry in order to discover the available endpoints for the registry.
Service Registry Data Model
The data model for the Service Registry consists of three data elements:
ServiceInfo- includes standard configuration information about a service that has been published in the registry
ServiceDescriptor- includes an XML form of the
ServiceConfigurationwhich consists of detailed service configuration information
ServiceEndpoint- includes both the
ServiceDescriptorfor a service
Each of these is a concrete immutable class which implements a contract interface that defines the various pieces of data on each and what the invariants are. Contract interfaces for each can be reviewed below:
As can been seen from this data model, the service endpoint is separated into two parts, the service information and the service descriptor. The
ServiceInfo contains the service descriptor id that can be used to load the descriptor when it is needed. This is important as it allows the client-side service bus functionality to load all the online
ServiceInfo from the registry and then only load the
ServiceDescriptor for services it actually needs to use. This was implemented similarly in previous versions of Rice but took advantage of lazy loading of proxy references in the OJB layer. Since this functionality has moved from the ORM layer up to the service layer (in order to facilitate version compatibility) we are implementing a "manual" lazy loading of this information through the service layer.
The Service Registry "Service"
ServiceRegistry interface defines the contract for the registry and specifies the various operations that are supported. A summary of supported operations include:
ServiceInfofor all of the services that are "online"
ServiceInfofor all online services with a particular name
- Get all
ServiceInfoin the registry, regardless of status
- Load the
ServiceDescriptorfor a given descriptor id
- Publish a
- Remove a
- Remove and publish as a single operation - used by the client-side registry "diff" process
- Update service status to the given status
- Take an instance offline based on the instance id - sets all statuses for services with that instance id to
The full specification for the
ServiceRegistry interface can be found here: http://fisheye.kuali.org/browse/~raw,r=20802/rice/trunk/ksb/api/src/main/java/org/kuali/rice/ksb/api/registry/ServiceRegistry.java
The concept of a service being "online" has to do with the status code on the service's
ServiceInfo. These are defined by the
ServiceEndpointStatus enumeration which defines three valid values:
- ONLINE - Indicates the service is online and available to receieve requests.
- OFFLINE - Indicates the service has been taken offline, most likely as the result of the instance hosting the service being taken offline (i.e. for maintenance or otherwise)
- DISABLED - Indicates the service has been disabled because the registry has detected that it, or it's host instance is defective or not processing requests properly.
Note that the
DISABLED status is not currently used by the KSB but is meant to be leveraged by future functionality which will help with keeping entries in the service registry as viable as possible.
The registry is accessible through a SOAP-based interface which allows for both java and non-java clients to both query for information about services that have been published, as well as publishing their own services to the registry.
Service Registry Discovery
ServiceRegistry is itself now a service, a process must be put in place to "bootstrap" a connection to the registry. For other services, the process to acquire information is to obtain it from the registry, however, for the registry itself we need to use another mechanism in order to bring it online. To accomplish this, the client application simply has to configure a URL in a similar fashion to the following:
Currently, this connector is only configured to understand a SOAP interface to the service registry which is secured by digital signatures. This is the only type of interface to the registry that the standalone server currently publishes.
Additionally, only a single URL to the registry can be configured at the current time. If someone wants to do load balancing amongst potential registry endpoints, then a hardware or software load balancer could be configured to do this.
ServiceBus handles maintaining a client application's "view" of the registry. It periodically synchronizes with the registry to find newly available services or ones that were removed. Additionally, a client application can "publish" it's own services here and this piece will handle installing those services into the
ServiceRegistry. Note that the
ServiceBus implementations runs embedded within the client application and is therefore only relevant for Java applications. Non-java applications can still interact with the remote
ServiceRegistry as discussed previously, but would need to code any behavior similar to the
ServiceBus by hand if they wanted to replicate it.
ServiceBus works with three main objects:
ServiceDefinnition - The
ServiceDefinition contains information about a service to be published, as well a reference to the service endpoint instance to publish. This definition is typically used in conjuction with the
ServiceBusExporter and is sent to the
ServiceExportManager to handle making an endpoint available for the service (see below for more information about both of these). There are generally different implementations of the service definition depending on the type of service being published (i.e.
ServiceConfiguration - The
ServiceConfiguration contains all information about the configuration of the service including service name, endpoint url, version, type, messaging configuration, security policies, as well as protocol specific configuration. There are generally different implementations of the service configuration, such as
RestServiceConfiguration, etc. These are serialized to an XML form and stored on the
ServiceDescriptor in the registry as was discussed previously.
Endpoint - The
Endpoint contains a reference to the
ServiceConfiguration for a service as well as a proxy to the service endpoint that can be invoked. This service can be cast to the appropriate service interface in order to invoke the desired operations.
ServiceBus is primarily interact with through a method called "getService" that takes a service name and returns a reference to a service with that name that can be invoked. The reference returned from this method is actually an
Endpoint but the service proxy can be obtained by invoking
getService() on the endpoint.
The bus will give priority to locally available services with the given name over remote services with the same name. If necessary (in the case that the service is remote), the service reference it returns will be wrapped in a proxy that handles failover which is triggered during invocation failure scenarios. When a failover event is encountered, the proxy will go back to the
ServiceBus to attempt to locate another endpoint with the same service name and attempt to invoke that instead. It will continue this process until it finds a valid service to invoke, if none of the services work, then the service invocation will fail.
Services are generally published by creating a
ServiceDefinition and "exporting" it using a
ServiceBusExporter. This is often done via spring configuration as in the following example showing the publication of the
ServiceBusExporter simply delegates to the "publishService" method on
ServiceBus which handles publishing and exporting the service. So using this operation on the bus is another way to publish a service directly without using a
ServiceBus Synchronization with the Registry
This synchronization process is handled by a component called the
ServiceRegistryDiffCalculator. This component contains the logic related to looking at a client application's local view of the registry and services it has published and making sure that the client's view is synchronized with the registry. It also ensures that the registry's understanding of services available at the client is consistent. It does this by interacting with the
The client-side bus will be synchronized with the remote
ServiceRegistry once upon startup (startup will fail if the synchronization fails or the remote registry is unavailable) and then subsequent synchronization is handled by a background thread which executes periodically. This refresh period defaults to 30 seconds but is configurable by setting a parameter as follows:
Integration with the ResourceLoader Framework
ResourceLoader framework integrates with the
ServiceBus by providing a special
ResourceLoader implementation which delegates it's "getService" method to the corresponding "getService" method on the
ServiceBus. This resource loader is placed on the bottom of the resource loader stack to ensure that potentially remoteable services are always accessed last before checking for locally available implementations of the service being looked up.
Additional ServiceBus Operations
Besides the operations already mentioned, there are many other operations on the
ServiceBus interface. The
ServiceBus specification includes operations like the following:
- Get the instance id for this service bus client (see terminology section below for more information on the instance id)
- Get all endpoints for a given service name
- Get the remote endpoints for a given service name
- Get the local endpoint for a given service name
- Get all local endpoints
- Get an available endpoint for a given service name
- Get a service proxy for the given service name
- Publish a service
- Remove a service
- Manually invoke synchronization with the registry
The full specification of the
ServiceBus interface can be found here: http://fisheye.kuali.org/browse/~raw,r=20493/rice/trunk/ksb/api/src/main/java/org/kuali/rice/ksb/api/bus/ServiceBus.java
Service Export Manager
ServiceExportManager is a component that is communicated with during service publishing through the
ServiceBus. It interfaces with a set of
ServiceExporter implementations to create an endpoint proxy that can receive and process incoming requests to invoke that service endpoint. This proxy handles things like security and ultimately delegates the call down to the underlying service implementation that was published.
ServiceExportManager also knows how to translate an incoming URL from an
HttpServletRequest into a service endpoint to invoke. The
KSBDispatcherServlet then delegates incoming requests down to the
ServiceExportManager in order to invoke the appropriate endpoint during service invocation.
Other Important Terminology
To wrap up a few other pieces, there is some other important terminology. All applications have an application id (this was previously called service namespace). An application id is an identifier for an application, though it's intended that more than one instance of an application could exist (i.e. In the typical clustered use case). There is additionally the concept of an "instance", every instance has an instanceId which uniquely identifies it. By default (if not manually set) this value is generated as a combination of the server's ip address and the application id (effectively meaning that you can't deploy the same application more than once on the same ip unless you customize how the instance id is generated).
The KSB is configured using the
KSBConfigurer class. It supports two run modes:
- LOCAL - local run mode should only be used by the KSB registry server and/or the Rice standalone server, in this mode the application connects via direct database connections to the registry
- REMOTE - the run mode that all client applications should use, this connects to the service registry through a remote SOAP interface
There are additionally some other configuration parameters that can be configured in different ways to control how the KSB operates:
KSB Impact when moving from 1.0.x to 2.0
Many of the changes made to the KSB for this refactoring may be impacting to existing code. A summary is below:
Moved to org.kuali.rice.ksb.api.bus.support.ServiceBusExporter
Moved to org.kuali.rice.ksb.api.bus.support.PropertyConditionalServiceBusExporter
Moved to org.kuali.rice.ksb.api.bus.support.JavaServiceDefinition
Moved to org.kuali.rice.ksb.api.bus.support.RestServiceDefinition
Moved to org.kuali.rice.ksb.api.bus.support.SoapServiceDefinition
Use org.kuali.rice.ksb.api.bus.ServiceBus instead
Moved to org.kuali.rice.ksb.api.bus.support.PropertyConditionalServiceBusExporter