Business transaction management (BTM) is a promising new development in general-purpose enterprise software. Most large companies are devoting significant resources to the problem of reliable, consistent integration of application services.
BTM offers previously inaccessible levels of application coordination and process synchronization, radically simplifying the design and implementation of transactional business processes. Business process management (BPM) needs to be enriched by BTM for users to see the potential value of BPM realized in practice.
XML is already widely deployed as a useful lingua franca enabling the creation of canonical data standards for particular industries, trading communities, and information exchanges. The extended family of Web services standards (clustered around the leading duo of SOAP and WSDL) is gaining growing acceptance as an important way of providing interoperable connectivity between heterogeneous systems. Many organizations are also examining the use of BPM technologies, exemplified by the current OASIS initiative, Web Services Business Process Execution Language (WS BPEL). Increasingly, attention is turning to the special problems associated with building transactional business processes and reliable, composable services. This is where BTM technology comes into its own.
In this article - which follows the excellent introductions by Mark Little and Jim Webber to WS-Coordination, WS-Transaction, OASIS Business Transaction Protocol, and BPEL (Web Services Journal, Vol. 3, issues 5-8) - I'm going to look at the rationale for and current status of BTM, and how vendors and users are thinking about the integration or fusion of BTM with BPM, particularly in the OASIS BPEL standardization effort. BPEL, as a special-purpose programming language designed to make processes portable across different vendors' execution engines, can become a very useful standard programming interface for business transactions in the Web services world.
Why Is BTM Needed?
Companies would like to minimize the costly, slow, and risky business of "reconcile and repair." Automating or improving the internal flow of transaction processing, spanning multiple disparate applications, is a major issue for most corporations: the finance industry's focus on straight-through processing is an emblematic case. The business model of the Internet (which can cynically be summarized as "make your customers your clerks") continues to drive efforts to offer automated transactional platforms or portals directly to consumers and to business clients. This push towards "self-care" makes reliable, consistent, and recoverable composition of back-end services more important, as inconsistencies and failures become quickly visible outside the call center or the operations room. Equally, customers and suppliers who transact business electronically need well-formed protocols which model the interactions of their business relationship: they want near-instant certainty and "assured common knowledge" about the outcomes (good and bad) of those interactions.
Automated business transactions are a new category, wider than historical data-centric local or distributed transactions. This "third generation" of transaction management builds out from core transactional technology, particularly the concept of a multi-phase distributed outcome ("two-phase commit" in conventional database/messaging transactions). Business transactions may be of split-second duration, but may also be long-running, i.e., may endure for periods that exceed the anticipated service cycles of participant systems. Business transactions retain the driving ambition of consistency. However, they are more flexible and sophisticated in their means - and often more modest in their goals, reflecting the imperfect determinism of real business processes (which are necessarily designed to cope with innumerable variations and with incremental and partial successes).
Both of these features are relevant to the work of the OASIS WS BPEL Technical Committee, whose charter foresees work over a minimum period of nine months (from May this year to January next) to achieve a recognized standard. In the remainder of this article I'll concentrate on the integration of a transactional coordination protocol with BPEL.
Exception and Recovery Handling in BPEL Today
The current WS BPEL 1.1 draft specification (jointly submitted by IBM, Microsoft, BEA, SAP, and Siebel) incorporates the notion of processes that can contain nested sub-processes or scopes, to any depth. Processes are expected to carry out "real work" (work which affects persistent records and external systems) by invoking WSDL-defined operations. While BPEL processes can define and manipulate variables, these variables are designed for use in tracking process state and controlling conditional flows. If processes wish to communicate with other processes then they can do so using "partner links," which essentially create sets of WSDL-defined services as access points for bilateral message sequences between defined roles in a collaborative exchange.
Processes are initiated by receiving inbound Web service invocations. BPEL, therefore, expects that external stimuli will arrive and substantive processing work will be invoked as Web service documents, typically delivered using SOAP messages.
Processes and scopes can declare fault handlers and compensation handlers, in addition to the normal sequence of "happy path" activities. A fault handler is designed to deal with exceptions during normal processing. A compensation handler is designed to reverse the effect of a completed scope, and can only be invoked when the normal work of the scope is finished (this gives the compensation handler a clean starting point for its reversal activity). Compensation handlers can be invoked by the fault handler of an enclosing scope - which can decide which compensations to process, and in which order - in order to roll back partial work in the event of failure.
Integrating BTM with BPEL BPM
Vendor and user companies in the WS BPEL committee, including Choreology Ltd, are discussing how BPEL will use BTM. (In this context BTM tends to be equated with WS-Transaction Business Activity [WS-T BA]. As we shall see, some features of OASIS BTP must also be taken into account, and the use of WS-Coordination must be made more precise.)
To set the scene, it's useful to look first at the relationship of BPEL processes to Web services. Figure 1 shows a BPEL process, a scope (sub-process) external Web services, and a WS client. Note that a BPEL process may receive messages from a Web service client. It can also call out to Web services, including other BPEL processes that offer Web service interfaces. BPEL processes can also execute nested scopes. (Scopes are effectively processes that happen to be initiated by a colocated process, and have direct access to control variables declared in enclosing scopes/processes.)
BTM features are superimposed on this basic structure. Figure 1 shows a BTM coordination service. (This might be "physically" located within a BPEL execution engine, or operate as a freestanding Web service.) It also shows the use of a business transaction context to connect or relate participant services and processes to a particular business transaction.
In looking at the integration of BTM into WS BPEL the technical committee is likely to consider four main areas:
In another article, I'll discuss the specific changes proposed for BPEL in order to address these questions.
Sidebar
BTM: Protocols and Standards
Full-scale BTM software needs to implement interoperable protocols that define three phases of any transactional interaction (whether integrating internal systems, or automating external trades and reconciliations).
A coordination protocol requires three related sub-protocols: a control protocol, which creates and terminates a coordination or transaction (present in BTP); a propagation protocol, which allows a unique transaction identity to be used to bind participating services to a coordination service (this sub-protocol is mostly defined by WS-Coordination); and an outcome protocol, which allows a coordination service to reliably transmit the instructions of a controlling application to the participants, even in the event of temporary process, processor or network failures. WS-T, BTP, and WS-TXM, contain very similar outcome protocols.
A coordination protocol is typically useful only if it is inherently reliable, utilizing persistent checkpoints and message retries to survive planned or unplanned interruptions in service.
Current BTM products and standards focus on Phase Two, Coordinated Outcome. WS BPEL may prove a useful vehicle for creating a new, wider view of the scope of business tranaction management.