Rule of Thumb for Configuring Timeout in Oracle SOA Infrastructure

Introduction

While designing integration solutions that involve multiple distributed systems, it is often critical to understand the transactional capability of the various participants and configure the transaction settings appropriately. One such critical setting that often leads to unrecoverable errors in Oracle SOA infrastructure is the Transaction Timeout setting.

This topic has been discussed in isolation in various blogs, but here we’ll provide a holistic view of the various transaction timeout settings that are part of the SOA infrastructure, the relationship between each of them and how they affect each other in the wider scheme of things.

We have pulled this summary information together after working on numerous real world client projects and consolidating information from various sources including Oracle Support and Oracle documentation.This consolidated information has now been incorporated into the Rubicon Red Reference Architecture, used on all our projects.

The Sum of its Parts

Since an integration solution spans across multiple distributed applications, proper analysis should occur to ensure the transaction timeout setting is set to an appropriate value across each system, avoiding unexpected and inconsistent distributed transaction outcomes. Even though each web service invocation is primarily impacted by the connection and read timeout settings, the encompassing transactional behaviour of the overall business service can be impacted by additional timeout settings within the Oracle SOA infrastructure. They are:

  • DISTRIBUTED_LOCK_TIMEOUT configured in the database
  • WebLogic application service domain JTA timeout
  • SOA Data Source XA transaction timeout configuration
  • Transaction timeout setting across the various application component(s) that participate in the distributed transaction
  • soa-infra application EJB Timeout Settings
  • syncMaxWaitTime configuration under the BPEL infrastructure
DISTRIBUTED_LOCK_TIMEOUT Setting

The DISTRIBUTED_LOCK_TIMEOUT parameter specifies the number of seconds a distributed transaction waits for a lock at the database level. The timeout value set for this parameter must be greater than any other timeout value set for the transaction participants.

The value for the DISTRIBUTED_LOCK_TIMEOUT parameter can be verified by running the SQL command "show parameter distributed_lock_timeout;" in the database.

WebLogic Domain JTA Timeout Setting

The timeout settings for an "active" transaction after which it is rolled back. The default transaction timeout is 30 seconds, which can be too low for most cases. The rule that must be applied is:

DISTRIBUTED_LOCK_TIMEOUT > JTA Timeout
Stuck Thread Timeout  > JTA Timeout
SOA Data Source XA Transaction Timeout Setting

For all the configured XA data sources, the XA Transaction Timeout property must be 'enabled' and the XA Transaction Timeout must be set to '0' seconds. Setting the XATransactionTimeout to '0' seconds, will default to the global transaction timeout.

soa-infra Application EJB Timeout Setting

A general rule to follow is ensuring that the WebLogic Server JTA timeout (either global, specific to an EJB, or for individual transactions) is set to a lower value than the shortest timeout value configured/set for any participating XA resources (e.g., XA Transaction Timeout for Oracle XA JDBC connections). Failing to do this can cause an unexpected and inconsistent distributed transaction outcome (i.e. a participating XA Resource timing out before WebLogic Server JTA as the distributed transaction coordinator).

Whenever EJB Container Managed Transactions with JTA is used, the EJB Transaction Timeout overrides the JTA Transaction Time Out. The rule that must be applied is:

DISTRIBUTED_LOCK_TIMEOUT > EJB Timeout > JTA Timeout
Oracle BPEL Infrastructure syncMaxWaitTime Setting

The syncMaxWaitTime property applies to durable processes and should always be lower than the soa-infra EJB transaction timeouts, otherwise it will get interrupted by the EJB timeout resulting in unexpected behaviour.
The consolidated rule that must be applied is:

DISTRIBUTED_LOCK_TIMEOUT > EJB Timeout > JTA Timeout > syncMaxWaitTime
Web Services Invocation Timeout Setting – The Impact of intermediaries

Considering a business service can, in turn, invoke multiple services, the timeout setting configured between the interactions with each of the intermediary services will impact the overall timeout behaviour of the business service. The rule that must be applied on the read timeout of the business service is:

Business Service HTTP Read-Timeout > Sum of (HTTP Connection 
+ Read Timeout of the intermediaries)

Summary

Service Oriented Architecture aims at providing integration across loosely coupled components, and thus requiring the need to address non-functional requirements like timeouts. Timeout settings at each layer of the integration solution should be carefully configured by analysing their interdepencies. This article consolidate the various timeout settings required in Oracle SOA infrastructure and provides the rationale and best practice around how their should be configured.