Search

Home

Home

Managing Database Transactions in a Project

To ensure data consistency in the event of a project execution failure, the GRAVITY system includes a built-in module for managing database connections. This module maintains separate connection registries for each process being executed.

The GRAVITY system ensures (except when the Separate transaction mode or Transaction mode=Use separate transaction is explicitly set on operators) that all database queries are executed within a transaction. In the event of a project execution error, the system rolls back all database changes made during the project execution. If the project completes successfully, all changes are committed.

If the database allows it, the GRAVITY system initiates transactions with the Read uncommitted isolation level, enabling so-called dirty reads by other software — for example, DLL libraries invoked in the DLL CALL FUNCTIONDLL CALL FUNCTION operator or services called in the REST API CALL REST API CALL operator.

GENERAL METHOD OF TRANSACTION MANAGEMENT

Each GRAVITY system operator that uses a database connection communicates with the database connection management module before processing begins. During execution, the operator queries the management module to check whether the database connection defined for it already exists in the process connection registry.

If it does, the module returns the existing connection.

If the connection does not exist, the management module creates a new connection using the configuration provided by the operator, starts a transaction on that connection, adds it to the connection registry, and then returns it to the operator.

Once the operator receives the connection, it uses it to execute the SQL command on the database.

A connection registry is maintained for each project execution. After processing is completed, a decision is sent to the connection management module on how to handle the database transactions associated with the connections listed in the registry of the executed process.

Depending on the project’s completion status, the module receives an instruction to either commit or roll back the transactions (confirming or reverting the changes made in the databases). Finally, the database connections are closed, and the connection registry for the executed project is cleared.

This behavior means that if a GRAVITY project contains multiple operators using the same database connection, all those operators will operate within a single transaction. An exception may occur for the CALL SQLCALL SQL operator, which can be executed in a separate transaction (when the Separate transaction option is selected).

When a project uses multiple databases, the system creates separate transactions for each database used in the project and maintains them until the project is completed. Depending on the project’s outcome, these transactions are either committed or rolled back.

icon
The above description applies when: - the Separate transaction mode is not enabled for the CALL SQLCALL SQL operator, - the Transaction mode for the INPUT OTHER PROJECTINPUT OTHER PROJECT and OTHER PROJECT CALL OTHER PROJECT CALL operators is set to Use separate transaction.

OPERATION OF THE SEPARATE TRANSACTION MODE

Enabling the Separate transaction mode available on the CALL SQLCALL SQL operator causes the connection management module not to pass an existing connection from the connection registry to the operator, but instead to always create a new connection with a new transaction, which is completed (committed or rolled back) after data processing on the operator finishes. Such a connection is not added to the connection registry and will not be used by other operators.

As a result, changes made to the database by the CALL SQLCALL SQL operator cannot be rolled back if the project fails due to an error in later steps of the process. However, an error occurring during the execution of an operator with Separate transaction enabled will cause the changes made by that operator, as well as the changes made through all connections present in the process connection registry, to be rolled back.

icon
A new connection with a separate transaction is used to execute the SQL statement for all records on the input data bus of the CALL SQLCALL SQL operator.
icon
Use the Separate transaction mode with caution and only in situations where no other option is available.

OPERATION OF THE TRANSACTION MODE

The Transaction mode can be configured in the INPUT OTHER PROJECTINPUT OTHER PROJECT and OTHER PROJECT CALL OTHER PROJECT CALL operators, which allow another project to be invoked within the currently executing project.

icon
For the OTHER PROJECT CALL OTHER PROJECT CALL operator, the project can be invoked as many times as there are records on the data bus.

The Transaction mode can have two values:

  • Use transaction from main project → Using this mode causes the database connection registry to be passed to the invoked process. This means that operators in the invoked project use the database connections defined in the parent project. If a connection does not exist in the registry, it will be created and added to it. After the child process finishes, the transaction is not completed. The connection registry returns to the main process together with the connections created in the child process. Committing or rolling back the transaction after the main process finishes also applies to the transactions created in the child processes.
  • Use separate transaction → Using this mode causes the child process to behave as if it were invoked as a separate process. The connection registry is not passed to the invoked process, and the connection management module maintains a separate connection registry for the child process invocation. After the child process finishes, the transactions of that process are committed or rolled back depending on the process completion status.
  • icon
    There can be as many child process invocations as there are records on the data bus (using the OTHER PROJECT CALL OTHER PROJECT CALL operator), and each invocation is treated as a separate process execution.
    icon
    If an error occurs in the main process or in one of the child invocations, it will not be possible to roll back the changes made by child processes that have already been executed.

OTHER DATA CONSISTENCY ISSUES

When designing processes, it should be taken into account that a project may use operators that invoke external services (INPUT REST APIINPUT REST API, REST API CALL REST API CALL ) or external code. (DLL CALL FUNCTIONDLL CALL FUNCTION, CALL EXECALL EXE). Changes introduced by invoked services or external code in databases or other parts of IT environments are not controlled by the GRAVITY environment. Such changes cannot be rolled back if a project execution error occurs at a later stage of processing.