To ensure data consistency in the event of a project execution failure, the GRAVITY system includes a built-in module for managing database connections. This module maintains separate connection registries for each process being executed.
The GRAVITY system ensures (except when the Separate transaction mode or Transaction mode=Use separate transaction is explicitly set on operators) that all database queries are executed within a transaction. In the event of a project execution error, the system rolls back all database changes made during the project execution. If the project completes successfully, all changes are committed.
If the database allows it, the GRAVITY system initiates transactions with the Read uncommitted isolation level, enabling so-called dirty reads by other software — for example, DLL libraries invoked in the DLL CALL FUNCTION operator or services called in the
REST API CALL operator.
GENERAL METHOD OF TRANSACTION MANAGEMENT
Each GRAVITY system operator that uses a database connection communicates with the database connection management module before processing begins. During execution, the operator queries the management module to check whether the database connection defined for it already exists in the process connection registry.
If it does, the module returns the existing connection.
If the connection does not exist, the management module creates a new connection using the configuration provided by the operator, starts a transaction on that connection, adds it to the connection registry, and then returns it to the operator.
Once the operator receives the connection, it uses it to execute the SQL command on the database.
A connection registry is maintained for each project execution. After processing is completed, a decision is sent to the connection management module on how to handle the database transactions associated with the connections listed in the registry of the executed process.
Depending on the project’s completion status, the module receives an instruction to either commit or roll back the transactions (confirming or reverting the changes made in the databases). Finally, the database connections are closed, and the connection registry for the executed project is cleared.
This behavior means that if a GRAVITY project contains multiple operators using the same database connection, all those operators will operate within a single transaction. An exception may occur for the CALL SQL operator, which can be executed in a separate transaction (when the Separate transaction option is selected).
When a project uses multiple databases, the system creates separate transactions for each database used in the project and maintains them until the project is completed. Depending on the project’s outcome, these transactions are either committed or rolled back.
OPERATION OF THE SEPARATE TRANSACTION MODE
Enabling the Separate transaction mode available on the CALL SQL operator causes the connection management module not to pass an existing connection from the connection registry to the operator, but instead to always create a new connection with a new transaction, which is completed (committed or rolled back) after data processing on the operator finishes. Such a connection is not added to the connection registry and will not be used by other operators.
As a result, changes made to the database by the CALL SQL operator cannot be rolled back if the project fails due to an error in later steps of the process. However, an error occurring during the execution of an operator with Separate transaction enabled will cause the changes made by that operator, as well as the changes made through all connections present in the process connection registry, to be rolled back.
OPERATION OF THE TRANSACTION MODE
The Transaction mode can be configured in the INPUT OTHER PROJECT and
OTHER PROJECT CALL operators, which allow another project to be invoked within the currently executing project.
The Transaction mode can have two values:
- Use transaction from main project → Using this mode causes the database connection registry to be passed to the invoked process. This means that operators in the invoked project use the database connections defined in the parent project. If a connection does not exist in the registry, it will be created and added to it. After the child process finishes, the transaction is not completed. The connection registry returns to the main process together with the connections created in the child process. Committing or rolling back the transaction after the main process finishes also applies to the transactions created in the child processes.
- Use separate transaction → Using this mode causes the child process to behave as if it were invoked as a separate process. The connection registry is not passed to the invoked process, and the connection management module maintains a separate connection registry for the child process invocation. After the child process finishes, the transactions of that process are committed or rolled back depending on the process completion status.
OTHER DATA CONSISTENCY ISSUES
When designing processes, it should be taken into account that a project may use operators that invoke external services (INPUT REST API,
REST API CALL ) or external code. (
DLL CALL FUNCTION,
CALL EXE). Changes introduced by invoked services or external code in databases or other parts of IT environments are not controlled by the GRAVITY environment. Such changes cannot be rolled back if a project execution error occurs at a later stage of processing.