OUTPUT MONGODB

The OUTPUT MONGODB operator enables writing data to a MongoDB database. At the operator’s input, there is a data stream in the form of a data bus, from which the data is transformed into documents compliant with the MongoDB collection model. The operator allows the processed records to be efficiently saved to a specified collection in the MongoDB database.

The operator supports various data types and format conversions in accordance with the MongoDB document structure.

Use cases of the OUTPUT MONGODB operator:

• persistent storage of data processing results in the form of documents,
• integration of the GRAVITY system with MongoDB databases,
• enabling further use of the data by applications that rely on MongoDB,
• flexible mapping of data bus bits to document fields within a collection.

The operator is essential wherever data produced by processing workflows in GRAVITY must be permanently stored and made available in a MongoDB NoSQL environment.

Example of a GRAVITY project using the OUTPUT MONGODB operator:

SELECTING AND CREATING A MONGODB CONNECTION

The OUTPUT MONGODB operator requires a connection to a MongoDB database. If you already have a connection defined, you can select it from the available connections in the Connection field on the Connection tab.

After selecting a connection, the connection definition fields will be populated according to the selected connection’s configuration.

You can modify the connection details using the Edit button.

Keep in mind that if the connection is already used by another operator, that operator will also use the modified connection.

If you do not yet have a MongoDB connection defined in the system, you can always add one using the Add button.

After entering the details of the new connection, you can test it using the Test connection button and save it in the system.

After saving, the connection will be available in the system under the name specified in the Name field.

A detailed description of how to create the connection can be found here.

OPERATOR CONFIGURATION

In the first step, you must define the structure of the document that will be sent to the MongoDB database.

Select the DEFINE BSON SCHEME option on the Document format tab, which helps you clearly create the document structure based on the data bus.

You build the format by using drag-and-drop actions from the Busbar area to the BSON nodes area—placing nodes where you want them—and vice versa, or by using the navigation buttons.

The BSON format editor is similar to the JSON format editor. For more information, see the JSON Format Editor chapter.

When configuring a BSON object, you must mark the data bus bit that acts as the primary key in the input data for the OUTPUT MONGODB operator (when Action for document = Always add new document is selected, marking a primary key is not required).

The data bus bit designated as the primary key must be included in the BSON object being created. By clicking the bit, you can set the Is PK field in the Element properties area.

The primary key is used when adding data to a collection in Action for document = Add new document or update existing mode.

There is a specific case where the primary key of the created document has the same name as the field used by MongoDB as its unique primary key (_id). This field stores a value of type ObjectId. If you name a document field _id and it is not marked as the primary key, it will be ignored and not added to the document. If it is marked as the primary key and represents a MongoDB server–generated ObjectId, that value will be used as the document’s key, and no new key will be generated. If it contains data that does not match the ObjectId type, an error will be returned.

The next step is to define where and how the data will be written to the MongoDB database (the Properties tab).

• Collection name → Selects the collection to which documents will be written. You can add a new collection to the database by manually entering its name in the field.
• Action for document → Defines how data is written to the database. You have two options to choose from, and the way duplicate documents are handled depends on the option you select:
• Always add new document

Use this option when you want every document created by the operator to be saved as a new document in the MongoDB database.

The operator writes every document generated based on the BSON format configuration to the target database. Primary key configuration in the BSON editor is not required.

• Add new document or update existing

Use this option when you want to modify existing documents or add new ones that do not yet exist in the MongoDB database.

The operator determines, based on the bits that make up the primary key of the created document, whether the document already exists in the selected MongoDB collection.

→ If so, it updates the existing document.

→ If not, it adds a new document to the database.

• Action for the same document → An option available only for Add new document or update existing. This option allows you to permit modifications to existing documents by selecting Pass, or prevent them by selecting Skip.
• Pass → The operator passes all documents to the database for processing.
• Skip → The operator passes to the database only those documents that do not yet exist in the database (based on primary key fields).
• Bit for result object id → The data bus bit to which the ObjectId key generated by the MongoDB database (the document’s _id field) is written.