INPUT MONGODB

The INPUT MONGODB operator allows you to retrieve data from a MongoDB database, a non-relational NoSQL database system. The data source consists of documents stored in the selected collection, which the operator transforms into a GRAVITY data stream and places on the data bus as records.

The operator allows you to precisely define data retrieval conditions by applying filters (e.g., in JSON format or using the wizard) and specifying result sorting based on any collection keys. This enables you to limit and organize the retrieved data directly at the MongoDB query level.

For each retrieved document, a data-bus record is created, and the bit (column) structure is automatically generated based on the structure of the BSON documents.

Additionally, the operator supports the use of project parameters and global variables to dynamically build queries or filtering conditions. This enables seamless integration of MongoDB with other systems and supports the implementation of complex ETL and analytical scenarios.

SELECTING AND CREATING A MONGODB CONNECTION

The INPUT MONGODB operator requires a connection to a MongoDB database. If you have already created a connection, you can select it from the list of connections on the Connection tab, in the Connection field.

After selecting a connection, the connection definition fields will be filled in according to the selected connection’s configuration.

You can modify the connection details by clicking the Edit button.

Remember that if the connection is already used by other operators, they will also operate using the modified connection.

If you have not yet defined your MongoDB connection in the system, you can add one at any time by clicking the Add button.

After entering the details of the new connection, you can test it using the Test connection button and then save it in the system.

After saving, the connection will be available in the system under the name you entered in the Name field.

A detailed description of creating a connection can be found here.

SELECTING A DATA COLLECTION

The operator retrieves data from a specific data collection. You can select the collection on the Properties tab, in the Collection name field.

If the list is empty or a collection is missing, use the refresh button (marked in red in the screenshot above).

To build the operator’s output data bus, you need to provide the structure of the documents stored in the collection. You can supply it in three ways:

• by manually pasting a sample document into the Bson Template,
• if the collection is not empty, click Load from database — the operator will retrieve the document structure directly from the database,
• by loading a document from a file using the Load from file button.

After providing a sample document, you can generate a data tree, which will be used to create the output data bus. To do this, click the Generate tree button.

The system will generate a tree of document fields based on the sample document’s structure. The tree will be available on the Node selection tab and will contain all fields found in the document.

Based on the selected fields, the output data bus will be created. You can select or deselect fields at any time.

Use the Test button to verify that the connection works correctly and to preview the retrieved data.

If the BSON document contains nested objects and you select fields from both the main object and the nested objects, the data bus will contain as many records as there are nested objects in total, and the values of the bits corresponding to the main object will be duplicated for each nested element.

FILTERING DATA

The operator allows you to define filters that determine which documents will be retrieved from the selected collection (on the Filter tab). Filters are defined as a tree structure, with information about how conditions are combined (AND, OR) at each level and between nested filter levels.

Use the Add group button to create additional nested filter levels. The Add expression button adds a filter expression at the current level.

Clicking Generate query shows how the filter was constructed.

If you are an advanced user and want to build the filter in query mode, clear the Use tree filter mode toggle.

SORTING AND LIMITING DATA

The Sort and limit tab lets you sort data by the selected fields. You can sort by multiple fields; the data will be sorted according to their order and sort direction.

Clicking the Add sort button adds a new sort condition. To choose a field, click the field area in the sort row, and do the same to set the sort direction.

If you enter a value in the Limit field, the number of retrieved documents will be restricted to that value.