The TEXT RECOGNIZE operator is used to recognize and standardize text values based on an external or automatically generated dictionary. Its primary purpose is to eliminate different variants of the same value that may appear in input data as a result of human errors, language differences, abbreviations, typos, or inconsistent naming.
The operator works by:
- comparing the value of a given text bit with a reference dictionary,
- if no direct match is found, identifying the most likely dictionary equivalent (based on textual similarity),
- replacing the original value with its standardized version.
The operator uses a so-called universal dictionary, which can be:
- explicitly defined by the user—for example, by specifying a table that maps values,
- automatically built—based on an analysis of value frequency in the data, allowing dominant values to be identified and the dictionary to be built dynamically.
TEXT RECOGNIZE is particularly useful in projects where data comes from different systems or sources, or is entered manually—wherever textual consistency and minimization of interpretation errors are required.
An example of a GRAVITY project using the TEXT RECOGNIZE operator:
OPERATOR CONFIGURATION
During configuration, you specify the column to be recognized and the recognition action variant.
If you select the recognition variant based on a fixed dictionary (Standardized with dictionary), you need to additionally specify the dataset passed to the END OF PATH (the Temporary output field), which contains the universal dictionary, as well as the column linked to the analyzed column (Temporary output column).
If you select the dictionary-free variant (Standardized without dictionary), the dictionary will be built based on the data from the selected column. If a given character string appears more than three times in the column, it will be added to the generated dictionary.