Text-to-Bus-Bits Conversion Editor

The text-format conversion editor in GRAVITY enables the transformation of textual data contained in a single data bus bit into a structure compliant with GRAVITY’s internal data model—that is, into sets of records and bits.

The editor was designed for analyzing and parsing structured textual data, such as document fragments, EDI messages, data in custom formats, or hierarchically nested texts, but it can also be used to decompose standard formats such as XML or JSON. When building a conversion, you must provide a sample input text and define markers that:

define the nesting levels of the data structure (that is, how records are grouped),
and indicate which fragments of the text correspond to specific data (bus bits).

As a result of the conversion, the data is split into records according to the first level of nesting. Subsequent nesting levels are expanded into repeated records, enriched with additional bits derived from the deeper levels of the structure.

This processing makes it possible to automatically interpret texts with complex structures and transform them into a consistent, tabular GRAVITY data layout, ready for further processing by other operators.

The editor is available in the following operators in the GRAVITY system:

CONVERT FORMAT TO BITS

BASIC INFORMATION

Text-to–data-bus bit conversion is performed by entering arbitrary text provided by the person configuring the process. Markers are inserted into the text, allowing it to be interpreted during processing. You have access to group markers and bit markers.

Two types of markers are distinguished:

group (nesting) markers → allow groups (repeating blocks) to be defined in the data format being created.

A group can be understood as a block of text (including bit markers) that is interpreted as repeating in the converted text. During text processing, the operator interprets the text between group markers as a repeating fragment of the data format. You can think of a group as a record, and the bit markers within it as columns. If a group contains subgroups, the number of output bits on the data bus will be the sum of the bits from all groups, and the number of records will be the sum of records across the group tree (nesting levels). The values of bits from parent nesting levels will be duplicated across records originating from deeper nesting levels.

At least one group is required in the text.

#S-group_name# → group start marker, e.g. #S-G1#
#E-group_name# → group end marker, e.g. #E-G1#

bit (data) markers → define locations in the text where variable values appear (i.e., values that are not a fixed part of the text). Each such value results in a bit on the output bus. Bit markers consist of the bit names to be created on the output bus, enclosed by start and end delimiters. The default delimiters are { and }, but they can be changed in the operator that invokes the editor. An example of a bit marker is {bit_name}.

Because the conversion may be performed on textual data without a predefined format, the system strictly enforces the exact occurrence of characters in the text that are not enclosed in data markers. As a result, fixed text must match character by character (including whitespace), while text that may vary must be enclosed in a data marker (these may be empty markers without a name, e.g. {}), even if you do not intend to use the extracted data. It is not possible to convert text whose structure may change. Leaving variable text without data markers will cause a text conversion error that may be difficult to detect.

Each use of a bus bit in the text must appear within a group marker.

The conversion editor consists of a main editing area, where you configure the text format, and a preview showing the bits that will be created during text conversion.

The User format definition area is the main workspace where you build the conversion of text into bus bits. This area contains buttons that assist in building the conversion.

Load from template → Loads the text to be converted from the Text template field of the operator that invokes the editor.
Load file → Loads text from a file.
Create group → Inserts a data group marker. If you select text in the editor, the group marker will wrap the entire selected text. At least one data group is required in the defined format.
Ungroup → Removes the group marker from the selected text.
Insert empty token → Inserts an empty bus bit marker.
Show specificators → Displays basic bit formatting options. Bit formats can be dragged into the text editor area. A bit marker with the selected formatting will then be inserted. The bit name must be entered manually.
Save → Saves the format schema to the component that invoked the editor.
Save to file → Saves the format to a file.

TEXT CONVERSION EXAMPLE

For simplicity, let’s assume you want to convert data from text in XML format, but it can be any text format—for example, HTML, EDI, EPP, CSV, or even completely unstructured text.

Sample text

<?xml version="1.0" encoding="utf-8"?>
<root>
  <order>
    <order_id>202325</order_id>
    <order_year>2023</order_year>
    <order_nr>25</order_nr>
    <number>25</number>
    <order_date>2023-02-13T00:00:00</order_date>
    <customer>9</customer>
    <customer_name>Company Id 9</customer_name>
    <position>
      <position_no>1</position_no>
      <article_id>8</article_id>
      <article></article>
      <quantity>2</quantity>
      <unit_price>10</unit_price>
      <total>20</total>
      <currency>PLN</currency>
    </position>
    <position>
      <position_no>2</position_no>
      <article_id>9</article_id>
      <article></article>
      <quantity>2</quantity>
      <unit_price>50</unit_price>
      <total>100</total>
      <currency>PLN</currency>
    </position>
  </order>
  <order>
    <order_id>202320</order_id>
    <order_year>2023</order_year>
    <order_nr>20</order_nr>
    <number>10</number>
    <order_date>2023-02-22T00:00:00</order_date>
    <customer>9</customer>
    <customer_name>Company Id 9</customer_name>
    <position>
      <position_no>1</position_no>
      <article_id>2</article_id>
      <article>Krasnystaw Natural Yogurt</article>
      <quantity>10000</quantity>
      <unit_price>1.95</unit_price>
      <total>19500</total>
      <currency>PLN</currency>
    </position>
  </order>
  <order>
    <order_id>202319</order_id>
    <order_year>2023</order_year>
    <order_nr>19</order_nr>
    <number>9</number>
    <order_date>2023-02-22T00:00:00</order_date>
    <customer>5</customer>
    <customer_name>Company Id 5</customer_name>
    <position>
      <position_no>1</position_no>
      <article_id>6</article_id>
      <article>Onion</article>
      <quantity>3000</quantity>
      <unit_price>0.5</unit_price>
      <total>1500</total>
      <currency>PLN</currency>
    </position>
    <position>
      <position_no>2</position_no>
      <article_id>7</article_id>
      <article>Carrot</article>
      <quantity>1950</quantity>
      <unit_price>0.8</unit_price>
      <total>1560</total>
      <currency>PLN</currency>
    </position>
  </order>
</root>

The first step is to simplify the text that will be inserted into the editor.

Since this text contains order data and their line items, we can observe that its structure is repetitive. Therefore, it is sufficient to use a single element from each level. Remember to include every structural element.

The simplified text will look as follows:

<?xml version="1.0" encoding="utf-8"?>
<root>
  <order>
    <order_id>202325</order_id>
    <order_year>2023</order_year>
    <order_nr>25</order_nr>
    <number>25</number>
    <order_date>2023-02-13T00:00:00</order_date>
    <customer>9</customer>
    <customer_name>Company Id 9</customer_name>
    <position>
      <position_no>1</position_no>
      <article_id>8</article_id>
      <article></article>
      <quantity>2</quantity>
      <unit_price>10</unit_price>
      <total>20</total>
      <currency>PLN</currency>
    </position>
  </order>
</root>

You can of course use the Load from template or Load file button to load the text and simplify it directly in the editor.

The next step is to define the group representing the order header and its data. To do this, select the text corresponding to the header definition and insert a group using the Create group button.

Data markers will be inserted, and you will see the first data group on the bit map.

Next, select and insert bit markers into the header data. The bit marker must be placed exactly at the position where the data is expected to appear in the processed text.

You can select the position of the value in the text and use the Insert empty token, button, then enter the name of the bit marker.

Perform this operation for all data in the text, regardless of whether you actually need the values or not.

Note that the bit map shows which bits will make up the bus.

Perform the same steps for the order line items—mark the item group and insert bit markers at the positions where the values occur.

In this way, the text conversion is complete. There is no remaining element in the text that is variable and has not been enclosed in a bit marker. The output bus will consist of the bits shown in the bit map, and the number of records on the data bus will be equal to the total number of order line items (group 2). The bits representing the order header will have their values duplicated across all corresponding line items.