Werk #12391: HW/SW Inventory History: Improve handling of changed inventory table rows

Previously table rows were compared successively (if tables had the same length) which might have led to unexpected delta tree results. Moreover the comparison of two tables (and other operations like merging or filtering) was expensive.

In order to speed up these operations and dump above delta tree heuristic, the internal table structure changes.

Tables of old or legacy inventory plugins are migrated on-the-fly.

If migrated tables are compared during delta tree calculations there are no "changed" entries anymore. Instead compared rows are treated as "new", "old" or "identical".

Delta tree calculations of tables generated from newer inventory plugins are more exact and don't need the heuristic anymore.

Technical details:

Previous delta tree heuristic:

Assume the following rows which are compared.

old_rows = [
    {
        "col1": "value 11",
        "col2": "old value 12",
        ...
    },
    {
        "col1": "value 21",
        "col2": "value 22",
        ...
    },
]

new_rows = [
    {
        "col1": "value 01",
        "col2": "value 02",
        ...
    },
    {
        "col1": "value 11",
        "col2": "new value 12",
        ...
    },
    {
        "col1": "value 21",
        "col2": "value 22",
        ...
    },
]

First both lists were filtered, ie. find rows which

• are in both lists,
• are only in the old list,
• are only in the new list.

Then if the remaining lists have the same length both lists are compared successively, ie. compare first old row with first new row, compare second old row with second new row, and so on. These comparisons generated "changed" entries in the delta tree.

If the remainings lists have not the same length then all remaining old rows are treated as {[removed}} and all remaining new rows are treated as "new".

The delta tree result of the above example is:

• one removed row,
• one identical row,
• two new rows.

Obviously this is not what we expect. The result should be:

• one identical row,
• one changed row entry: "old value 12" -> "new value 12",
• one new row.

Advantages: Especially the above filtering of both lists is expensive. We also want to make these operations more reliable. For this we need a row identifier which can be calculated from the "key_columns" declared in newer inventory plugins:

TableRow(
    path=["path", "to", "node"],
    key_columns = {
        "col1": "value 11",
        ...
    },
    inventory_columns = {
        "col2": "value 12",
        ...
    },
    ...
)

Then the internal table structure looks like:

rows = {
    ('value 11', ...): {
        "col1": "value 11",
        "col2": "value 12",
        ...
    },
    ('value 21', ...): {
        "col1": "value 21",
        "col2": "value 22",
        ...
    },
}

To the list of all Werks