Mapping Arbitrary Data to OTLP AnyValue

Status: Development

This document defines how to map (convert) arbitrary data (e.g. in-memory objects) to OTLP’s AnyValue.

The mapping is needed when OpenTelemetry needs to convert a value produced outside OpenTelemetry into a value that can be exported using an OTLP exporter, or otherwise be converted to be used inside OpenTelemetry boundaries. Example use cases are the following:

• In the Logging SDKs, to convert values received from logging libraries into OpenTelemetry representation.
• In the Collector, to convert values received from various data sources into pdata internal representation.

Converting to AnyValue

AnyValue is capable of representing primitive and structured data of certain types.

Implementations that have source data in any form, such as in-memory objects or data coming from other formats that needs to be converted to AnyValue SHOULD follow the rules described below.

Primitive Values

Integer Values

Integer values which are within the range of 64 bit signed numbers [-2^63..2^63-1] SHOULD be converted to AnyValue’s int_value field.

Integer values which are outside the range of 64 bit signed numbers SHOULD be converted to AnyValue’s string_value field using decimal representation.

Enumerations

Values, which belong to a limited enumerated set (e.g. a Java enum), SHOULD be converted to AnyValue’s string_value field with the value of the string set to the symbolic name of the enumeration.

If it is not possible to obtain the symbolic name of the enumeration, the implementation SHOULD map enumeration values to AnyValue’s int_value field set to the enum’s ordinal value, when such an ordinal number is naturally obtainable.

If it is also not possible to obtain the ordinal value, the enumeration SHOULD be converted to AnyValue’s bytes_value field in any manner that the implementation deems reasonable.

Floating Point Values

Floating point values which are within the range and precision of IEEE 754 64-bit floating point numbers (including IEEE 32-bit floating point values) SHOULD be converted to AnyValue’s double_value field.

Floating point values which are outside the range or precision of IEEE 754 64-bit floating point numbers (e.g. IEEE 128-bit floating bit values) SHOULD be converted to AnyValue’s string_value field using decimal floating point representation.

String Values

String values which are valid UTF-8 sequences SHOULD be converted to AnyValue’s string_value field.

String values which are not valid Unicode sequences SHOULD be converted to AnyValue’s bytes_value with the bytes representing the string in the original order and format of the source string.

Byte Sequences

Byte sequences (e.g. Go’s []byte slice or raw byte content of a file) SHOULD be converted to AnyValue’s bytes_value field.

Composite Values

Array Values

Values that represent ordered sequences of other values (such as arrays, vectors, ordered lists, slices) SHOULD be converted to AnyValue’s array_value field. String values and byte sequences are an exception from this rule (see above).

The rules described in this document should be applied recursively to each element of the array.

Associative Arrays With Unique Keys

Values that represent associative arrays with unique keys (also often known as maps, dictionaries or key-value stores) SHOULD be converted to AnyValue’s kvlist_value field.

If the keys of the source array are not strings, they MUST be converted to strings by any means available, often via a toString() or stringify functions available in programming languages. The conversion function MUST be chosen in a way that ensures that the resulting string keys are unique in the target array.

The value part of each element of the source array SHOULD be converted to AnyValue recursively.

For example a JSON object {"a": 123, "b": "def"} SHOULD be converted to

AnyValue{
    kvlist_value:KeyValueList{
        values:[
            KeyValue{key:"a",value:AnyValue{int_value:123}},
            KeyValue{key:"b",value:AnyValue{string_value:"def"}},
        ]
    }
}

The rules described in this document should be applied recursively to each value of the associative array.

Associative Arrays With Non-Unique Keys

Values that represent an associative arrays with non-unique keys where multiple values may be associated with the same key (also sometimes known as multimaps, multidicts) SHOULD be converted to AnyValue’s kvlist_value field.

The resulting kvlist_value field MUST list each key only once and the value of each element of kvlist_value field MUST be an array represented using AnyValue’s array_value field, each element of the array_value representing one value of source array associated with the given key.

For example an associative array shown in the following table:

SHOULD be converted to:

AnyValue{
    kvlist_value:KeyValueList{
        values:[
            KeyValue{
                key:"abc",
                value:AnyValue{array_value:ArrayValue{values[
                    AnyValue{int_value:123}
                ]}}
            },
            KeyValue{
                key:"def",
                value:AnyValue{array_value:ArrayValue{values[
                    AnyValue{string_value:"foo"},
                    AnyValue{string_value:"bar"}
                ]}}
            },
        ]
    }
}

The rules described in this document should be applied recursively to each value of the associative array.

Sets

Unordered collections of unique values (such as Java Sets, C++ sets, Python Sets) SHOULD be converted to AnyValue’s array_value field, where each element of the set becomes an element of the array.

The rules described in this document should be applied recursively to each value of the set.

Other Values

Any other values not listed above SHOULD be converted to AnyValue’s string_value field if the source data can be serialized to a string (can be stringified) using toString() or stringify functions available in programming languages.

If the source data cannot be serialized to a string then the value SHOULD be converted AnyValue’s bytes_value field by serializing it into a byte sequence by any means available.

If the source data cannot be serialized neither to a string nor to a byte sequence then it SHOULD by converted to an empty AnyValue.

Empty Values

If the source data has no type associated with it and is empty, null, nil or otherwise indicates absence of data it SHOULD be converted to an empty AnyValue, where all the fields are unset.

Empty values which has a type associated with them (e.g. empty associative array) SHOULD be converted using the corresponding rules defined for the types above.