MessagePackStreamingDecoder¶

A streaming-safe MessagePack decoder that never corrupts the underlying reader on insufficient data.

Unlike MessagePackDecoder, which assumes all data is available and will corrupt the reader's state on partial reads, this class peeks at the format byte and any length fields before consuming any bytes. If insufficient data is available, it returns NotEnoughData with zero bytes consumed, allowing the caller to append more data and retry.

Limits protect against denial-of-service attacks where a malicious payload claims enormous sizes for variable-length values or deeply nested containers. By default, conservative limits are applied (1 MB for str/bin/ext, 131,072 for array/map counts, 512 for container nesting depth). When a value exceeds its limit, next() returns LimitExceeded with zero bytes consumed.

The decoder automatically tracks container nesting depth. When next() returns a MessagePackArray or MessagePackMap, the depth counter increments. As the caller reads elements and containers are exhausted, the depth counter decrements automatically. Use depth() to inspect the current nesting level.

When validate_utf8 is true, decoded str values are validated for UTF-8 correctness. If a str value contains invalid UTF-8 byte sequences, next() returns InvalidUtf8 instead of the decoded string. The bytes have been consumed from the reader and decoding can continue. By default, validation is off for backward compatibility.

Usage:

// Default conservative limits, no UTF-8 validation:
let decoder = MessagePackStreamingDecoder
decoder.append(chunk1)
match decoder.next()
| let v: U32 => // got a value
| NotEnoughData => // need more data, append and retry
| LimitExceeded => // value too large or too deep, reject
| InvalidData => // corrupt stream, abort
end

// Custom limits:
let limits = MessagePackDecodeLimits(
  where max_str_len' = 4096,
        max_depth' = 16)
let decoder = MessagePackStreamingDecoder(limits)

// With UTF-8 validation:
let decoder = MessagePackStreamingDecoder(
  where validate_utf8' = true)
match decoder.next()
| InvalidUtf8 => // str value had invalid UTF-8
end

// No limits:
let decoder = MessagePackStreamingDecoder(
  MessagePackDecodeLimits.unlimited())

Container types (arrays and maps) return header objects (MessagePackArray / MessagePackMap) containing the element count. The caller is responsible for subsequently reading that many values.

class ref MessagePackStreamingDecoder

Constructors¶

create¶

[Source]

new ref create(
  limits: MessagePackDecodeLimits val = reference,
  validate_utf8': Bool val = false)
: MessagePackStreamingDecoder ref^

Parameters¶

limits: MessagePackDecodeLimits val = reference
validate_utf8': Bool val = false

Returns¶

MessagePackStreamingDecoder ref^

Public Functions¶

append¶

[Source]

Append data to the internal reader. Call this as chunks arrive.

fun ref append(
  data: (String val | Array[U8 val] val))
: None val

Parameters¶

data: (String val | Array[U8 val] val)

Returns¶

None val

depth¶

[Source]

Returns the current container nesting depth. Depth increases when next() returns MessagePackArray or MessagePackMap, and decreases automatically as elements are consumed.

fun box depth()
: USize val

Returns¶

USize val

next¶

[Source]

Attempt to decode the next MessagePack value.

Returns one of: - A MessagePackValue if a complete value was decoded - NotEnoughData if more bytes are needed (no bytes consumed) - LimitExceeded if the value exceeds a configured size limit (no bytes consumed) - InvalidData if the format byte is invalid (0xC1). The invalid byte is NOT consumed. The caller must stop calling next() after receiving InvalidData — the stream is corrupt and cannot be resynced.

fun ref next()
: (None val | Bool val | U8 val | 
    U16 val | U32 val | U64 val | 
    I8 val | I16 val | I32 val | 
    I64 val | F32 val | F64 val | 
    String val | Array[U8 val] val | MessagePackArray val | 
    MessagePackMap val | MessagePackExt val | MessagePackTimestamp val | 
    NotEnoughData val | InvalidData val | InvalidUtf8 val | 
    LimitExceeded val)

Returns¶

(None val | Bool val | U8 val | U16 val | U32 val | U64 val | I8 val | I16 val | I32 val | I64 val | F32 val | F64 val | String val | Array[U8 val] val | MessagePackArray val | MessagePackMap val | MessagePackExt val | MessagePackTimestamp val | NotEnoughData val | InvalidData val | InvalidUtf8 val | LimitExceeded val)

skip¶

[Source]

Advances past one complete MessagePack value without decoding it. For containers (arrays and maps), skips all contained elements.

Returns None on success, NotEnoughData if more bytes are needed (no bytes consumed), InvalidData if the format byte is invalid (no bytes consumed), or LimitExceeded if the number of values traversed exceeds the configured max_skip_values limit (no bytes consumed).

Like next(), a successful skip decrements the parent container's remaining element count when called inside a container.

fun ref skip()
: (None val | NotEnoughData val | InvalidData val | 
    LimitExceeded val)

Returns¶

(None val | NotEnoughData val | InvalidData val | LimitExceeded val)