Materialize v0.86

v0.86

Sources and sinks

Add support for handling batched events in the webhook source via the new JSON ARRAY format.

CREATE SOURCE webhook_source_json_batch IN CLUSTER my_cluster FROM WEBHOOK
BODY FORMAT JSON ARRAY
INCLUDE HEADERS;

POST webhook_source_json_batch
[
  { "event_type": "a" },
  { "event_type": "b" },
  { "event_type": "c" }
]

SELECT COUNT(body) FROM webhook_source_json_batch;
----
3

Decrease memory utilization for unpacking Kafka headers. Use the new map_build function to turn all headers exposed via INCLUDE HEADERS into a map, which makes it easier to extract header values.

SELECT
    id,
    seller,
    item,
    convert_from(map_build(headers)->'client_id', 'utf-8') AS client_id,
    map_build(headers)->'encryption_key' AS encryption_key,
FROM kafka_metadata;

 id | seller |        item        | client_id |    encryption_key
----+--------+--------------------+-----------+----------------------
  2 |   1592 | Custom Art         |        23 | \x796f75207769736821
  3 |   1411 | City Bar Crawl     |        42 | \x796f75207769736821

SQL

Add support for new SQL functions:

Function Description

map_build Builds a map from a list of records whose fields are two elements, the first of which is text.

map_agg Aggregate keys and values (including nulls) as a map.

Function	Description
`map_build`	Builds a map from a list of records whose fields are two elements, the first of which is `text`.
`map_agg`	Aggregate keys and values (including nulls) as a map.

Bug fixes and other improvements

Mitigate queue saturation is Kafka sinks (#24871).
Fix a correctness issue with subqueries that referred to ungrouped columns when columns of the same name existed in an outer scope (#24354).
Fix casts from interval to time for large negative intervals (#24795).
Prevent INSERTs with table references in VALUES in transactions (#24697).