Materialize v0.73

v0.73.0

Sources and sinks

• Private preview. Allow propagating comments in materialized views to the Avro schema of Kafka sinks, as well as manually specifying comments using the new [KEY|VALUE] DOC ON [TYPE|COLUMN] <identifier> connection option.

  Example:

  CREATE TABLE t (c1 int, c2 text);
  COMMENT ON TABLE t IS 'materialize comment on t';
  COMMENT ON COLUMN t.c2 IS 'materialize comment on t.c2';
  
  CREATE SINK avro_sink
    IN CLUSTER my_io_cluster
    FROM t
    INTO KAFKA CONNECTION kafka_connection (TOPIC 'test_avro_topic')
    KEY (c1)
    FORMAT AVRO USING CONFLUENT SCHEMA REGISTRY CONNECTION csr_connection
    (
      DOC ON TYPE t = 'top-level comment for avro record in both key and value schemas',
      KEY DOC ON COLUMN t.c1 = 'comment on column only in key schema',
      VALUE DOC ON COLUMN t.c1 = 'comment on column only in value schema'
    )
    ENVELOPE UPSERT;
  

  
   **Key schema:**
   ```json
   {
     "type": "record",
     "name": "row",
     "doc": "this is a materialized view",
     "fields" : [
       {"name": "a", "type": "string", "doc": "this is column a"},
       {"name": "b", "type": "string"}
     ]
   }{
     "type": "record",
     "name": "row",
     "doc": "top-level comment for avro record in both key and value schemas",
     "fields": [
       {
         "name": "c1",
         "type": [
           "null",
           "int"
         ],
         "doc": "comment on column only in key schema"
       }
     ]
   }
  

  
   **Value schema:**
  
   ```json
   {
     "type": "record",
     "name": "envelope",
     "doc": "top-level comment for avro record in both key and value schemas",
     "fields": [
       {
         "name": "c1",
         "type": [
           "null",
           "int"
         ],
         "doc": "comment on column only in value schema"
       },
       {
         "name": "c2",
         "type": [
           "null",
           "string"
         ],
         "doc": "materialize comment on t.c2"
       }
     ]
   }
  

Bug fixes and other improvements

• Allow the value of the SASL MECHANISM option for Kafka connections to be specified in any case style. Previously, Materialize only accepted uppercase case style (as required by librdkafka).