Snowflake-Labs/snowflake-event-table-to-otlp-exporter v0.2.0

v0.2.0

Repository: Snowflake-Labs/snowflake-event-table-to-otlp-exporter

Tag: v0.2.0

Published: 2026-08-11T13:20:09Z

Prerelease: no

Release notes:

Breaking Changes

• [Snowpark Deployments] The stored procedure now has 7 arguments instead of 6 (options JSON appended). You will need to drop and recreate it (instead of updating in-place). 15_snowpark_post-deploy.sql proc-grant DDL, smoke-test CALL, and task CALL updated; 99_cleanup.sql DROP PROCEDURE signature updated accordingly.
• [Developer Tooling] Dev SQL files (50_dev_telemetry_gen.sql) moved under dev/sql/ to separate development/test resources from production deployment scripts.

Known Issues

• A stream delta with many rows sharing the same RESOURCE_ATTRIBUTES produces a single OTLP document that can exceed librdkafka's 1 MB message.max.bytes limit, causing MSG_SIZE_TOO_LARGE and stalling the pipeline. A fix for this is planned for the next release.

Added

• `options` JSON parameter (7th arg) on PUSH_STREAM_TO_KAFKA (default '{}') — allows passing secondary/debug config settings without exposing further top-level args. See [Snowflake Deployment](docs/deploy-snowflake.md#procedure-options-json) for more details. Similar config values are exposed as ENVs in the local runner as described in [Local Config](docs/local-configuration.md).
• PrivateLink egress support for the Snowpark path via new ET2OTLP_NET_RULE_TYPE session variable in sql/00_config.sql (default HOST_PORT; set to PRIVATE_HOST_PORT for PrivateLink NLB brokers). 10_snowpark_pre-deploy.sql now uses it to build the network rule.
• `ET2OTLP_KAFKA_SSL_INSECURE` env var (local runner) and "ssl_insecure": true in the proc options JSON — disables TLS certificate verification for dev/test brokers where the broker cert does not match the DNS name (e.g. MSK behind an NLB). Loud WARNING at startup; never enable in production.
• `ET2OTLP_KAFKA_DEBUG` env var (local runner) and "kafka_debug": "..." in the proc options JSON — enables librdkafka wire-level debug contexts (broker,security,protocol, etc.), validated against the librdkafka 2.2.0 context list at startup.
• Kafka producer error_cb — routes hard librdkafka failures (TLS/SASL/handshake) into Python logging at ERROR (visible in the account event table for the Snowpark path even without DEBUG).
• Additional debug logging throughout pipeline.py, kafka.py, local_runner.py, and sproc_handler.py (stream gate, transaction boundaries, route stats, Kafka delivery per-topic counts, delivery callbacks).
• Startup log.debug in local runner for full Kafka config (client_id, topic_prefix, ssl_insecure, kafka_debug, SASL details).
• Producer-config secret redaction (sasl.password, sasl.username, ssl.key.password) via a new shared logging_config module (redact_producer_config).
• A [script](dev/scripts/check_snowflake_channel_versions.py) to help manage PyPI vs Anaconda dependency names + versions, wired into .pre-commit-config.yaml.

Changed

• dev/docker-compose.yaml was renamed to dev/compose.yaml to match modern naming standards.
• local_runner.py startup logging replaced inline basicConfig call with configure_logging() from the new logging_config module.
• sproc_handler.py startup log now includes bootstrap server, security protocol, and options value; error log now records exception type alongside message.
• produce_all now logs per-topic message counts at INFO before flush and logs delivery success at DEBUG via _on_delivery.
• snowflake-connector-python promoted from dev-only to a direct runtime dependency (pinned >=4.6.0,<=4.6.0) for explicit channel alignment.
• Bump dependencies to Snowflake Anaconda channel max:
• snowflake-snowpark-python <=1.53.0
• snowflake-connector-python <=4.6.0
• Documentation updates and improvements.

Fixed

• RBAC Fix: EXECUTE TASK / EXECUTE MANAGED TASK grants moved from SVC_RL to RUNTIME_AR (access-role-first convention); smoke-test CALL now runs as SVC_RL to validate least-privilege instead of ACCOUNTADMIN.
• RBAC isolation when the source is the built-in SNOWFLAKE.TELEMETRY.EVENTS, the required SNOWFLAKE.EVENTS_ADMIN application role is now granted to a dedicated ET2OTLP_EVENTS_ADMIN_AR access role (not READ_AR). This keeps the elevated privilege (which also carries TRUNCATE/DELETE) out of the general read path. Custom event tables are unaffected.

Full Changelog: https://github.com/Snowflake-Labs/snowflake-event-table-to-otlp-exporter/compare/v0.1.0...v0.2.0