We use cookies to ensure that we give you the best experience on our website. By continuing to use the website you agree for the use of cookies for better website performance and a personalized experience.
Accept

Latest
Apache Druid Release – GitHub & Downloads.

Get installation supportDownload latest release

GitHub Release Notes Summary

34.0.0
33.0.0
32.0.0
31.0.0
30.0.0
29.0.0
28.0.0

Key Features

Key features icon
1. Query Context Management:
  • Added support for SET statements in queries (via console or API).
  • Context parameters like timeout, useCache, and engine can now be defined directly in SQL workflows.
2. Cloning Historicals (Experimental):
  • Coordinators can configure clones of Historicals to mirror assignments from source servers.
  • Useful for rolling upgrades and seamless failover scenarios.
  • New query context parameter cloneQueryMode introduced (prefer, exclude, include).
  • New Coordinator APIs added for clone management.
3. Embedded Kill Tasks on the Overlord (Experimental):
  • Kill tasks now run directly on Overlords without consuming task slots.
  • Faster execution and optimized metadata queries.
  • Supports incremental cleanup with reduced locking overhead.
  • New metrics introduced to monitor kill operations.
4. Preferred Tier Selection for Brokers:
  • Brokers can prioritize Historicals from specific tiers (for example, same availability zone).
  • Retains fallback capability if preferred tier is unavailable.
5. Multi-Stream Supervisors (Experimental):
  • Multiple supervisors can ingest into the same datasource using unique IDs.
  • Requires enabling useConcurrentLocks for supervisor specs.
  • Provides more flexible streaming ingestion setups.
6. Dart Engine Enhancements:
  • Dart now uses the standard /druid/v2/sql endpoint instead of its own.
  • Supports querying real-time tasks by default, configurable via includeSegmentSource.
  • Compatible with both synchronous and asynchronous queries.
  • Introduces new Dart- and MSQ-specific query metrics.

Improvements

Improvements icon
1. Web Console Enhancements:
  • Task filtering by error and tiered replications to offline tiers.
  • Improved SQL and JSON autocompletion.
  • Console now relies on Overlord APIs for segment management.
2. Ingestion:
  • Batch and streaming ingestion concurrency improved.
  • useMaxMemoryEstimates config removed; accurate estimates are now default.
  • Streaming ingestion auto-detects maximum columns to merge.
3. Querying:
  • json_merge() updated to be SQL-compliant when handling nulls.
  • Added big decimal aggregations in MSQ engine.
  • More reliable query handling when segments temporarily disappear.
  • groupBy queries now replace topN in some SQL scenarios for better performance.
4. Cluster Management & Monitoring:
  • Configurable subtask timeout (subTaskTimeoutMillis).
  • New APIs for clone status and broker configuration sync.
  • Enhanced task cancellation for MSQ tasks.
  • Additional metrics for cloning, streaming ingestion, Kafka consumers, kill tasks, and MSQ/Dart queries.
  • Improved logging noise reduction and Prometheus metric coverage.
5. Dependency Updates:
  • Multiple dependencies upgraded, including Kafka (3.9.1), AWS SDK (1.12.784), Curator (5.8.0), Jackson (2.18.4), Guava (32.1.3), SLF4J (2.0.16), and Netty (4.1.122.Final).

Bug Fixes

Bug fixes icon
1. Query Execution & Reliability:
  • Fixed partial result handling when Historicals temporarily lack segments.
  • Corrected SQL functions (MV_OVERLAP, MV_CONTAINS) to align with native filters.
  • Prevented SQL queries that cannot be parsed from being logged or emitting metrics.
2. Ingestion:
  • Coordinators can configure clones of Historicals to mirror assignments from source servers.Seekable stream supervisors (Kafka, Kinesis, Rabbit) can no longer change input streams mid-operation, preventing inconsistent behavior.
3. Console & Cluster Management:
  • Improved auto-completion accuracy and API reliability.
  • Enhanced task log streaming for Indexers.
  • Various fixes to ensure task cancellation, auditing, and API status consistency.
4. Metrics & Logging:
  • Fixed inconsistencies in task/metric emission.
  • Fixed rolling log rotation handling.
  • Adjusted logging levels (e.g., cancellation moved from WARN → INFO).

Details

Details icon
For detailed information, refer to the full release notes.

Key Features

1. Turbo Segment Loading:
  • Coordinator Dynamic Config now supports turboLoadingNodes, speeding up segment loads on Historicals.
  • Trades loading speed for some query performance if overused.
2. Overlord Compaction APIs (Experimental):
  • Enhanced support to manage compaction statuses and configs via new endpoints.
  • Works even if compaction supervisors are disabled.
3. Scheduled Batch Ingestion (Experimental):
  • Supports cron-based scheduling using either Unix or Quartz syntax.
  • Allows automated REPLACE queries for batch ingestion using MSQ engine.
4. Improved S3 Uploads:
  • Uses AWS S3 Transfer Manager by default, reducing upload time.
  • Configurable via druid.storage.transfer.* settings.
5. Web Console Enhancements:
  • Syntax highlighting for MERGE INTO.
  • Timezone support in Explore view.
  • Markdown export for results.
  • Multi-select filters and status suggestions added.

Improvements

1. Querying:
  • GROUP BY and ORDER BY now handle NULL values properly.
  • Performance boost for large argument functions (e.g., complex CASE statements).
  • /results API can prompt browser downloads using the filename query parameter.
2. Ingestion:
  • skipRestartIfUnmodified=true to prevent unnecessary supervisor restarts.
  • Streaming ingestion auto-detects maximum columns to merge.
3. Cluster Management & Monitoring:
  • Compaction Supervisors: Replaces older runtime props with new config-based approach.
  • Segment Metadata Caching: Improves performance via druid.manager.segments.useCache.
  • Kill Task Interval: Now defaults to a max of 30 days per task for performance.
4. Metrics & Monitoring:
  • Kafka & Kinesis Lag Metrics: Track ingestion delay in milliseconds.
  • Custom Prometheus Histogram Buckets now configurable.
  • New metrics for segment cache transactions and ingestion throughput (bytes processed).
5. Kubernetes Support:
  • Namespace isolation for pods.
  • Lazy loading of pod templates.
  • Larger payloads supported with HDFS storage.
  • Task pod name customization available.

Bug Fixes

Bug Fixes and Deprecations:
  • Metadata Queries now return maxIngestedEventTime for improved real-time visibility.
  • Improved query distribution when using druid.broker.balancer.type=connectionCount.
  • Deprecated Config Cleanup: Several outdated properties have been officially removed.
  • Docker now registers services by canonical hostname by default in Kubernetes.

Details

For detailed information, refer to the full release notes.

Key Features

1. New Overlord APIs:
  • Migrated segment management from Coordinator to Overlord, deprecating old Coordinator APIs.
  • Introduced new APIs for marking segments as used/unused at various levels.
2. Realtime Query Processing for Multi-Value Strings:
  • Improved real-time query performance and consistency.
  • Fixed topN query inconsistencies by aligning real-time and published segment behaviors.
3. Join Hints in MSQ Task Engine Queries:
  • Enabled SQL hints for granular control over JOIN query types.
  • Improved query performance with per-join-level optimization.
4. Java Support Updates:
  • Removed Java 8 support.
  • Deprecated Java 11; Java 17 is now recommended.
5. Hadoop-Based Ingestion Deprecation:
  • Deprecated Hadoop ingestion in favor of SQL-based ingestion.
6. Web Console Enhancements:
  • Enhanced Explore view and segment timeline.
  • Added timezone picker, UNNEST autocomplete, and resizable panels.

Improvements

1. Ingestion Enhancements:
  • CSV/TSV Parsing introduced tryParseNumbers for configurable numeric input handling.
  • Memory Optimization reduced buffer usage in non-query tasks.
  • Streaming Efficiency added maxColumnsToMerge for controlled segment merging.
2. Query Optimization:
  • Window Query Updates deprecated legacy fields for window queries.
  • Query Prioritization implemented segment-age-based prioritization.
  • Error Reporting enhanced feedback for incomplete queries.
3. Cluster Management:
  • Metadata IO Reduction optimized segment allocation to reduce overhead.
  • Autoscaling Enhancements improved efficiency in task publishing.
  • Leadership Recovery accelerated Overlord recovery following ZooKeeper restarts.
4. Data Management:
  • Compaction Improvements enabled sorting of non-time columns.
  • Schema Handling improved consistency in datasource schemas.
5. Metrics & Monitoring:
  • GroupBy Query Metrics added detailed tracking for merge buffer usage and query spills.
  • CgroupV2 Monitoring introduced CPU, disk, and memory monitoring.
  • Ingestion & Query Metrics expanded statsd metrics for performance tracking.
6. Extensions & Compatibility:
  • Delta Lake enhanced decimal type support and snapshot filtering.
  • gRPC Queries introduced a gRPC API for SQL and native queries.
  • Iceberg integrated AWS Glue Iceberg catalog support.

Bug Fixes

  • Fixed incorrect handling of null values in real-time segment processing.
  • Resolved query failures due to incorrect boolean logic handling.
  • Improved handling of JSON and SQL-based ingestion headers.
  • Addressed segment publishing delays when re-submitting supervisors.
  • Enhanced resilience to ZooKeeper-induced service leadership changes.
  • Fixed real-time segment metric reporting inconsistencies.
  • Various UI and usability bug fixes in the web console.

Details

For detailed information, refer to the full release notes.

Key Features

1. Compaction:
  • Compaction supervisors introduced for more flexible scheduling and better visibility.
  • GA support for concurrent append and replace operations.
  • MSQ task engine-based auto-compaction for higher performance.
2. Window Functions:
  • Window functions are now generally available in both native and MSQ engines.
  • No longer require special query context flags.
3. Projections (Experimental):
  • Pre-aggregated segment data that reduces computation and I/O at query time.
  • Currently supports JSON ingestion with MSQ and Dart engines.
  • Designed to significantly accelerate query performance.
4. Dart Query Engine (Experimental):
  • New distributed asynchronous runtime for high-complexity, ad-hoc queries.
  • Supports joins, subqueries, CTEs, and high-cardinality group-bys.
  • Provides low latency without needing external engines like Spark or Presto.
5. Data Lake Integrations:
  • Delta Lake: Support for complex types (structs, arrays, maps) and snapshot versions.
  • Iceberg: Support for REST catalogs and case-sensitive column handling.
6. Storage:
  • Compression support for complex metric columns (lz4, zstd, etc.).
  • Flexible segment sorting by dimensions other than __time for storage optimization.

Improvements

1. Web Console:
  • Graph visualization for query stages and new CPU counters.
  • Copy query results as SQL for reproducibility.
  • Explore view enhancements: editable source queries, stateful URLs, on-the-fly measures.
  • Added support for Kinesis and Delta Lake ingestion.
  • JSON display improvements, datasource search, and better schema discovery.
2. Ingestion:
  • Optimized broadcast datasource loading.
  • Expanded format support: CSV in Kafka, Delta complex types.
  • Faster S3 writes for MSQ output.
  • Overlord performance improvements and Hadoop on Kubernetes support.
3. Querying:
  • Support for querying cold datasources with centralized schemas.
  • Added SQL DIV function and improved equality filter behavior.
  • Guardrails for window queries and enhanced error messaging.
  • Improved query filtering, vectorization, and aggregation consistency.
4. Cluster Management & APIs:
  • Cluster-level compaction API.
  • Configurable string multi-value handling.
  • New APIs for early streaming task handoff and viewing task locks.
  • Guardrail for max task payload size.
5. Metrics & Monitoring:
  • Expanded Prometheus metrics coverage.
  • New Cgroup CPU and disk usage metrics.
  • Segment loading rate tracking with moving averages.
  • Added subquery and indexer task failure metrics.
6. Extensions & Kubernetes:
  • Dynamic pod template selection for MiddleManager-less ingestion.
  • Iceberg REST catalog and Delta snapshot ingestion.
  • Reduced logging overhead in multiple extensions.

Bug Fixes

1. Ingestion & Tasks:
  • Fixed failures during rolling upgrades.
  • Corrected Parquet reader column filtering.
  • Addressed compact segment NPEs and Kafka CSV ingestion issues.
  • JSON display improvements, datasource search, and better schema discovery.
2. Query Execution:
  • Fixed array comparison logic and subquery filtering.
  • Resolved window function partitioning and aggregation inconsistencies.
  • Corrected frame writer capacity issues for large queries.
  • Improved error messages for exceeded row limits.
3. Web Console & UX:
  • Fixed NPEs in numeric columns.
  • Corrected tombstone segment visibility in console.
  • Fixed JSON value formatting in query results.
4. Cluster & Storage:
  • Fixed metadata query performance for unused segments.
  • Improved tombstone refresh logic and segment reuse.
  • Corrected hostname resolution in Docker startup scripts.

Details

For detailed information, refer to the full release notes.

Key Features

1. Ingestion & Concurrency:
  • Concurrent append and replace: safely run streaming appends alongside replace/compaction tasks via concurrent locks—no ingestion lag or task failures.
2. Query Capabilities:
  • Grouping on complex columns and nested arrays: supported in both native queries and MSQ.
  • Window functions in MSQ: run window functions in the MSQ task engine without requiring a GROUP BY.
3. Coordination & Storage:
  • Removed ZooKeeper-based segment loading: Coordinator improvements and HTTP-based segment loading streamline operations and performance.
4. MSQ & Cloud Outputs:
  • Google Cloud Storage export: write MSQ query results directly to GCS via google().
5. Ecosystem Extensions:
  • RabbitMQ ingestion (community): manage RabbitMQ indexing tasks with exactly-once delivery and full partitioning through super streams.

Improvements

1. Web Console:
  • Enhanced Supervisors view with dynamic queries and additional columns.
  • Better search across tables and columns.
  • Smoother sampling flows for Kafka input format and lookups.
2. General Ingestion:
  • Azure input source: ingest from multiple storage accounts.
  • More robust memory management and richer logging for S3, Azure, and other sources.
3. Querying:
  • Dynamic table append with TABLE(APPEND(...)).
  • SCALAR_IN_ARRAY function for concise array membership checks.
  • Optimized filter partitioning for AND filters to improve performance.
4. Cluster Management:
  • Faster retrieval of active task status from in-memory caches.
5. Data Management:
  • Tuned default Coordinator parameters for better efficiency.
6. Metrics & Monitoring:
  • New metrics for unused segment counts.
  • Custom Kafka emitter dimensions.
  • Prometheus emitter enhancements.

Bug Fixes

1. Web Console:
  • Corrected task duration display and Azure icon rendering.
  • Improved UX in manual capabilities mode and stabilized the query timer.
2. Data Management & Ingestions:
  • Fixed handling of segment IDs and ingestion task reports.
  • Resolved issues with concurrent replace intervals during compaction.
  • Better serialization and clearer logs across several task types.
3. Query Processing:
  • Addressed null handling in boolean logic and stability with high-cardinality data.
  • Fixed MSQ task report inconsistencies and improved error messages for failures.
4. Extensions & Kubernetes:
  • Larger ingestion payload support for Microsoft Azure.
  • Improved CPU core configuration for Kubernetes Peons.

Details

For detailed information, refer to the full release notes.

Key Features

1. MSQ Enhancements:
  • Export statements (experimental): write query results from the MSQ task engine to external destinations.
  • MSQ task engine-based auto-compaction for higher performance.
2. SQL Extensions:
  • PIVOT and UNPIVOT operators (experimental): transform rows into columns and vice versa.
  • Range support in window functions (experimental): includes both strict and non-strict modes.
3. Joins & Aggregations:
  • Arbitrary join conditions for INNER joins, improving compatibility with tools like Tableau.
  • First and Last aggregators now support numeric types (double, float, long).

Improvements

1. Ingestion & Concurrency:
  • Concurrent append and replace (experimental): automatically determines task lock type.
  • System fields available in input sources (e.g., S3 URIs).
  • Changed storage format for columns containing only empty/null arrays to ARRAY<LONG>.
  • Improved kill tasks and segment allocation processes.
2. Audit & Logging:
  • REST API endpoints now log audit events for better monitoring and security.
  • Enhanced JSON parser error logging.
3. Web Console:
  • Support for array types in ingestion wizards.
  • Enhanced Explore view with improved time chart and auto-granularity.
  • Faster query detail archive loading and upgraded lookup dialog.
4. Querying:
  • Added IPv6_MATCH and JSON_QUERY_ARRAY SQL functions.
  • Improved ANY_VALUE function and native arrayContainsElement filter.
  • Better lookup filter performance and nested JSON column indexing.
5. Data Management:
  • Tombstone segments now have zero core partitions, simplifying removal.
  • Improved segment retrieval performance.
6. Extensions:
  • Prometheus emitter improvements:
    — Deletes metrics from push gateway on task shutdown.
    — Configurable shutdown delay for smoother operation.

Bug Fixes

1. Ingestion & Tasks:
  • Fixed issues in segment allocation and ingestion task reporting.
  • Resolved race conditions affecting ingestion stability.
2. Query Execution:
  • Improved handling of nested array fields and JSON functions.
  • Fixed errors in SQL planning for filter expressions.
  • Enhanced error handling for multiple query scenarios.
3. Data Management:
  • Addressed problems in segment retrieval workflows.
  • Improved handling of null values and timestamp extraction.

Details

For detailed information, refer to the full release notes.

Key Features

1. Enhanced SQL Compliance:
  • Improved SQL engine for analytics.
  • Graduated async queries, deep storage querying, and UNNEST to GA.
  • Added UNION ALL support in multi-stage query engine (MSQ).
2. Ingestion System Enhancements:
  • Default nested column ingestion for stability.
  • Multi-topic Kafka ingestion with regex matching.
3. Experimental Features:
  • Introduced window functions in SQL.
  • Experimental support for concurrent append and replace.

Improvements

1. Async Query and Deep Storage:
  • MSQ engine fetches real-time data for improved query results.
  • Added Azure BLOB storage support for fault-tolerant storage.
2. Array and UNNEST Enhancements:
  • SQL-based ingestion supports true arrays.
  • General availability of SQL UNNEST for flexible reporting.
3. Performance Optimizations:
  • Improved segment file handling for better query performance.
4. ANSI SQL Support Upgrade:
  • Upgrade to Calcite 1.35 for improved query planning.
  • Enabled NULL support and ANSI SQL standard behaviors.

Bug Fixes

  • Various bug fixes in query planning and correctness.
  • Addressed issues with compaction tasks and late arrival data.

Details

For detailed information, refer to the full release notes.

Download the latest stable version of Apache Druid from official sources

Need help with your installation or upgrade?

Book free consultationSee pricing

Installation

If you are new to Apache Druid:

Before beginning, make sure your system requirements for the installation are met:

  • Linux, Mac OS X, or other Unix-like OS (Windows is not supported).
  • Java 8u92+, 11, or 17.
  • Python 3 (preferred) or Python 2.

You will need a workstation or virtual server with 6 GiB of RAM.

Proceed with Apache Druid installation option that fits your needs.

If you would like to upgrade Apache Druid:

Review upgrade notes and incompatible changes on GitHub before you start your Druid upgrade.

Community

Connect and engage with the Druid community across various channels to stay informed, seek assistance, and connect with users, committers, and enthusiasts:

Explore Deep.BI on GitHubFollow Deep.BI on LinkedIn

Expert Apache Druid Support by Deep.BI

Need professional help deploying or managing Apache Druid? 
Deep.BI offers:

Advantage icon
Installation and upgrade assistance
Advantage icon
Optimized performance tuning
Advantage icon
Troubleshooting and monitoring
Advantage icon
Support responses in under 24 hours

License

The Apache License Version 2.0 is an open-source license that grants users a perpetual, worldwide, non-exclusive, no-charge, royalty-free, and irrevocable copyright license to reproduce, modify, distribute, and sublicense the work and its derivatives.

Let's help you

with installation or upgrade

Contact information:
Email:
druid@deep.bi
Phone:
+1 ‪(415) 966-1931‬
Click to reveal number

Get in touch and we'll reach back out to you soon!

Contact information
Email:
druid@deep.bi
Phone:
+1 ‪(415) 966-1931‬
Click to reveal number