Apache Pinot is a real-time distributed OLAP datastore for low-latency analytics over streaming and batch data. Core runtime roles: broker (query routing), server (segment storage/execution), controller (cluster metadata/management), minion (async tasks).
| Directory | Purpose |
|---|---|
pinot-broker |
Broker query planning and scatter-gather |
pinot-controller |
Controller APIs, table/segment metadata, Helix management |
pinot-server |
Server query execution, segment loading, indexing |
pinot-minion |
Background tasks (segment conversion, purge, etc.) |
pinot-common / pinot-spi |
Shared utils, config, and SPI interfaces |
pinot-segment-local / pinot-segment-spi |
Segment generation, indexes, storage |
pinot-query-planner / pinot-query-runtime |
Multi-stage query engine (MSQE) |
pinot-connectors |
External tooling to connect to Pinot |
pinot-plugins |
All Pinot plugins (input formats, filesystems, stream/batch ingestion, metrics, etc.) |
pinot-tools |
CLI and quickstart scripts |
pinot-integration-tests |
End-to-end validation suites |
pinot-distribution |
Packaging artifacts |
When locating a plugin implementation, consult kb/plugin-modules.md.
Use AGENTS.md for the build JDK, artifact bytecode targets, and scoped build/test commands. Use kb/skills/run-test.md for targeted tests and kb/skills/quickstart.md to start and verify a local cluster.
- Run
./mvnw spotless:applyto auto-format code. - Run
./mvnw checkstyle:checkto validate style. Checkstyle config is inconfig/checkstyle.xml. - Run
./mvnw license:formatto add license headers to new files. - Run
./mvnw license:checkto validate license headers. - Always use the Maven wrapper (
./mvnw) rather than a systemmvn.
- Add class-level Javadoc for new classes; describe behavior and thread-safety.
- Use Javadoc syntax supported by the owning module's configured source release; see AGENTS.md.
- Keep Apache 2.0 license headers on all new source files.
- Preserve backward compatibility across mixed-version broker/server/controller.
- Prefer imports over fully qualified class names (e.g., use
import com.foo.Barand refer toBar, notcom.foo.Barinline). - Prefer
List.of(),Set.of(), andMap.of()for non-null immutable collection literals. Checkstyle blocksCollections.emptyList(),Collections.emptySet(), andCollections.emptyMap(); useList.of(),Set.of(), andMap.of()instead. Do not add blanket bans forCollections.singleton*; use them only when an element/key/value argument is intentionally null becauseList.of(null),Set.of(null), andMap.of(...)with null keys or values throwNullPointerException. Before replacing empty collection factories, check whether the value flows to mutating callers. Seekb/code-review-principles.mdC7.12. - Prefer targeted unit tests; use integration tests when behavior crosses roles.
- Avoid deprecated APIs in new code. If you must reference one (e.g., for backward-compat serialization or to test the deprecated path), justify it with a comment.
- Do not include
Co-authored-bytrailers that reference AI tools (e.g., Claude, Copilot).- Why: These trailers propagate into squash-merge commits on GitHub, making the project history appear AI-authored rather than human-authored.
- Fix: Omit the
Co-authored-byline entirely when committing.
Before pushing, follow kb/skills/precommit.md for affected-module formatting, license headers,
checkstyle, and license validation. That procedure defines applicability, execution order, evidence reuse, and when
compiler warning checks are needed. Fix failures within the authorized scope; do not push with applicable checks failing.
Claude Code users can invoke /precommit.
- Query changes often touch broker planning and server execution; verify both.
- Segment/index changes usually live under
pinot-segment-localandpinot-segment-spi. - Config or API changes should update relevant configs and docs where applicable.
Review the changed behavior and relevant contracts before declaring implementation complete. Follow kb/agents/code-reviewer.md for requested full reviews and changes that benefit from independent domain review, especially compatibility, concurrency, security, and distributed state changes. Small changes can be reviewed directly. Reuse review evidence while the relevant diff and assumptions remain unchanged. Fix confirmed CRITICAL findings before proceeding; address MAJOR findings or explain why they can be deferred.
- This is a large multi-module Maven project. Building the entire project takes a long time — prefer building only the modules you need with
-pl <module> -am. - When running tests, use
-Dtest=ClassNameto run a specific test class rather than the full suite. - Mixed-version compatibility matters — do not break wire protocols or serialization formats without careful consideration.