Pulsar

<!-- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. -->

Pulsar is a distributed pub-sub messaging platform with a very flexible messaging model and an intuitive client API.

Learn more about Pulsar at https://pulsar.apache.org

Main features

• Horizontally scalable (Millions of independent topics and millions of messages published per second)
• Strong ordering and consistency guarantees
• Low latency durable storage
• Topic and queue semantics
• Load balancer
• Designed for being deployed as a hosted service:
  • Multi-tenant
  • Authentication
  • Authorization
  • Quotas
  • Support mixing very different workloads
  • Optional hardware isolation
• Keeps track of consumer cursor position
• REST API for provisioning, admin and stats
• Geo replication
• Transparent handling of partitioned topics
• Transparent batching of messages

Repositories

This repository is the main repository of Apache Pulsar. Pulsar PMC also maintains other repositories for components in the Pulsar ecosystem, including connectors, adapters, and other language clients.

• Pulsar Core

Helm Chart

• Pulsar Helm Chart

Ecosystem

• Pulsar Adapters

Clients

• .NET/C# Client
• C++ Client
• Go Client
• NodeJS Client
• Python Client
• Reactive Java Client

Dashboard & Management Tools

• Dekaf UI
• Pulsar Manager

Website

• Pulsar Site

CI/CD

• Pulsar CI

Archived/Halted

• Pulsar Connectors (bundled as pulsar-io)
• Pulsar Translation
• Pulsar SQL (Pulsar Presto Connector) (bundled as pulsar-sql)
• Ruby Client

Pulsar Runtime Java Version Recommendation

Note:

When using Java versions, it is recommended to use a recent version of a particular Java release (17 or 21) with the most recent bug fixes and security patches. For example, the JVM bug JDK-8351933 can cause stability issues in Pulsar. JDK-8351933 was fixed in Java 17.0.17+ and Java 21.0.8+. Pulsar Docker images come with the most recent Java version at the time of release.

When using the Pulsar Java client, it is recommended to set specific system properties and JVM options to allow optimal performance.

pulsar ver >= 4.1 and master branch

| Component | Java Version | |-----------------|:------------:| | Broker | 21 | | Functions / IO | 21 | | CLI | 17 or 21 | | Java Client | 17 or 21 |

Docker image Java runtime: 21

3.3 <= pulsar ver <= 4.0

| Component | Java Version | |-----------------|:---------------:| | Broker | 21 | | Functions / IO | 21 | | CLI | 17 or 21 | | Java Client | 8, 11, 17 or 21 |

Docker image Java runtime: 21

2.10 <= pulsar ver <= 3.0

| Component | Java Version | |----------------|:-------------:| | Broker | 17 | | Functions / IO | 17 | | CLI | 17 | | Java Client | 8 or 11 or 17 |

Docker image Java runtime: 17

2.8 <= pulsar ver <= 2.10

| Component | Java Version | |-----------------|:------------:| | Broker | 11 | | Functions / IO | 11 | | CLI | 8 or 11 | | Java Client | 8 or 11 |

pulsar ver < 2.8

| Component | Java Version | |-------------|:------------:| | All | 8 or 11 |

Build Pulsar

Requirements

• JDK

  | Pulsar Version | JDK Version | |------------------|:--------------------------------------------------------------------------------:| | master and 4.0+ | JDK 21 | | 2.11 + | JDK 17 | | 2.8 / 2.9 / 2.10 | JDK 11 | | 2.7 - | JDK 8 |

• zip

There is also a guide for setting up the tooling for building Pulsar.

Note:

This project includes a Gradle Wrapper so no separate Gradle installation is needed. Use ./gradlew on Linux/macOS and gradlew.bat on Windows.

Build

Compile and assemble:

$ ./gradlew assemble

Compile and assemble individual module:

$ ./gradlew :pulsar-broker:assemble

Minimal build (core modules only, skips external connectors and tiered storage handlers)

$ ./gradlew assemble -PcoreModules

Run Unit Tests:

$ ./gradlew test

Run Individual Unit Test:

$ ./gradlew :pulsar-client-original:test --tests "ConsumerBuilderImplTest"

Run Selected Test packages:

$ ./gradlew :pulsar-broker:test --tests "org.apache.pulsar.*"

Start standalone Pulsar service:

$ bin/pulsar standalone

Check https://pulsar.apache.org for documentation and examples.

Build custom docker images

The commands used in the Apache Pulsar release process can be found in the release process documentation.

Here are some general instructions for building custom docker images:

• Java 21 is the recommended JDK version since branch-4.0.

The following command builds the docker images apachepulsar/pulsar-all:latest and apachepulsar/pulsar:latest:

$ ./gradlew docker-all

After the images are built, they can be tagged and pushed to your custom repository. Here's an example of a bash script that tags the docker images with the current version and git revision and pushes them to localhost:32000/apachepulsar.

image_repo_and_project=localhost:32000/apachepulsar
pulsar_version=$(src/get-pulsar-version.sh)
gitrev=$(git rev-parse HEAD | colrm 10)
tag="${pulsar_version}-${gitrev}"
echo "Using tag $tag"
docker tag apachepulsar/pulsar-all:latest ${image_repo_and_project}/pulsar-all:$tag
docker push ${image_repo_and_project}/pulsar-all:$tag
docker tag apachepulsar/pulsar:latest ${image_repo_and_project}/pulsar:$tag
docker push ${image_repo_and_project}/pulsar:$tag

Setting up your IDE

Read https://pulsar.apache.org/contribute/setup-ide for setting up IntelliJ IDEA or Eclipse for developing Pulsar.

Documentation

Note:

For how to make contributions to Pulsar documentation, see Pulsar Documentation Contribution Guide.

Contact

Mailing lists

• The mailing lists are the primary contact for the Apache Pulsar project.
• Your email to the mailing list might be placed in a moderation queue if you haven't joined the mailing list by subscribing before posting.

Note

Please note that security-related issues or concerns should not be reported in public channels. Follow the instructions in the Security Policy to contact the ASF Security Team and the Apache Pulsar PMC directly.

| Name | Scope | Subscribe | Unsubscribe | Archives | |:---------------------------