PIP 130: Apply redelivery backoff policy for ack timeout #13528

PIP 130: Apply redelivery backoff policy for ack timeout

• Status: DISCUSS
• Author: Penghui Li
• Issue:
• Pull Request:
• Mailing list discussion:
• Release: 2.10.0

Motivation

PIP 106 https://github.com/apache/pulsar/wiki/PIP-106%3A-Negative-acknowledgment-backoff
introduced negative acknowledgment message redelivery backoff which allows users to achieve
more flexible message redelivery delay time control. But the redelivery backoff policy only
apply to the negative acknowledgment API, for users who use ack timeout to trigger the message
redelivery, not the negative acknowledgment API, they can't use the new features introduced by
PIP 106.

So the proposal is to apply the message redelivery policy for the ack timeout mechanism.
Users can specify an ack timeout redelivery backoff, for example, apply an exponential backoff
with 10 seconds ack timeout:

client.newConsumer()
    .ackTimeout(10, TimeUnit.SECOND)
    .ackTimeoutRedeliveryBackoff(
        ExponentialRedeliveryBackoff.builder()
            .minDelayMs(1000)
            .maxDelayMs(60000).build());
    .subscribe();

The message redelivery behavior should be:

Goal

Add an API to the Java Client to provide the ability to specify the ack timeout message redelivery
backoff and the message redelivery behavior should abide by the redelivery backoff policy.

API Changes

1. Change NegativeAckRedeliveryBackoff to RedeliveryBackoff, so that we can use the
   MessageRedeliveryBackoff for both negative acknowledgment API and ack timeout message redelivery.

2. Change NegativeAckRedeliveryExponentialBackoff to ExponentialRedeliveryBackoff, and add multiplier
   for RedeliveryExponentialBackoff with default value 2.

ExponentialRedeliveryBackoff.builder()
.minDelayMs(1000)
.maxDelayMs(60000)
.multiplier(5)
.build()

3. Add ackTimeoutRedeliveryBackoff method for the ConsumerBuilder:

client.newConsumer()
    .ackTimeout(10, TimeUnit.SECOND)
    .ackTimeoutRedeliveryBackoff(
        ExponentialRedeliveryBackoff.builder()
            .minDelayMs(1000)
            .maxDelayMs(60000).build());
    .subscribe();

Compatibility and migration plan

Since the negative acknowledgment message, redelivery backoff has not been released yet,
so we can modify the API directly.

Tests plan

• Verify the introduced multiplier of ExponentialRedeliveryBackoff
• Verify the ack timeout message redelivery work as expected