Draft support for 8.8 arrays - #3076

* draft support for 8.8 arrays

* - mark API as experimental
- add keyspace notification tests
- tidying
- docs

* clarify how last-items interacts with ring buffers

* fix CI netfx compilation

* use ValuePairInterleavedProcessorBase to ensure that jagged vs flat doesn't impact us (RESP2 vs RESP3)

* make life even easier

* add Array to signature prefix list

* stabilize hotkeys CI

* fix last-minute ARGREP result change

* WIP v3 (#3035)

* WIP v3 merge attempt

* fix refs/externs

* nits

* ascii hash cleanup

* remove unused local function

* v3 remove pipelines.sockets.unofficial (#3042)

* remove Pipelines.Sockets.Unofficial

* wip

* remove the last traces of Pipelines.Sockets.Unofficial

* fix deps

* WIP: deps

* fix build deps

* rev GitHubActionsTestLogger

* - fix delegate [UnsafeAccessor] usage
- implement Mono pattern

* merge (incomplete)

* merge

* missed a sync-context await

* fix missing configureawait usage

* missing VT consumption

* checking CI fail

* versioning

* don't test WriteMode.Sync/Async yet

* failing test for #3048

* upstream #3049 into v3

* release notes

* fix broken config test

* re-add channel handling (merge)

* optimize pub/sub check for server-defined channels

* prefer WriteBulkString("..."u8) => WriteRaw(RESPu8)

* optimize CommandMap by pre-generating all the RESP chunks

* Move CI to 8.8 rc1 (#3079)

* update CI to 8.8 rc1

* and the dockerfile

* Draft support for 8.8 arrays (#3076)

* draft support for 8.8 arrays

* - mark API as experimental
- add keyspace notification tests
- tidying
- docs

* clarify how last-items interacts with ring buffers

* fix CI netfx compilation

* use ValuePairInterleavedProcessorBase to ensure that jagged vs flat doesn't impact us (RESP2 vs RESP3)

* make life even easier

* add Array to signature prefix list

* stabilize hotkeys CI

* fix last-minute ARGREP result change

* CI: graceful skip if DEBUG not available

* Alternative AwaitableMutex implementation for netfx (#3081)

* implement AwaitableMutex for netfx

* optimize successful sync take path

* words

* optimize async path for immediate acquisition

* re-implement TCP keep-alive

* default RESP3 everywhere (#3084)

* default RESP3 everywhere

* Retain <v6 RESP2 logic; default to v6

* move ship file entries

* improve logging for ReadNotificationAsync

* stabilize CI for RetryPolicyFailureCases

* merge

* broken merge (unshipped.txt)

* add all the files

* build fix

* merge drift

* CI: allow more time for Echo

* use Task machinery for teardown test

* Drag benchmark from respite branch

* typo

* "classic"

* BufferedStreamWriter threading tests and fixes

* more clarifications on BufferedStreamWriter

* change default mode

* investigating parse bug (Garnet SPOP?)

* categorise all simulate-connection-failure tests; start working through failures

* more simulate-failure tests

* update `aoc` test expectation (fast-fail assert in test connect)

* optimize ConfigurationOptions fields

* Update CI workflow to include v3 branch for packaging

* Fix missing Me() key usage

* be explicit about test TFMs

* merge the sync and async write implementations into a shared implementation that can transition from sync to async if needed

* 3.0 release

* counting is hard

IncrexMessageBase (#3065) and ArrayGrepMessage (#3076) derived from Message
directly, held a key, and never overrode GetHashSlot, so they reported
NoSlot. On a cluster that sends the command to an arbitrary node and leans
on -MOVED: an extra round trip for most calls, and a RedisServerException
under CommandFlags.NoRedirect.

Both now derive from Message.CommandKeyBase, as their single-key siblings
do, which supplies the slot (and the key in CommandAndKey).

Affects the StringIncrement overloads that take an Expiration (INCREX) and
ArrayGrep. No public API change.

…eyed routing tests, SER1xx experiment block (#3355)

* Make the dedicated subscription connection lazy (#3310)

The dedicated pub/sub connection (always under RESP2; under RESP3 unless
SharedSubscriptionConnection is enabled) was opened eagerly as part of
activating every endpoint, so a client that never subscribes to anything
still paid for a second socket per node, and a second reconnect per node
on every failover. Seen on AMR, whose options provider disables the
configuration channel: 3.4.x showed two RESP3 connections where 3.0.x
showed one.

ActivateServer now opens that connection only when there is something to
subscribe to straight away: a configuration channel, or subscriptions that
already exist (and may need to move to this endpoint). Otherwise it is
created on first use, via the existing GetBridge paths; with the default
backlog policy the first SUBSCRIBE queues until it is up.

The connect wait ("second leg") and the maintenance handoff deadline
treated an absent subscription bridge as a disconnected one, which would
have left connects hanging; they now consider a connection that has not
been asked for as not required. The endpoint-state connect log says
"NotRequired" for that case instead of "Disconnected".

Tests: AzureManagedRedisConnectionCount now asserts one connection before
pub/sub and the expected count after; a new test covers the eager case
when a configuration channel is set; two tests that relied on the eager
subscription connection now subscribe first.

* Command coverage ledger and audit skill (#3311)

* Command coverage ledger and audit skill

Adds docs/CommandCoverage.md: every upstream Redis command the library does
not expose as a typed API, grouped and with a recorded decision (policy,
not applicable, deferred, todo, deprecated, enum-only, implemented). Seeded
from a diff of redis/redis src/commands (477 specs, @6a5fee67) against the
RedisCommand enum and source; it carries forward the dispositions from
#2055 (blocking is policy, Functions and ACL are deferred) and replaces
that hand-maintained list.

Adds the audit-command-coverage skill and its script, which re-runs the
diff on request and reports only what changed: new upstream commands with
no decision, ledger rows the code now covers, rows the spec no longer has,
and enum members the spec does not know. The script exits non-zero when
the ledger needs editing.

Closes #2055.

* Ledger: CLIENT TRACKING is v4, cluster/COMMAND introspection is administration, CLUSTER SHARDS note corrected

* Ledger: CLIENT PAUSE/UNPAUSE are operator tooling, not applicable

* Update RedisType.Bitmap and the keyspace-event tokens to reflect current server values (#3312)

* Object-type metadata for native bitmaps (redis/redis#15331), plus the tokens that diff turned up

RedisType gains Bitmap ("bitmap", the TYPE reply for a native roaring
bitmap, 8.12) and Gcra ("gcra", already in the server's type-name table).
KeyNotificationType gains SetBit (emitted under class $ for string bitmaps
and class b for native ones) and the other event names the server emits
that the enum lacked: arinsert, armset, arring, arseek, gcra, pfadd,
sexpire, spersist, sinterstore, zrangestore, and keymiss (class m, side
effect group).

Fixes the Copy token: the server has only ever emitted "copy_to", so the
"copy" token never parsed and a RedisChannel.KeyEvent(KeyNotificationType.Copy)
subscription never fired. The member and its value are unchanged.

Both parsers are generated from [AsciiHash]; tests extend the token tables.

* Drop Gcra: the type was withdrawn before release, so there is nothing for the token to match

* Add sunionstore/sdiffstore events (from the keyspace-notifications docs); drop the history comment on Copy

* Fix ConnectionGroupMember string-ctor dropping the name (#3314)

* Forward the name argument in ConnectionGroupMember's string constructor

The string overload chained to the ConfigurationOptions constructor without passing `name`, so every
member built from a configuration string (the idiom used throughout docs/Failover.md) was named after
its first endpoint by Init() instead of the name the caller gave it. Regression test added.

* Parenthesize a Debug-only assertion that trips SA1407

`endSegmentLength == 0 & endSegmentCommitted == 0` in DebugAssertValid sits inside `#if DEBUG`, so CI
(Release) never compiles it and never saw the StyleCop error; any local Debug build of RESPite failed
with TreatWarningsAsErrors. Introduced by ee47e53 after the StyleCop bump in aa1e5c0.

* Parse unbracketed IPv6 ip:port from the server (#1422) (#3307)

* Parse unbracketed IPv6 ip:port from the server (#1422)

CLUSTER NODES, MOVED/ASK and CLIENT LIST write IPv6 addresses without
brackets, i.e. "2a06:4e80:3:a::a:7000". The RFC 3986 parser used for
configuration strings reads that as a (valid!) eight-group address with
no port, so cluster nodes ended up as "[2a06:4e80:3:a::a:7000]:0" and
redirections and client addresses were equally wrong.

Add Format.TryParseServerEndPoint for the server's "ip:port" form: when
there is no ']' and more than one ':', split at the last colon and
require both halves to parse; anything else goes to the existing parser
unchanged, so configuration parsing is not touched. Use it at the three
sites that receive server-emitted text.

The sentinel +switch-master handler gets host and port as separate
tokens and was gluing them back together with ':' before parsing; pass
them to the (host, port) overload instead.

* Parse unbracketed IPv6 in MOVING/SMIGRATED endpoints

ParseMigrationEndPoint used the configuration-string parser, so an unbracketed IPv6 address parsed as an address with port 0, which the placeholder check then discarded. Use TryParseServerEndPoint, as the other server-emitted sites do; tests for the MOVING successor and SMIGRATED source/target.

* Profile commands sent in a batch (#2175) (#3313)

Co-authored-by: mohsinaliwork686-rgb <229973101+mohsinaliwork686-rgb@users.noreply.github.com>

* public-api.py: apply *REMOVED* at promotion; promote the API surface since 3.4.0 (#3316)

* public-api.py: apply *REMOVED* entries at promotion instead of copying them

The analyzer records a vanished shipped signature as a *REMOVED* line in
the unshipped file. Promotion moved every unshipped line across verbatim,
so a promotion after any such change would have left the shipped file
with both the dead signature and a *REMOVED* copy of it, and the analyzer
would then fail the build on the dead one (RS0017).

Promotion now consumes the marker: the named signature is dropped from the
shipped file, the marker is not carried over, and a marker that names
nothing shipped is an error. A *REMOVED* line already sitting in a shipped
file is also reported. The "+N promoted" note gains "-N removed", and an
unshipped file holding only removals is still emptied.

* Promote the shipped API surface: everything since 3.4.0

eng/public-api.py --promote. Fifty-two StackExchange.Redis entries (the
lag-aware availability surface, FailbackHealthCheck, HealthCheckProbeScope,
the 8.12-independent RedisType and KeyNotificationType tokens) and the
net6.0 SetUserPemCertificate overload move to shipped; the three parameter
renames from #3296 (muxer -> multiplexer on the Azure providers'
AfterConnectAsync, other -> obj on RESPite AsciiHash.Equals) are applied
rather than copied. Every unshipped file is now empty.

* Split third-party notices out of LICENSE so GitHub detects MIT (#3317)

* Split third-party notices out of LICENSE so GitHub detects MIT

The LICENSE file carried ~1.1KB of third-party notes after the MIT text,
which drops it below licensee's similarity threshold; GitHub reports the
repo license as "Other" (SPDX NOASSERTION). Keep LICENSE as the bare MIT
text and move the notes, verbatim, to THIRD-PARTY-NOTICES.md.

* Correct the third-party notices

- Redis: BSD-3 only through 7.2; RSALv2/SSPLv1 from 7.4, AGPLv3 added in
  8.0; point at redis.io/legal/licenses (the old /topics/license URL now
  redirects to the docs landing page).
- redis-doc: repo is archived under redis/redis-doc; note that redis/docs,
  its successor, is CC BY-NC-SA for newer content.
- Drop the Redis-64 and BookSleeve notes: neither is part of the
  development solution any more.
- Add the code actually incorporated from other projects (dotnet/runtime,
  dotnet/corefx, aspnet/BasicMiddleware) and the test-only redis-sharp copy,
  none of which the old notes mentioned.

* The read loop stops matching replies once teardown owns the response queue; a close with commands in flight no longer completes a call with the previous call's reply (#3330)

RecordConnectionFailed drains _writtenAwaitingResponse one message at a
time, completing each outside the lock, while the read loop can still be
matching replies that were already in the receive buffer. Each reply the
loop matched after the drain had taken a message went to the message
*after* the one it answered: GET key N completed, successfully, with key
N-1's value, and every reply behind it was off by one. Reproduced on
main with the reporter's loop: 616 wrong replies in 20,000 calls.

_isShutdown was already set under the queue's lock before the drain, and
the loop already consulted it - but only once the queue was empty.
Checking it before the dequeue closes the race: a reply arriving after
teardown has begun is dropped with the connection, and the message it
answered is failed by the drain as before.

The test re-creates the scenario against the real server (20 rounds of
500 in-flight GETs closed with allowCommandsToComplete: false) and
asserts that no call completes with another call's value; without the
fix it sees ~200 in a run.

Fixes #3329

* CLUSTER SLOT-STATS (8.2): per-slot usage statistics on IServer, experimental SER012 (#3341)

* CLUSTER SLOT-STATS (8.2): per-slot usage statistics on IServer, experimental SER012

IServer.ClusterSlotStatsBySlotRange(Async) and ClusterSlotStatsOrderBy(Async), modelled
on the CLUSTER SLOTS file layout and the HOTKEYS field handling: node-local read, lenient
nominal parsing, config-gated metrics as nullable properties (absent != zero), unknown
metrics preserved as Metadata, cpu-usec surfaced as TimeSpan. RedisFeatures.ClusterSlotStats
gates on 8.2 RC1. Promotes the CommandCoverage 'deferred' row.

* The storage-kind tripwire acknowledges SLOT-STATS and SLOTSRANGE: two new >8-byte literals, 28 -> 30

* Check the maximum operation count in HeartbeatConsistencyCheckPingsAsync (#3342)

* In-repo links follow the org move to seredis (#3326): github.com/StackExchange/StackExchange.Redis -> github.com/seredis/StackExchange.Redis everywhere, including the nupkg RepositoryUrl/PackageReleaseNotes metadata (#3344)

* MOVING handoff: hold new work in the backlog while the in-flight commands drain, so the recycle happens within a round trip instead of at the window end (#3347)

* MOVING handoff: hold new work in the backlog while the in-flight commands drain, so the recycle happens within a round trip instead of at the window end

The drain waited for "no caller work", which under steady traffic never comes: there is always a command in flight, so the recycle waited out the whole announced window and failed whatever was in flight at that instant. Measured live (endpoint rebind, single proxy, ~900 ops/s): recycle at +20.0s against a MOVING at +6.0s, 16 failed commands, missed-deadline warning.

While a handoff drains, the bridge holds a caller's work in the backlog instead of writing it to the connection about to be replaced; the backlog processor does not flush while the hold stands; the drain waits only for what is already written, capped at half the relaxed timeout (bounded by half the remaining window, floor one second); the connection is then recycled as before, the hold cleared, and the held work flushed to the replacement by the ordinary reconnect path. Timeouts are relaxed for the hold and the reconnect. A FailFast policy does not abandon held work for the handoff's own disconnect. Same scenario on this branch: recycle at +6.2s against a MOVING at +5.9s, 0 failed of 97k.

Deliberately a flag and nothing more: no per-message timeout cover (with relaxation off and a timeout shorter than the hold, held commands can time out in the backlog, as on any reconnect), and no retire under the write lock (a writer that reaches the lock after the recycle gets a write fault, as before). Also: the drain budget is recomputed after the DNS poll, a second MOVING during a handoff updates the successor pin rather than being dropped, and the pin lasts the window so the second bridge dials the same successor.

Tests: steady load (with and without relaxation), slow command inside the cap, blocking command past the cap; plus a fault-injector tier measurement that records the live timeline.

* OneLogicalEventFromEveryNodeRaisesOneEvent: wait for the one event rather than counting at the instant the second node relaxed; the callback is dispatched after the relaxation, so the count could be read as zero (flaked on main at ~1 in 6)

* INCREX and ARGREP are routed by key on a cluster (#3352)

IncrexMessageBase (#3065) and ArrayGrepMessage (#3076) derived from Message
directly, held a key, and never overrode GetHashSlot, so they reported
NoSlot. On a cluster that sends the command to an arbitrary node and leans
on -MOVED: an extra round trip for most calls, and a RedisServerException
under CommandFlags.NoRedirect.

Both now derive from Message.CommandKeyBase, as their single-key siblings
do, which supplies the slot (and the key in CommandAndKey).

Affects the StringIncrement overloads that take an Expiration (INCREX) and
ArrayGrep. No public API change.

---------

Co-authored-by: Mohsin Ali <mohsin.ali.work686@gmail.com>
Co-authored-by: mohsinaliwork686-rgb <229973101+mohsinaliwork686-rgb@users.noreply.github.com>
Co-authored-by: Nikolay Zdravkov <unsafeptr@gmail.com>