Visitar URL original
[Bug] Embedded NATS redelivers postprocessing events for missing uploads with no stop · Issue #3799 · opencloud-eu/opencloud · GitHub
Skip to content

[Bug] Embedded NATS redelivers postprocessing events for missing uploads with no stop #3799

Open

Description

@suse-coder

Describe the bug

Embedded JetStream keeps replaying StartPostprocessingStep / PostprocessingFinished for upload sessions that no longer exist. Consumers log ERR_UPLOAD_NOT_FOUND in a loop. That flood also hits brand-new uploads. Clearing the NATS store unblocked one upload, then the same errors returned.

Steps to reproduce

  1. After a crash, have leftover postprocessing events whose upload sessions are gone.
  2. Restart and upload a new file.
  3. mv /var/lib/opencloud/nats aside, restart, upload again.
  4. Do not run uploads sessions --processing --resume on the leftover list (it republishes the same events).

Expected behavior

Events for missing uploads are not redelivered forever. New uploads are not blocked by that flood.

Actual behavior

NATS answered healthy (127.0.0.1:9233, "jetstream":true). Store: 4.3 GiB / 1012 files.

{"level":"error","service":"storage-users","driver":"decomposed","event":"StartPostprocessingStep","uploadid":"76804efe-765b-4de2-b5bf-7f9a19fa877b","error":"ERR_UPLOAD_NOT_FOUND: upload not found","line":".../decomposedfs.go:501","message":"Failed to get upload"}
{"level":"error","service":"storage-users","driver":"decomposed","event":"PostprocessingFinished","uploadid":"76804efe-765b-4de2-b5bf-7f9a19fa877b","error":"ERR_UPLOAD_NOT_FOUND: upload not found","line":".../decomposedfs.go:307","message":"Failed to get upload"}

After moving the store: one new upload completed, then ERR_UPLOAD_NOT_FOUND returned for later uploads, including entirely new ones. --resume on the leftover list republishes those events. No nats CLI in the image.

Setup

Details

Image=docker.io/opencloudeu/opencloud-rolling:8.1.0
STORAGE_USERS_DRIVER=decomposed
OC_ASYNC_UPLOADS=true
OC_EXCLUDE_RUN_SERVICES=idp,idm,search
POSTPROCESSING_STEPS=
# embedded NATS 127.0.0.1:9233, JetStream on /var/lib/opencloud/nats

Additional context

Same Helm/env on a test cluster: uploads OK, tiny NATS store. JetStream is on CephFS; no protocol-level confirmation (no nats CLI).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions