Repository navigation
python_template_launcher.go:37: Failed to initialize fluentd logger #13213
Description
Activity
This is an example, my real image is hardened and running with nonroot user and no access to /var/log.
FROM nvcr.io/nvidia/cuda:12.6.1-cudnn-devel-ubuntu24.04
ARG WORKDIR=/template
WORKDIR ${WORKDIR}COPY requirements.txt .
COPY .py ./
RUN apt-get update
# Install Python and other system dependencies.
&& apt-get install -y --no-install-recommends software-properties-common
&& add-apt-repository ppa:deadsnakes/ppa -y
&& apt-get install -y --no-install-recommends
curl g++ python3.11-dev python3.11-venv apt-transport-https ca-certificates gnupg
&& rm -rf /var/lib/apt/lists/
&& update-alternatives --install /usr/bin/python python /usr/bin/python3.11 11
&& curl https://bootstrap.pypa.io/get-pip.py | python
# Install the pipeline requirements.
&& pip install --no-cache-dir -r requirements.txt
&& pip checkCOPY --from=gcr.io/cloud-dataflow/v1beta3/beam_python3.11_sdk:2.61.0 /opt/apache/beam /opt/apache/beam
COPY --from=gcr.io/dataflow-templates-base/python311-template-launcher-base:latest /opt/google/dataflow/python_template_launcher /opt/google/dataflow/python_template_launcherCOPY main.py .
COPY pyproject.toml .
COPY setup.py .
COPY src srcRUN pip install .
ENV FLEX_TEMPLATE_PYTHON_PY_FILE="${WORKDIR}/main.py"
RUN pip freeze
RUN pip listENTRYPOINT [ "/opt/apache/beam/boot" ]
@pbonito thanks so much for the information. I can see the challenge with the current sample for custom Docker images. We're investigating what the right path is here. If you have a support contract with Google, please create an issue in the support console as well. Thank you!
In which file did you encounter the issue?
python-docs-samples/dataflow/flex-templates/pipeline_with_dependencies
Did you change the file? If so, how?
Yes, used a different base image in docker file
Describe the issue
Jon start but I got following error:
2025/03/06 17:38:45.885541 python_template_launcher.go:37: Failed to initialize fluentd logger: failed to create log file /var/log/dataflow/template_launcher/runner-json.log: open /var/log/dataflow/template_launcher/runner-json.log: permission denied
python_template_launcher fails if running with non root user or an user that have no write access to /var/log