Accessing tables as files, query construction and out-of-band modification in HTTP interface - #105249

Implements the spec from
#46925 (comment).

Adds settings to interpret URL path components as a database, table, format,
compression, and filters (hive partitioning style), plus settings for query
construction (`select`, `order`, `sort`, `filter`, `page`) and format/compression
overrides (`format`, `input_format`, `output_format`, `default_format`,
`compression`).

When the URL path identifies a table, the request is processed as
`SELECT * FROM database.table`, and combined with the query-construction settings
the wrapped query becomes `SELECT [select] FROM (...) [WHERE filter] [ORDER BY order]`.
When both a path table and a `query` URL parameter are present, the table is
exposed via `implicit_table_at_top_level` so the query can omit `FROM`.

Binary or compressed responses set `Content-Disposition: attachment` so a
browser downloads them with a sensible filename.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

…nstraints

The dynamic-query handler now accepts an optional `url_prefix` element in its
`<http_handlers>` rule. When set, the factory adds a non-strict-path filter so
the handler matches any URL starting with the prefix, and the handler itself
strips the prefix from the URL path before parsing the remainder as
`database/table.format[.compression]` (or hive-partition filters).

Example config:

    <http_handlers>
        <rule>
            <url_prefix>/api/v1</url_prefix>
            <handler><type>dynamic_query_handler</type></handler>
        </rule>
        <defaults/>
    </http_handlers>

A request `GET /api/v1/default/hits.CSV` is then processed as if the path were
`/default/hits.CSV` — the `default` database, the `hits` table, and the `CSV`
output format.

Also extends the `<changeable_in_readonly/>` set in the shipped default profile
to all query-construction settings (`format`, `input_format`, `output_format`,
`compression`, `limit`, `offset`, `page`, `select`, `order`, `sort`, `filter`,
`implicit_table_at_top_level`) so they remain settable on HTTP GET and from
`readonly = 1` user profiles.

The stateless test grows a configured `/api/v1` handler (via
`tests/config/config.d/http_url_prefix.xml`) and a few smoke-test cases.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

…rsing

Addresses issues called out by the automated review on
#105249:

* Register the Prometheus handler before the dynamic-query catch-all in
  `addDefaultHandlersFactory`, so `/metrics` is no longer shadowed when path
  features are enabled.

* `Settings::addToProgramOptions` now skips setting names that are already
  registered as command-line options (mirroring
  `MergeTreeSettings::addToProgramOptionsIfNotPresent`). Fixes the
  `option '--database' is ambiguous` error from the fast-test job after the
  new `database` setting was added.

* Move URL path parsing in `HTTPHandler::processQuery` to *after*
  `applySettingsChanges`, and gate on the per-context `http_allow_*` settings
  rather than the server-default settings. When all path features are off in
  the authenticated user's settings, the URL path is ignored entirely.

* `parseURLParameterAsFilter` now handles the case where HTMLForm splits on
  the operator's `=`. `?a!=2`, `?a>=2`, `?a<=2` are now reassembled to
  `a != 2`, `a >= 2`, `a <= 2` respectively. Operators without an `=` at all
  (`?a>2`, `?a<>2`, `?f(x)>3`) keep using the full-expression fallback.

* Add an "Ok" comment to the catch-all in `HTTPQueryConstructor.cpp`
  (silently falling back to "not binary" for unknown formats); the style
  check refuses bare `catch (...)` blocks.

* Test additions: per-user disable of path features ignores the URL path;
  `?a!=2`, `?a>=2`, `?a<>2` non-encoded operator forms.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* Drop the global-defaults gate on the dynamic-query handler's catch-all
  filter and always route arbitrary paths to it (after strict handlers).
  Routing happens before authentication so we cannot read per-user settings
  here; instead, the per-user `http_allow_*_as_path` settings are honored
  inside `HTTPHandler::processQuery`, where path parsing is gated on the
  authenticated context.

* When path parsing is invoked but the path contains components that no
  enabled feature can claim (e.g. a non-filter, non-table component while
  `http_allow_database_as_path` is off), return an empty `HTTPPathInfo`
  instead of throwing — the URL path is effectively ignored and the
  request proceeds as if at the root URL. Without this, requests like
  `/hits` with `http_allow_table_as_file=0` would fail with a confusing
  "Database name in path is disabled" error.

* Path-derived `default_format` now defers to an explicit `default_format`
  setting (in addition to `format` and `output_format`). Matches the
  documented precedence: explicit overrides win over the file extension.

* Style: rephrase the `parseURLParameterAsFilter` doc comment to avoid
  `!",` which trips the `check_cpp` style rule banning that punctuation
  sequence ("looks dirty, unconfident").

* Add a regression test for the format-precedence rule.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

`limit` and `offset` settings are widened from `UInt64` to `Double`, so they
can hold negative or fractional values. The same applies to the new `page`
setting. The values now flow through to SQL `LIMIT` / `OFFSET` clauses,
where ClickHouse's existing native support for negative `LIMIT` ("last N
rows") and fractional `LIMIT` / `OFFSET` (a share of the result) takes over.

Page semantics:
- Positive page `N` with limit `L`: offset = `L * (N - 1)`; limit stays `L`.
- Negative page `N` with limit `L`: limit becomes `-L` and offset is
  `L * (N + 1)`, so `page=-1` is the last page, `page=-2` the second-to-last,
  and so on. This composes with ClickHouse's negative-`LIMIT` semantics:
  `LIMIT -L OFFSET 0` returns the last `|L|` rows; `LIMIT -L OFFSET -L`
  returns the rows before those, etc.

Call sites that consume `Setting::limit` / `Setting::offset` in
`InterpreterSelectWithUnionQuery`, `QueryTreeBuilder`, and
`ClusterProxy::executeQuery` are updated to use `Float64` arithmetic and to
emit `ASTLiteral(Field(double))` so the resulting `LIMIT` / `OFFSET`
expressions carry the user's value verbatim. The legacy `LimitStep` /
`OffsetStep` fallback (only reached for `WITH FILL` / `WITH TIES`) is
preserved for non-negative integer values.

Test additions: `?limit=-1`, `?limit=0.5`, `?limit=0.5&offset=0.5`,
`?limit=1&page=-1`, `?limit=1&page=-2`.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

…ault_format always changeable in readonly

The previous fast-test failure was caused by `Settings::addToProgramOptionsAsMultitokens`
not having the duplicate-skip logic that `Settings::addToProgramOptions` already had.
With `--allow_repeated_settings` (which the CI passes), the client's `--database`
CLI option collided with the new `database` setting and boost::program_options reported
the option as ambiguous on every command. Applies the same skip logic to the
multitokens path. Also catches `boost::bad_any_cast` in `addToClientOptions`: the
collision means the value stored under the shared name belongs to the client option
(`String`), not the multitokens setting (`Strings`), and we should not try to read
it as the setting.

For backward compatibility, `database` and `default_format` are now always
changeable in `readonly` mode regardless of the user profile's `<constraints>`.
Operators upgrading from a version where these were special URL parameters did
not need a `<changeable_in_readonly/>` entry, and breaking those scenarios on
upgrade would be unfriendly. The setting-profile `<constraints>` block remains
the canonical way to widen *other* settings; this hardcoded list (`database`,
`default_format`) only ensures that the former special-parameter behavior is
preserved.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

…ptions into matching settings

`--database` (and its `-d` short alias), `--format` (`-f`), `--input-format`,
and `--output-format` are now mirrored into the corresponding `database`,
`format`, `input_format`, and `output_format` settings on `cmd_settings`. This
means the value is sent to the server as a regular per-query setting in
addition to seeding the TCP connection's initial database / the client's
default-format selection. The user-facing effect: `clickhouse-client
--database=foo` behaves the same way as passing `?database=foo` over HTTP.

The settings registration still uses the duplicate-skip logic added earlier
(`--database` / `--format` collide with the settings' auto-registered names),
so `boost::program_options` keeps the client-side option as the canonical
parser for those names; we just copy the value across after parsing.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

…outing

Three follow-ups to address the fast-test failures on the previous head:

1. `--param_database=...` was failing every TCP request with
   `Cannot parse quoted string: expected opening quote ''', got 'd'`.
   The client serializes query parameters by calling `Settings::set(name, value)`
   for each one; when the name happens to be a real declared setting (now
   including `database`), the value lands in a typed `SettingFieldString` whose
   `toString()` returns the raw value, whereas custom (undeclared) settings'
   `toString()` SQL-quotes it. The server's `Settings::toNameToNameMap()` then
   called `readQuoted` on the value, which choked on the unquoted form.
   Make `toNameToNameMap` accept both forms — if the string starts with a `'`,
   parse as a quoted SQL literal; otherwise pass it through verbatim.

2. `?profile=readonly` (and similar) was being treated as a SQL filter when
   `http_allow_filters_as_unrecognized_url_parameters` was on, because
   `profile` is not a `DECLARE`d setting (it's handled specially by
   `Context::setSetting`). Add `profile` to the early "known name" carve-out
   in `HTTPHandler::processQuery` so it flows through the normal settings
   pipeline and triggers profile loading.

3. The dynamic-query catch-all filter was too permissive and hijacked
   single-component-no-extension paths like `/sashboards`, breaking
   `02842_suggest_http_page_in_error_message` (which expects a "Maybe you
   meant /dashboard" hint from `NotFoundHandler`). Narrow the extended
   routing to paths that *look* like a database/table reference:
   non-empty query string, multiple `/` separators, a `.` in the last
   component, or a comparison operator (`= < > !`). Simple typo paths fall
   through to `NotFoundHandler` again.

Test config: drop `http_allow_filters_as_unrecognized_url_parameters` from
the global default profile (it would otherwise turn typos / `?profile=`
into filters and break tests like `02152_invalid_setting_with_hints_...`).
The 04245 test enables it per-request for the cases that exercise it.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

…defaults; avoid rule-name collision

Three issues uncovered by CI:

1. Tests like `01399_http_request_headers` were hanging on
   `curl -H 'X-ClickHouse-Database: header_test' -d 'SHOW TABLES'`.
   The header now flows through the `database` setting; after
   `applySettingsChanges` we call `setCurrentDatabase` to apply it. That call
   used to happen *after* `query_scope = QueryScope::create(context)`, so an
   `UNKNOWN_DATABASE` exception from `setCurrentDatabase` would unwind through
   an in-flight query scope and leave the HTTP response buffers in a state
   that never finalized — the curl just timed out. Move the
   database-resolution block *before* `QueryScope::create`, so the exception
   unwinds cleanly and the client sees the expected `Code: 81`.

2. The test config installs a configured dynamic-query handler under
   `/api/v1` (used by `04245_http_table_as_file`'s `url_prefix` test). The
   factory iterates `config.keys("http_handlers")` which returns child
   elements in unspecified order, and on Poco's hash-backed map this often
   put `defaults` first. The built-in catch-all dynamic handler registered
   by `defaults` would then match `/api/v1/...` before the configured
   `rule_api_v1` ever ran. Defer the `defaults` block in
   `createHandlersFactoryFromConfig` until after every `rule*` child has
   been processed, so user-configured rules always take precedence over the
   catch-all.

3. The test config splits `<http_handlers>` across two `config.d/` files
   (`handlers.yaml` for the `/upyachka` redirect, `http_url_prefix.xml` for
   the `/api/v1` prefix). Both used the element name `<rule>`, and the
   ClickHouse XML config merger treats same-named siblings as merging-by-name,
   so one overwrote the other. Rename my XML's element to `rule_api_v1` —
   the factory accepts any element name that starts with `rule`.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

…20260816