Visitar URL original
fix(adapter-commons): validate query operators nested in arrays (#3700) · feathersjs/feathers@304a1aa · GitHub
Skip to content

Commit 304a1aa

Browse files
marshallswainmarshallswain
authored
fix(adapter-commons): validate query operators nested in arrays (#3700)
* fix(adapter-commons): validate query operators nested in arrays validateQueryProperty skipped array values because isPlainObject excludes arrays. That let unknown $ operators through when wrapped in an extra array level, e.g. $or: [[{ $where: '1==1' }]]. Recurse into arrays so the operator allow-list applies at every level. * fix: validate filter values and tighten ObjectIdSchema filterQuery left $select, $or, $and, and $sort object values unvalidated, so unknown $ operators could ride through those filters. ObjectIdSchema accepted any object, which let operator documents pass querySyntax on generated MongoDB services. Validate those filter values with validateQueryProperty. The objectid keyword now accepts ObjectId instances and rejects other objects, and ObjectIdSchema uses that for its object branch. * fix(schema): allow $exists on queryProperty after ObjectIdSchema tighten Tightening ObjectIdSchema dropped { _id: { $exists: true } }, which only passed because the old any-object branch treated the operator document as an id. Add $exists as a boolean operator on queryProperty so it is allowed on every querySyntax field, including ObjectIds. * fix(schema): allow null in $ne, $in, and $nin query operators { userId: { $ne: null } } is the usual Mongo and SQL "field is set" query. After tightening ObjectIdSchema it only passed when the field type already included null. Accept null on $ne/$in/$nin in queryProperty so ObjectId, number, and string fields all allow it. * docs: show querySyntax extensions for adapter-specific queries The default syntax already covers $ne/$in/$nin null and $exists. Document how to add $like/$regex/array operators, how to allow { userId: null } on ObjectId query fields, and how $meta/$slice map to service operators rather than querySyntax. * fix: allow null equality, projection ops, and Mongo $regex { userId: null } is a real IS NULL query; accept null on the queryProperty equality branch the same way as $ne/$in/$nin. Allow $meta/$slice/$elemMatch inside object $select and $sort (not as field operators). Add more with projectionOperators. @feathersjs/mongodb defaults operators to include $regex and $options. Generated Mongo query schemas do the same on the primary string field. * revert: drop new query syntax obligations Keep the allow-list tightenings. Remove $exists, implicit null equality, default Mongo $regex, and projectionOperators so those stay app-level extensions via querySyntax or operators, not new supported syntax. * test: cover typical querySyntax and operators extensions Assert $regex/$options, $like, $exists, $meta, and nullable ObjectId fields still work when the app opts in, and that unlisted operators such as $where stay rejected. * fix: reject spoofed ObjectId JSON and correct querySyntax example isObjectId now requires instanceof ObjectId, or _bsontype plus a non-Object constructor and toHexString, so `{ _bsontype: 'ObjectId' }` does not pass ObjectIdSchema. Attach $regex/$options to text in the TypeBox Pick example. * test(mongodb): isolate validated $regex find from service create CI failed because people.create() returned no _id after earlier aggregation tests mutated the shared service. Insert the fixture through the collection so the test only covers validateQuery + $regex.
1 parent e541397 commit 304a1aa

16 files changed

Lines changed: 473 additions & 55 deletions

File tree

‎docs/api/databases/knex.md‎

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -113,19 +113,21 @@ In addition to the [common querying mechanism](./querying.md), this adapter also
113113
```ts
114114
const messageQuerySchema = Type.Intersect(
115115
[
116-
// This will additionally allow querying for `{ name: { $ilike: 'Dav%' } }`
117116
querySyntax(messageQueryProperties, {
118117
name: {
119-
$ilike: Type.String()
118+
$like: Type.String(),
119+
$notlike: Type.String(),
120+
$ilike: Type.String() // PostgreSQL
120121
}
121122
}),
122-
// Add additional query properties here
123-
Type.Object({})
123+
Type.Object({}, { additionalProperties: false })
124124
],
125125
{ additionalProperties: false }
126126
)
127127
```
128128

129+
More extension examples are in [querySyntax](../schema/typebox.md#querysyntax). `{ age: null }` and `{ age: { $ne: null } }` work when the query property type includes `null` (see the `age` field in the adapter tests).
130+
129131
### $like
130132

131133
Find all records where the value matches the given string pattern. The following query retrieves all messages that start with `Hello`:

‎docs/api/databases/mongodb.md‎

Lines changed: 16 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -215,13 +215,13 @@ new MongoDBService({
215215

216216
<BlockQuote type="warning" label="Important">
217217

218-
Note that in a normal application all MongoDB specific operators have to explicitly be added to the [TypeBox query schema](../schema/typebox.md#query-schemas) or [JSON query schema](../schema/schema.md#querysyntax).
218+
Note that in a normal application all MongoDB specific operators have to explicitly be added to the [TypeBox query schema](../schema/typebox.md#querysyntax) or [JSON query schema](../schema/schema.md#querysyntax).
219219

220220
</BlockQuote>
221221

222222
There are two ways to perform search queries with MongoDB:
223223

224-
- Perform basic Regular Expression matches using the `$regex` filter.
224+
- Perform basic Regular Expression matches using the `$regex` operator.
225225
- Perform full-text search using the `$search` filter.
226226

227227
### Basic Regex Search
@@ -234,6 +234,19 @@ You can perform basic search using regular expressions with the `$regex` operato
234234
}
235235
```
236236

237+
Allow those operators on the properties that need them:
238+
239+
```ts
240+
querySyntax(messageQueryProperties, {
241+
text: {
242+
$regex: Type.String(),
243+
$options: Type.String()
244+
}
245+
})
246+
```
247+
248+
If you also use [`validateQuery(schema, { skipSanitize: false })`](../schema/validators.md#keeping-adapter-sanitization), list them on the service as well: `operators: ['$regex', '$options']`.
249+
237250
### Full-Text Search
238251

239252
See the MongoDB documentation for instructions on performing full-text search using the `$search` operator:
@@ -456,7 +469,7 @@ validator.addKeyword(keywordObjectId)
456469

457470
### ObjectIdSchema
458471

459-
Both, `@feathersjs/typebox` and `@feathersjs/schema` export an `ObjectIdSchema` helper that creates a schema which can be both, a MongoDB ObjectId or a string that will be converted with the `objectid` keyword:
472+
Both, `@feathersjs/typebox` and `@feathersjs/schema` export an `ObjectIdSchema` helper that creates a schema which can be a MongoDB ObjectId instance or a string that will be converted with the `objectid` keyword. Arbitrary objects — including query operator documents like `{ $ne: null }` or `{ $where: '…' }` — are not valid ObjectIds.
460473

461474
```ts
462475
import { ObjectIdSchema } from '@feathersjs/typebox' // or '@feathersjs/schema'

‎docs/api/schema/schema.md‎

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -171,7 +171,7 @@ const userQuery: UserQuery = {
171171
}
172172
```
173173

174-
Additional special query properties [that are not already included in the query syntax](../databases/querying.md) like `$ilike` can be added like this:
174+
Additional operators that are [not already in the common query syntax](../databases/querying.md) (`$like`, `$regex`, …) are added per property. Only add operators your adapter supports. See [TypeBox querySyntax](./typebox.md#querysyntax) for more examples.
175175

176176
```ts
177177
import { querySyntax } from '@feathersjs/schema'
@@ -184,9 +184,7 @@ export const userQuerySchema = {
184184
properties: {
185185
...querySyntax(userSchema.properties, {
186186
email: {
187-
$ilike: {
188-
type: 'string'
189-
}
187+
$ilike: { type: 'string' }
190188
}
191189
} as const)
192190
}

‎docs/api/schema/typebox.md‎

Lines changed: 16 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -101,30 +101,38 @@ const messageQuerySchema = querySyntax(messageQueryProperties)
101101
type MessageQuery = Static<typeof messageQuerySchema>
102102
```
103103
104-
Additional special query properties [that are not already included in the query syntax](../databases/querying.md) like `$ilike` can be added like this:
104+
Additional operators that are [not already in the common query syntax](../databases/querying.md) must be added per property. Only add operators your adapter actually supports.
105105
106106
```ts
107-
import { querySyntax } from '@feathersjs/typebox'
107+
import { querySyntax, Type } from '@feathersjs/typebox'
108108

109-
// Schema for allowed query properties
110109
const messageQueryProperties = Type.Pick(messageSchema, ['id', 'text', 'createdAt', 'userId'], {
111110
additionalProperties: false
112111
})
112+
113113
const messageQuerySchema = Type.Intersect(
114114
[
115-
// This will additionally allow querying for `{ name: { $ilike: 'Dav%' } }`
116115
querySyntax(messageQueryProperties, {
117-
name: {
118-
$ilike: Type.String()
116+
text: {
117+
$like: Type.String(),
118+
$notlike: Type.String(),
119+
$ilike: Type.String(), // PostgreSQL
120+
$regex: Type.String(),
121+
$options: Type.String()
119122
}
120123
}),
121-
// Add additional query properties here
122-
Type.Object({})
124+
Type.Object({}, { additionalProperties: false })
123125
],
124126
{ additionalProperties: false }
125127
)
126128
```
127129

130+
That allows `{ text: { $like: 'Hello%' } }` and `{ text: { $regex: 'feathers', $options: 'i' } }`.
131+
132+
`$ne: null` and `{ userId: null }` are allowed when the **query** property type includes `null` (for example `Type.Union([Type.Number(), Type.Null()])` or `Type.Union([ObjectIdSchema(), Type.Null()])`). That is a field type, not a new operator. Do not change the create/patch data schema unless you also want to store nulls.
133+
134+
Mongo `$meta` / `$slice` in object `$select` or `$sort` are not part of the common syntax. On the adapter sanitizer path, list them on the existing service `operators` option if you need them. `querySyntax` `$select` remains a string array.
135+
128136
To allow additional query properties outside of the query syntax use the intersection type:
129137

130138
```ts

‎docs/api/schema/validators.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -113,7 +113,7 @@ This is intentional. Schema validation and the legacy sanitizer are alternative
113113

114114
- Prefer [`querySyntax`](./typebox.md#querysyntax) (or the [JSON schema helpers](./schema.md#query-helpers)) so only the common operators are allowed on each property.
115115
- Set `additionalProperties: false` on query objects so unknown keys (including unexpected `$` operators) are rejected. Generated applications already do this.
116-
- Only add extra operators (for example `$ilike` or `$regex`) when your adapter supports them and your application needs them.
116+
- Only add extra operators (for example `$ilike` or `$regex`) when your adapter supports them and your application needs them. Copy-paste examples: [querySyntax](./typebox.md#querysyntax).
117117
- Avoid permissive schemas such as `additionalProperties: true` or an open object on external query validation unless you intentionally want clients to send those keys.
118118

119119
<BlockQuote type="warning" label="TypeBox and JSON Schema defaults">

‎docs/guides/cli/service.schemas.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -105,7 +105,7 @@ export const messageQueryValidator = getValidator(messageQuerySchema, queryValid
105105
export const messageQueryResolver = resolve<MessageQuery, HookContext>({})
106106
```
107107

108-
To add additional operators like `$like` see the [querySyntax](../../api/schema/typebox.md#querysyntax) documentation. You can also add your own query parameters in the `Type.Object({}, { additionalProperties: false })` definition.
108+
To add additional operators like `$like` or `$regex`, see [querySyntax](../../api/schema/typebox.md#querysyntax). You can also add your own query parameters in the `Type.Object({}, { additionalProperties: false })` definition.
109109

110110
<BlockQuote type="warning" label="Important">
111111

‎packages/adapter-commons/src/query.ts‎

Lines changed: 15 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,10 @@ const parse = (value: any) => (typeof value !== 'undefined' ? parseInt(value, 10
88
const isPlainObject = (value: any) => _.isObject(value) && value.constructor === {}.constructor
99

1010
const validateQueryProperty = (query: any, operators: string[] = []): Query => {
11+
if (Array.isArray(query)) {
12+
return query.map((value) => validateQueryProperty(value, operators))
13+
}
14+
1115
if (!isPlainObject(query)) {
1216
return query
1317
}
@@ -17,11 +21,7 @@ const validateQueryProperty = (query: any, operators: string[] = []): Query => {
1721
throw new BadRequest(`Invalid query parameter ${key}`, query)
1822
}
1923

20-
const value = query[key]
21-
22-
if (isPlainObject(value)) {
23-
query[key] = validateQueryProperty(value, operators)
24-
}
24+
query[key] = validateQueryProperty(query[key], operators)
2525
}
2626

2727
return {
@@ -91,41 +91,44 @@ export const OPERATORS = ['$in', '$nin', '$lt', '$lte', '$gt', '$gte', '$ne', '$
9191

9292
export const FILTERS: FilterSettings = {
9393
$skip: (value: any) => parse(value),
94-
$sort: (sort: any): { [key: string]: number } => {
94+
$sort: (sort: any, { operators }: FilterQueryOptions): { [key: string]: any } => {
9595
if (typeof sort !== 'object' || Array.isArray(sort)) {
9696
return sort
9797
}
9898

9999
return Object.keys(sort).reduce(
100100
(result, key) => {
101-
result[key] = typeof sort[key] === 'object' ? sort[key] : parse(sort[key])
101+
result[key] =
102+
typeof sort[key] === 'object' && sort[key] !== null
103+
? validateQueryProperty(sort[key], operators)
104+
: parse(sort[key])
102105

103106
return result
104107
},
105-
{} as { [key: string]: number }
108+
{} as { [key: string]: any }
106109
)
107110
},
108111
$limit: (_limit: any, { paginate }: FilterQueryOptions) => getLimit(_limit, paginate),
109-
$select: (select: any) => {
112+
$select: (select: any, { operators }: FilterQueryOptions) => {
110113
if (Array.isArray(select)) {
111114
return select.map((current) => `${current}`)
112115
}
113116

114-
return select
117+
return validateQueryProperty(select, operators)
115118
},
116119
$or: (or: any, { operators }: FilterQueryOptions) => {
117120
if (Array.isArray(or)) {
118121
return or.map((current) => validateQueryProperty(current, operators))
119122
}
120123

121-
return or
124+
return validateQueryProperty(or, operators)
122125
},
123126
$and: (and: any, { operators }: FilterQueryOptions) => {
124127
if (Array.isArray(and)) {
125128
return and.map((current) => validateQueryProperty(current, operators))
126129
}
127130

128-
return and
131+
return validateQueryProperty(and, operators)
129132
}
130133
}
131134

0 commit comments

Comments
 (0)