Filtering, Sorting, and Searching

Filtering Resources

Filtering is the process of narrowing down a collection based on specific property values. In REST, we represent these as query parameters appended to the URL. The reasoning here is to adhere to the stateless nature of HTTP while allowing the server to generate a precise SQL 'WHERE' clause. When a client requests `/users?active=true`, the server interprets the key-value pair to filter the data source before serialization. This reduces the amount of data transferred over the network, which is critical for mobile clients or low-bandwidth environments. By keeping the interface predictable, you allow developers to compose complex queries by chaining parameters. It is crucial to define a strict schema for which fields are filterable to prevent performance degradation, such as preventing users from filtering on unindexed columns, which would force the database to perform an expensive full table scan during every API request.

// Example: Filtering active users via GET request
// Endpoint: GET /users?active=true
const users = await database.collection('users').find({ 
  active: true // Server translates query param to database query
}).toArray();

Sorting Collections

Sorting allows the client to define the order of the returned collection, typically by a specific field and direction (ascending or descending). This is essential because standard resource listing endpoints may return items in an arbitrary database order. By using a standard parameter like 'sort=createdAt:desc', the API provides the client control over the presentation layer without needing a different endpoint for every possible ordering permutation. The reasoning behind this is to minimize client-side post-processing; it is far more efficient for a database to perform an indexed sort than for a web application to sort a massive array of objects in memory. You should document which fields support sorting to ensure clients do not attempt to sort by non-indexable or high-cardinality fields that would lead to memory pressure on the server during the sorting operation.

// Example: Sorting products by price descending
// Endpoint: GET /products?sort=price:desc
const sortParams = { price: -1 }; // -1 indicates descending
const products = await database.collection('products').find().sort(sortParams).toArray();

Searching Textual Data

Searching differs from filtering because it typically involves fuzzy matching or partial string comparisons across one or more fields. While filtering is exact, searching often relies on database-level full-text indexing. The reasoning for separating search from filtering is the underlying performance cost. Searching through large blobs of text or multiple fields requires different storage structures, such as inverted indices. If you treat every search as a general-purpose filter, you will eventually encounter severe performance bottlenecks as your dataset grows. Providing a dedicated 'q' parameter, like `/posts?q=rest+api`, signals to the client that this operation may be more resource-intensive. Always sanitize these inputs to prevent injection attacks and ensure the server enforces a minimum character length to prevent users from triggering broad, inefficient searches that scan every single row in the database index.

// Example: Searching posts for a specific term
// Endpoint: GET /posts?q=API+design
const query = request.query.q;
const results = await database.collection('posts').find({ 
  $text: { $search: query } // Uses full-text index for efficiency
}).toArray();

Combining Multiple Parameters

The true power of query parameters lies in their composability. A well-designed API allows clients to filter, search, and sort simultaneously within a single request. The reasoning for this is to satisfy complex UI requirements, such as a dashboard that displays 'Active users (filtered) who joined recently (sorted) and match a name pattern (searched)'. When combining these, ensure your internal logic processes the filters first to reduce the dataset size, then performs the search, and finally applies the sort to the result set. This order of operations maximizes performance because it minimizes the number of records that must be held in the sorting buffer. Always validate that the combination of parameters does not violate security policies, such as users requesting data they are not authorized to view, even if their query parameters are technically valid.

// Example: Combined filtering, searching, and sorting
// GET /users?active=true&q=John&sort=name:asc
const query = { active: true, $text: { $search: 'John' } };
const users = await database.collection('users').find(query).sort({ name: 1 }).toArray();

Pagination as a Necessary Constraint

No matter how good your filtering and searching are, you must implement pagination to protect the server from unbounded requests. A client might provide a filter that returns millions of results, crashing the service if the server tries to return them all in a single response body. The reasoning here is to provide a safety valve; by enforcing a 'limit' and 'offset' (or cursor-based) pagination, you guarantee that the response size remains constant regardless of the underlying data size. This effectively completes the triad of search, sort, and filter by ensuring that the result set is manageable. Without pagination, your sorting and searching logic will eventually lead to catastrophic system failure during peak usage times, as the server will run out of memory attempting to buffer the entire collection for the network transport layer.

// Example: Applying limit and offset for pagination
// GET /items?limit=20&offset=40
const limit = parseInt(request.query.limit) || 10;
const offset = parseInt(request.query.offset) || 0;
const items = await database.collection('items').find().skip(offset).limit(limit).toArray();

Interview questions

All REST API Design interview questions →

Check yourself

Take the full REST API Design quiz →