☰

Query DSL Component

Elasticsearch Query DSL: Match, Range, Bool and Filter Queries

Type the word ford in a search box, and you expect cars made by Ford. Add a price limit, and you expect only the cheap ones. Hide the black ones, and prefer the ones with a sunroof. All of that is one search in Elasticsearch, and the way you write it is the Query DSL.

The Query DSL is a language built from JSON. You describe what you want in a query object, and send it to the search API. Each query type does one job. Some find words in text, some compare numbers, some check that a field exists, and the bool query joins them together with and, or and not logic.

This guide goes through the main query types, in the same order as the original page: match_all, match, multi_match, exists, match_phrase, range, sorting, the bool query with must, must_not and should, and filters. The old page showed each one only as a picture. Here, every query has its full text, so you can copy it, and the matching documents are listed so you can check your own result. We also explain query context against filter context, and the term query mistake that catches most beginners.

What is Query DSL in Elasticsearch?

DSL stands for domain specific language. In Elasticsearch, the Query DSL is the set of JSON queries that you put in the query part of a search request. The old page said that searches are done using a query component, and that is the idea. A search request looks like this:

kibana dev tools
GET cars/_search
{
  "query": {
    "match": { "make": "ford" }
  }
}

The word cars is the index, _search is the search API, and everything inside query is the Query DSL. The Elastic docs split queries into two kinds of use, which we explain next.

Query Context and Filter Context

This idea explains why Query DSL has both match and filter, so it is worth a minute.

Query context compared with filter context
Aspect Query context Filter context
Question it asks How well does this document match? Does this document match, yes or no?
Score Calculates a relevance score, shown in _score No score
Best for Full text search, where some matches are better than others Exact values, numbers, dates and statuses
Caching Not cached Frequently used filters are cached
Where it applies The query parameter, and the must and should clauses of bool The filter and must_not clauses of bool, and constant_score

The rule of thumb in the Elastic docs is simple. Use query context when a condition should change the score. Use filter context for everything else.

Set Up Sample Data

To make every example copyable, here is a small index of 10 cars. Run these two requests in Kibana Dev Tools. The make and model fields are text, with a keyword sub field. The description is text. The sunroof field is only present on 4 cars, which we need for the exists example. Refresh makes the documents searchable at once.

kibana dev tools
PUT cars
{
  "mappings": {
    "properties": {
      "make":        { "type": "text", "fields": { "keyword": { "type": "keyword" } } },
      "model":       { "type": "text", "fields": { "keyword": { "type": "keyword" } } },
      "description": { "type": "text" },
      "color":       { "type": "keyword" },
      "year":        { "type": "integer" },
      "price":       { "type": "integer" },
      "sunroof":     { "type": "boolean" }
    }
  }
}
kibana dev tools
POST cars/_bulk?refresh=true
{"index": {"_id": "1"}}
{"make": "Toyota", "model": "Corolla", "year": 2022, "price": 21000, "color": "white", "description": "Reliable family sedan with low running cost"}
{"index": {"_id": "2"}}
{"make": "Toyota", "model": "Camry", "year": 2023, "price": 28000, "color": "black", "description": "Comfortable executive sedan with hybrid engine", "sunroof": true}
{"index": {"_id": "3"}}
{"make": "Toyota", "model": "Fortuner", "year": 2021, "price": 41000, "color": "white", "description": "Powerful diesel SUV for rough roads"}
{"index": {"_id": "4"}}
{"make": "Toyota", "model": "Yaris", "year": 2020, "price": 14000, "color": "silver", "description": "Compact city hatchback easy to park"}
{"index": {"_id": "5"}}
{"make": "Honda", "model": "City", "year": 2022, "price": 19000, "color": "silver", "description": "Spacious family sedan with good mileage"}
{"index": {"_id": "6"}}
{"make": "Honda", "model": "Civic", "year": 2023, "price": 26000, "color": "black", "description": "Sporty sedan with turbo engine", "sunroof": true}
{"index": {"_id": "7"}}
{"make": "Honda", "model": "Amaze", "year": 2021, "price": 12000, "color": "white", "description": "Compact sedan for city driving"}
{"index": {"_id": "8"}}
{"make": "Ford", "model": "Figo", "year": 2020, "price": 11000, "color": "white", "description": "Small city hatchback with low price"}
{"index": {"_id": "9"}}
{"make": "Ford", "model": "Endeavour", "year": 2022, "price": 44000, "color": "grey", "description": "Powerful diesel SUV with off road ability", "sunroof": true}
{"index": {"_id": "10"}}
{"make": "BMW", "model": "X1", "year": 2023, "price": 52000, "color": "black", "description": "Premium compact SUV with turbo engine", "sunroof": true}
The sample index of 10 cars
ID Make Model Year Price Color Description
1 Toyota Corolla 2022 21000 white Reliable family sedan with low running cost
2 Toyota Camry 2023 28000 black Comfortable executive sedan with hybrid engine
3 Toyota Fortuner 2021 41000 white Powerful diesel SUV for rough roads
4 Toyota Yaris 2020 14000 silver Compact city hatchback easy to park
5 Honda City 2022 19000 silver Spacious family sedan with good mileage
6 Honda Civic 2023 26000 black Sporty sedan with turbo engine
7 Honda Amaze 2021 12000 white Compact sedan for city driving
8 Ford Figo 2020 11000 white Small city hatchback with low price
9 Ford Endeavour 2022 44000 grey Powerful diesel SUV with off road ability
10 BMW X1 2023 52000 black Premium compact SUV with turbo engine

Sunroof is set to true on the cars with ID 2, 6, 9 and 10. The other six have no sunroof field at all. In each example below, I list the matching documents. A real response has more fields, such as _id, _score and _source for each hit. When no sort is given, hits come back with the best score first, so the order on your screen may differ from my tables.

1. Match_All Query

The match_all query matches every document in the index. It is the default when you send no query. The old page said that it returns the configurations of the index. That is not right. It returns documents only, and each gets a score of 1.0.

Screenshot from the original page of a match_all query on the vehicles index
kibana dev tools
GET cars/_search
{
  "query": { "match_all": {} }
}

Matching documents:

Matching documents for Match_All Query
ID Make Model Price Color
1 Toyota Corolla 21000 white
2 Toyota Camry 28000 black
3 Toyota Fortuner 41000 white
4 Toyota Yaris 14000 silver
5 Honda City 19000 silver
6 Honda Civic 26000 black
7 Honda Amaze 12000 white
8 Ford Figo 11000 white
9 Ford Endeavour 44000 grey
10 BMW X1 52000 black

All 10 documents match. Every hit gets a score of 1.0.

If you only want the number of documents, use the count API with the same query. It returns the count and no hits:

kibana dev tools
GET cars/_count
{
  "query": { "match_all": {} }
}
output
{
  "count": 10
}

2. Match Query

The match query is the standard full text search. Elasticsearch splits your search text into words, lower cases them, and finds documents that have those words in the field. The old page said that the query checks the same field pattern, which does not explain this. Here it finds the Ford cars. Notice that I typed ford in lower case. The make is stored as Ford, but because the field is analysed, the capital letter does not matter.

Screenshot from the original page of a match query for the value ford in the make field
kibana dev tools
GET cars/_search
{
  "query": { "match": { "make": "ford" } }
}

Matching documents:

Matching documents for Match Query
ID Make Model Price Color
8 Ford Figo 11000 white
9 Ford Endeavour 44000 grey

More Than One Word

With more than one word, the match query uses OR by default. A document with any of the words matches, and the one with more words scores higher.

kibana dev tools
GET cars/_search
{
  "query": { "match": { "description": "family sedan" } }
}

Matching documents:

Matching documents for More Than One Word
ID Make Model Price Color
1 Toyota Corolla 21000 white
2 Toyota Camry 28000 black
5 Honda City 19000 silver
6 Honda Civic 26000 black
7 Honda Amaze 12000 white

To require all the words, set the operator to and:

kibana dev tools
GET cars/_search
{
  "query": {
    "match": {
      "description": { "query": "family sedan", "operator": "and" }
    }
  }
}

Matching documents:

Matching documents for More Than One Word, request 2
ID Make Model Price Color
1 Toyota Corolla 21000 white
5 Honda City 19000 silver

3. Multi_match Query

The multi_match query runs the same search text on several fields. Here it looks for the word city in both the model and the description. Some cars match in the model, and some match in the description.

Screenshot from the original page of a multi_match query on the make and make_desc fields
kibana dev tools
GET cars/_search
{
  "query": {
    "multi_match": {
      "query": "city",
      "fields": [ "model", "description" ]
    }
  }
}

Matching documents:

Matching documents for Multi_match Query
ID Make Model Price Color
4 Toyota Yaris 14000 silver
5 Honda City 19000 silver
7 Honda Amaze 12000 white
8 Ford Figo 11000 white

Car 5 matches because its model is City. Cars 4, 7 and 8 match because the word city is in the description.

4. Exists Query

The exists query finds documents that have a value in a field. The old page used it with the count API, which is a good way to answer how many documents have this field. Here it counts the cars that have a sunroof field.

Screenshot from the original page of an exists query on the condition field, run with the _count endpoint
kibana dev tools
GET cars/_count
{
  "query": { "exists": { "field": "sunroof" } }
}
output
{
  "count": 4
}

To see the cars themselves, use the search API:

kibana dev tools
GET cars/_search
{
  "query": { "exists": { "field": "sunroof" } }
}

Matching documents:

Matching documents for Exists Query
ID Make Model Price Color
2 Toyota Camry 28000 black
6 Honda Civic 26000 black
9 Ford Endeavour 44000 grey
10 BMW X1 52000 black

A field counts as missing when it is not in the document, or when its value is null or an empty list. A value of false still exists.

5. Match_phrase Query

The match_phrase query finds the words together, in the same order. This is different from match, which finds the words anywhere. Here it finds the exact phrase diesel SUV.

Screenshot from the original page of a match_phrase query for the phrase This car has very good in the cars_desc field
kibana dev tools
GET cars/_search
{
  "query": { "match_phrase": { "description": "diesel SUV" } }
}

Matching documents:

Matching documents for Match_phrase Query
ID Make Model Price Color
3 Toyota Fortuner 41000 white
9 Ford Endeavour 44000 grey

The order of the words matters. The reverse phrase finds nothing:

kibana dev tools
GET cars/_search
{
  "query": { "match_phrase": { "description": "SUV diesel" } }
}

Matching documents: none.

A plain match query with the same words finds more, because it also finds car 10, which has the word SUV but not the word diesel:

kibana dev tools
GET cars/_search
{
  "query": { "match": { "description": "SUV diesel" } }
}

Matching documents:

Matching documents for Match_phrase Query, request 2
ID Make Model Price Color
3 Toyota Fortuner 41000 white
9 Ford Endeavour 44000 grey
10 BMW X1 52000 black

6. Range Query

The range query finds numbers or dates between limits. It uses four operators:

The four range operators
Operator Meaning
gt Greater than
gte Greater than or equal to
lt Less than
lte Less than or equal to
Screenshot from the original page of a range query for prices from 30000 to 100000

Cars that cost from 20000 to 30000, with both ends included:

kibana dev tools
GET cars/_search
{
  "query": {
    "range": { "price": { "gte": 20000, "lte": 30000 } }
  }
}

Matching documents:

Matching documents for Range Query
ID Make Model Price Color
1 Toyota Corolla 21000 white
2 Toyota Camry 28000 black
6 Honda Civic 26000 black

With gt and lt, the ends are left out. Car 1 costs exactly 21000 and car 2 costs 28000, so they are not in the result:

kibana dev tools
GET cars/_search
{
  "query": {
    "range": { "price": { "gt": 21000, "lt": 28000 } }
  }
}

Matching documents:

Matching documents for Range Query, request 2
ID Make Model Price Color
6 Honda Civic 26000 black

7. Sort Along With a Query

By default, results come back with the best score first. Use sort to order them by a field instead. The old page sorted by price in descending order. Here are the three most expensive cars:

Screenshot from the original page of a search that sorts the price field in descending order
kibana dev tools
GET cars/_search
{
  "size": 3,
  "query": { "match_all": {} },
  "sort": [ { "price": { "order": "desc" } } ]
}

Matching documents:

Matching documents for Sort Along With a Query
ID Make Model Price Color
10 BMW X1 52000 black
9 Ford Endeavour 44000 grey
3 Toyota Fortuner 41000 white

Sort on a number, a date or a keyword field. Sorting on a text field fails by default, for the same reason that aggregations do. Use the keyword sub field, such as make.keyword.

Paging With from and size

The size setting sets how many hits come back, and the default is 10. The from setting skips that number of hits. Here are the hits from the fourth to the sixth, in price order:

kibana dev tools
GET cars/_search
{
  "from": 3,
  "size": 3,
  "query": { "match_all": {} },
  "sort": [ { "price": "desc" } ]
}

Matching documents:

Matching documents for Paging With from and size
ID Make Model Price Color
2 Toyota Camry 28000 black
6 Honda Civic 26000 black
1 Toyota Corolla 21000 white

Return Only Some Fields

The _source setting limits which fields appear in each hit. This keeps the response small.

kibana dev tools
GET cars/_search
{
  "_source": [ "make", "model", "price" ],
  "query": { "range": { "price": { "gte": 40000 } } }
}

Matching documents:

Matching documents for Return Only Some Fields
ID Make Model Price Color
3 Toyota Fortuner 41000 white
9 Ford Endeavour 44000 grey
10 BMW X1 52000 black

8. Bool Query: Must, Must_not and Should

The bool query joins other queries with logic. The old page listed three clauses, and then had filter in a separate section, with the list numbered 1, 2 and 1. In fact bool has four clauses. Here they are together:

The four clauses of a bool query
Clause What it does Scoring Works like
must The document must match Adds to the score AND
should The document may match. A match adds to the score Adds to the score OR
filter The document must match Ignored, and can be cached AND
must_not The document must not match Ignored, and can be cached NOT

Here is one query that uses three of them. It looks for sedans, leaves out the black ones, and prefers Toyotas.

Screenshot from the original page of a bool query with a must clause on the vehicles index
Screenshot from the original page of a bool query with must_not and should clauses on the vehicles index
kibana dev tools
GET cars/_search
{
  "query": {
    "bool": {
      "must":     [ { "match": { "description": "sedan" } } ],
      "must_not": [ { "term":  { "color": "black" } } ],
      "should":   [ { "match": { "make": "toyota" } } ]
    }
  }
}

Matching documents:

Matching documents for Bool Query: Must, Must_not and Should
ID Make Model Price Color
1 Toyota Corolla 21000 white
5 Honda City 19000 silver
7 Honda Amaze 12000 white

Corolla matches the should clause too, so it gets the highest score and comes first. The order of the other two depends on their scores.

Check this by hand. The sedans are cars 1, 2, 5, 6 and 7. Removing the black ones (2 and 6) leaves 1, 5 and 7.

A Note on minimum_should_match

The Elastic docs say that a should clause is optional when the query also has a must or a filter clause. If a bool query has only should clauses, at least one of them must match, because the default value of minimum_should_match is then 1. Otherwise, the default is 0. Here the query has only should clauses, so a car must be white or silver:

kibana dev tools
GET cars/_search
{
  "query": {
    "bool": {
      "should": [
        { "term": { "color": "white" } },
        { "term": { "color": "silver" } }
      ]
    }
  }
}

Matching documents:

Matching documents for A Note on minimum_should_match
ID Make Model Price Color
1 Toyota Corolla 21000 white
3 Toyota Fortuner 41000 white
4 Toyota Yaris 14000 silver
5 Honda City 19000 silver
7 Honda Amaze 12000 white
8 Ford Figo 11000 white

9. Filter Query

A filter requires a match, like must, but it does not change the score, and Elasticsearch can cache it. It is the right choice for exact values, such as a color, a status or a price limit. Here is a query for white cars that cost 20000 or less.

Screenshot from the original page of a filter clause inside a bool query
kibana dev tools
GET cars/_search
{
  "query": {
    "bool": {
      "filter": [
        { "term":  { "color": "white" } },
        { "range": { "price": { "lte": 20000 } } }
      ]
    }
  }
}

Matching documents:

Matching documents for Filter Query
ID Make Model Price Color
7 Honda Amaze 12000 white
8 Ford Figo 11000 white

Every hit has a score of 0, because a filter does not score.

The old page said that filter clauses are faster than any other component. That is too strong. The Elastic docs say that frequently used filters are cached, and that filter and must_not run without scoring. This often helps, but how much depends on your data and your cluster. Test on your own data before you claim a speed up.

Filter and Must Together

The most useful pattern is to put the search words in must, so that they score, and the exact conditions in filter. Here we look for sedans that cost 20000 or less:

kibana dev tools
GET cars/_search
{
  "query": {
    "bool": {
      "must":   [ { "match": { "description": "sedan" } } ],
      "filter": [ { "range": { "price": { "lte": 20000 } } } ]
    }
  }
}

Matching documents:

Matching documents for Filter and Must Together
ID Make Model Price Color
5 Honda City 19000 silver
7 Honda Amaze 12000 white
Screenshot from the original page of a bool query that uses must and must_not conditions inside a filter clause

constant_score

When you want a filter on its own, with the same score for every hit, wrap it in constant_score. The Elastic docs say this gives every match a score of 1.0.

kibana dev tools
GET cars/_search
{
  "query": {
    "constant_score": {
      "filter": { "term": { "color": "black" } }
    }
  }
}

Matching documents:

Matching documents for constant_score
ID Make Model Price Color
2 Toyota Camry 28000 black
6 Honda Civic 26000 black
10 BMW X1 52000 black

Every hit has the same score of 1.0.

A Common Mistake: Term Query on a Text Field

The term query looks for the exact value that you type, and does not change it. A text field stores words in lower case. So if you search the make field for Ford with a capital F, nothing matches:

kibana dev tools
GET cars/_search
{
  "query": { "term": { "make": "Ford" } }
}

Matching documents: none.

Zero hits. The text field stores the lower case word ford, and a term query does not change the case of what you type.

For an exact match on a name, use the keyword sub field, which stores the value as it is:

kibana dev tools
GET cars/_search
{
  "query": { "term": { "make.keyword": "Ford" } }
}

Matching documents:

Matching documents for A Common Mistake: Term Query on a Text Field
ID Make Model Price Color
8 Ford Figo 11000 white
9 Ford Endeavour 44000 grey

The rule is easy. Use match on text fields, and term on keyword fields, numbers and dates.

Which Query Should You Use?

Which query to use for each job
You want to Use
Get every document match_all
Search words in a text field match
Search the same words in several fields multi_match
Find words together, in order match_phrase
Find an exact value, such as a color or a code term on a keyword field
Find documents that have a field exists
Find numbers or dates between limits range
Join conditions with and, or and not bool
Narrow results without changing the score filter inside bool

Common Query DSL Errors and How to Fix Them

Common Query DSL errors, their usual cause and the fix
Problem Usual cause Fix
A term query finds nothing The field is text, and the value has capital letters Use the keyword sub field, or use match
Only 10 hits come back The default size is 10 Set size, or use from and size for pages
The result order looks random No sort was given, so hits follow the score Add a sort
A sort on a text field fails Text fields are not meant for sorting Sort on the keyword sub field
A should clause does nothing The query also has a must or filter, so should is optional Use minimum_should_match, or move it to must
The exists query misses a document The value is null or an empty list Remember that null counts as missing
The JSON request is rejected A missing comma, or a curly quote copied from a web page Check the commas, and use straight quotes

Practice Exercises

  1. Create the cars index, and find all cars made by Honda with a match query.
  2. Find the cars that have the words compact and sedan together, in that order, with match_phrase.
  3. Find cars from 2022 or later, that cost less than 30000, using a range query on each field inside a bool filter.
  4. Sort all cars by year in descending order, and then by price in ascending order.
  5. Write a bool query that must match the word turbo in the description, must not be black, and should prefer a sunroof.

Once these nine queries feel natural, you can build almost any search from them. Start with a match for the words, add a filter for the exact conditions, and add a sort. Then look at aggregations, which summarise the same results into counts and averages. A good next step is to run these requests in Kibana Dev Tools against your own index, and change one thing at a time.

Popular Tags: