Skip to content

docs: document which query_by field controls query tokenization - #495

Open
tharropoulos wants to merge 2 commits into
typesense:masterfrom
tharropoulos:query-by-callout
Open

docs: document which query_by field controls query tokenization#495
tharropoulos wants to merge 2 commits into
typesense:masterfrom
tharropoulos:query-by-callout

Conversation

@tharropoulos

Copy link
Copy Markdown
Collaborator

Change Summary

  • add a :::warning callout under the Query parameters table in api/search.md explaining the search query is tokenized once using the highest-weighted query_by field's locale, token_separators, symbols_to_index and stemming
  • note in the callout that those tokens are reused for every queried field while each field is still indexed, faceted and highlighted with its own settings
  • append a short pointer to the query_by cell linking down to the callout
  • clarify that the highest-weighted field is the first listed in query_by unless query_by_weights reorders precedence
  • add pointers on the token_separators, symbols_to_index, locale and stem settings in api/collections.md marking them index-time only while query parsing uses the top query_by field
  • add a matching callout to the guide/locale.md multilingual guide covering how the top query_by field decides tokenization when fields use different locales
  • replicate the api/search.md and api/collections.md notes across versions 28.0 through 30.2

PR Checklist

…query_by field

Add pointers to the token_separators, symbols_to_index, locale and stem field settings clarifying that they apply at index time, and that query parsing instead uses the highest-weighted query_by field. Applies across versions 28.0 through 30.2.
…ighted query_by field

Add a callout under the Query parameters table explaining that the search query is tokenized once using the highest-weighted query_by field's locale, token_separators, symbols_to_index and stemming, and that each field is still indexed with its own settings. Also add a short pointer in the query_by cell. Applies across versions 28.0 through 30.2.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant