What tools do we use to collect and visualize data?
We use Segment, a platform for data collection and management, to collect usage data. We then feed that data into Amplitude, a tool for graphing and highlighting data, so that we can build visualizations according to our needs.What kind of data do we collect?
Our data collection is focused on the following categories:- System metrics, such as the technical specs of the device running Meilisearch, the software version, and the OS
- Performance metrics, such as the success rate of search requests and the average latency
- Usage metrics, aimed at evaluating our newest features. These change with each new version
- Identify or track users
- Collect personal information such as IP addresses, email addresses, or website URLs
- Store data from documents added to a Meilisearch instance
Why collect telemetry data?
We collect telemetry data for only two reasons: so that we can improve our product, and so that we can continue working on this project full-time. In order to create a better product, we need reliable quantitative information. The data we collect helps us fix bugs, evaluate the success of features, and better understand our users’ needs. We also need to prove that people are actually using Meilisearch. Usage metrics help us justify our existence to investors so that we can keep this project alive.Why should you trust us?
Don’t trust us—hold us accountable. We feel that it is understandable, and in fact wise, to be distrustful of tech companies when it comes to your private data. That is why we attempt to maintain complete transparency about our data collection, provide an opt-out, and enable users to request the deletion of all their collected data at any time. In the absence of global data protection laws, we believe that this is the only ethical way to approach data collection. No company is perfect. If you ever feel that we are being anything less than 100% transparent or collecting data that is infringing on your personal privacy, please let us know by emailing our dedicated account: [email protected]. Similarly, if you discover a data rights initiative or data protection tool that you think is relevant to us, please share it. We are passionate about this subject and take it very seriously.How to disable data collection
Data collection can be disabled at any time by setting a command-line option or environment variable, then restarting the Meilisearch instance.- Command-line option
- Environment variable
- With systemctl
How to delete all collected data
We, the Meilisearch team, provide an email address so that users can request the complete removal of their data from all of our tools. To do so, send an email to [email protected] containing the unique identifier generated for your Meilisearch installation (Instance UID when launching Meilisearch). Any questions regarding the management of the data we collect can also be sent to this email address.
Exhaustive list of all collected data
Whenever an event is triggered that collects some piece of data, Meilisearch does not send it immediately. Instead, it bundles it with other data in a batch of up to500kb. Batches are sent either every hour, or after reaching 500kb—whichever occurs first. This is done in order to improve performance and reduce network traffic.
This list is liable to change with every new version of Meilisearch. It’s not because we’re trying to be sneaky! It’s because when we add new features we need to collect additional data points to see how they perform.
| Metric name | Description | Example |
|---|---|---|
context.app.version | Meilisearch version number | 1.3.0 |
infos.env | Value of --env/MEILI_ENV | production |
infos.db_path | true if --db-path/MEILI_DB_PATH is specified | true |
infos.import_dump | true if --import-dump is specified | true |
infos.dump_dir | true if --dump-dir/MEILI_DUMP_DIR is specified | true |
infos.ignore_missing_dump | true if --ignore-missing-dump is activated | true |
infos.ignore_dump_if_db_exists | true if --ignore-dump-if-db-exists is activated | true |
infos.import_snapshot | true if --import-snapshot is specified | true |
infos.schedule_snapshot | Value of --schedule_snapshot/MEILI_SCHEDULE_SNAPSHOT if set, otherwise None | 86400 |
infos.snapshot_dir | true if --snapshot-dir/MEILI_SNAPSHOT_DIR is specified | true |
infos.ignore_missing_snapshot | true if --ignore-missing-snapshot is activated | true |
infos.ignore_snapshot_if_db_exists | true if --ignore-snapshot-if-db-exists is activated | true |
infos.http_addr | true if --http-addr/MEILI_HTTP_ADDR is specified | true |
infos.http_payload_size_limit | Value of --http-payload-size-limit/MEILI_HTTP_PAYLOAD_SIZE_LIMIT in bytes | 336042103 |
infos.log_level | Value of --log-level/MEILI_LOG_LEVEL | debug |
infos.max_indexing_memory | Value of --max-indexing-memory/MEILI_MAX_INDEXING_MEMORY in bytes | 336042103 |
infos.max_indexing_threads | Value of --max-indexing-threads/MEILI_MAX_INDEXING_THREADS in integer | 4 |
infos.log_level | Value of --log-level/MEILI_LOG_LEVEL | debug |
infos.ssl_auth_path | true if --ssl-auth-path/MEILI_SSL_AUTH_PATH is specified | false |
infos.ssl_cert_path | true if --ssl-cert-path/MEILI_SSL_CERT_PATH is specified | false |
infos.ssl_key_path | true if --ssl-key-path/MEILI_SSL_KEY_PATH is specified | false |
infos.ssl_ocsp_path | true if --ssl-ocsp-path/MEILI_SSL_OCSP_PATH is specified | false |
infos.ssl_require_auth | Value of --ssl-require-auth/MEILI_SSL_REQUIRE_AUTH as a boolean | false |
infos.ssl_resumption | true if --ssl-resumption/MEILI_SSL_RESUMPTION is specified | false |
infos.ssl_tickets | true if --ssl-tickets/MEILI_SSL_TICKETS is specified | false |
system.distribution | Distribution on which Meilisearch is launched | Arch Linux |
system.kernel_version | Kernel version on which Meilisearch is launched | 5.14.10 |
system.cores | Number of cores | 24 |
system.ram_size | Total RAM capacity. Expressed in KB | 16777216 |
system.disk_size | Total capacity of the largest disk. Expressed in Bytes | 1048576000 |
system.server_provider | Value of MEILI_SERVER_PROVIDER environment variable | AWS |
stats.database_size | Database size. Expressed in Bytes | 2621440 |
stats.indexes_number | Number of indexes | 2 |
start_since_days | Number of days since instance was launched | 365 |
user_agent | User-agent header encountered during API calls | [“Meilisearch Ruby (2.1)”, “Ruby (3.0)“] |
requests.99th_response_time | Highest latency from among the fastest 99% of successful search requests | 57ms |
requests.total_succeeded | Total number of successful requests | 3456 |
requests.total_failed | Total number of failed requests | 24 |
requests.total_received | Total number of received search requests | 3480 |
requests.total_degraded | Total number of searches canceled after reaching search time cut-off | 100 |
requests.total_used_negative_operator | Count searches using either a negative word or a negative phrase operator | 173 |
sort.with_geoPoint | true if the sort rule _geoPoint is specified | true |
sort.avg_criteria_number | Average number of sort criteria among all search requests containing the sort parameter | 2 |
filter.with_geoBoundingBox | true if the filter rule _geoBoundingBox is specified | false |
filter.with_geoRadius | true if the filter rule _geoRadius is specified | false |
filter.most_used_syntax | Most used filter syntax among all search requests containing the filter parameter | string |
filter.on_vectors | true if the filter rule includes _vector | false |
q.max_terms_number | Highest number of terms given for the q parameter | 5 |
pagination.max_limit | Highest value given for the limit parameter | 60 |
pagination.max_offset | Highest value given for the offset parameter | 1000 |
formatting.max_attributes_to_retrieve | Maximum number of attributes to retrieve | 100 |
formatting.max_attributes_to_highlight | Maximum number of attributes to highlight | 100 |
formatting.highlight_pre_tag | true if highlightPreTag is specified | false |
formatting.highlight_post_tag | true if highlightPostTag is specified | false |
formatting.max_attributes_to_crop | Maximum number of attributes to crop | 100 |
formatting.crop_length | true if cropLength is specified | false |
formatting.crop_marker | true if cropMarker is specified | false |
formatting.show_matches_position | true if showMatchesPosition is used in this batch | false |
facets.avg_facets_number | Average number of facets | 10 |
primary_key | Name of primary key when explicitly set. Otherwise null | id |
payload_type | All values encountered in the Content-Type header, including invalid ones | [“application/json”, “text/plain”, “application/x-ndjson”] |
index_creation | true if a document addition or update request triggered index creation | true |
ranking_rules.words_position | Position of the words ranking rule if any, otherwise null | 1 |
ranking_rules.typo_position | Position of the typo ranking rule if any, otherwise null | 2 |
ranking_rules.proximity_position | Position of the proximity ranking rule if any, otherwise null | 3 |
ranking_rules.attribute_position | Position of the attribute ranking rule if any, otherwise null | 4 |
ranking_rules.sort_position | Position of the sort ranking rule | 5 |
ranking_rules.exactness_position | Position of the exactness ranking rule if any, otherwise null | 6 |
ranking_rules.values | A string representing the ranking rules without the custom asc-desc rules | ”words, typo, attribute, sort, exactness” |
sortable_attributes.total | Number of sortable attributes | 3 |
sortable_attributes.has_geo | true if _geo is set as a sortable attribute | true |
filterable_attributes.total | Number of filterable attributes | 3 |
filterable_attributes.has_geo | true if _geo is set as a filterable attribute | false |
filterable_attributes.has_patterns | true if filterableAttributes uses attributePatterns | true |
searchable_attributes.total | Number of searchable attributes | 4 |
searchable_attributes.with_wildcard | true if * is specified as a searchable attribute | false |
per_task_uid | true if a uids is used to fetch a particular task resource | true |
filtered_by_uid | true if tasks are filtered by the uids query parameter | false |
filtered_by_index_uid | true if tasks are filtered by the indexUids query parameter | false |
filtered_by_type | true if tasks are filtered by the types query parameter | false |
filtered_by_status | true if tasks are filtered by the statuses query parameter | false |
filtered_by_canceled_by | true if tasks are filtered by the canceledBy query parameter | false |
filtered_by_before_enqueued_at | true if tasks are filtered by the beforeEnqueuedAt query parameter | false |
filtered_by_after_enqueued_at | true if tasks are filtered by the afterEnqueuedAt query parameter | false |
filtered_by_before_started_at | true if tasks are filtered by the beforeStartedAt query parameter | false |
filtered_by_after_started_at | true if tasks are filtered by the afterStartedAt query parameter | false |
filtered_by_before_finished_at | true if tasks are filtered by the beforeFinishedAt query parameter | false |
filtered_by_after_finished_at | true if tasks are filtered by the afterFinishedAt query parameter | false |
typo_tolerance.enabled | true if typo tolerance is enabled | true |
typo_tolerance.disable_on_attributes | true if at least one value is defined for disableOnAttributes | false |
typo_tolerance.disable_on_words | true if at least one value is defined for disableOnWords | false |
typo_tolerance.min_word_size_for_typos.one_typo | The defined value for the minWordSizeForTypos.oneTypo parameter | 5 |
typo_tolerance.min_word_size_for_typos.two_typos | The defined value for the minWordSizeForTypos.twoTypos parameter | 9 |
pagination.max_total_hits | The defined value for the pagination.maxTotalHits property | 1000 |
faceting.max_values_per_facet | The defined value for the faceting.maxValuesPerFacet property | 100 |
distinct_attribute.set | true if a field name is specified | false |
distinct | true if a distinct was specified in an aggregated list of requests | true |
proximity_precision.set | true if the setting has been manually set. | false |
proximity_precision.value | byWord or byAttribute. | byWord |
facet_search.set | facetSearch has been changed by the user | true |
facet_search.value | facetSearch value set by the user | true |
prefix_search.set | prefixSearch has been changed by the user | true |
prefix_search.value | prefixSearch value set by the user | indexingTime |
displayed_attributes.total | Number of displayed attributes | 3 |
displayed_attributes.with_wildcard | true if * is specified as a displayed attribute | false |
stop_words.total | Number of stop words | 3 |
separator_tokens.total | Number of separator tokens | 3 |
non_separator_tokens.total | Number of non-separator tokens | 3 |
dictionary.total | Number of words in the dictionary | 3 |
synonyms.total | Number of synonyms | 3 |
per_index_uid | true if the uid is used to fetch an index stat resource | false |
searches.avg_search_count | The average number of search queries received per call for the aggregated event | 4.2 |
searches.total_search_count | The total number of search queries received for the aggregated event | 16023 |
indexes.avg_distinct_index_count | The average number of queried indexes received per call for the aggregated event | 1.2 |
indexes.total_distinct_index_count | The total number of distinct index queries for the aggregated event | 6023 |
indexes.total_single_index | The total number of calls when only one index is queried | 2007 |
matching_strategy.most_used_strategy | Most used word matching strategy | last |
infos.with_configuration_file | true if the instance is launched with a configuration file | false |
infos.experimental_composite_embedders | true if the compositeEmbedders feature is set to true for this instance | false |
infos.experimental_contains_filter | true if the containsFilter experimental feature is enabled | false |
infos.experimental_edit_documents_by_function | true if the editDocumentsByFunction experimental feature is enabled | false |
infos.experimental_enable_metrics | true if --experimental-enable-metrics is specified at launch | false |
infos.experimental_embedding_cache_entries | Size of configured embedding cache | 100 |
infos.experimental_multimodal | true when multimodal search feature is enabled | true |
infos.experimental_no_edition_2024_for_settings | true if instance disabled new indexer | false |
infos.experimental_replication_parameters | true if --experimental-replication-parameters is specified at launch | false |
infos.experimental_reduce_indexing_memory_usage | true if --experimental-reduce-indexing-memory-usage is specified at launch | false |
infos.experimental_logs_mode | human or json depending on the value specified | human |
infos.experimental_enable_logs_route | true if --experimental-enable-logs-route is specified at launch | false |
infos.gpu_enabled | true if Meilisearch was compiled with CUDA support | false |
swap_operation_number | Number of swap operations | 2 |
pagination.most_used_navigation | Most used search results navigation | estimated |
per_document_id | true if the DELETE /indexes/:indexUid/documents/:documentUid endpoint was used | false |
per_filter | true if POST /indexes/:indexUid/documents/fetch, GET /indexes/:indexUid/documents/, or POST /indexes/:indexUid/documents/delete endpoints were used | false |
clear_all | true if DELETE /indexes/:indexUid/documents endpoint was used | false |
per_batch | true if the POST /indexes/:indexUid/documents/delete-batch endpoint was used | false |
facets.total_distinct_facet_count | Total number of distinct facets queried for the aggregated event | false |
facets.additional_search_parameters_provided | true if additional search parameters were provided for the aggregated event | false |
faceting.sort_facet_values_by_star_count | true if all fields are set to be sorted by count | false |
faceting.sort_facet_values_by_total | The number of different values that were set | 10 |
scoring.show_ranking_score | true if showRankingScore used in the aggregated event | true |
scoring.show_ranking_score_details | true if showRankingScoreDetails was used in the aggregated event | true |
scoring.ranking_score_threshold | true if rankingScoreThreshold was specified in an aggregated list of requests | true |
attributes_to_search_on.total_number_of_uses | Total number of queries where attributesToSearchOn is set | 5 |
vector.max_vector_size | Highest number of dimensions given for the vector parameter in this batch | 1536 |
vector.retrieve_vectors | true if the retrieve_vectors parameter has been used in this batch. | false |
hybrid.enabled | true if hybrid search been used in the aggregated event | true |
hybrid.semantic_ratio | true if semanticRatio was used in this batch, otherwise false | false |
hybrid.total_media | Aggregated number of search requests where media is not null | 42 |
embedders.total | Numbers of defined embedders | 2 |
embedders.sources | An array representing the different provided sources | [“huggingFace”, “userProvided”] |
embedders.document_template_used | A boolean indicating if one of the provided embedders has a custom template defined | true |
embedders.document_template_max_bytes | A value indicating the largest value for document TemplateMaxBytes across all embedder | 400 |
embedders.binary_quantization_used | true if the user updated the binary quantized field of the embedded settings | false |
infos.task_queue_webhook | true if the instance is launched with a task queue webhook | false |
infos.experimental_search_queue_size | Size of the search queue | 750 |
infos.experimental_dumpless_upgrade | true if instance is launched with the parameter | true |
locales | List of locales used with /search and /settings routes | [“fra”, “eng”] |
federation.use_federation | true when at least one multi-search request contains a top-level federation object | false |
network_has_self | true if the network object has a non-null self field | true |
network_size | Number of declared remotes | 0 |
network | true when the network experimental feature is enabled | true |
experimental_network | true when the network experimental feature is enabled | true |
remotes.total_distinct_remote_count | Sum of the number of distinct remotes appearing in each search request of the aggregate | 48 |
remotes.avg_distinct_remote_count | Average number of distinct remotes appearing in a search request of the aggregate | 2.33 |
multimodal | true when multimodal search is enabled via the /experimental-features route | true |
export.total_received | Number of exports received in this batch | 152 |
export.has_api_key | Number of exports with an API Key set | 89 |
export.avg_index_patterns | Average number of index patterns set per export | 3.2 |
export.avg_patterns_with_filter | Average number of index patterns with filters per export | 1.7 |
export.avg_payload_size | Average payload size per export | 512 |
webhooks_created | Number of webhooks created in an instance | 2 |
webhooks.updated | Number of times all webhooks in an instance have been updated | 5 |
with_vector_filter | true when a document fetch request used a vector filter | false |