Many operations in Meilisearch are processed asynchronously. These API requests are not handled immediately—instead, Meilisearch places them in a queue and processes them in the order they were received.

​
Which operations are asynchronous?

Every operation that might take a long time to be processed is handled asynchronously. Processing operations asynchronously allows Meilisearch to handle resource-intensive tasks without impacting search performance.

Currently, these are Meilisearch’s asynchronous operations:

  • Creating an index
  • Updating an index
  • Swapping indexes
  • Deleting an index
  • Updating index settings
  • Adding documents to an index
  • Updating documents in an index
  • Deleting documents from an index
  • Canceling a task
  • Deleting a task
  • Creating a dump
  • Creating snapshots

​
Understanding tasks

When an API request triggers an asynchronous process, Meilisearch creates a task and places it in a task queue.

​
Task objects

Tasks are objects containing information that allow you to track their progress and troubleshoot problems when things go wrong.

A task object includes data not present in the original request, such as when the request was enqueued, the type of request, and an error code when the task fails:

{
    "uid": 1,
    "indexUid": "movies",
    "status": "enqueued",
    "type": "documentAdditionOrUpdate",
    "canceledBy": null,
    "details": {
        "receivedDocuments": 67493,
        "indexedDocuments": null
    },
    "error": null,
    "duration": null,
    "enqueuedAt": "2021-08-10T14:29:17.000000Z",
    "startedAt": null,
    "finishedAt": null
}

For a comprehensive description of each task object field, consult the task API reference.

​
Summarized task objects

When you make an API request for an asynchronous operation, Meilisearch returns a summarized version of the full task object.

{
  "taskUid": 0,
  "indexUid": "movies",
  "status": "enqueued",
  "type": "indexCreation",
  "enqueuedAt": "2021-08-11T09:25:53.000000Z"
}

Use the summarized task’s taskUid to track the progress of a task.

​
Task status

Tasks always contain a field indicating the task’s current status. This field has one of the following possible values:

  • enqueued: the task has been received and will be processed soon
  • processing: the task is being processed
  • succeeded: the task has been successfully processed
  • failed: a failure occurred when processing the task. No changes were made to the database
  • canceled: the task was canceled

succeeded, failed, and canceled tasks are finished tasks. Meilisearch keeps them in the task database but has finished processing these tasks. It is possible to configure a webhook to notify external services when a task is finished.

enqueued and processing tasks are unfinished tasks. Meilisearch is either processing them or will do so in the future.

​
Global tasks

Some task types are not associated with a particular index but apply to the entire instance. These tasks are called global tasks. Global tasks always display null for the indexUid field.

Meilisearch considers the following task types as global:

In a protected instance, your API key must have access to all indexes ("indexes": [*]) to view global tasks.

​
Task queue

After creating a task, Meilisearch places it in a queue. Enqueued tasks are processed one at a time, following the order in which they were requested.

When the task queue reaches its limit (about 10GiB), it will throw a no_space_left_on_device error. Users will need to delete tasks using the delete tasks endpoint to continue write operations.

​
Task queue priority

Meilisearch considers certain tasks high-priority and always places them at the front of the queue.

The following types of tasks are always processed as soon as possible:

  1. taskCancelation
  2. taskDeletion
  3. snapshotCreation
  4. dumpCreation

All other tasks are processed in the order they were enqueued.

​
Task workflow

When you make a request for an asynchronous operation, Meilisearch processes all tasks following the same steps:

  1. Meilisearch creates a task, puts it in the task queue, and returns a summarized task object. Task status set to enqueued
  2. When your task reaches the front of the queue, Meilisearch begins working on it. Task status set to processing
  3. Meilisearch finishes the task. Status set to succeeded if task was successfully processed, or failed if there was an error

Terminating a Meilisearch instance in the middle of an asynchronous operation is completely safe and will never adversely affect the database.

​
Task batches

Meilisearch processes tasks in batches, grouping tasks for the best possible performance. In most cases, batching should be transparent and have no impact on the overall task workflow. Use the /batches route to obtain more information on batches and how they are processing your tasks.

​
Canceling tasks

You can cancel a task while it is enqueued or processing by using the cancel tasks endpoint. Doing so changes a task’s status to canceled.

Tasks are not canceled when you terminate a Meilisearch instance. Meilisearch discards all progress made on processing tasks and resets them to enqueued. Task handling proceeds as normal once the instance is relaunched.

​
Deleting tasks

Finished tasks remain visible in the task list. To delete them manually, use the delete tasks route.

Meilisearch stores up to 1M tasks in the task database. If enqueuing a new task would exceed this limit, Meilisearch automatically tries to delete the oldest 100K finished tasks. If there are no finished tasks in the database, Meilisearch does not delete anything and enqueues the new task as usual.

​
Examples

Suppose you add a new document to your instance using the add documents endpoint and receive a taskUid in response.

When you query the get task endpoint using this value, you see that it has been enqueued:

{
    "uid": 1,
    "indexUid": "movies",
    "status": "enqueued",
    "type": "documentAdditionOrUpdate",
    "canceledBy": null,
    "details": {
        "receivedDocuments": 67493,
        "indexedDocuments": null
    },
    "error": null,
    "duration": null,
    "enqueuedAt": "2021-08-10T14:29:17.000000Z",
    "startedAt": null,
    "finishedAt": null
}

Later, you check the task’s progress one more time. It was successfully processed and its status changed to succeeded:

{
    "uid": 1,
    "indexUid": "movies",
    "status": "succeeded",
    "type": "documentAdditionOrUpdate",
    "canceledBy": null,
    "details": {
            "receivedDocuments": 67493,
            "indexedDocuments": 67493
    },
    "error": null,
    "duration": "PT1S",
    "enqueuedAt": "2021-08-10T14:29:17.000000Z",
    "startedAt": "2021-08-10T14:29:18.000000Z",
    "finishedAt": "2021-08-10T14:29:19.000000Z"
}

Had the task failed, the response would have included a detailed error object:

{
    "uid": 1,
    "indexUid": "movies",
    "status": "failed",
    "type": "documentAdditionOrUpdate",
    "canceledBy": null,
    "details": {
            "receivedDocuments": 67493,
            "indexedDocuments": 0
    },
    "error": {
        "message": "Document does not have a `:primaryKey` attribute: `:documentRepresentation`.",
        "code": "internal",
        "type": "missing_document_id",
        "link": "https://docs.meilisearch.com/errors#missing-document-id"
    },
    "duration": "PT1S",
    "enqueuedAt": "2021-08-10T14:29:17.000000Z",
    "startedAt": "2021-08-10T14:29:18.000000Z",
    "finishedAt": "2021-08-10T14:29:19.000000Z"
}

If the task had been canceled while it was enqueued or processing, it would have the canceled status and a non-null value for the canceledBy field.

After a task has been deleted, trying to access it returns a task_not_found error.

Was this page helpful?