Why FT.DROPINDEX with DD option did not delete all my documents?

Question

Why FT.DROPINDEX with DD option did not delete all my documents?

Answer

There may have been an index deletion while the index was indexing asynchronously.

If an index creation is still running (FT.CREATE is running asynchronously), only the document hashes that have already been indexed are deleted. The document hashes left to be indexed remain in the database. To check the completion of the indexing, use FT.INFO.

The same issue can be caused by an error at indexation time. If an error happens, the document is not indexed and won't be deleted by the FT.DROPINDEX command with the DD option set. Check the following example.

An index is created for numeric values.

FT.CREATE numeric_idx PREFIX 1 cnt: SCHEMA val NUMERIC

Add two documents.

HSET cnt:1 val 5
HSET cnt:2 val "word"

It is expected that the second document won't be indexed, because the value of val is not a number. Check it with:

FT.INFO numeric_idx

And confirm that there is a document indexed and that there was a failure.

 9) num_docs
10) "1"
...
41) hash_indexing_failures
42) "1"

Now, remove the index and the documents and verify that only one document was removed.

127.0.0.1:6379> FT.DROPINDEX numeric_idx DD
OK
127.0.0.1:6379> EXISTS cnt:1
(integer) 0
127.0.0.1:6379> EXISTS cnt:2
(integer) 1

References

Refer to the documentation to learn more about FT.DROPINDEX.