Delete huge field from index

Elastic Search
1

Hi,

due to a bug in our software, we indexed a field with many thousands lines. Now we are short on disk space, but still need the index. What is the best way in ES 8, to get rid of the huge field to get back the disk space? Index size is about 5TB.

A reindex without the field can be problematic, because we don't have that much space left and the index is a monthly index and we are still writing to it.

It should be possible with "_update_by_query" and delete the field with

"script": "ctx._source.remove('name_of_field')",

right? But if so, will I get back the used disk space? Do I need a force merge after deleting it?

Any other ideas?
Thanks!

2

It will be slow and will consume a lot of IO.

Not immediately. It will first consume much more disk space.

Yes. And you can use: only_expunge_deletes option.

Reindexing would be better...
If you still have some disk space, may be "wait" for the month to end and your "old" "bad" indices will be removed???

3

Isnt only_expunge_deletes for deleted documents? Or does it mean, behind the scenes, ES is writing a new document and the old is going to be deleted afterwards?

Unfortunately we need to store the index for 14 months.
But I will try to make some room to reindex it. I also think this is the cleanest solution.

Thanks for your help.

4

That's correct.