Funkwhale unusable after running inplace import as a cronjob (#1068) · Issues · funkwhale / funkwhale

Funkwhale unusable after running inplace import as a cronjob

Steps to reproduce

Run manage import_files "/music/**/*" --in-place --recursive --async --no-input as a cronjob over an extended period of time

What happens?

Upload entries with status Pending/Error accumulate in the database to in my case now over 8 Million. That makes Funkwhale very slow and sometimes unusable. The majority of the entries shows the status pending. However, actually new files in the directory still get added to the database correctly. So, with every import the entry count with status "Pending" increases by roughly the number of files existing in the specified directory. I am aware that i am importing every file with the inplace import command (the folder mostly contains mp3 and flac files, only few .jpgs etc.), but shouldn't funkwhale be able to handle this? I tried to delete the Pending and Error Entries, but it is not possible to delete that amount via the Django Web Admin, and I have no idea how to do it with the manage command.

What is expected?

When using inplace import, only new files should be registered in the database at all.

Context

Funkwhale version: 0.20.1 in a Docker environment

## Steps to reproduce

1. Run `manage import_files "/music/**/*" --in-place --recursive --async --no-input` as a cronjob over an extended period of time

## What happens?

Upload entries with status Pending/Error accumulate in the database to in my case now over 8 Million. That makes Funkwhale very slow and sometimes unusable. The majority of the entries shows the status pending. However, actually new files in the directory still get added to the database correctly. So, with every import the entry count with status "Pending" increases by roughly the number of files existing in the specified directory.
I am aware that i am importing every file with the inplace import command (the folder mostly contains mp3 and flac files, only few .jpgs etc.), but shouldn't funkwhale be able to handle this? 
I tried to delete the Pending and Error Entries, but it is not possible to delete that amount via the Django Web Admin, and I have no idea how to do it with the manage command.

## What is expected?

When using inplace import, only new files should be registered in the database at all.

## Context

Funkwhale version: 0.20.1 in a Docker environment

Edited Mar 29, 2020 by git1873

To upload designs, you'll need to enable LFS. More information