CLI importer improvements
For context, someone is making some attempts at importing a 30TB library in Funkwhale via CLI. This MR is a results of direct discussion with this person to ensure the importer works and doesn't crash on such large libraries.
This mainly improves the filesystem crawler,
- Handle import in batches, to avoid OOM issues
Optimization of the
check_inplace_filescommand to avoid OOM issues
- Better UX with CLI importer, by removing the need to use glob patterns