Skip to content
GitLab
Projects Groups Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
  • Sign in / Register
  • funkwhale funkwhale
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
    • Locked Files
  • Issues 406
    • Issues 406
    • List
    • Boards
    • Service Desk
    • Milestones
    • Iterations
    • Requirements
  • Merge requests 22
    • Merge requests 22
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
    • Test Cases
  • Deployments
    • Deployments
    • Releases
  • Packages and registries
    • Packages and registries
    • Package Registry
    • Container Registry
    • Infrastructure Registry
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Code review
    • Insights
    • Issue
    • Repository
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • funkwhalefunkwhale
  • funkwhalefunkwhale
  • Issues
  • #1676
Closed
Open
Issue created Jan 17, 2022 by f1reflyyyylmao@f1reflyyyylmao

Database growing way too much (music_upload)

Steps to reproduce

Unfortunately I don't have enough time to trial-and-error a reproduction. I'll try to describe what lead to this situation.

My funkwhale is about a year old now and my library consists of ~4600 Files (according to find -type f), totalling 39 GiB, there's probably some other non-music files sprinkled in. Funkwhale, however, tells me my Library consist of 292994 Songs, according to the number that shows up in the library management view.

Because I use syncthing to get my music library that I manage with https://beets.io synced to my music server which resides on a vps in a datacenter, I don't want to manually tell funkwhale to keep it updated. So I set up a script that is ran by cron every three hours that performs some maintenance:

import_files --recursive --in-place --no-input --username $myuser $my_library_uuid /path/to/my/synced/music/
check_inplace_files --no-dry-run
prune_library --no-dry-run --artists --albums --tracks
rebuild_music_permissions

Which worked well for me so far.

What happens?

I was recently greeted by a 500 Server error, which was caused by postgresql crashing due to the harddisk usage reaching 100%. When I inspect my funkwhale database in postgresql, it tells me the music_upload table totals 28GiB.

What is expected?

The music_upload tables's size stays reasonably low, or functionality is added to prune it.

Context

Funkwhale version(s) affected: 1.1.4

  • OS: Debian 10.10

I'd added some logs first but those are 100% irrelevant to this case. I decided to inspect the contents of music_upload and a single row(!) for a song takes 1.1MiB to store on disk when writing it out: dump

Edited Jan 17, 2022 by f1reflyyyylmao
Assignee
Assign to
Time tracking