Replace traceback with more user friendly output when manual upgrade DB migration fails due to missing album cover files
/label Type: New feature Status: Need triage
What is the problem you are facing?
I upgraded a manual installation from 20.1 to 21.0. The database migration crashed because of a missing album cover jpg. A traceback was printed. I copied another album cover cp existing.jpg missing.jpg
and the migration command now worked as expected.
Instead of the traceback I suggest a more user friendly error message. Some thing like "The migration failed. The album cover path-to-file is missing."
Another idea is that the migration resolves this on it's own by removing the reference to the file, or creates a dummy file representing missing album art.
As for why the image was missing, it is very likely that I unknowingly messed up while editing an album using the Django GUI. So I don't think this crash was caused by a bug.
What are the possible drawbacks or issues with the requested changes?
Simplified output might not contain enough information to solve the problem. The traceback needs to be available in a log file or with a command option.
Context
- Funkwhale version(s) affected: 20.1 upgrade to 21.0
- OS: Ubuntu 18.04.4 LTS
Crash output:
sudo -u funkwhale -H -E /srv/funkwhale/virtualenv/bin/python api/manage.py migrate
/srv/funkwhale/virtualenv/bin/python: can't open file 'api/manage.py': [Errno 2] No such file or directory
root@funkwhale:/srv/funkwhale/data/media/albums/covers/2019/02/17# cd -
/srv/funkwhale
root@funkwhale:/srv/funkwhale# sudo -u funkwhale -H -E /srv/funkwhale/virtualenv/bin/python api/manage.py migrate
2020-05-30 01:38:09,837 funkwhale_api.config INFO Loaded env file at /srv/funkwhale/config/.env
2020-05-30 01:38:09,838 funkwhale_api.config INFO Running with no plugins
Operations to perform:
Apply all migrations: account, admin, audio, auth, authtoken, common, contenttypes, dynamic_preferences, favorites, federation, history, moderation, music, playlists, radios, requests, sessions, sites, socialaccount, tags, users
Running migrations:
Applying music.0043_album_cover_attachment...Creating attachments for 276 album covers, this may take a while…
Traceback (most recent call last):
File "/srv/funkwhale/api/funkwhale_api/music/migrations/0043_album_cover_attachment.py", line 23, in create_attachments
size = album.cover.size
File "/srv/funkwhale/virtualenv/lib/python3.6/site-packages/django/db/models/fields/files.py", line 69, in size
return self.storage.size(self.name)
File "/srv/funkwhale/virtualenv/lib/python3.6/site-packages/django/core/files/storage.py", line 327, in size
return os.path.getsize(self.path(name))
File "/usr/lib/python3.6/genericpath.py", line 50, in getsize
return os.stat(filename).st_size
FileNotFoundError: [Errno 2] No such file or directory: '/srv/funkwhale/data/media/albums/covers/2019/02/17/b6b3ae3c5-1779-4f18-aef3-c2f4dc9c9788.jpg'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "api/manage.py", line 27, in <module>
execute_from_command_line(sys.argv)
File "/srv/funkwhale/virtualenv/lib/python3.6/site-packages/django/core/management/__init__.py", line 401, in execute_from_command_line
utility.execute()
File "/srv/funkwhale/virtualenv/lib/python3.6/site-packages/django/core/management/__init__.py", line 395, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/srv/funkwhale/virtualenv/lib/python3.6/site-packages/django/core/management/base.py", line 328, in run_from_argv
self.execute(*args, **cmd_options)
File "/srv/funkwhale/virtualenv/lib/python3.6/site-packages/django/core/management/base.py", line 369, in execute
output = self.handle(*args, **options)
File "/srv/funkwhale/virtualenv/lib/python3.6/site-packages/django/core/management/base.py", line 83, in wrapped
res = handle_func(*args, **kwargs)
File "/srv/funkwhale/virtualenv/lib/python3.6/site-packages/django/core/management/commands/migrate.py", line 233, in handle
fake_initial=fake_initial,
File "/srv/funkwhale/virtualenv/lib/python3.6/site-packages/django/db/migrations/executor.py", line 117, in migrate
state = self._migrate_all_forwards(state, plan, full_plan, fake=fake, fake_initial=fake_initial)
File "/srv/funkwhale/virtualenv/lib/python3.6/site-packages/django/db/migrations/executor.py", line 147, in _migrate_all_forwards
state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial)
File "/srv/funkwhale/virtualenv/lib/python3.6/site-packages/django/db/migrations/executor.py", line 245, in apply_migration
state = migration.apply(state, schema_editor)
File "/srv/funkwhale/virtualenv/lib/python3.6/site-packages/django/db/migrations/migration.py", line 124, in apply
operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
File "/srv/funkwhale/virtualenv/lib/python3.6/site-packages/django/db/migrations/operations/special.py", line 190, in database_forwards
self.code(from_state.apps, schema_editor)
File "/srv/funkwhale/api/funkwhale_api/music/migrations/0043_album_cover_attachment.py", line 26, in create_attachments
print(" Warning: cover file wasn't found in storage: {}".format(e.__class__))
NameError: name 'e' is not defined
Fix:
root@funkwhale:/srv/funkwhale# cd /srv/funkwhale/data/media/albums/covers/2019/02/17/
root@funkwhale:/srv/funkwhale/data/media/albums/covers/2019/02/17# ls -l b6b3ae3c5-1779-4f18-aef3-c2f4dc9c9788.jpg
ls: cannot access 'b6b3ae3c5-1779-4f18-aef3-c2f4dc9c9788.jpg': No such file or directory
root@funkwhale:/srv/funkwhale/data/media/albums/covers/2019/02/17# sudo -u funkwhale cp bfff250bd-6c12-4f11-ab9e-a84568a74afb.jpg b6b3ae3c5-1779-4f18-aef3-c2f4dc9c9788.jpg
root@funkwhale:/srv/funkwhale/data/media/albums/covers/2019/02/17# ls -l b6b3ae3c5-1779-4f18-aef3-c2f4dc9c9788.jpg
-rw-r--r-- 1 funkwhale funkwhale 84127 May 30 01:38 b6b3ae3c5-1779-4f18-aef3-c2f4dc9c9788.jpg
root@funkwhale:/srv/funkwhale/data/media/albums/covers/2019/02/17# cd -
/srv/funkwhale
root@funkwhale:/srv/funkwhale# sudo -u funkwhale -H -E /srv/funkwhale/virtualenv/bin/python api/manage.py migrate
2020-05-30 01:39:10,327 funkwhale_api.config INFO Loaded env file at /srv/funkwhale/config/.env
2020-05-30 01:39:10,328 funkwhale_api.config INFO Running with no plugins
Operations to perform:
Apply all migrations: account, admin, audio, auth, authtoken, common, contenttypes, dynamic_preferences, favorites, federation, history, moderation, music, playlists, radios, requests, sessions, sites, socialaccount, tags, users
Running migrations:
Applying music.0043_album_cover_attachment...Creating attachments for 276 album covers, this may take a while…
Commiting changes…
OK
Applying music.0044_full_text_search...Creating database trigger music_artist_update_body_text on music_artist…
CREATE TRIGGER music_artist_update_body_text
BEFORE INSERT OR UPDATE
ON music_artist
FOR EACH ROW
EXECUTE PROCEDURE
tsvector_update_trigger(body_text, 'pg_catalog.english', name)
Creating database trigger music_track_update_body_text on music_track…
CREATE TRIGGER music_track_update_body_text
BEFORE INSERT OR UPDATE
ON music_track
FOR EACH ROW
EXECUTE PROCEDURE
tsvector_update_trigger(body_text, 'pg_catalog.english', title, copyright)
Creating database trigger music_album_update_body_text on music_album…
CREATE TRIGGER music_album_update_body_text
BEFORE INSERT OR UPDATE
ON music_album
FOR EACH ROW
EXECUTE PROCEDURE
tsvector_update_trigger(body_text, 'pg_catalog.english', title)
Populating search index for Artist…
Populating search index for Track…
Populating search index for Album…
OK
Applying music.0045_full_text_search_stop_words...Create non stopword dictionary and search configuration…
Dropping database trigger music_artist_update_body_text on music_artist…
Creating database trigger music_artist_update_body_text on music_artist…
Dropping database trigger music_track_update_body_text on music_track…
Creating database trigger music_track_update_body_text on music_track…
Dropping database trigger music_album_update_body_text on music_album…
Creating database trigger music_album_update_body_text on music_album…
Updating search index for Artist…
Updating search index for Track…
Updating search index for Album…
OK
Applying music.0046_auto_20200113_1018... OK
Applying music.0047_auto_20200116_1246... OK
Applying music.0048_auto_20200120_0900... OK
Applying music.0049_auto_20200122_1020... OK
Applying music.0050_auto_20200129_1344... OK
Applying music.0051_auto_20200319_1249... OK
Applying users.0017_actor_avatar...Creating attachments for 0 user avatars, this may take a while…
Commiting changes…
OK