Library scan error
Steps to reproduce
There are two issues. One may be hard to reproduce.
- A user on server djembe.babos.land starts a scan on a lib of server music.ppom.me
What happens?
Scan systematically hangs at the same percentage of scanned tracks.
Context
Both instances are running 1.3.1.
djembe.babos.land celery-worker logs:
[2023-10-24 20:56:19,726: WARNING/ForkPoolWorker-2] [celery] Error during task 1c0b4c1a-9e5b-4548-9dbd-efd711863019: Invalid actor payload: <!DOCTYPE html>
<html lang="en">
... default index.html ...
</html>
[2023-10-24 20:56:19,727: WARNING/ForkPoolWorker-2] Traceback (most recent call last):
[2023-10-24 20:56:19,728: WARNING/ForkPoolWorker-2] File "/venv/lib/python3.10/site-packages/requests/models.py", line 971, in json
return complexjson.loads(self.text, **kwargs)
[2023-10-24 20:56:19,728: WARNING/ForkPoolWorker-2] File "/usr/lib/python3.10/json/__init__.py", line 346, in loads
return _default_decoder.decode(s)
[2023-10-24 20:56:19,728: WARNING/ForkPoolWorker-2] File "/usr/lib/python3.10/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
[2023-10-24 20:56:19,728: WARNING/ForkPoolWorker-2] File "/usr/lib/python3.10/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
[2023-10-24 20:56:19,728: WARNING/ForkPoolWorker-2] json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
[2023-10-24 20:56:19,728: WARNING/ForkPoolWorker-2]
During handling of the above exception, another exception occurred:
[2023-10-24 20:56:19,728: WARNING/ForkPoolWorker-2] Traceback (most recent call last):
[2023-10-24 20:56:19,728: WARNING/ForkPoolWorker-2] File "/app/funkwhale_api/federation/actors.py", line 23, in get_actor_data
return response.json()
[2023-10-24 20:56:19,728: WARNING/ForkPoolWorker-2] File "/venv/lib/python3.10/site-packages/requests/models.py", line 975, in json
raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
[2023-10-24 20:56:19,728: WARNING/ForkPoolWorker-2] requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
[2023-10-24 20:56:19,729: WARNING/ForkPoolWorker-2]
During handling of the above exception, another exception occurred:
[2023-10-24 20:56:19,729: WARNING/ForkPoolWorker-2] Traceback (most recent call last):
[2023-10-24 20:56:19,729: WARNING/ForkPoolWorker-2] File "/venv/lib/python3.10/site-packages/celery/app/trace.py", line 451, in trace_task
R = retval = fun(*args, **kwargs)
[2023-10-24 20:56:19,729: WARNING/ForkPoolWorker-2] File "/venv/lib/python3.10/site-packages/celery/app/trace.py", line 734, in __protected_call__
return self.run(*args, **kwargs)
[2023-10-24 20:56:19,729: WARNING/ForkPoolWorker-2] File "/venv/lib/python3.10/site-packages/celery/app/autoretry.py", line 34, in run
return task._orig_run(*args, **kwargs)
[2023-10-24 20:56:19,729: WARNING/ForkPoolWorker-2] File "/app/funkwhale_api/taskapp/celery.py", line 55, in inner
return function(*args, **kwargs)
[2023-10-24 20:56:19,729: WARNING/ForkPoolWorker-2] File "/app/funkwhale_api/music/tasks.py", line 128, in scan_library_page
upload = item_serializer.save(library=library_scan.library)
[2023-10-24 20:56:19,729: WARNING/ForkPoolWorker-2] File "/venv/lib/python3.10/site-packages/rest_framework/serializers.py", line 212, in save
self.instance = self.create(validated_data)
[2023-10-24 20:56:19,729: WARNING/ForkPoolWorker-2] File "/usr/lib/python3.10/contextlib.py", line 79, in inner
return func(*args, **kwds)
[2023-10-24 20:56:19,729: WARNING/ForkPoolWorker-2] File "/app/funkwhale_api/federation/serializers.py", line 1620, in create
).create(validated_data["track"])
[2023-10-24 20:56:19,729: WARNING/ForkPoolWorker-2] File "/app/funkwhale_api/federation/serializers.py", line 1509, in create
references[url] = actors.get_actor(url)
[2023-10-24 20:56:19,729: WARNING/ForkPoolWorker-2] File "/app/funkwhale_api/federation/actors.py", line 40, in get_actor
data = get_actor_data(fid)
[2023-10-24 20:56:19,730: WARNING/ForkPoolWorker-2] File "/app/funkwhale_api/federation/actors.py", line 25, in get_actor_data
raise ValueError(f"Invalid actor payload: {response.text}")
[2023-10-24 20:56:19,730: WARNING/ForkPoolWorker-2] ValueError: Invalid actor payload: <!DOCTYPE html>
<html lang="en">
... default index.html ...
</html>
[2023-10-24 20:56:19,730: ERROR/ForkPoolWorker-2] Task music.scan_library_page[1c0b4c1a-9e5b-4548-9dbd-efd711863019] raised unexpected: ValueError('Invalid actor payload: <!DOCTYPE html>\n<html lang="en">... default index.html ...</html>\n')
Traceback (most recent call last):
File "/venv/lib/python3.10/site-packages/requests/models.py", line 971, in json
return complexjson.loads(self.text, **kwargs)
File "/usr/lib/python3.10/json/__init__.py", line 346, in loads
return _default_decoder.decode(s)
File "/usr/lib/python3.10/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python3.10/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/app/funkwhale_api/federation/actors.py", line 23, in get_actor_data
return response.json()
File "/venv/lib/python3.10/site-packages/requests/models.py", line 975, in json
raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/venv/lib/python3.10/site-packages/celery/app/trace.py", line 451, in trace_task
R = retval = fun(*args, **kwargs)
File "/venv/lib/python3.10/site-packages/celery/app/trace.py", line 734, in __protected_call__
return self.run(*args, **kwargs)
File "/venv/lib/python3.10/site-packages/celery/app/autoretry.py", line 34, in run
return task._orig_run(*args, **kwargs)
File "/app/funkwhale_api/taskapp/celery.py", line 55, in inner
return function(*args, **kwargs)
File "/app/funkwhale_api/music/tasks.py", line 128, in scan_library_page
upload = item_serializer.save(library=library_scan.library)
File "/venv/lib/python3.10/site-packages/rest_framework/serializers.py", line 212, in save
self.instance = self.create(validated_data)
File "/usr/lib/python3.10/contextlib.py", line 79, in inner
return func(*args, **kwds)
File "/app/funkwhale_api/federation/serializers.py", line 1620, in create
).create(validated_data["track"])
File "/app/funkwhale_api/federation/serializers.py", line 1509, in create
references[url] = actors.get_actor(url)
File "/app/funkwhale_api/federation/actors.py", line 40, in get_actor
data = get_actor_data(fid)
File "/app/funkwhale_api/federation/actors.py", line 25, in get_actor_data
raise ValueError(f"Invalid actor payload: {response.text}")
ValueError: Invalid actor payload: <!DOCTYPE html>
<html lang="en">
... default html ...
</html>
music.ppom.me funkwhale-api logs:
2023-10-24 20:56:56,168 django.request ERROR Internal Server Error: /api/v1/federation/actors/clement@music.ppom.me/
Traceback (most recent call last):
File "/venv/lib/python3.10/site-packages/asgiref/sync.py", line 534, in thread_handler
raise exc_info[1]
File "/venv/lib/python3.10/site-packages/django/core/handlers/exception.py", line 38, in inner
response = await get_response(request)
File "/venv/lib/python3.10/site-packages/django/core/handlers/base.py", line 233, in _get_response_async
response = await wrapped_callback(request, *callback_args, **callback_kwargs) Oct 24 22:56:56 musi docker-funkwhale-api-start[5083]: File "/venv/lib/python3.10/site-packages/asgiref/sync.py", line 479, in __call__
ret: _R = await loop.run_in_executor(
File "/venv/lib/python3.10/site-packages/asgiref/current_thread_executor.py", line 40, in run
result = self.fn(*self.args, **self.kwargs)
File "/venv/lib/python3.10/site-packages/asgiref/sync.py", line 538, in thread_handler
return func(*args, **kwargs)
File "/usr/lib/python3.10/contextlib.py", line 79, in inner
return func(*args, **kwds)
File "/venv/lib/python3.10/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
return view_func(*args, **kwargs)
File "/venv/lib/python3.10/site-packages/rest_framework/viewsets.py", line 125, in view
return self.dispatch(request, *args, **kwargs)
File "/venv/lib/python3.10/site-packages/rest_framework/views.py", line 509, in dispatch
response = self.handle_exception(exc)
File "/venv/lib/python3.10/site-packages/rest_framework/views.py", line 469, in handle_exception
self.raise_uncaught_exception(exc)
File "/venv/lib/python3.10/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception
raise exc
File "/venv/lib/python3.10/site-packages/rest_framework/views.py", line 506, in dispatch
response = handler(request, *args, **kwargs)
File "/venv/lib/python3.10/site-packages/rest_framework/mixins.py", line 54, in retrieve
instance = self.get_object()
File "/app/funkwhale_api/federation/api_views.py", line 294, in get_object
return queryset.get(preferred_username=username, domain_id=domain)
File "/venv/lib/python3.10/site-packages/django/db/models/query.py", line 435, in get Oct 24 22:56:56 musi docker-funkwhale-api-start[5083]: raise self.model.DoesNotExist(
funkwhale_api.federation.models.Actor.DoesNotExist: Actor matching query does not exist.
Conclusion
So there are 2 issues I think:
- The
Actor.DoesNotExist
exception is not catched by music.ppom.me- music.ppom.me returns the default HTML
- djembe.babos.land can't parse the JSON (because it's HTML)
- There is this unknown clement@music.ppom.me:
- Doing a full-text search in djembe.babos.land AND in music.ppom.me SQL dumps, no clement@music.ppom.me can be found.
- There is a clement@music.intbh.space though, but how would it change its
familyinstance name?