...
 
Commits (200)
......@@ -10,6 +10,142 @@ This changelog is viewable on the web at https://docs.funkwhale.audio/changelog.
.. towncrier
0.18.2 (2019-02-13)
-------------------
Upgrade instructions are available at
https://docs.funkwhale.audio/index.html
Enhancements:
- Added a 'fix_federation_ids' management command to deal with protocol/domain issues in federation
IDs after deployments (#706)
- Can now use a local file with FUNKWHALE_SPA_HTML_ROOT to avoid sending an HTTP request (#705)
Bugfixes:
- Downgraded channels dependency to 2.1.6 to fix denied uploads (#697)
- Fixed cards display issues on medium/small screens (#707)
- Fixed Embed component name that could lead to issue when developping on OSX (#696)
- Fixed resizing issues for album cards on artist pages (#694)
0.18.1 (2019-01-29)
-------------------
Upgrade instructions are available at
https://docs.funkwhale.audio/index.html
Fix Gzip compression to avoid BREACH exploit [security] [manual action required]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
In the 0.18 release, we've enabled Gzip compression by default for various
content types, including HTML and JSON. Unfortunately, enabling Gzip compression
on such content types could make BREACH-type exploits possible.
We've removed the risky content-types from our nginx template files, to ensure new
instances are safe, however, if you already have an instance, you need
to double check that your host nginx virtualhost do not include the following
values for the ``gzip_types`` settings::
application/atom+xml
application/json
application/ld+json
application/activity+json
application/manifest+json
application/rss+xml
application/xhtml+xml
application/xml
For convenience, you can also replace the whole setting with the following snippet::
gzip_types
application/javascript
application/vnd.geo+json
application/vnd.ms-fontobject
application/x-font-ttf
application/x-web-app-manifest+json
font/opentype
image/bmp
image/svg+xml
image/x-icon
text/cache-manifest
text/css
text/plain
text/vcard
text/vnd.rim.location.xloc
text/vtt
text/x-component
text/x-cross-domain-policy;
Many thanks to @jibec for the report!
Fix Apache configuration file for 0.18 [manual action required]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The way front is served has changed since 0.18. The Apache configuration can't serve 0.18 properly, leading to blank screens.
If you are on an Apache setup, you will have to replace the `<Location "/api">` block with the following::
<Location "/">
# similar to nginx 'client_max_body_size 100M;'
LimitRequestBody 104857600
ProxyPass ${funkwhale-api}/
ProxyPassReverse ${funkwhale-api}/
</Location>
And add some more `ProxyPass` directives so that the `Alias` part of your configuration file looks this way::
ProxyPass "/front" "!"
Alias /front /srv/funkwhale/front/dist
ProxyPass "/media" "!"
Alias /media /srv/funkwhale/data/media
ProxyPass "/staticfiles" "!"
Alias /staticfiles /srv/funkwhale/data/static
In case you are using custom css and theming, you also need to match this block::
ProxyPass "/settings.json" "!"
Alias /settings.json /srv/funkwhale/custom/settings.json
ProxyPass "/custom" "!"
Alias /custom /srv/funkwhale/custom
Enhancements:
- Added name attributes on all inputs to improve UX, especially with password managers (#686)
- Disable makemigrations in production and misleading message when running migrate (#685)
- Display progress during file upload
- Hide pagination when there is only one page of results (#681)
- Include shared/public playlists in Subsonic API responses (#684)
- Use proper locale for date-related/duration strings (#670)
Bugfixes:
- Fix transcoding of in-place imported tracks (#688)
- Fixed celery worker defaulting to development settings instead of production
- Fixed crashing Django admin when loading track detail page (#666)
- Fixed list icon alignement on landing page (#668)
- Fixed overescaping issue in notifications and album page (#676)
- Fixed wrong number of affected elements in bulk action modal (#683)
- Fixed wrong URL in documentation for funkwhale_proxy.conf file when deploying using Docker
- Make Apache configuration file work with 0.18 changes (#667)
- Removed potential BREACH exploit because of Gzip compression (#678)
- Upgraded kombu to fix an incompatibility with redis>=3
Documentation:
- Added user upload documentation at https://docs.funkwhale.audio/users/upload.html
0.18 "Naomi" (2019-01-22)
-------------------------
......@@ -189,19 +325,11 @@ in the server block, then reload your nginx server::
gzip_vary on;
gzip_types
application/atom+xml
application/javascript
application/json
application/ld+json
application/activity+json
application/manifest+json
application/rss+xml
application/vnd.geo+json
application/vnd.ms-fontobject
application/x-font-ttf
application/x-web-app-manifest+json
application/xhtml+xml
application/xml
font/opentype
image/bmp
image/svg+xml
......
......@@ -353,12 +353,174 @@ Internationalization
--------------------
We're using https://github.com/Polyconseil/vue-gettext to manage i18n in the project.
When working on the front-end, any end-user string should be translated
using either ``<translate>yourstring</translate>`` or ``$gettext('yourstring')``
function.
When working on the front-end, any end-user string should be marked as a translatable string,
with the proper context, as described below.
Translations in HTML
^^^^^^^^^^^^^^^^^^^^
Translations in HTML use the ``<translate>`` tag::
<template>
<div>
<h1><translate :translate-context="'Content/Profile/Header'">User profile</translate></h1>
<p>
<translate
:translate-context="'Content/Profile/Paragraph'"
:translate-params="{username: 'alice'}">
You are logged in as %{ username }
</translate>
</p>
<p>
<translate
:translate-context="'Content/Profile/Paragraph'"
translate-plural="You have %{ count } new messages, that's a lot!"
:translate-n="unreadMessagesCount"
:translate-params="{count: unreadMessagesCount}">
You have 1 new message
</translate>
</p>
</div>
</template>
Anything between the `<translate>` and `</translate>` delimiters will be considered as a translatable string.
You can use variables in the translated string via the ``:translate-params="{var: 'value'}"`` directive, and reference them like this:
``val value is %{ value }``.
For pluralization, you need to use ``translate-params`` in conjunction with ``translate-plural`` and ``translate-n``:
- ``translate-params`` should contain the variable you're using for pluralization (which is usually shown to the user)
- ``translate-n`` should match the same variable
- The ``<translate>`` delimiters contain the non-pluralized version of your string
- The ``translate-plural`` directive contains the pluralized version of your string
Translations in javascript
^^^^^^^^^^^^^^^^^^^^^^^^^^
Translations in javascript work by calling the ``this.$*gettext`` functions::
export default {
computed: {
strings () {
let tracksCount = 42
let playButton = this.$pgettext('Sidebar/Player/Button/Verb, Short', 'Play')
let loginMessage = this.$pgettext('*/Login/Message', 'Welcome back %{ username }')
let addedMessage = this.$npgettext('*/Player/Message', 'One track was queued', '%{ count } tracks were queued', tracksCount)
console.log(this.$gettextInterpolate(addedMessage, {count: tracksCount}))
console.log(this.$gettextInterpolate(loginMessage, {username: 'alice'}))
}
}
}
The first argument of the ``$pgettext`` and ``$npgettext`` functions is the string context.
Contextualization
^^^^^^^^^^^^^^^^^
Translation contexts provided via the ``translate-context`` directive and the ``$pgettext`` and ``$npgettext`` are never shown to end users
but visible by Funkwhale translators. They help translators where and how the strings are used,
especially with short or ambiguous strings, like ``May``, which can refer a month or a verb.
While we could in theory use free form context, like ``This string is inside a button, in the main page, and is a call to action``,
Funkwhale use a hierarchical structure to write contexts and keep them short and consistents accross the app. The previous context,
rewritten correctly would be: ``Content/Home/Button/Call to action``.
This hierarchical structure is made of several parts:
- The location part, which is required and refers to the big blocks found in Funkwhale UI where the translated string is displayed:
- ``Content``
- ``Footer``
- ``Head``
- ``Menu``
- ``Popup``
- ``Sidebar``
- ``*`` for strings that are not tied to a specific location
- The feature part, which is required, and refers to the feature associated with the translated string:
- ``About``
- ``Admin``
- ``Album``
- ``Artist``
- ``Embed``
- ``Home``
- ``Login``
- ``Library``
- ``Moderation``
- ``Player``
- ``Playlist``
- ``Profile``
- ``Favorites``
- ``Notifications``
- ``Radio``
- ``Search``
- ``Settings``
- ``Signup``
- ``Track``
- ``Queue``
- ``*`` for strings that are not tied to a specific feature
- The component part, which is required and refers to the type of element that contain the string:
- ``Button``
- ``Card``
- ``Checkbox``
- ``Dropdown``
- ``Error message``
- ``Form``
- ``Header``
- ``Help text``
- ``Hidden text``
- ``Icon``
- ``Input``
- ``Image``
- ``Label``
- ``Link``
- ``List item``
- ``Menu``
- ``Message``
- ``Paragraph``
- ``Placeholder``
- ``Tab``
- ``Table``
- ``Title``
- ``Tooltip``
- ``*`` for strings that are not tied to a specific component
The detail part, which is optional and refers to the contents of the string itself, such as:
- ``Adjective``
- ``Call to action``
- ``Noun``
- ``Short``
- ``Unit``
- ``Verb``
Here are a few examples of valid context hierarchies:
- ``Sidebar/Player/Button``
- ``Content/Home/Button/Call to action``
- ``Footer/*/Help text``
- ``*/*/*/Verb, Short``
- ``Popup/Playlist/Button``
- ``Content/Admin/Table.Label/Short, Noun (Value is a date)``
It's possible to nest multiple component parts to reach a higher level of detail. The component parts are then separated by a dot:
- ``Sidebar/Queue/Tab.Title``
- ``Content/*/Button.Title``
- ``Content/*/Table.Header``
- ``Footer/*/List item.Link``
- ``Content/*/Form.Help text``
Collecting translatable strings
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
If you want to ensure your translatable strings are correctly marked for translation,
you can try to extract them.
Extraction is done by calling ``yarn run i18n-extract``, which
will pull all the strings from source files and put them in a PO file.
will pull all the strings from source files and put them in a PO files.
You can then inspect the PO files to ensure everything is fine (but don't commit them, it's not needed).
Contributing to the API
-----------------------
......
......@@ -25,15 +25,18 @@ RUN \
RUN mkdir /requirements
COPY ./requirements/base.txt /requirements/base.txt
# hack around https://github.com/pypa/pip/issues/6158#issuecomment-456619072
ENV PIP_DOWNLOAD_CACHE=/noop/
RUN \
echo 'fixing requirements file for alpine' && \
sed -i '/Pillow/d' /requirements/base.txt && \
\
\
echo 'installing pip requirements' && \
pip3 install --no-cache-dir --upgrade pip && \
pip3 install --no-cache-dir setuptools wheel && \
pip3 install --no-cache-dir -r /requirements/base.txt
pip3 install --upgrade pip && \
pip3 install setuptools wheel && \
pip3 install -r /requirements/base.txt && \
rm -rf $PIP_DOWNLOAD_CACHE
ARG install_dev_deps=0
COPY ./requirements/*.txt /requirements/
......
......@@ -40,6 +40,12 @@ v1_patterns += [
r"^manage/",
include(("funkwhale_api.manage.urls", "manage"), namespace="manage"),
),
url(
r"^moderation/",
include(
("funkwhale_api.moderation.urls", "moderation"), namespace="moderation"
),
),
url(
r"^federation/",
include(
......
......@@ -94,6 +94,9 @@ FEDERATION_MUSIC_NEEDS_APPROVAL = env.bool(
)
# XXX: deprecated, see #186
FEDERATION_ACTOR_FETCH_DELAY = env.int("FEDERATION_ACTOR_FETCH_DELAY", default=60 * 12)
FEDERATION_SERVICE_ACTOR_USERNAME = env(
"FEDERATION_SERVICE_ACTOR_USERNAME", default="service"
)
ALLOWED_HOSTS = env.list("DJANGO_ALLOWED_HOSTS", default=[]) + [FUNKWHALE_HOSTNAME]
# APP CONFIGURATION
......
# -*- coding: utf-8 -*-
__version__ = "0.18"
__version__ = "0.18.2"
__version_info__ = tuple(
[
int(num) if num.isdigit() else num
......
import os
from django.core.management.base import CommandError
from django.core.management.commands.makemigrations import Command as BaseCommand
class Command(BaseCommand):
def handle(self, *apps_label, **options):
"""
Running makemigrations in production can have desastrous consequences.
We ensure the command is disabled, unless a specific env var is provided.
"""
force = os.environ.get("FORCE") == "1"
if not force:
raise CommandError(
"Running makemigrations on your Funkwhale instance can have desastrous"
" consequences. This command is disabled, and should only be run in "
"development environments."
)
return super().handle(*apps_label, **options)
from django.core.management.commands.migrate import Command as BaseCommand
def patch_write(buffer):
"""
Django is trying to help us when running migrate, by checking we don't have
model changes not included in migrations. Unfortunately, running makemigrations
on production instances create unwanted migrations and corrupt the database.
So we disabled the makemigrations command, and we're patching the
write method to ensure misleading messages are never shown to the user,
because https://github.com/django/django/blob/2.1.5/django/core/management/commands/migrate.py#L186
does not leave an easy way to disable them.
"""
unpatched = buffer.write
def p(message, *args, **kwargs):
if "'manage.py makemigrations'" in message or "not yet reflected" in message:
return
return unpatched(message, *args, **kwargs)
setattr(buffer, "write", p)
class Command(BaseCommand):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
patch_write(self.stdout)
......@@ -56,6 +56,10 @@ def serve_spa(request):
def get_spa_html(spa_url):
if spa_url.startswith("/"):
# we try to open a local file
with open(spa_url) as f:
return f.read()
cache_key = "spa-html:{}".format(spa_url)
cached = caches["local"].get(cache_key)
if cached:
......
......@@ -147,3 +147,49 @@ def order_for_search(qs, field):
this function will order the given qs based on the length of the given field
"""
return qs.annotate(__size=models.functions.Length(field)).order_by("__size")
def recursive_getattr(obj, key, permissive=False):
"""
Given a dictionary such as {'user': {'name': 'Bob'}} and
a dotted string such as user.name, returns 'Bob'.
If the value is not present, returns None
"""
v = obj
for k in key.split("."):
try:
v = v.get(k)
except (TypeError, AttributeError):
if not permissive:
raise
return
if v is None:
return
return v
def replace_prefix(queryset, field, old, new):
"""
Given a queryset of objects and a field name, will find objects
for which the field have the given value, and replace the old prefix by
the new one.
This is especially useful to find/update bad federation ids, to replace:
http://wrongprotocolanddomain/path
by
https://goodprotocalanddomain/path
on a whole table with a single query.
"""
qs = queryset.filter(**{"{}__startswith".format(field): old})
# we extract the part after the old prefix, and Concat it with our new prefix
update = models.functions.Concat(
models.Value(new),
models.functions.Substr(field, len(old) + 1, output_field=models.CharField()),
)
return qs.update(**{field: update})
class SkipFilterForGetObject:
def get_object(self, *args, **kwargs):
setattr(self.request, "_skip_filters", True)
return super().get_object(*args, **kwargs)
def filter_queryset(self, queryset):
if getattr(self.request, "_skip_filters", False):
return queryset
return super().filter_queryset(queryset)
from django_filters import rest_framework as filters
from funkwhale_api.common import fields
from funkwhale_api.moderation import filters as moderation_filters
from . import models
class TrackFavoriteFilter(filters.FilterSet):
class TrackFavoriteFilter(moderation_filters.HiddenContentFilterSet):
q = fields.SearchFilter(
search_fields=["track__title", "track__artist__name", "track__album__title"]
)
......@@ -13,3 +12,6 @@ class TrackFavoriteFilter(filters.FilterSet):
class Meta:
model = models.TrackFavorite
fields = ["user", "q"]
hidden_content_fields_mapping = moderation_filters.USER_FILTER_CONFIG[
"TRACK_FAVORITE"
]
......@@ -9,6 +9,8 @@ from django.db.models import Q
from funkwhale_api.common import channels
from funkwhale_api.common import utils as funkwhale_utils
recursive_getattr = funkwhale_utils.recursive_getattr
logger = logging.getLogger(__name__)
PUBLIC_ADDRESS = "https://www.w3.org/ns/activitystreams#Public"
......@@ -89,9 +91,9 @@ def should_reject(id, actor_id=None, payload={}):
media_types = ["Audio", "Artist", "Album", "Track", "Library", "Image"]
relevant_values = [
recursive_gettattr(payload, "type", permissive=True),
recursive_gettattr(payload, "object.type", permissive=True),
recursive_gettattr(payload, "target.type", permissive=True),
recursive_getattr(payload, "type", permissive=True),
recursive_getattr(payload, "object.type", permissive=True),
recursive_getattr(payload, "target.type", permissive=True),
]
# if one of the payload types match our internal media types, then
# we apply policies that reject media
......@@ -343,7 +345,7 @@ class OutboxRouter(Router):
return activities
def recursive_gettattr(obj, key, permissive=False):
def recursive_getattr(obj, key, permissive=False):
"""
Given a dictionary such as {'user': {'name': 'Bob'}} and
a dotted string such as user.name, returns 'Bob'.
......@@ -366,7 +368,7 @@ def recursive_gettattr(obj, key, permissive=False):
def match_route(route, payload):
for key, value in route.items():
payload_value = recursive_gettattr(payload, key)
payload_value = recursive_getattr(payload, key)
if payload_value != value:
return False
......
......@@ -5,8 +5,9 @@ from django.conf import settings
from django.utils import timezone
from funkwhale_api.common import preferences, session
from funkwhale_api.users import models as users_models
from . import models, serializers
from . import keys, models, serializers
logger = logging.getLogger(__name__)
......@@ -28,7 +29,7 @@ def get_actor_data(actor_url):
def get_actor(fid, skip_cache=False):
if not skip_cache:
try:
actor = models.Actor.objects.get(fid=fid)
actor = models.Actor.objects.select_related().get(fid=fid)
except models.Actor.DoesNotExist:
actor = None
fetch_delta = datetime.timedelta(
......@@ -42,3 +43,23 @@ def get_actor(fid, skip_cache=False):
serializer.is_valid(raise_exception=True)
return serializer.save(last_fetch_date=timezone.now())
def get_service_actor():
name, domain = (
settings.FEDERATION_SERVICE_ACTOR_USERNAME,
settings.FEDERATION_HOSTNAME,
)
try:
return models.Actor.objects.select_related().get(
preferred_username=name, domain__name=domain
)
except models.Actor.DoesNotExist:
pass
args = users_models.get_actor_data(name)
private, public = keys.get_key_pair()
args["private_key"] = private.decode("utf-8")
args["public_key"] = public.decode("utf-8")
args["type"] = "Service"
return models.Actor.objects.create(**args)
import cryptography
import logging
import datetime
from django.contrib.auth.models import AnonymousUser
from rest_framework import authentication, exceptions as rest_exceptions
from django.utils import timezone
from rest_framework import authentication, exceptions as rest_exceptions
from funkwhale_api.moderation import models as moderation_models
from . import actors, exceptions, keys, signing, utils
from . import actors, exceptions, keys, signing, tasks, utils
logger = logging.getLogger(__name__)
......@@ -57,6 +59,15 @@ class SignatureAuthentication(authentication.BaseAuthentication):
actor = actors.get_actor(actor_url, skip_cache=True)
signing.verify_django(request, actor.public_key.encode("utf-8"))
# we trigger a nodeinfo update on the actor's domain, if needed
fetch_delay = 24 * 3600
now = timezone.now()
last_fetch = actor.domain.nodeinfo_fetch_date
if not last_fetch or (
last_fetch < (now - datetime.timedelta(seconds=fetch_delay))
):
tasks.update_domain_nodeinfo(domain_name=actor.domain.name)
actor.domain.refresh_from_db()
return actor
def authenticate(self, request):
......
......@@ -69,6 +69,7 @@ def create_user(actor):
@registry.register
class DomainFactory(NoUpdateOnCreate, factory.django.DjangoModelFactory):
name = factory.Faker("domain_name")
nodeinfo_fetch_date = factory.LazyFunction(lambda: timezone.now())
class Meta:
model = "federation.Domain"
......
from django.core.management.base import BaseCommand, CommandError
from funkwhale_api.common import utils
from funkwhale_api.federation import models as federation_models
from funkwhale_api.music import models as music_models
MODELS = [
(music_models.Artist, ["fid"]),
(music_models.Album, ["fid"]),
(music_models.Track, ["fid"]),
(music_models.Upload, ["fid"]),
(music_models.Library, ["fid", "followers_url"]),
(
federation_models.Actor,
[
"fid",
"url",
"outbox_url",
"inbox_url",
"following_url",
"followers_url",
"shared_inbox_url",
],
),
(federation_models.Activity, ["fid"]),
(federation_models.Follow, ["fid"]),
(federation_models.LibraryFollow, ["fid"]),
]
class Command(BaseCommand):
help = """
Find and replace wrong protocal/domain in local federation ids.
Use with caution and only if you know what you are doing.
"""
def add_arguments(self, parser):
parser.add_argument(
"old_base_url",
type=str,
help="The invalid url prefix you want to find and replace, e.g 'http://baddomain'",
)
parser.add_argument(
"new_base_url",
type=str,
help="The url prefix you want to use in place of the bad one, e.g 'https://gooddomain'",
)
parser.add_argument(
"--noinput",
"--no-input",
action="store_false",
dest="interactive",
help="Do NOT prompt the user for input of any kind.",
)
parser.add_argument(
"--no-dry-run",
action="store_false",
dest="dry_run",
help="Commit the changes to the database",
)
def handle(self, *args, **options):
results = {}
old_prefix, new_prefix = options["old_base_url"], options["new_base_url"]
for kls, fields in MODELS:
results[kls] = {}
for field in fields:
candidates = kls.objects.filter(
**{"{}__startswith".format(field): old_prefix}
)
results[kls][field] = candidates.count()
total = sum([t for k in results.values() for t in k.values()])
self.stdout.write("")
if total:
self.stdout.write(
self.style.WARNING(
"Will replace {} found occurences of '{}' by '{}':".format(
total, old_prefix, new_prefix
)
)
)
self.stdout.write("")
for kls, fields in results.items():
for field, count in fields.items():
self.stdout.write(
"- {}/{} {}.{}".format(
count, kls.objects.count(), kls._meta.label, field
)
)
else:
self.stdout.write(
"No objects found with prefix {}, exiting.".format(old_prefix)
)
return
if options["dry_run"]:
self.stdout.write(
"Run this command with --no-dry-run to perform the replacement."
)
return
self.stdout.write("")
if options["interactive"]:
message = (
"Are you sure you want to perform the replacement on {} objects?\n\n"
"Type 'yes' to continue, or 'no' to cancel: "
).format(total)
if input("".join(message)) != "yes":
raise CommandError("Command canceled.")
for kls, fields in results.items():
for field, count in fields.items():
self.stdout.write(
"Replacing {} on {} {}…".format(field, count, kls._meta.label)
)
candidates = kls.objects.all()
utils.replace_prefix(candidates, field, old=old_prefix, new=new_prefix)
self.stdout.write("")
self.stdout.write(self.style.SUCCESS("Done!"))
# Generated by Django 2.1.5 on 2019-01-30 09:26
import django.contrib.postgres.fields.jsonb
from django.db import migrations, models
import django.db.models.deletion
import funkwhale_api.common.validators
import funkwhale_api.federation.models
class Migration(migrations.Migration):
dependencies = [
('federation', '0016_auto_20181227_1605'),
]
operations = [
migrations.RemoveField(
model_name='actor',
name='old_domain',
),
migrations.AddField(
model_name='domain',
name='service_actor',
field=models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.SET_NULL, related_name='managed_domains', to='federation.Actor'),
),
migrations.AlterField(
model_name='domain',
name='name',
field=models.CharField(max_length=255, primary_key=True, serialize=False, validators=[funkwhale_api.common.validators.DomainValidator()]),
),
migrations.AlterField(
model_name='domain',
name='nodeinfo',
field=django.contrib.postgres.fields.jsonb.JSONField(blank=True, default=funkwhale_api.federation.models.empty_dict, max_length=50000),
),
]
......@@ -46,7 +46,9 @@ class FederationMixin(models.Model):
class ActorQuerySet(models.QuerySet):
def local(self, include=True):
return self.exclude(user__isnull=include)
if include:
return self.filter(domain__name=settings.FEDERATION_HOSTNAME)
return self.exclude(domain__name=settings.FEDERATION_HOSTNAME)
def with_current_usage(self):
qs = self
......@@ -92,7 +94,13 @@ class Domain(models.Model):
creation_date = models.DateTimeField(default=timezone.now)
nodeinfo_fetch_date = models.DateTimeField(default=None, null=True, blank=True)
nodeinfo = JSONField(default=empty_dict, max_length=50000, blank=True)
service_actor = models.ForeignKey(
"Actor",
related_name="managed_domains",
on_delete=models.SET_NULL,
null=True,
blank=True,
)
objects = DomainQuerySet.as_manager()
def __str__(self):
......
......@@ -11,6 +11,7 @@ from requests.exceptions import RequestException
from funkwhale_api.common import preferences
from funkwhale_api.common import session
from funkwhale_api.common import utils as common_utils
from funkwhale_api.music import models as music_models
from funkwhale_api.taskapp import celery
......@@ -18,6 +19,7 @@ from . import keys
from . import models, signing
from . import serializers
from . import routes
from . import utils
logger = logging.getLogger(__name__)
......@@ -184,9 +186,27 @@ def update_domain_nodeinfo(domain):
nodeinfo = {"status": "ok", "payload": fetch_nodeinfo(domain.name)}
except (requests.RequestException, serializers.serializers.ValidationError) as e:
nodeinfo = {"status": "error", "error": str(e)}
service_actor_id = common_utils.recursive_getattr(
nodeinfo, "payload.metadata.actorId", permissive=True
)
try:
domain.service_actor = (
utils.retrieve_ap_object(
service_actor_id,
queryset=models.Actor,
serializer_class=serializers.ActorSerializer,
)
if service_actor_id
else None
)
except (serializers.serializers.ValidationError, RequestException) as e:
logger.warning(
"Cannot fetch system actor for domain %s: %s", domain.name, str(e)
)
domain.nodeinfo_fetch_date = now
domain.nodeinfo = nodeinfo
domain.save(update_fields=["nodeinfo", "nodeinfo_fetch_date"])
domain.save(update_fields=["nodeinfo", "nodeinfo_fetch_date", "service_actor"])
def delete_qs(qs):
......
from funkwhale_api.moderation import filters as moderation_filters
from . import models
class ListeningFilter(moderation_filters.HiddenContentFilterSet):
class Meta:
model = models.Listening
hidden_content_fields_mapping = moderation_filters.USER_FILTER_CONFIG[
"LISTENING"
]
fields = ["hidden"]
......@@ -7,7 +7,7 @@ from funkwhale_api.activity import record
from funkwhale_api.common import fields, permissions
from funkwhale_api.music.models import Track
from funkwhale_api.music import utils as music_utils
from . import models, serializers
from . import filters, models, serializers
class ListeningViewSet(
......@@ -25,6 +25,7 @@ class ListeningViewSet(
IsAuthenticatedOrReadOnly,
]
owner_checks = ["write"]
filterset_class = filters.ListeningFilter
def get_serializer_class(self):
if self.request.method.lower() in ["head", "get", "options"]:
......
......@@ -2,6 +2,7 @@ import memoize.djangocache
import funkwhale_api
from funkwhale_api.common import preferences
from funkwhale_api.federation import actors
from . import stats
......@@ -19,6 +20,7 @@ def get():
"openRegistrations": preferences.get("users__registration_enabled"),
"usage": {"users": {"total": 0, "activeHalfyear": 0, "activeMonth": 0}},
"metadata": {
"actorId": actors.get_service_actor().fid,
"private": preferences.get("instance__nodeinfo_private"),
"shortDescription": preferences.get("instance__short_description"),
"longDescription": preferences.get("instance__long_description"),
......
......@@ -28,3 +28,10 @@ class InstancePolicyAdmin(admin.ModelAdmin):
"summary",
]
list_select_related = True
@admin.register(models.UserFilter)
class UserFilterAdmin(admin.ModelAdmin):
list_display = ["uuid", "user", "target_artist", "creation_date"]
search_fields = ["target_artist__name", "user__username", "user__email"]
list_select_related = True
......@@ -2,6 +2,8 @@ import factory
from funkwhale_api.factories import registry, NoUpdateOnCreate
from funkwhale_api.federation import factories as federation_factories
from funkwhale_api.music import factories as music_factories
from funkwhale_api.users import factories as users_factories
@registry.register
......@@ -21,3 +23,17 @@ class InstancePolicyFactory(NoUpdateOnCreate, factory.DjangoModelFactory):
for_actor = factory.Trait(
target_actor=factory.SubFactory(federation_factories.ActorFactory)
)
@registry.register
class UserFilterFactory(NoUpdateOnCreate, factory.DjangoModelFactory):
user = factory.SubFactory(users_factories.UserFactory)
target_artist = None
class Meta:
model = "moderation.UserFilter"
class Params:
for_artist = factory.Trait(
target_artist=factory.SubFactory(music_factories.ArtistFactory)
)
from django.db.models import Q
from django_filters import rest_framework as filters
USER_FILTER_CONFIG = {
"ARTIST": {"target_artist": ["pk"]},
"ALBUM": {"target_artist": ["artist__pk"]},
"TRACK": {"target_artist": ["artist__pk", "album__artist__pk"]},
"LISTENING": {"target_artist": ["track__album__artist__pk", "track__artist__pk"]},
"TRACK_FAVORITE": {
"target_artist": ["track__album__artist__pk", "track__artist__pk"]
},
}
def get_filtered_content_query(config, user):
final_query = None
for filter_field, model_fields in config.items():
query = None
ids = user.content_filters.values_list(filter_field, flat=True)
for model_field in model_fields:
q = Q(**{"{}__in".format(model_field): ids})
if query:
query |= q
else:
query = q
final_query = query
return final_query
class HiddenContentFilterSet(filters.FilterSet):
"""
A filterset that include a "hidden" param:
- hidden=true : list user hidden/filtered objects
- hidden=false : list all objects user hidden/filtered objects
- not specified: hidden=false
Usage:
class MyFilterSet(HiddenContentFilterSet):
class Meta:
hidden_content_fields_mapping = {'target_artist': ['pk']}
Will map UserContentFilter.artist values to the pk field of the filtered model.
"""
hidden = filters.BooleanFilter(field_name="_", method="filter_hidden_content")
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.data = self.data.copy()
self.data.setdefault("hidden", False)
def filter_hidden_content(self, queryset, name, value):
user = self.request.user
if not user.is_authenticated:
# no filter to apply
return queryset
config = self.__class__.Meta.hidden_content_fields_mapping
final_query = get_filtered_content_query(config, user)
if value is True:
return queryset.filter(final_query)
else:
return queryset.exclude(final_query)
# Generated by Django 2.1.5 on 2019-02-13 09:27
from django.conf import settings
from django.db import migrations, models
import django.db.models.deletion
import django.utils.timezone
import uuid
class Migration(migrations.Migration):
dependencies = [
migrations.swappable_dependency(settings.AUTH_USER_MODEL),
("music", "0037_auto_20190103_1757"),
("moderation", "0001_initial"),
]
operations = [
migrations.CreateModel(
name="UserFilter",
fields=[
(
"id",
models.AutoField(
auto_created=True,
primary_key=True,
serialize=False,
verbose_name="ID",
),
),
("uuid", models.UUIDField(default=uuid.uuid4, unique=True)),
(
"creation_date",
models.DateTimeField(default=django.utils.timezone.now),
),
(
"target_artist",
models.ForeignKey(
on_delete=django.db.models.deletion.CASCADE,
related_name="user_filters",
to="music.Artist",
),
),
(
"user",
models.ForeignKey(
on_delete=django.db.models.deletion.CASCADE,
related_name="content_filters",
to=settings.AUTH_USER_MODEL,
),
),
],
),
migrations.AlterUniqueTogether(
name="userfilter", unique_together={("user", "target_artist")}
),
]
......@@ -73,3 +73,22 @@ class InstancePolicy(models.Model):
return {"type": "actor", "obj": self.target_actor}
if self.target_domain_id:
return {"type": "domain", "obj": self.target_domain}
class UserFilter(models.Model):
uuid = models.UUIDField(default=uuid.uuid4, unique=True)
creation_date = models.DateTimeField(default=timezone.now)
target_artist = models.ForeignKey(
"music.Artist", on_delete=models.CASCADE, related_name="user_filters"
)
user = models.ForeignKey(
"users.User", on_delete=models.CASCADE, related_name="content_filters"
)
class Meta:
unique_together = ("user", "target_artist")
@property
def target(self):
if self.target_artist:
return {"type": "artist", "obj": self.target_artist}
from rest_framework import serializers
from funkwhale_api.music import models as music_models
from . import models
class FilteredArtistSerializer(serializers.ModelSerializer):
class Meta:
model = music_models.Artist
fields = ["id", "name"]
class TargetSerializer(serializers.Serializer):
type = serializers.ChoiceField(choices=["artist"])
id = serializers.CharField()
def to_representation(self, value):
if value["type"] == "artist":
data = FilteredArtistSerializer(value["obj"]).data
data.update({"type": "artist"})
return data
def to_internal_value(self, value):
if value["type"] == "artist":
field = serializers.PrimaryKeyRelatedField(
queryset=music_models.Artist.objects.all()
)
value["obj"] = field.to_internal_value(value["id"])
return value
class UserFilterSerializer(serializers.ModelSerializer):
target = TargetSerializer()
class Meta:
model = models.UserFilter
fields = ["uuid", "target", "creation_date"]
read_only_fields = ["uuid", "creation_date"]
def validate(self, data):
target = data.pop("target")
if target["type"] == "artist":
data["target_artist"] = target["obj"]
return data
from rest_framework import routers
from . import views
router = routers.SimpleRouter()
router.register(r"content-filters", views.UserFilterViewSet, "content-filters")
urlpatterns = router.urls
from django.db import IntegrityError
from rest_framework import mixins
from rest_framework import permissions
from rest_framework import response
from rest_framework import status
from rest_framework import viewsets
from . import models
from . import serializers
class UserFilterViewSet(
mixins.ListModelMixin,
mixins.CreateModelMixin,
mixins.RetrieveModelMixin,
mixins.DestroyModelMixin,
viewsets.GenericViewSet,
):
lookup_field = "uuid"
queryset = (
models.UserFilter.objects.all()
.order_by("-creation_date")
.select_related("target_artist")
)
serializer_class = serializers.UserFilterSerializer
permission_classes = [permissions.IsAuthenticated]
ordering_fields = ("creation_date",)
def create(self, request, *args, **kwargs):
try:
return super().create(request, *args, **kwargs)
except IntegrityError:
content = {"detail": "A content filter already exists for this object"}
return response.Response(content, status=status.HTTP_400_BAD_REQUEST)
def get_queryset(self):
qs = super().get_queryset()
return qs.filter(user=self.request.user)
def perform_create(self, serializer):
serializer.save(user=self.request.user)
......@@ -20,7 +20,7 @@ class AlbumAdmin(admin.ModelAdmin):
class TrackAdmin(admin.ModelAdmin):
list_display = ["title", "artist", "album", "mbid"]
search_fields = ["title", "artist__name", "album__title", "mbid"]
list_select_related = True
list_select_related = ["album__artist", "artist"]
@admin.register(models.ImportBatch)
......
......@@ -2,12 +2,13 @@ from django_filters import rest_framework as filters
from funkwhale_api.common import fields
from funkwhale_api.common import search
from funkwhale_api.moderation import filters as moderation_filters
from . import models
from . import utils
class ArtistFilter(filters.FilterSet):
class ArtistFilter(moderation_filters.HiddenContentFilterSet):
q = fields.SearchFilter(search_fields=["name"])
playable = filters.BooleanFilter(field_name="_", method="filter_playable")
......@@ -17,13 +18,14 @@ class ArtistFilter(filters.FilterSet):
"name": ["exact", "iexact", "startswith", "icontains"],
"playable": "exact",
}
hidden_content_fields_mapping = moderation_filters.USER_FILTER_CONFIG["ARTIST"]
def filter_playable(self, queryset, name, value):
actor = utils.get_actor_from_request(self.request)
return queryset.playable_by(actor, value)
class TrackFilter(filters.FilterSet):
class TrackFilter(moderation_filters.HiddenContentFilterSet):
q = fields.SearchFilter(search_fields=["title", "album__title", "artist__name"])
playable = filters.BooleanFilter(field_name="_", method="filter_playable")
......@@ -36,6 +38,7 @@ class TrackFilter(filters.FilterSet):
"album": ["exact"],
"license": ["exact"],
}
hidden_content_fields_mapping = moderation_filters.USER_FILTER_CONFIG["TRACK"]
def filter_playable(self, queryset, name, value):
actor = utils.get_actor_from_request(self.request)
......@@ -85,13 +88,14 @@ class UploadFilter(filters.FilterSet):
return queryset.playable_by(actor, value)