Merge tag '0.9' into develop

0.9 (2018-04-17)
----------------

Features:

- Add internationalization support (#5)
- Can now follow and import music from remote libraries (#136, #137)

Enhancements:

- Added a i18n-extract yarn script to extract strings to PO files (#162)
- User admin now includes signup and last login dates (#148)
- We now use a proper user agent including instance version and url during
  outgoing requests

Federation is here!
^^^^^^^^^^^^^^^^^^^

This is for real this time, and includes:

- Following other Funkwhale libraries
- Importing tracks from remote libraries (tracks are hotlinked, and only cached for a short amount of time)
- Searching accross federated catalogs

Note that by default, federation is opt-in, on a per-instance basis:
instances will request access to your catalog, and you can accept or refuse
those requests. You can also revoke the access at any time.

Documentation was updated with relevant instructions to use and benefit
from this new feature: https://docs.funkwhale.audio/federation.html

Preparing internationalization
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Funkwhale's front-end as always been english-only, and this is a barrier
to new users. The work make Funkwhale's interface translatable was started
in this release by Baptiste. Although nothing is translated yet,
this release includes behind the stage changes that will make it possible in
the near future.

Many thank to Baptiste for the hard work and for figuring out a proper solution
to this difficult problem.

Upgrade path
^^^^^^^^^^^^

In addition to the usual instructions from
https://docs.funkwhale.audio/upgrading.html, non-docker users will have
to setup an additional systemd unit file for recurrent tasks.

This was forgotten in the deployment documentation, but recurrent tasks,
managed by the celery beat process, will be needed more and more in subsequent
releases. Right now, we'll be using to clear the cache for federated music files
and keep disk usage to a minimum.

In the future, they will also be needed to refetch music metadata or federated
information periodically.

Celery beat can be enabled easily::

    curl -L -o "/etc/systemd/system/funkwhale-beat.service" "https://code.eliotberriot.com/funkwhale/funkwhale/raw/develop/deploy/funkwhale-beat.service"
    # Also edit /etc/systemd/system/funkwhale.target
    # and ensure the Wants= line contains the following:
    # Wants=funkwhale-server.service funkwhale-worker.service funkwhale-beat.service
    nano /etc/systemd/system/funkwhale.target
    # reload configuration
    systemctl daemon-reload

Docker users already have celerybeat enabled.

CHANGELOG

@@ -3,6 +3,79 @@ Changelog
 
 .. towncrier
 
+0.9 (2018-04-17)
+----------------
+
+Features:
+
+- Add internationalization support (#5)
+- Can now follow and import music from remote libraries (#136, #137)
+
+
+Enhancements:
+
+- Added a i18n-extract yarn script to extract strings to PO files (#162)
+- User admin now includes signup and last login dates (#148)
+- We now use a proper user agent including instance version and url during
+  outgoing requests
+
+
+Federation is here!
+^^^^^^^^^^^^^^^^^^^
+
+This is for real this time, and includes:
+
+- Following other Funkwhale libraries
+- Importing tracks from remote libraries (tracks are hotlinked, and only cached for a short amount of time)
+- Searching accross federated catalogs
+
+Note that by default, federation is opt-in, on a per-instance basis:
+instances will request access to your catalog, and you can accept or refuse
+those requests. You can also revoke the access at any time.
+
+Documentation was updated with relevant instructions to use and benefit
+from this new feature: https://docs.funkwhale.audio/federation.html
+
+Preparing internationalization
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Funkwhale's front-end as always been english-only, and this is a barrier
+to new users. The work make Funkwhale's interface translatable was started
+in this release by Baptiste. Although nothing is translated yet,
+this release includes behind the stage changes that will make it possible in
+the near future.
+
+Many thank to Baptiste for the hard work and for figuring out a proper solution
+to this difficult problem.
+
+Upgrade path
+^^^^^^^^^^^^
+
+In addition to the usual instructions from
+https://docs.funkwhale.audio/upgrading.html, non-docker users will have
+to setup an additional systemd unit file for recurrent tasks.
+
+This was forgotten in the deployment documentation, but recurrent tasks,
+managed by the celery beat process, will be needed more and more in subsequent
+releases. Right now, we'll be using to clear the cache for federated music files
+and keep disk usage to a minimum.
+
+In the future, they will also be needed to refetch music metadata or federated
+information periodically.
+
+Celery beat can be enabled easily::
+
+    curl -L -o "/etc/systemd/system/funkwhale-beat.service" "https://code.eliotberriot.com/funkwhale/funkwhale/raw/develop/deploy/funkwhale-beat.service"
+    # Also edit /etc/systemd/system/funkwhale.target
+    # and ensure the Wants= line contains the following:
+    # Wants=funkwhale-server.service funkwhale-worker.service funkwhale-beat.service
+    nano /etc/systemd/system/funkwhale.target
+    # reload configuration
+    systemctl daemon-reload
+
+Docker users already have celerybeat enabled.
+
+
 0.8 (2018-04-02)
 ----------------
 
@@ -71,27 +144,16 @@ and add the following snippets::
 This will ensure federation endpoints will be reachable in the future.
 You can of course skip this part if you know you will not federate your instance.
 
-A new ``FEDERATION_ENABLED`` env var have also been added to control wether
+A new ``FEDERATION_ENABLED`` env var have also been added to control whether
 federation is enabled or not on the application side. This settings defaults
-to True, which should have no consequencies at the moment, since actual
+to True, which should have no consequences at the moment, since actual
 federation is not implemented and the only available endpoints are for
 testing purposes.
 
 Add ``FEDERATION_ENABLED=false`` to your .env file to disable federation
 on the application side.
 
-The last step involves generating RSA private and public keys for signing
-your instance requests on the federation. This can be done via::
-
-    # on docker setups
-    docker-compose run --rm api python manage.py generate_keys --no-input
-
-    # on non-docker setups
-    source /srv/funkwhale/virtualenv/bin/activate
-    source /srv/funkwhale/load_env
-    python manage.py generate_keys --no-input
-
-To test and troobleshoot federation, we've added a bot account. This bot is available at @test@yourinstancedomain,
+To test and troubleshoot federation, we've added a bot account. This bot is available at @test@yourinstancedomain,
 and sending it "/ping", for example, via Mastodon, should trigger
 a response.
 

api/config/settings/common.py

@@ -13,6 +13,8 @@ from __future__ import absolute_import, unicode_literals
 from urllib.parse import urlsplit
 import os
 import environ
+from celery.schedules import crontab
+
 from funkwhale_api import __version__
 
 ROOT_DIR = environ.Path(__file__) - 3  # (/a/b/myfile.py - 3 = /)
@@ -334,6 +336,16 @@ CELERY_BROKER_URL = env(
 # Your common stuff: Below this line define 3rd party library settings
 CELERY_TASK_DEFAULT_RATE_LIMIT = 1
 CELERY_TASK_TIME_LIMIT = 300
+CELERYBEAT_SCHEDULE = {
+    'federation.clean_music_cache': {
+        'task': 'funkwhale_api.federation.tasks.clean_music_cache',
+        'schedule': crontab(hour='*/2'),
+        'options': {
+            'expires': 60 * 2,
+        },
+    }
+}
+
 import datetime
 JWT_AUTH = {
     'JWT_ALLOW_REFRESH': True,

api/funkwhale_api/__init__.py

 # -*- coding: utf-8 -*-
-__version__ = '0.8'
+__version__ = '0.9'
 __version_info__ = tuple([int(num) if num.isdigit() else num for num in __version__.replace('-', '.', 1).split('.')])

changes/changelog.d/136.feature

-Can now follow and import music from remote libraries (#136, #137)

changes/changelog.d/148.enhancement

-User admin now includes signup and last login dates (#148)

changes/changelog.d/162-script.enhancement

-Added a i18n-extract yarn script to extract strings to PO files (#162)

changes/changelog.d/5.feature

-Add internationalization support (#5)

changes/changelog.d/user-agent.enhancement

-We now use a proper user agent including instance version and url during
-outgoing requests

deploy/funkwhale-beat.service

+[Unit]
+Description=Funkwhale celery beat process
+After=redis.service postgresql.service
+PartOf=funkwhale.target
+
+[Service]
+User=funkwhale
+# adapt this depending on the path of your funkwhale installation
+WorkingDirectory=/srv/funkwhale/api
+EnvironmentFile=/srv/funkwhale/config/.env
+ExecStart=/srv/funkwhale/virtualenv/bin/celery -A funkwhale_api.taskapp beat -l INFO
+
+[Install]
+WantedBy=multi-user.target

deploy/funkwhale.target

 [Unit]
 Description=Funkwhale
-Wants=funkwhale-server.service funkwhale-worker.service
+Wants=funkwhale-server.service funkwhale-worker.service funkwhale-beat.service

docs/federation.rst

@@ -7,11 +7,26 @@ and share its own library with an instance C.
 
 We support various levels of controls for federation-related features.
 
+Managing federation
+-------------------
+
+Federation management is only available to instance admins and users
+who have the proper permissions. You can disable federation completely
+at the instance level by setting the FEDERATION_ENABLED environment variable
+to False.
+
+On the front end, assuming you have the proper permission, you will see
+a "Federation" link in the sidebar.
+
+
 Acquire music via federation
 ----------------------------
 
 Instance libraries are protected by default. To access another instance
-library, you have to follow it. This follow request will be sent to
+library, you have to follow it. Each funkwhale instance gets a dedicated
+ActivityPub Actor you can follow via the username "library@yourinstance.domain".
+
+When submitted, a follow request will be sent to
 the other instance which can accept or deny it. Once your follow request
 is accepted, you can start browsing the other instance library
 and import music from it.
@@ -36,5 +51,7 @@ Federation is enabled by default, but requires manually approving
 each other instance asking for access to library. This is by design,
 to ensure your library is not shared publicly without your consent.
 
-However, we offer a configuration option to alter this behaviour and
-disable the manual approval part.
+However, if you're confident about federating publicly without manual approval,
+you can set the FEDERATION_MUSIC_NEEDS_APPROVAL environment variable to false.
+Follow requests will be accepted automatically and followers
+given access to your library without manual intervention.

docs/installation/systemd.rst

@@ -13,11 +13,13 @@ First, download the sample unitfiles:
     curl -L -o "/etc/systemd/system/funkwhale.target" "https://code.eliotberriot.com/funkwhale/funkwhale/raw/|version|/deploy/funkwhale.target"
     curl -L -o "/etc/systemd/system/funkwhale-server.service" "https://code.eliotberriot.com/funkwhale/funkwhale/raw/|version|/deploy/funkwhale-server.service"
     curl -L -o "/etc/systemd/system/funkwhale-worker.service" "https://code.eliotberriot.com/funkwhale/funkwhale/raw/|version|/deploy/funkwhale-worker.service"
+    curl -L -o "/etc/systemd/system/funkwhale-beat.service" "https://code.eliotberriot.com/funkwhale/funkwhale/raw/|version|/deploy/funkwhale-beat.service"
 
 This will download three unitfiles:
 
 - ``funkwhale-server.service`` to launch the funkwhale web server
 - ``funkwhale-worker.service`` to launch the funkwhale task worker
+- ``funkwhale-beat.service`` to launch the funkwhale task beat (this is for recurring tasks)
 - ``funkwhale.target`` to easily stop and start all of the services at once
 
 You can of course review and edit them to suit your deployment scenario