Commit 4de78995 authored by Eliot Berriot's avatar Eliot Berriot 💬

Fix #862: replaced Daphne by Gunicorn/Uvicorn

parent 2ddd721f
......@@ -129,6 +129,7 @@ test_api:
only:
- branches
before_script:
- apk add make
- cd api
- sed -i '/Pillow/d' requirements/base.txt
- pip3 install -r requirements/base.txt
......
......@@ -44,7 +44,7 @@ RUN \
if [ "$install_dev_deps" = "1" ] ; then echo "Installing dev dependencies" && pip3 install --no-cache-dir -r /requirements/local.txt -r /requirements/test.txt ; else echo "Skipping dev deps installation" ; fi
ENTRYPOINT ["./compose/django/entrypoint.sh"]
CMD ["./compose/django/daphne.sh"]
CMD ["./compose/django/server.sh"]
COPY . /app
WORKDIR /app
#!/bin/bash -eux
python /app/manage.py collectstatic --noinput
daphne -b 0.0.0.0 -p 5000 config.asgi:application --proxy-headers
gunicorn config.asgi:application -w ${FUNKWHALE_WEB_WORKERS-1} -k uvicorn.workers.UvicornWorker -b 0.0.0.0:5000
......@@ -8,3 +8,4 @@ postgresql-client
python3-dev
libldap2-dev
libsasl2-dev
make
......@@ -50,6 +50,8 @@ python-magic==0.4.15
channels==2.1.6
channels_redis>=2.3,<2.4
daphne>=2.2,<2.3
uvicorn
gunicorn
cryptography>=2,<3
# requests-http-signature==0.0.3
......
Replaced Daphne by Gunicorn/Uvicorn to improve stability, flexibility and performance (#862)
......@@ -6,106 +6,22 @@ Next release notes
Those release notes refer to the current development branch and are reset
after each release.
Edits on tracks, albums and artists
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Funkwhale was a bit annoying when it camed to metadata. Tracks, albums and artists profiles
were created from audio file tags, but basically immutable after that (unless you had
admin access to Django's UI, which wasn't ideal to do this kind of changes).
Replaced Daphne by Gunicorn/Uvicorn [manual action required, non-docker only]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
With this release, everyone can suggest changes on track, album and artist pages. Users
with the "library" permission can review suggested edits in a dedicated interface
and apply/reject them.
To improve the performance, stability and reliability of Funkwhale's web processes,
we now recommend using Gunicorn and Uvicorn instead of Daphne. This combination unlock new use cases such as:
Approved edits are broadcasted via federation, to ensure other instances get the information
too.
- zero-downtime upgrades
- configurable number of web worker processes
Not all fields are currently modifiable using this feature. Especially, it's not possible
to suggest a new album cover, or reassign a track to a different album or artist. Those will
be implemented in a future release.
Based on our benchmarks, Gunicorn/Unicorn is also faster and more stable under higher workloads compared to Daphne.
Admin UI for tracks, albums, artists, libraries and uploads
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
To benefit from this enhancement on existing instances, you need to add ``FUNKWHALE_WEB_WORKERS=1`` in your ``.env`` file
(use a higher number if you want to have more web worker processes).
As part of our ongoing effort to make Funkwhale easier to manage for instance owners,
this release includes a brand new administration interface to deal with:
Then, edit your ``/etc/systemd/system/funkwhale-server.service`` and replace the ``ExecStart=`` line with
``ExecStart=/srv/funkwhale/virtualenv/bin/gunicorn config.asgi:application -w ${FUNKWHALE_WEB_WORKERS} -k uvicorn.workers.UvicornWorker -b ${FUNKWHALE_API_IP}:${FUNKWHALE_API_PORT}``
- tracks
- albums
- artists
- libraries
- uploads
You can use this UI to quickly search for any object, delete objects in batch, understand
where they are coming from etc. This new UI should remove the need to go through Django's
admin in the vast majority of cases (but also includes a link to Django's admin when needed).
Artist hiding in the interface
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
It's now possible for users to hide artists they don't want to see.
Content linked to hidden artists will not show up in the interface anymore. Especially:
- Hidden artists tracks are removed from the current queue
- Starting a playlist will skip tracks from hidden artists
- Recently favorited, recently listened and recently added widgets on the homepage won't include content from hidden artists
- Radio suggestions will exclude tracks from hidden artists
- Hidden artists won't appear in Subsonic apps
Results linked to hidden artists will continue to show up in search results and their profile page remains accessible.
OAuth2 authorization for better integration with third-party apps
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Funkwhale now support the OAuth2 authorization and authentication protocol which will allow
third-party apps to interact with Funkwhale on behalf of users.
This feature makes it possible to build third-party apps that have the same capabilities
as Funkwhale's Web UI. The only exception at the moment is for actions that requires
special permissions, such as modifying instance settings or moderation (but this will be
enabled in a future release).
If you want to start building an app on top of Funkwhale's API, please check-out
`https://docs.funkwhale.audio/api.html`_ and `https://docs.funkwhale.audio/developers/authentication.html`_.
Better error handling and display during import
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Funkwhale should now be more resilient to missing tags in imported files, and give
you more insights when something goes wrong, including the specific tags that were missing
or invalid, and additional debug information to share in your support requests.
This information is available in all pages that list uploads, when clicking on the button next to the upload status.
Support for S3-compatible storages to store media files
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Storing all media files on the Funkwhale server itself may not be possible or desirable
in all scenarios. You can now configure Funkwhale to store those files in a S3
bucket instead.
Check-out `https://docs.funkwhale.audio/admin/external-storages.html`_ if you want to use
this feature.
Prune library command
^^^^^^^^^^^^^^^^^^^^^
Users are often surprised by Funkwhale's tendency to keep track, album and artist
metadata even if no associated files exist.
To help with that, we now offer a ``prune_library`` management command you can run
to purge your database from obsolete entries. `Please refer to our documentation
for usage instructions <https://docs.funkwhale.audio/admin/commands.html#pruning-library>`_.
Check in-place files command
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
When using in-place import with a living audio library, you'll quite often rename or
remove files from the file system. Unfortunately, Funkwhale keeps a reference to those
files in the database, which results in unplayable tracks.
To help with that, we now offer a ``check_inplace_files`` management command you can run
to purge your database from obsolete files. `Please refer to our documentation
for usage instructions <https://docs.funkwhale.audio/admin/commands.html#remove-obsolete-files-from-database>`_.
Then reload the configuration change with ``sudo systemctl daemon-reload`` and ``sudo systemctl restart funkwhale-server``.
......@@ -25,8 +25,8 @@ funkwhale_server_user=funkwhale
funkwhale_server_env=$(cat /usr/local/www/funkwhale/config/.env | grep -v ^# | xargs)
command_interpreter="/usr/local/www/funkwhale/virtualenv/bin/python3"
command="/usr/local/www/funkwhale/virtualenv/bin/daphne"
command_args="-b 127.0.0.1 -p 5000 config.asgi:application --proxy-headers \
command="/usr/local/www/funkwhale/virtualenv/bin/gunicorn"
command_args="config.asgi:application -w 4 -k uvicorn.workers.UvicornWorker -b 127.0.0.1:5000 \
>> /var/log/funkwhale/${name##funkwhale_}.log 2>&1 &"
run_rc_command "$1"
......@@ -3,9 +3,9 @@
NAME=funkwhaleserver
PIDFILE=/var/run/$NAME.pid
USER=funkwhale
DAEMON_ARGS="-b 127.0.0.1 -p 5000 config.asgi:application --proxy-headers "
Daphne=/srv/funkwhale/virtualenv/bin/daphne
WORKDIR=/srv/funkwhale/api
DAEMON_ARGS="config.asgi:application -w 4 -k uvicorn.workers.UvicornWorker -b 127.0.0.1:5000 "
Gunicorn=/srv/funkwhale/virtualenv/bin/gunicorn
WORKDIR=/srv/funkwhale/api
depend() {
need net redis postgresql nginx funkwhale_beat funkwhale_worker
......@@ -16,7 +16,7 @@ start() {
cd /srv/funkwhale/api
set -a && source /srv/funkwhale/config/.env && set +a
echo 'Starting Funkwhale Server'
start-stop-daemon --start --user $USER --make-pidfile --pidfile $PIDFILE -d $WORKDIR --exec $Daphne -- $DAEMON_ARGS >> /var/log/funk/daphne.log 2>&1&
start-stop-daemon --start --user $USER --make-pidfile --pidfile $PIDFILE -d $WORKDIR --exec $Gunicorn -- $DAEMON_ARGS >> /var/log/funk/server.log 2>&1&
echo 'Funkwhale Server started'
echo
eend $?
......
......@@ -34,7 +34,9 @@ FUNKWHALE_VERSION=latest
# example: FUNKWHALE_API_PORT=5678
FUNKWHALE_API_IP=127.0.0.1
FUNKWHALE_API_PORT=5000
# The number of web workers to start in parallel. Higher means you can handle
# more concurrent requests, but also leads to higher CPU/Memory usage
FUNKWHALE_WEB_WORKERS=1
# Replace this by the definitive, public domain you will use for
# your instance
FUNKWHALE_HOSTNAME=yourdomain.funkwhale
......@@ -168,4 +170,4 @@ AWS_STORAGE_BUCKET_NAME=
# If you are using Amazon S3 to serve media directly, you will need to specify your region
# name in order to access files. Example:
# AWS_S3_REGION_NAME=eu-west-2
# AWS_S3_REGION_NAME=
# AWS_S3_REGION_NAME=
......@@ -8,7 +8,6 @@ User=funkwhale
# adapt this depending on the path of your funkwhale installation
WorkingDirectory=/srv/funkwhale/api
EnvironmentFile=/srv/funkwhale/config/.env
ExecStart=/srv/funkwhale/virtualenv/bin/daphne -b ${FUNKWHALE_API_IP} -p ${FUNKWHALE_API_PORT} config.asgi:application --proxy-headers
ExecStart=/srv/funkwhale/virtualenv/bin/gunicorn config.asgi:application -w ${FUNKWHALE_WEB_WORKERS} -k uvicorn.workers.UvicornWorker -b ${FUNKWHALE_API_IP}:${FUNKWHALE_API_PORT}
[Install]
WantedBy=multi-user.target
......@@ -24,7 +24,7 @@ On Debian-like systems, you can install them using:
# Install dependencies
sudo apt-get install curl python3-pip python3-venv git unzip libldap2-dev libsasl2-dev
# Funkwhale dependencies
sudo apt install build-essential ffmpeg libjpeg-dev libmagic-dev libpq-dev postgresql-client python3-dev
sudo apt install build-essential ffmpeg libjpeg-dev libmagic-dev libpq-dev postgresql-client python3-dev make
On Arch Linux and its derivatives:
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment