Commit 0913b716 authored by Eliot Berriot's avatar Eliot Berriot 💬

Merge branch 'release/0.16.1'

parents 6e9f40e3 123569b9
Pipeline #1841 passed with stages
in 2 minutes and 31 seconds
......@@ -7,7 +7,7 @@ C_FORCE_ROOT=true
FUNKWHALE_HOSTNAME=localhost
FUNKWHALE_PROTOCOL=http
PYTHONDONTWRITEBYTECODE=true
WEBPACK_DEVSERVER_PORT=8080
VUE_PORT=8080
MUSIC_DIRECTORY_PATH=/music
BROWSABLE_API_ENABLED=True
FORWARDED_PROTO=http
......@@ -72,16 +72,17 @@ api/media
api/staticfiles
api/static
api/.pytest_cache
api/celerybeat-*
# Front
oldfront/node_modules/
front/static/translations
front/node_modules/
front/dist/
front/npm-debug.log*
front/yarn-debug.log*
front/yarn-error.log*
front/test/unit/coverage
front/test/e2e/reports
front/tests/unit/coverage
front/tests/e2e/reports
front/selenium-debug.log
docs/_build
......
......@@ -156,7 +156,6 @@ test_api:
tags:
- docker
test_front:
stage: test
image: node:9
......@@ -166,7 +165,7 @@ test_front:
- branches
script:
- yarn install
- yarn run unit
- yarn test:unit
cache:
key: "funkwhale__front_dependencies"
paths:
......@@ -179,7 +178,6 @@ test_front:
tags:
- docker
build_front:
stage: build
image: node:9
......@@ -192,8 +190,8 @@ build_front:
- yarn run i18n-compile
# this is to ensure we don't have any errors in the output,
# cf https://code.eliotberriot.com/funkwhale/funkwhale/issues/169
- yarn run build | tee /dev/stderr | (! grep -i 'ERROR in')
- chmod -R 750 dist
- yarn build | tee /dev/stderr | (! grep -i 'ERROR in')
- chmod -R 755 dist
cache:
key: "funkwhale__front_dependencies"
paths:
......@@ -210,7 +208,6 @@ build_front:
tags:
- docker
pages:
stage: test
image: python:3.6
......
......@@ -10,7 +10,82 @@ This changelog is viewable on the web at https://docs.funkwhale.audio/changelog.
.. towncrier
0.16 (unreleased)
0.16.1 (2018-08-19)
-------------------
Upgrade instructions are available at
https://docs.funkwhale.audio/upgrading.html
Features:
- Make funkwhale themable by loading external stylesheets (#456)
Enhancements:
- Add link to admin on "Staff member" button (#202)
- Can now add a description to radios and better radio cards (#331)
- Display track duration in track tables (#461)
- More permissive default permissions for front-end files (#388)
- Simpler configuration and toolchain for the front-end using vue-cli (!375)
- Use Howler to manage audio instead of our own dirty/untested code (#392)
Bugfixes:
- Fix alignment issue on top bar in Admin tabs (#395)
- Fix Apache2 permission issue preventing `/media` folder from being served
correctly (#389)
- Fix loading on browse page lists causing them to go down, and dimming over
the top bar (#468)
- Fixed (again): administration section not showing up in sidebar after login
(#245)
- Fixed audio mimetype not showing up on track detail and list (#459)
- Fixed broken audio playback on Chrome and invisible volume control (#390)
- Fixed broken federation import on big imports due to missing transaction
logic (#397)
- Fixed crash on artist pages when no cover is available (#457)
- Fixed favorited status of tracks not appearing in interface (#398)
- Fixed invitation code not prefilled in form when accessing invitation link
(#476)
- Fixed typos in scheduled tasks configuration (#487)
- Removed release date error in case of empty date (#478)
- Removed white on white artist button on hover, on Album page (#393)
- Smarter date parsing during import by replacing arrow with pendulum (#376)
- Display public playlists properly for anonymous users (#488)
i18n:
- Added portuguese, spanish and german translations
Custom themes for Funkwhale
^^^^^^^^^^^^^^^^^^^^^^^^^^^
If you ever wanted to give a custom look and feel to your instance, this is now possible.
Check https://docs.funkwhale.audio/configuration.html#theming if you want to know more!
Fix Apache2 configuration file for media block [Manual action required]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The permission scope on the current Apache2 configuration file is too narrow, preventing thumbnails from being served.
On Apache2 setups, you have to replace the following line::
<Directory /srv/funkwhale/data/media/albums>
with::
<Directory /srv/funkwhale/data/media>
You can now restart your server::
sudo systemctl restart apache2
0.16 (2018-07-22)
-----------------
Upgrade instructions are available at
......@@ -232,7 +307,7 @@ In the end, the ``volumes`` directives of your containers should look like that:
Removed Cacheops dependency
---------------------------
^^^^^^^^^^^^^^^^^^^^^^^^^^^
We removed one of our dependency named django-cacheops. It was unly used in a few places,
and not playing nice with other dependencies.
......
......@@ -23,7 +23,7 @@ As the front-end can work with any Funkwhale server, you can work with the front
and make it talk with an existing instance (like the demo one, or you own instance, if you have one).
If even that is too much for you, you can also make your changes without any development environment,
and open a merge request. We will be able to to review your work easily by spawning automatically a
and open a merge request. We will be able to review your work easily by spawning automatically a
live version of your changes, thanks to Gitlab Review apps.
Setup front-end only development environment
......@@ -43,7 +43,7 @@ Setup front-end only development environment
4. Launch the development server::
# this will serve the front-end on http://localhost:8000
WEBPACK_DEVSERVER_PORT=8000 yarn dev
VUE_PORT=8000 yarn serve
5. Make the front-end talk with an existing server (like https://demo.funkwhale.audio),
by clicking on the corresponding link in the footer
......@@ -264,7 +264,7 @@ When working on federation with traefik, ensure you have this in your ``env``::
# This will ensure we don't bind any port on the host, and thus enable
# multiple instances of funkwhale to be spawned concurrently.
WEBPACK_DEVSERVER_PORT_BINDING=
VUE_PORT_BINDING=
# This disable certificate verification
EXTERNAL_REQUESTS_VERIFY_SSL=false
# this ensure you don't have incorrect urls pointing to http resources
......@@ -466,12 +466,12 @@ Running tests
To run the front-end test suite, use the following command::
docker-compose -f dev.yml run --rm front yarn run unit
docker-compose -f dev.yml run --rm front yarn test:unit
We also support a "watch and test" mode were we continually relaunch
tests when changes are recorded on the file system::
docker-compose -f dev.yml run --rm front yarn run unit-watch
docker-compose -f dev.yml run --rm front yarn test:unit -w
The latter is especially useful when you are debugging failing tests.
......
......@@ -26,7 +26,7 @@ Contribute
----------
Contribution guidelines as well as development installation instructions
are outlined in `CONTRIBUTING <CONTRIBUTING>`_.
are outlined in `CONTRIBUTING <CONTRIBUTING.rst>`_.
Translate
^^^^^^^^^
......
......@@ -344,9 +344,9 @@ CELERY_BROKER_URL = env(
# Your common stuff: Below this line define 3rd party library settings
CELERY_TASK_DEFAULT_RATE_LIMIT = 1
CELERY_TASK_TIME_LIMIT = 300
CELERYBEAT_SCHEDULE = {
CELERY_BEAT_SCHEDULE = {
"federation.clean_music_cache": {
"task": "funkwhale_api.federation.tasks.clean_music_cache",
"task": "federation.clean_music_cache",
"schedule": crontab(hour="*/2"),
"options": {"expires": 60 * 2},
}
......
# -*- coding: utf-8 -*-
__version__ = "0.16"
__version__ = "0.16.1"
__version_info__ = tuple(
[
int(num) if num.isdigit() else num
......
......@@ -796,6 +796,8 @@ class LibraryTrackActionSerializer(common_serializers.ActionSerializer):
jobs.append(job)
music_models.ImportJob.objects.bulk_create(jobs)
music_tasks.import_batch_run.delay(import_batch_id=batch.pk)
funkwhale_utils.on_commit(
music_tasks.import_batch_run.delay, import_batch_id=batch.pk
)
return {"batch": {"id": batch.pk}}
import arrow
import datetime
import mutagen
import pendulum
from django import forms
NODEFAULT = object()
......@@ -101,6 +102,11 @@ class FirstUUIDField(forms.UUIDField):
return super().to_python(value)
def get_date(value):
parsed = pendulum.parse(str(value))
return datetime.date(parsed.year, parsed.month, parsed.day)
VALIDATION = {
"musicbrainz_artistid": FirstUUIDField(),
"musicbrainz_albumid": FirstUUIDField(),
......@@ -118,7 +124,7 @@ CONF = {
"title": {},
"artist": {},
"album": {},
"date": {"field": "date", "to_application": lambda v: arrow.get(v).date()},
"date": {"field": "date", "to_application": get_date},
"musicbrainz_albumid": {},
"musicbrainz_artistid": {},
"musicbrainz_recordingid": {"field": "musicbrainz_trackid"},
......@@ -134,7 +140,7 @@ CONF = {
"title": {},
"artist": {},
"album": {},
"date": {"field": "date", "to_application": lambda v: arrow.get(v).date()},
"date": {"field": "date", "to_application": get_date},
"musicbrainz_albumid": {"field": "MusicBrainz Album Id"},
"musicbrainz_artistid": {"field": "MusicBrainz Artist Id"},
"musicbrainz_recordingid": {"field": "MusicBrainz Track Id"},
......@@ -148,10 +154,7 @@ CONF = {
"title": {"field": "TIT2"},
"artist": {"field": "TPE1"},
"album": {"field": "TALB"},
"date": {
"field": "TDRC",
"to_application": lambda v: arrow.get(str(v)).date(),
},
"date": {"field": "TDRC", "to_application": get_date},
"musicbrainz_albumid": {"field": "MusicBrainz Album Id"},
"musicbrainz_artistid": {"field": "MusicBrainz Artist Id"},
"musicbrainz_recordingid": {
......@@ -172,10 +175,7 @@ CONF = {
"title": {},
"artist": {},
"album": {},
"date": {
"field": "date",
"to_application": lambda v: arrow.get(str(v)).date(),
},
"date": {"field": "date", "to_application": get_date},
"musicbrainz_albumid": {},
"musicbrainz_artistid": {},
"musicbrainz_recordingid": {"field": "musicbrainz_trackid"},
......
import datetime
import os
import shutil
import tempfile
import uuid
import arrow
import markdown
import pendulum
from django.conf import settings
from django.core.files import File
from django.core.files.base import ContentFile
......@@ -125,9 +124,7 @@ def import_artist(v):
def parse_date(v):
if len(v) == 4:
return datetime.date(int(v), 1, 1)
d = arrow.get(v).date()
d = pendulum.parse(v).date()
return d
......@@ -147,7 +144,7 @@ class AlbumQuerySet(models.QuerySet):
class Album(APIModelMixin):
title = models.CharField(max_length=255)
artist = models.ForeignKey(Artist, related_name="albums", on_delete=models.CASCADE)
release_date = models.DateField(null=True)
release_date = models.DateField(null=True, blank=True)
release_group_id = models.UUIDField(null=True, blank=True)
cover = VersatileImageField(
upload_to="albums/covers/%Y/%m/%d", null=True, blank=True
......
......@@ -35,7 +35,7 @@ djangorestframework>=3.7,<3.8
djangorestframework-jwt>=1.11,<1.12
oauth2client<4
google-api-python-client>=1.6,<1.7
arrow>=0.12,<0.13
pendulum>=2,<3
persisting-theory>=0.2,<0.3
django-versatileimagefield>=1.9,<1.10
django-filter>=1.1,<1.2
......
import arrow
import pendulum
import pytest
from django.urls import reverse
from django.utils import timezone
......@@ -455,7 +455,7 @@ def test_library_actor_handle_create_audio(mocker, factories):
assert lt.title == a["metadata"]["recording"]["title"]
assert lt.artist_name == a["metadata"]["artist"]["name"]
assert lt.album_title == a["metadata"]["release"]["title"]
assert lt.published_date == arrow.get(a["published"])
assert lt.published_date == pendulum.parse(a["published"])
def test_library_actor_handle_create_audio_autoimport(mocker, factories):
......@@ -494,7 +494,7 @@ def test_library_actor_handle_create_audio_autoimport(mocker, factories):
assert lt.title == a["metadata"]["recording"]["title"]
assert lt.artist_name == a["metadata"]["artist"]["name"]
assert lt.album_title == a["metadata"]["release"]["title"]
assert lt.published_date == arrow.get(a["published"])
assert lt.published_date == pendulum.parse(a["published"])
batch = music_models.ImportBatch.objects.latest("id")
......
import arrow
import pendulum
import pytest
from django.core.paginator import Paginator
......@@ -492,7 +492,7 @@ def test_activity_pub_audio_serializer_to_library_track(factories):
assert lt.title == audio["metadata"]["recording"]["title"]
assert lt.artist_name == audio["metadata"]["artist"]["name"]
assert lt.album_title == audio["metadata"]["release"]["title"]
assert lt.published_date == arrow.get(audio["published"])
assert lt.published_date == pendulum.parse(audio["published"])
def test_activity_pub_audio_serializer_to_library_track_no_duplicate(factories):
......
......@@ -12,6 +12,7 @@ from funkwhale_api.federation import (
views,
webfinger,
)
from funkwhale_api.music import tasks as music_tasks
@pytest.mark.parametrize(
......@@ -398,7 +399,7 @@ def test_library_track_action_import(factories, superuser_api_client, mocker):
lt2 = factories["federation.LibraryTrack"](library=lt1.library)
lt3 = factories["federation.LibraryTrack"]()
factories["federation.LibraryTrack"](library=lt3.library)
mocked_run = mocker.patch("funkwhale_api.music.tasks.import_batch_run.delay")
mocked_run = mocker.patch("funkwhale_api.common.utils.on_commit")
payload = {
"objects": "all",
......@@ -416,7 +417,9 @@ def test_library_track_action_import(factories, superuser_api_client, mocker):
assert batch.jobs.count() == 2
for i, job in enumerate(batch.jobs.all()):
assert job.library_track == imported_lts[i]
mocked_run.assert_called_once_with(import_batch_id=batch.pk)
mocked_run.assert_called_once_with(
music_tasks.import_batch_run.delay, import_batch_id=batch.pk
)
def test_local_actor_detail(factories, api_client):
......
......@@ -122,3 +122,11 @@ def test_mbid_clean_keeps_only_first(field_name):
result = field.to_python("/".join([u1, u2]))
assert str(result) == u1
@pytest.mark.parametrize(
"raw,expected",
[("2017", datetime.date(2017, 1, 1)), ("2017-12-31", datetime.date(2017, 12, 31))],
)
def test_date_parsing(raw, expected):
assert metadata.get_date(raw) == expected
......@@ -111,7 +111,7 @@ Define MUSIC_DIRECTORY_PATH /srv/funkwhale/data/music
Require all granted
</Directory>
<Directory /srv/funkwhale/data/media/albums>
<Directory /srv/funkwhale/data/media>
Options FollowSymLinks
AllowOverride None
Require all granted
......
version: '3'
version: "3"
services:
front:
......@@ -8,22 +8,22 @@ services:
- .env
environment:
- "HOST=0.0.0.0"
- "WEBPACK_DEVSERVER_PORT=${WEBPACK_DEVSERVER_PORT-8080}"
- "VUE_PORT=${VUE_PORT-8080}"
ports:
- "${WEBPACK_DEVSERVER_PORT_BINDING-8080:}${WEBPACK_DEVSERVER_PORT-8080}"
- "${VUE_PORT_BINDING-8080:}${VUE_PORT-8080}"
volumes:
- './front:/app'
- '/app/node_modules'
- './po:/po'
- "./front:/app"
- "/app/node_modules"
- "./po:/po"
networks:
- federation
- internal
labels:
traefik.backend: "${COMPOSE_PROJECT_NAME-node1}"
traefik.frontend.rule: "Host:${COMPOSE_PROJECT_NAME-node1}.funkwhale.test,${NODE_IP-127.0.0.1}"
traefik.enable: 'true'
traefik.federation.protocol: 'http'
traefik.federation.port: "${WEBPACK_DEVSERVER_PORT-8080}"
traefik.enable: "true"
traefik.federation.protocol: "http"
traefik.federation.port: "${VUE_PORT-8080}"
postgres:
env_file:
......@@ -53,9 +53,9 @@ services:
context: ./api
dockerfile: docker/Dockerfile.test
links:
- postgres
- redis
command: celery -A funkwhale_api.taskapp worker -l debug
- postgres
- redis
command: celery -A funkwhale_api.taskapp worker -l debug -B
environment:
- "FUNKWHALE_HOSTNAME=${FUNKWHALE_HOSTNAME-localhost}"
- "FUNKWHALE_HOSTNAME_SUFFIX=funkwhale.test"
......@@ -98,7 +98,7 @@ services:
- .env
image: nginx
environment:
- "WEBPACK_DEVSERVER_PORT=${WEBPACK_DEVSERVER_PORT-8080}"
- "VUE_PORT=${VUE_PORT-8080}"
- "COMPOSE_PROJECT_NAME=${COMPOSE_PROJECT_NAME- }"
- "FUNKWHALE_HOSTNAME=${FUNKWHALE_HOSTNAME-localhost}"
links:
......@@ -120,20 +120,20 @@ services:
volumes:
- ".:/app/"
ports:
- '35730:35730'
- '8001:8001'
- "35730:35730"
- "8001:8001"
api-docs:
image: swaggerapi/swagger-ui
environment:
- "API_URL=/swagger.yml"
ports:
- '8002:8080'
- "8002:8080"
volumes:
- "./docs/swagger.yml:/usr/share/nginx/html/swagger.yml"
networks:
internal:
? internal
federation:
external:
name: federation
#!/bin/bash -eux
FORWARDED_PORT="$WEBPACK_DEVSERVER_PORT"
FORWARDED_PORT="$VUE_PORT"
COMPOSE_PROJECT_NAME="${COMPOSE_PROJECT_NAME// /}"
if [ -n "$COMPOSE_PROJECT_NAME" ]; then
echo
......
......@@ -158,3 +158,79 @@ permissions are:
There is no dedicated interface to manage users permissions, but superusers
can login on the Django's admin at ``/api/admin/`` and grant permissions
to users at ``/api/admin/users/user/``.
Theming
-------
Funkwhale supports custom themes, which are great if you want to personnalize the
look and feel of your instance. Theming is achieved by declaring
additionnal stylesheets you want to load in the front-end.
Customize the settings
^^^^^^^^^^^^^^^^^^^^^^
In order to know what stylesheets to load, the front-end requests the following
url: ``https://your.instance/settings.json``. On typical deployments, this url
returns a 404 error, which is simply ignored.
However, if you return the appropriate payload on this url, you can make the magic
work. We will store the necessary files in the ``/srv/funkwhale/custom`` directory:
.. code-block:: shell
cd /srv/funkwhale/
mkdir custom
cat <<EOF > custom/settings.json
{
"additionalStylesheets": ["/custom/custom.css"]
}
EOF
cat <<EOF > custom/custom.css
body {
background-color: red;
}
EOF
By executing the previous commands, you will end up with two files in your ``/srv/funkwhale/custom``
directory:
- ``settings.json`` will tell the front-end what stylesheets you want to load (``/custom/custom.css`` in this example)
- ``custom.css`` will hold your custom CSS
The last step to make this work is to ensure both files are served by the reverse proxy.
On nginx, add the following snippet to your vhost config::
location /settings.json {
alias /srv/funkwhale/custom/settings.json;
}
location /custom {
alias /srv/funkwhale/custom;
}
On apache, use the following one::
Alias /settings.json /srv/funkwhale/custom/settings.json
Alias /custom /srv/funkwhale/custom
<Directory "/srv/funkwhale/custom">
Options FollowSymLinks
AllowOverride None
Require all granted
</Directory>
Once done, reload your reverse proxy, refresh Funkwhale in your web browser, and you should see
a red background.
.. note::
You can reference external urls as well in ``settings.json``, simply use
the full urls. Be especially careful with external urls as they may affect your users
privacy.
.. warning::
Loading additional stylesheets and CSS rules can affect the performance and
usability of your instance. If you encounter issues with the interfaces and use
custom stylesheets, try to disable those to ensure the issue is not caused
by your customizations.
.. include:: ../CONTRIBUTING
.. include:: ../CONTRIBUTING.rst
{
"presets": [
["env", { "modules": false }],
"stage-2"
],
"plugins": ["transform-runtime"],
"comments": false,
"env": {
"test": {
"presets": ["env", "stage-2"],
"plugins": [ "istanbul" ]
}
}
}
build/*.js
config/*.js
// http://eslint.org/docs/user-guide/configuring
module.exports = {
root: true,
parser: 'babel-eslint',
parserOptions: {
sourceType: 'module'
},
env: {
browser: true,
},
// https://github.com/feross/standard/blob/master/RULES.md#javascript-standard-style
extends: 'standard',
// required to lint *.vue files
plugins: [
'html'
],
// add your custom rules here
'rules': {
// allow paren-less arrow functions
'arrow-parens': 0,
// allow async-await
'generator-star-spacing': 0,
// allow debugger during development
'no-debugger': process.env.NODE_ENV === 'production' ? 2 : 0
}
}
.DS_Store
node_modules
/dist
# local env files
.env.local
.env.*.local
# Log files
npm-debug.log*
yarn-debug.log*
yarn-error.log*
# Editor directories and files
.idea
.vscode
*.suo
*.ntvs*
*.njsproj
*.sln
*.sw*
// https://github.com/michael-ciniawsky/postcss-load-config
module.exports = {
"plugins": {
// to edit target browsers: use "browserlist" field in package.json
"autoprefixer": {}
}
}
......@@ -5,9 +5,9 @@ RUN curl -L -o /usr/local/bin/jq https://github.com/stedolan/jq/releases/downloa
EXPOSE 8080
WORKDIR /app/
ADD package.json .
ADD package.json yarn.lock ./
RUN yarn install
COPY . .
CMD ["npm", "run", "dev"]
CMD ["yarn", "serve"]
module.exports = {
presets: [
'@vue/app'
]
}
require('./check-versions')()
process.env.NODE_ENV = '