Unverified Commit acf5cb92 authored by Agate's avatar Agate 💬

Merge branch 'release/0.21'

parents 5c10d8e3 16b6360d
......@@ -20,7 +20,7 @@ stages:
review_front:
interruptible: true
stage: review
image: node:11
image: node:12-buster
when: manual
allow_failure: true
variables:
......@@ -71,7 +71,7 @@ review_docs:
- cd docs
- apt-get update
- apt-get install -y graphviz
- pip install sphinx sphinx_rtd_theme
- pip install sphinx sphinx_rtd_theme django-environ django
script:
- ./build_docs.sh
cache:
......@@ -120,7 +120,7 @@ test_api:
interruptible: true
services:
- postgres:11
- redis:3
- redis:5
stage: test
image: funkwhale/funkwhale:develop
cache:
......@@ -131,12 +131,12 @@ test_api:
DATABASE_URL: "postgresql://postgres@postgres/postgres"
FUNKWHALE_URL: "https://funkwhale.ci"
DJANGO_SETTINGS_MODULE: config.settings.local
POSTGRES_HOST_AUTH_METHOD: trust
only:
- branches
before_script:
- apk add make
- apk add make git gcc python3-dev musl-dev
- cd api
- sed -i '/Pillow/d' requirements/base.txt
- pip3 install -r requirements/base.txt
- pip3 install -r requirements/local.txt
- pip3 install -r requirements/test.txt
......@@ -148,7 +148,7 @@ test_api:
test_front:
interruptible: true
stage: test
image: node:11
image: node:12-buster
before_script:
- cd front
only:
......@@ -170,7 +170,7 @@ test_front:
build_front:
stage: build
image: node:11
image: node:12-buster
before_script:
- curl -L -o /usr/local/bin/jq https://github.com/stedolan/jq/releases/download/jq-1.5/jq-linux64
- chmod +x /usr/local/bin/jq
......
......@@ -10,6 +10,320 @@ This changelog is viewable on the web at https://docs.funkwhale.audio/changelog.
.. towncrier
0.21 "Agate" (2020-04-24)
-------------------------
This 0.21 release is dedicated to Agate, to thank her, for both having created the Funkwhale project, being the current lead developer, and for her courage of coming out. Thank you Agate from all the members of the Funkwhale community <3
We are truly grateful as well to the dozens of people who contributed to this release with translations, development, documentation, reviews, design, testing, feedback, financial support, third-party projects and integrations… You made it possible!
Upgrade instructions are available at
https://docs.funkwhale.audio/admin/upgrading.html, there are also additional operations you need to execute, listed in the changelog below (search "Manual action").
Channels and podcasts
^^^^^^^^^^^^^^^^^^^^^
Funkwhale 0.21 includes a brand new feature: Channels!
Channels can be used as a replacement to public libraries,
to publish audio content, both musical and non-musical. They federate with other Funkwhale pods, but also other
fediverse software, in particular Mastodon, Pleroma, Friendica and Reel2Bits, meaning people can subscribe to your channel
from any of these software. To get started with publication, simply visit your profile and create a channel from there.
Each Funkwhale channel also comes with RSS feed that is compatible with existing podcasting applications, like AntennaPod
on Android and, within Funkwhale, you can also subscribe to any podcast from its RSS feed!
Many, many thanks to the numerous people who helped with the feature design, development and testing, and in particular
to the members of the working group who met every week for months in order to get this done, and the members of other third-party
projects who took the time to work with us to ensure compatibility.
Redesigned navigation, player and queue
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
This release includes a full redesign of our navigation, player and queue. Overall, it should provide
a better, less confusing experience, especially on mobile devices. This redesign was suggested
14 months ago, and took a while, but thanks to the involvement and feedback of many people, we got it done!
Improved search bar for searching remote objects
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The search bar now support fetching arbitrary objects using a URL. In particular, you can use this to quickly:
- Subscribe to a remote library via its URL
- Listen a public track from another pod
- Subscribe to a channel
Screening for sign-ups and custom sign-up form
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Instance admins can now configure their pod so that registrations required manual approval from a moderator. This
is especially useful on private or semi-private pods where you don't want to close registrations completely,
but don't want spam or unwanted users to join your pod.
When this is enabled and a new user register, their request is put in a moderation queue, and moderators
are notified by email. When the request is approved or refused, the user is also notified by email.
In addition, it's also possible to customize the sign-up form by:
- Providing a custom help text, in markdown format
- Including additional fields in the form, for instance to ask the user why they want to join. Data collected through these fields is included in the sign-up request and viewable by the mods
Federated reports
^^^^^^^^^^^^^^^^^
It's now possible to send a copy of a report to the server hosting the reported object, in order to make moderation easier and more distributed.
This feature is inspired by Mastodon's current design, and should work with at least Funkwhale and Mastodon servers.
Improved search performance
^^^^^^^^^^^^^^^^^^^^^^^^^^^
Our search engine went through a full rewrite to make it faster. This new engine is enabled
by default when using the search bar, or when searching for artists, albums and tracks. It leverages
PostgreSQL full-text search capabilities.
During our tests, we observed huge performance improvements after the switch, by an order of
magnitude. This should be especially perceptible on pods with large databases, more modest hardware
or hard drives.
We plan to remove the old engine in an upcoming release. In the meantime, if anything goes wrong,
you can switch back by setting ``USE_FULL_TEXT_SEARCH=false`` in your ``.env`` file.
Enforced email verification
^^^^^^^^^^^^^^^^^^^^^^^^^^^
The brand new ``ACCOUNT_EMAIL_VERIFICATION_ENFORCE`` setting can be used to make email verification
mandatory for your users. It defaults to ``false``, and doesn't apply to superuser accounts created through
the CLI.
If you enable this, ensure you have a SMTP server configured too.
More reliable CLI importer [manual action required]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Our CLI importer is now more reliable and less prone to Out-of-Memory issues, especially when scanning large libraries. (hundreds of GB or bigger)
We've also improved the directory crawling logic, so that you don't have to use glob patterns or specify extensions when importing. As a result, the syntax for providing directories to the command as changed slightly.
If you use the ``import_files`` command, this means you should replace scripts that look like this::
python api/manage.py import_files $LIBRARY_ID "/srv/funkwhale/data/music/**/*.ogg" "/srv/funkwhale/data/music/**/*.mp3" --recursive --noinput
By this::
python api/manage.py import_files $LIBRARY_ID "/srv/funkwhale/data/music/" --recursive --noinput
And Funkwhale will happily import any supported audio file from the specified directory.
User management through the server CLI
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
We now support user creation (incl. non-admin accounts), update and removal directly
from the server CLI. Typical use cases include:
- Changing a user password from the command line
- Creating or updating users from deployments scripts or playbooks
- Removing or granting permissions or upload quota to multiple users at once
- Marking multiple users as inactive
All user-related commands are available under the ``python manage.py fw users`` namespace.
Please refer to the `Admin documentation <https://docs.funkwhale.audio/admin/commands.html#user-management>`_ for
more information and instructions.
Progressive web app [Manual action suggested, non-docker only]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
We've made Funkwhale's Web UI a Progressive Web Application (PWA), in order to improve the user experience
during offline use, and on mobile devices.
In order to fully benefit from this change, if your pod isn't deployed using Docker, ensure
the following instruction is present in your nginx configuration::
location /front/ {
# Add the following line in the /front/ location
add_header Service-Worker-Allowed "/";
}
Postgres docker changed environment variable [manual action required, docker multi-container only]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
If you're running with docker and our multi-container setup, there was a breaking change starting in the 11.7 postgres image (https://github.com/docker-library/postgres/pull/658)
You need to add this to your .env file: ``POSTGRES_HOST_AUTH_METHOD=trust``
Newer deployments aren't affected.
Upgrade from Postgres 10 to 11 [manual action required, docker all-in-one only]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
With our upgrade to Alpine 3.10, the ``funkwhale/all-in-one`` image now includes PostgreSQL 11.
In order to update to Funkwhale 0.21, you will first need to uprade Funkwhale's PostgreSQL database, following the steps below::
# open a shell as the Funkwhale user
sudo -u funkwhale -H bash
# move to the funkwhale data directory
# (replace this with your own if you used a different path)
cd /srv/funkwhale/data
# stop the funkwhale container
docker stop funkwhale
# backup the database files
cp -r data/ ../postgres.bak
# Upgrade the database
docker run --rm \
-v $(pwd)/data:/var/lib/postgresql/10/data \
-v $(pwd)/upgraded-postgresql:/var/lib/postgresql/11/data \
-e PGUSER=funkwhale \
-e POSTGRES_INITDB_ARGS="-U funkwhale --locale C --encoding UTF8" \
tianon/postgres-upgrade:10-to-11
# replace the Postgres 10 files with Postgres 11 files
mv data/ postgres-10
mv upgraded-postgresql/ data
Once you have completed the Funkwhale upgrade with our regular instructions and everything works properly,
you can remove the backups/old files::
sudo -u funkwhale -H bash
cd /srv/funkwhale/data
rm -rf ../postgres.bak
rm -rf postgres-10
Full list of changes
^^^^^^^^^^^^^^^^^^^^
Features:
- Support for publishing and subscribing to podcasts (#170)
- Brand new navigation, queue and player redesign (#594)
- Can now browse a library content through the UI (#926)
- Federated reports (#1038)
- Screening for sign-ups (#1040)
- Make it possible to enforce email verification (#1039)
- Added a new radio based on another user listenings (#1060)
- User management through the server CLI
Enhancements:
- Added ability to reject library follows from notifications screen (#859)
- Added periodic background task and CLI command to associate genre tags to artists and albums based on identical tags found on corresponding tracks (#988)
- Added support for CELERYD_CONCURRENCY env var to control the number of worker processes (#997)
- Added the ability to sort albums by release date (#1013)
- Added two new radios to play your own content or a given library tracks
- Advertise list of known nodes on /api/v1/federation/domains and in nodeinfo if stats sharing is enabled
- Changed footer to use instance name if available, and append ellipses if instance URL/Name is too long (#1012)
- Favor local uploads when playing a track with multiple uploads (#1036)
- Include only local content in nodeinfo stats, added downloads count
- Make media and static files serving more reliable when reverse proxy X_FORWARDED_* headers are incorrect (#947)
- Order the playlist columns by modification date in the Browse tab (#775)
- Reduced size of funkwhale/funkwhale docker images thanks to multi-stage builds (!1042)
- Remember display settings in Album, Artist, Radio and Playlist views (#391)
- Removed unnecessary "Federation music needs approval" setting (#959)
- Replaced our slow research logic by PostgreSQL full-text search (#994)
- Support autoplay when loading embed frame from Mastodon and third-party websites (#1041)
- Support filtering playlist by name and several additional UX improvements in playlists modal (#974)
- Support modifying album cover art through the web UI (#588)
- Use a dedicated scope for throttling subsonic to avoid intrusive rate-limiting
- Use same markdown widget for all content fields (rules, description, reports, notes, etc.)
- CLI Importer is now more reliable and less resource-hungry on large libraries
- Add support custom domain for S3 storage
- Better placeholders for channels when there are no episodes or series
- Updated documentation for 0.21 release
- Improved performance and error handling when fetching remote attachments
Bugfixes:
- Added missing manuallyApprovesFollowers entry in JSON-LD contexts (#961)
- Fix issue with browser shortcuts such as search and focus URL not being recognised (#340, #985)
- Fixed admin dropdown not showing after login (#1042)
- Fixed an issue with celerybeat container failing to restart (#1004)
- Fixed invalid displayed number of tracks in playlist (#986)
- Fixed issue with recent results not being loaded from the API (#948)
- Fixed issue with sorting by album name not working (#960)
- Fixed short audio glitch when switching switching to another track with player paused (#970)
- Improved deduplication logic to prevent skipped files during import (#348, #474, #557, #740, #928)
- More resilient tag parsing with empty release date or album artist (#1037)
- More robust importer against malformed dates (#966)
- Removed "nodeinfo disabled" setting, as nodeinfo is required for the UI to work (#982)
- Replaced PDF icon by List icon in playlist placeholder (#943)
- Resolve an issue where disc numbers were not taken into consideration when playing an album from the album card (#1006)
- Set correct size for album covers in playlist cards (#680)
- Remove double spaces in ChannelForm
- Deduplicate tags in Audio ActivityPub representation
- Add support custom domain for S3 storage
- Fix #1079: fixed z-index issues with dropdowns (#1079 and #1075)
- Exclude external podcasts from library home
- Fixed broken channel save when description is too long
- Fixed 500 error when federation is disabled and application+json is requested
- Fixed minor subsonic API crash
- Fixed broken local profile page when allow-list is enabled
- Fixed issue with confirmation email not sending when signup-approval was enabled
- Ensure 0 quota on user is honored
- Fixed attachments URL not honoring media URL
- Fix grammar in msg string in TrackBase.vue
- Fix typo in SubscribeButton.vue
Translations:
- Arabic
- Catalan
- English (United Kingdom)
- German
- Hungarian
- Japanese
- Occitan
- Portuguese (Brazil)
- Russian
Contributors to this release (translation, development, documentation, reviews, design, testing, third-party projects):
- Agate
- annando
- Anton Strömkvist
- Audrey
- ButterflyOfFire
- Ciarán Ainsworth
- Creak
- Daniele Lira Mereb
- dashie
- Eloisa
- eorn
- Francesc Galí
- gerhardbeck
- GinnyMcQueen
- guillermau
- Haelwenn
- jinxx
- Jonathan Aylard
- Keunes
- M.G
- marzzzello
- Mathé Grievink
- Mélanie Chauvel
- Mjourdan
- Morgan Kesler
- Noe Gaumont
- Noureddine HADDAG
- Ollie
- Peter Wickenberg
- Quentin PAGÈS
- Renon
- Satsuki Yanagi
- Shlee
- SpcCw
- techknowlogick
- ThibG
- Tony Wasserka
- unklebonehead
- wakest
- wxcafé
- Xaloc
- Xosé M
0.20.1 (2019-10-28)
-------------------
......
......@@ -84,6 +84,18 @@ Visit https://dev.funkwhale.audio/funkwhale/funkwhale and clone the repository u
git clone ssh://git@dev.funkwhale.audio/funkwhale/funkwhale.git
cd funkwhale
.. note::
As of January 2020, the SSH fingerprints of our Gitlab server are the following::
$ ssh-keyscan dev.funkwhale.audio | ssh-keygen -lf -
# dev.funkwhale.audio:22 SSH-2.0-OpenSSH_7.4p1 Debian-10+deb9u6
# dev.funkwhale.audio:22 SSH-2.0-OpenSSH_7.4p1 Debian-10+deb9u6
# dev.funkwhale.audio:22 SSH-2.0-OpenSSH_7.4p1 Debian-10+deb9u6
2048 SHA256:WEZ546nkMhB9yV9lyDZZcEeN/IfriyhU8+mj7Cz/+sU dev.funkwhale.audio (RSA)
256 SHA256:dEhAo+1ImjC98hSqVdnkwVleheCulV8xIsV1eKUcig0 dev.funkwhale.audio (ECDSA)
256 SHA256:/AxZwOSP74hlNKCHzmu9Trlp9zVGTrsJOV+zet1hYyQ dev.funkwhale.audio (ED25519)
A note about branches
^^^^^^^^^^^^^^^^^^^^^
......@@ -709,3 +721,48 @@ The latter is especially useful when you are debugging failing tests.
.. note::
The front-end test suite coverage is still pretty low
Making a release
----------------
To make a new 3.4 release::
# setup
export NEXT_RELEASE=3.4 # replace with the next release number
export PREVIOUS_RELEASE=3.3 # replace with the previous release number
# ensure you have an up-to-date repo
git checkout develop # use master if you're doing a hotfix release
git pull
# compile changelog
towncrier --version $NEXT_RELEASE --yes
# polish changelog
# - update the date
# - look for typos
# - add list of contributors via `python3 scripts/get-contributions-stats.py develop $PREVIOUS_RELEASE`
nano CHANGELOG
# Set the `__version__` variable to $NEXT_RELEASE
nano nano api/funkwhale_api/__init__.py
# commit
git add .
git commit -m "Version bump and changelog for $NEXT_RELEASE"
# tag
git tag $NEXT_RELEASE
# publish
git push --tags && git push
# if you're doing a hotfix release from master
git checkout develop && git merge master && git push
# if you're doing a non-hotfix release, and a real release (not a real release) from develop
git checkout master && git merge develop && git push
Then, visit https://dev.funkwhale.audio/funkwhale/funkwhale/-/tags, copy-paste the changelog on the corresponding
tag, and announce the good news ;)
FROM alpine:3.8
FROM alpine:3.10 as builder
RUN \
echo 'installing dependencies' && \
apk add \
bash \
apk add --no-cache \
git \
gettext \
musl-dev \
gcc \
postgresql-dev \
python3-dev \
py3-psycopg2 \
py3-pillow \
libldap \
ffmpeg \
libpq \
libmagic \
libffi-dev \
make \
zlib-dev \
openldap-dev && \
\
jpeg-dev \
openldap-dev \
&& \
\
ln -s /usr/bin/python3 /usr/bin/python
# create virtual env for next stage
RUN python -m venv /venv
# emulate activation by prefixing PATH
ENV PATH="/venv/bin:$PATH" VIRTUAL_ENV=/venv
RUN mkdir /requirements
COPY ./requirements/base.txt /requirements/base.txt
# hack around https://github.com/pypa/pip/issues/6158#issuecomment-456619072
ENV PIP_DOWNLOAD_CACHE=/noop/
RUN \
echo 'fixing requirements file for alpine' && \
sed -i '/Pillow/d' /requirements/base.txt && \
\
\
echo 'installing pip requirements' && \
pip3 install --upgrade pip && \
pip3 install setuptools wheel && \
......@@ -44,6 +40,26 @@ COPY ./requirements/*.txt /requirements/
RUN \
if [ "$install_dev_deps" = "1" ] ; then echo "Installing dev dependencies" && pip3 install --no-cache-dir -r /requirements/local.txt -r /requirements/test.txt ; else echo "Skipping dev deps installation" ; fi
FROM alpine:3.10 as build-image
COPY --from=builder /venv /venv
# emulate activation by prefixing PATH
ENV PATH="/venv/bin:$PATH"
RUN apk add --no-cache \
libmagic \
bash \
gettext \
python3 \
jpeg-dev \
ffmpeg \
libpq \
&& \
\
ln -s /usr/bin/python3 /usr/bin/python
ENTRYPOINT ["./compose/django/entrypoint.sh"]
CMD ["./compose/django/server.sh"]
......
......@@ -4,6 +4,7 @@ from rest_framework import routers
from rest_framework.urlpatterns import format_suffix_patterns
from funkwhale_api.activity import views as activity_views
from funkwhale_api.audio import views as audio_views
from funkwhale_api.common import views as common_views
from funkwhale_api.common import routers as common_routers
from funkwhale_api.music import views
......@@ -21,6 +22,8 @@ router.register(r"uploads", views.UploadViewSet, "uploads")
router.register(r"libraries", views.LibraryViewSet, "libraries")
router.register(r"listen", views.ListenViewSet, "listen")
router.register(r"artists", views.ArtistViewSet, "artists")
router.register(r"channels", audio_views.ChannelViewSet, "channels")
router.register(r"subscriptions", audio_views.SubscriptionsViewSet, "subscriptions")
router.register(r"albums", views.AlbumViewSet, "albums")
router.register(r"licenses", views.LicenseViewSet, "licenses")
router.register(r"playlists", playlists_views.PlaylistViewSet, "playlists")
......@@ -28,6 +31,7 @@ router.register(
r"playlist-tracks", playlists_views.PlaylistTrackViewSet, "playlist-tracks"
)
router.register(r"mutations", common_views.MutationViewSet, "mutations")
router.register(r"attachments", common_views.AttachmentViewSet, "attachments")
v1_patterns = router.urls
subsonic_router = routers.SimpleRouter(trailing_slash=False)
......@@ -84,6 +88,9 @@ v1_patterns += [
url(r"^token/?$", jwt_views.obtain_jwt_token, name="token"),
url(r"^token/refresh/?$", jwt_views.refresh_jwt_token, name="token_refresh"),
url(r"^rate-limit/?$", common_views.RateLimitView.as_view(), name="rate-limit"),
url(
r"^text-preview/?$", common_views.TextPreviewView.as_view(), name="text-preview"
),
]
urlpatterns = [
......
This diff is collapsed.
from django import urls
from funkwhale_api.audio import spa_views as audio_spa_views
from funkwhale_api.federation import spa_views as federation_spa_views
from funkwhale_api.music import spa_views
......@@ -20,4 +22,24 @@ urlpatterns = [
spa_views.library_playlist,
name="library_playlist",
),
urls.re_path(
r"^library/(?P<uuid>[0-9a-f-]+)/?$",
spa_views.library_library,
name="library_library",
),
urls.re_path(
r"^channels/(?P<uuid>[0-9a-f-]+)/?$",
audio_spa_views.channel_detail_uuid,
name="channel_detail",
),
urls.re_path(
r"^channels/(?P<username>[^/]+)/?$",
audio_spa_views.channel_detail_username,
name="channel_detail",
),
urls.re_path(
r"^@(?P<username>[^/]+)/?$",
federation_spa_views.actor_detail_username,
name="actor_detail",
),
]
......@@ -40,3 +40,8 @@ if settings.DEBUG:
urlpatterns = [
path("api/__debug__/", include(debug_toolbar.urls))
] + urlpatterns
if "silk" in settings.INSTALLED_APPS:
urlpatterns = [
url(r"^api/silk/", include("silk.urls", namespace="silk"))
] + urlpatterns
# -*- coding: utf-8 -*-
__version__ = "0.20.1"
__version__ = "0.21"
__version_info__ = tuple(
[
int(num) if num.isdigit() else num
......
from funkwhale_api.common import admin
from . import models
@admin.register(models.Channel)
class ChannelAdmin(admin.ModelAdmin):
list_display = [