README.rst 9.03 KB
Newer Older
1
2
3
4
5
6
7
Funkwhale
=============

A self-hosted tribute to Grooveshark.com.

LICENSE: BSD

8
9
Getting help
------------
10

11
We offer various Matrix.org rooms to discuss about funkwhale:
12

13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
- `#funkwhale:matrix.org <https://riot.im/app/#/room/#funkwhale:matrix.org>`_ for general questions about funkwhale
- `#funkwhale-dev:matrix.org <https://riot.im/app/#/room/#funkwhale-dev:matrix.org>`_ for development-focused discussion

Please join those rooms if you have any questions!

Running the development version
-------------------------------

If you want to fix a bug or implement a feature, you'll need
to run a local, development copy of funkwhale.

We provide a docker based development environment, which should
be both easy to setup and work similarly regardless of your
development machine setup.

Instructions for bare-metal setup will come in the future (Merge requests
are welcome).

Installing docker and docker-compose
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

This is already cover in the relevant documentations:

- https://docs.docker.com/install/
- https://docs.docker.com/compose/install/

Cloning the project
^^^^^^^^^^^^^^^^^^^

Visit https://code.eliotberriot.com/funkwhale/funkwhale and clone the repository using SSH or HTTPS. Exemple using SSH::

    git clone ssh://git@code.eliotberriot.com:2222/funkwhale/funkwhale.git
    cd funkwhale


A note about branches
^^^^^^^^^^^^^^^^^^^^^

Next release development occurs on the "develop" branch, and releases are made on the "master" branch. Therefor, when submitting Merge Requests, ensure you are merging on the develop branch.


Working with docker
^^^^^^^^^^^^^^^^^^^

In developpement, we use the docker-compose file named ``dev.yml``, and this is why all our docker-compose commands will look like this::

    docker-compose -f dev.yml logs

If you do not want to add the ``-f dev.yml`` snippet everytime, you can run this command before starting your work::

    export COMPOSE_FILE=dev.yml


Building the containers
^^^^^^^^^^^^^^^^^^^^^^^

On your initial clone, or if there have been some changes in the
app dependencies, you will have to rebuild your containers. This is done
via the following command::
72
73
74
75

    docker-compose -f dev.yml build


76
77
78
79
80
81
82
83
84
85
86
87
88
Creating your env file
^^^^^^^^^^^^^^^^^^^^^^

We provide a working .env.dev configuration file that is suitable for
development. However, to enable customization on your machine, you should
also create a .env file that will hold your personal environment
variables (those will not be commited to the project).

Create it like this::

    touch .env


89
90
91
92
93
94
95
96
97
98
99
100
101
102
Database management
^^^^^^^^^^^^^^^^^^^

To setup funkwhale's database schema, run this::

    docker-compose -f dev.yml run --rm api python manage.py migrate

This will create all the tables needed for the API to run proprely.
You will also need to run this whenever changes are made on the database
schema.

It is safe to run this command multiple times, so you can run it whenever
you fetch develop.

103

104
105
Development data
^^^^^^^^^^^^^^^^
106

107
108
You'll need at least an admin user and some artists/tracks/albums to work
locally.
109

110
Create an admin user with the following command::
111

112
113
114
115
116
117
118
119
120
121
    docker-compose -f dev.yml run --rm api python manage.py createsuperuser

Injecting fake data is done by running the fllowing script::

    artists=25
    command="from funkwhale_api.music import fake_data; fake_data.create_data($artists)"
    echo $command | docker-compose -f dev.yml run --rm api python manage.py shell -i python

The previous command will create 25 artists with random albums, tracks
and metadata.
122
123
124
125
126
127
128


Launch all services
^^^^^^^^^^^^^^^^^^^

Then you can run everything with::

129
130
131
132
133
134
135
136
137
138
    docker-compose -f dev.yml up

This will launch all services, and output the logs in your current terminal window.
If you prefer to launch them in the background instead, use the ``-d`` flag, and access the logs when you need it via ``docker-compose -f dev.yml logs --tail=50 --follow``.

Once everything is up, you can access the various funkwhale's components:

- The Vue webapp, on http://localhost:8080
- The API, on http://localhost:8080/api/v1/
- The django admin, on http://localhost:8080/api/admin/
139
140
141


Running API tests
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
^^^^^^^^^^^^^^^^^

To run the pytest test suite, use the following command::

    docker-compose -f dev.yml run --rm api pytest

This is regular pytest, so you can use any arguments/options that pytest usually accept::

    # get some help
    docker-compose -f dev.yml run --rm api pytest -h
    # Stop on first failure
    docker-compose -f dev.yml run --rm api pytest -x
    # Run a specific test file
    docker-compose -f dev.yml run --rm api pytest tests/test_acoustid.py


Running front-end tests
^^^^^^^^^^^^^^^^^^^^^^^

To run the front-end test suite, use the following command::

    docker-compose -f dev.yml run --rm front yarn run unit

We also support a "watch and test" mode were we continually relaunch
tests when changes are recorded on the file system::

    docker-compose -f dev.yml run --rm front yarn run unit-watch

The latter is especially useful when you are debugging failing tests.

.. note::

    The front-end test suite coverage is still pretty low


Stopping everything
^^^^^^^^^^^^^^^^^^^

Once you're down with your work, you can stop running containers, if any, with::

    docker-compose -f dev.yml stop


Removing everything
^^^^^^^^^^^^^^^^^^^

If you want to wipe your development environment completely (e.g. if you want to start over from scratch), just run::

    docker-compose -f dev.yml down -v

This will wipe your containers and data, so please be careful before running it.
193

194
You can keep your data by removing the ``-v`` flag.
195
196


197
198
Typical workflow for a merge request
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
199

200
201
202
203
204
205
206
207
208
0. Fork the project if you did not already or if you do not have access to the main repository
1. Checkout the development branch and pull most recent changes: ``git checkout develop && git pull``
2. Create a dedicated branch for your work ``42-awesome-fix``. It is good practice to prefix your branch name with the ID of the issue you are solving.
3. Work on your stuff
4. Commit small, atomic changes to make it easier to review your contribution
5. Add a changelog fragment to summarize your changes: ``echo "Implemented awesome stuff (#42)" > changes/changelog.d/42.feature"``
6. Push your branch
7. Create your merge request
8. Take a step back and enjoy, we're really grateful you did all of this and took the time to contribute!
209
210


211
212
213
214
215
216
217
218
219
220
221
Internationalization
--------------------

When working on the front-end, any end-user string should be translated
using either ``<i18next path="yourstring">`` or the ``$t('yourstring')``
function.

Extraction is done by calling ``yarn run i18n-extract``, which
will pull all the strings from source files and put them in a PO file.


222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
Working with federation locally
-------------------------------

To achieve that, you'll need:

1. to update your dns resolver to resolve all your .dev hostnames locally
2. a reverse proxy (such as traefik) to catch those .dev requests and
   and with https certificate
3. two instances (or more) running locally, following the regular dev setup

Resolve .dev names locally
^^^^^^^^^^^^^^^^^^^^^^^^^^

If you use dnsmasq, this is as simple as doing::

    echo "address=/test/172.17.0.1" | sudo tee /etc/dnsmasq.d/test.conf
    sudo systemctl restart dnsmasq

If you use NetworkManager with dnsmasq integration, use this instead::

    echo "address=/test/172.17.0.1" | sudo tee /etc/NetworkManager/dnsmasq.d/test.conf
    sudo systemctl restart NetworkManager

Add wildcard certificate to the trusted certificates
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Simply copy bundled certificates::

    sudo cp docker/ssl/test.crt /usr/local/share/ca-certificates/
    sudo update-ca-certificates

This certificate is a wildcard for ``*.funkwhale.test``

Run a reverse proxy for your instances
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^


259
Create docker network
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
^^^^^^^^^^^^^^^^^^^^

Create the federation network::

    docker network create federation

Launch everything
^^^^^^^^^^^^^^^^^

Launch the traefik proxy::

    docker-compose -f docker/traefik.yml up -d

Then, in separate terminals, you can setup as many different instances as you
need::

    export COMPOSE_PROJECT_NAME=node2
    docker-compose -f dev.yml run --rm api python manage.py migrate
    docker-compose -f dev.yml run --rm api python manage.py createsuperuser
279
    docker-compose -f dev.yml up nginx api front nginx api celeryworker
280
281
282
283
284
285
286

Note that by default, if you don't export the COMPOSE_PROJECT_NAME,
we will default to node1 as the name of your instance.

Assuming your project name is ``node1``, your server will be reachable
at ``https://node1.funkwhale.test/``. Not that you'll have to trust
the SSL Certificate as it's self signed.
287
288
289
290
291
292
293
294

When working on federation with traefik, ensure you have this in your ``env``::

    # This will ensure we don't bind any port on the host, and thus enable
    # multiple instances of funkwhale to be spawned concurrently.
    WEBPACK_DEVSERVER_PORT_BINDING=
    # This disable certificate verification
    EXTERNAL_REQUESTS_VERIFY_SSL=false
295
296
    # this ensure you don't have incorrect urls pointing to http resources
    FUNKWHALE_PROTOCOL=https