funkwhale issueshttps://dev.funkwhale.audio/funkwhale/funkwhale/-/issues2024-03-27T15:46:34Zhttps://dev.funkwhale.audio/funkwhale/funkwhale/-/issues/1315Does not show follower-requests2024-03-27T15:46:34ZxundeenergieDoes not show follower-requests## Steps to reproduce
1. Go to mastodon or Pleroma, search for @user@yourfunkwhale.instance, choose user, klick "Follow"
2. Go to funkwhaleinstance, check notifications, check settings->user-requests
3. Go to https://<funkwhaleinstance>/...## Steps to reproduce
1. Go to mastodon or Pleroma, search for @user@yourfunkwhale.instance, choose user, klick "Follow"
2. Go to funkwhaleinstance, check notifications, check settings->user-requests
3. Go to https://<funkwhaleinstance>/api/admin/federation/follow/
4. Reload userpage in mastodon or pleroma
## What happens?
In Mastodon and pleroma you see, the following-request is pending
In funkwhale as user, you do not see any following-request
In django-backend you can see the following-request as "not approved"
Approve the following-request make a green check appear instead of the black "?"
- mastodon/pleroma-user is not shown in funkwhale as follower
- mastodon/pleroma stays at "following requested" for the funkwhale-user
- no notifications about new titles or playlists on mastodon/pleroma
## What is expected?
I expect that i can approve following-requests from user-interface, that the following-requests are shown (and approveable)
Mastodon and Pleroma should receive notifications, when i upload or change my music/titles/playlists-library
## Context
<!--
The version of your instance can be found on the footer : Source code (x.y)
-->
**Funkwhale version(s) affected**: 1.0.1 and develop
- Instance configuration: nginx, manually installed from git
Mastodon search for funkwhale user
![image](/uploads/86c8ba081e126f66cc99b4e4e8ee9aed/image.png)
Mastodon klick on "Follow"
![image](/uploads/21ddf353085dd3e9b26e8b228c80a145/image.png)
Mastodon reload this page
![image](/uploads/8b11a34729bfd959f8a0a5b12e92db09/image.png)
Funkwhale Notifications (one request is shown twice)
![image](/uploads/9f7271e7c3bb1c420f9704c36828eef6/image.png)
Funkwhale show Following requests
![image](/uploads/98eb68a22ec7d45a158f8d830908eb18/image.png)
Funkwhale Django backend showing Following request
![image](/uploads/dbfc049c85287dc7170d0499678d36b1/image.png)
![image](/uploads/ac3b0149287753cf43f65c67e79f9e35/image.png)
Funkwhale Django request approved and saved
![image](/uploads/fd1ac378f93f785e27d456f3ad23b964/image.png)
Mastodon reload again following-page for funkwhale-user
![image](/uploads/389ac95f181cc45215c2c7fdf03ebb62/image.png)https://dev.funkwhale.audio/funkwhale/funkwhale/-/issues/1301Option to submit podcast transript2021-06-17T13:06:35ZKuba OrlikOption to submit podcast transript## What is the problem you are facing?
I'm hosting a podcast on Funkwhale and recently we've started writing down transcripts of our episodes. Funkwhale does not offer a place to enter the text transcript of given episode.
## What are ...## What is the problem you are facing?
I'm hosting a podcast on Funkwhale and recently we've started writing down transcripts of our episodes. Funkwhale does not offer a place to enter the text transcript of given episode.
## What are the possible drawbacks or issues with the requested changes?
Can't see anyhttps://dev.funkwhale.audio/funkwhale/funkwhale/-/issues/1268Tracker music support2021-08-10T11:28:33ZDrqTracker music support<!--
Hi there! You are about to share feature request or an idea, and we want to thank you!
If it's the first time you post here, please take a moment to read our Code of Conduct
(https://funkwhale.audio/code-of-conduct/) and ensure yo...<!--
Hi there! You are about to share feature request or an idea, and we want to thank you!
If it's the first time you post here, please take a moment to read our Code of Conduct
(https://funkwhale.audio/code-of-conduct/) and ensure your issue respect our guidelines.
To ensure we can deal with your idea or request, please try to stick
to the following structure. You can leave the parts text between `<!- ->`
markers untouched, they won't be displayed in your final message.
Please do not edit the following line, it's used for automatic classification
-->
## What is the problem you are facing?
Inability to play, catalogue and share tracker music formats such as MOD, S3M, XM, IT and all such.
## What are the possible drawbacks or issues with the requested changes?
One possible drawback is while those are definitely are music formats, they are not sound formats - they need to be rendered before playing. However, there are libraries that can do it, like libopenmpt.
## Context
There are troves of wonderful music made by the hacker scene. From the 80s to present day, the demoscene and otherwise electronic musicians continue to create music in this format. And it's free! Most of the time it is.
Also, the size of those files often measures in kilobytes, because they were made for systems that are really cramped for space and memory. They are unbelivably tiny. That means you can have a formidable library of music in the peanuts of space.
There is a site that publishes tracker music: https://modarchive.org - but frankly I find it lacking in almost every sense compared to any Funkwhale instance. This music needs proper catalogization, and, if I may be so bold, popularization through sharing and streaming.https://dev.funkwhale.audio/funkwhale/funkwhale/-/issues/1256Let album release date be only year2021-07-31T15:10:37ZPorrumentzioLet album release date be only yearCurrently, `v 0.21.2`, the release date of album **must** be YYYY-MM-DD, but many times an album only has the year. I think letting the release date be only year (YYYY) would be an improvement.Currently, `v 0.21.2`, the release date of album **must** be YYYY-MM-DD, but many times an album only has the year. I think letting the release date be only year (YYYY) would be an improvement.https://dev.funkwhale.audio/funkwhale/funkwhale/-/issues/1218Smart Playlists2022-09-07T10:07:11ZAMoonRabbitSmart Playlists<!--
Hi there! You are about to share feature request or an idea, and we want to thank you!
If it's the first time you post here, please take a moment to read our Code of Conduct
(https://funkwhale.audio/code-of-conduct/) and ensure yo...<!--
Hi there! You are about to share feature request or an idea, and we want to thank you!
If it's the first time you post here, please take a moment to read our Code of Conduct
(https://funkwhale.audio/code-of-conduct/) and ensure your issue respect our guidelines.
To ensure we can deal with your idea or request, please try to stick
to the following structure. You can leave the parts text between `<!- ->`
markers untouched, they won't be displayed in your final message.
Please do not edit the following line, it's used for automatic classification
-->
## Feature description
Implementation of smart playlists based on moods. Similar to Sony's SensMe channels, playlists are generated based on the mood of tracks. For example, there would be chilled, energetic, upbeat, Slow etc, or time of day.
## What are the possible drawbacks or issues with the requested changes?
This would require that the current library is analysed, measuring each track to determine where it lies in the mood spectrum so will need additional entries in the database, and additional file storage too for saving the information. May also add to system resource usage on existing libraries, but otherwise wouldn't expect it to be too much of an impact if it's included in the upload process too.
## Context
https://helpguide.sony.net/dmp/nwza10/v1/en/contents/TP0000410539.html#:~:text=SensMe%E2%84%A2%20Channels%20is%20a,Walkman%20or%20on%20the%20Walkman.
https://en.wikipedia.org/wiki/SensMehttps://dev.funkwhale.audio/funkwhale/funkwhale/-/issues/1048Unaccent search is not working anymore when full text search is enabled2023-04-14T15:21:34ZAgateUnaccent search is not working anymore when full text search is enabledAs reported today on matrix. We may want to have a look at https://stackoverflow.com/a/38713813/2844093 to see if we can combine `unaccent` and `full-text-seach`As reported today on matrix. We may want to have a look at https://stackoverflow.com/a/38713813/2844093 to see if we can combine `unaccent` and `full-text-seach`https://dev.funkwhale.audio/funkwhale/funkwhale/-/issues/1046Use cover hash to deduplicate albums during import2022-07-19T19:46:41ZAgateUse cover hash to deduplicate albums during importSome albums don't have an album artist but have the same cover hash. When possible, we should use that to avoid creating multiple versions of the same album.
cf https://dev.funkwhale.audio/funkwhale/funkwhale/issues/1045Some albums don't have an album artist but have the same cover hash. When possible, we should use that to avoid creating multiple versions of the same album.
cf https://dev.funkwhale.audio/funkwhale/funkwhale/issues/1045https://dev.funkwhale.audio/funkwhale/funkwhale/-/issues/1027Enhance Recently added radio2022-01-28T15:47:09ZEleosEnhance Recently added radio<!--
Hi there! You are about to share feature request or an idea, and we want to thank you!
If it's the first time you post here, please take a moment to read our Code of Conduct
(https://funkwhale.audio/code-of-conduct/) and ensure yo...<!--
Hi there! You are about to share feature request or an idea, and we want to thank you!
If it's the first time you post here, please take a moment to read our Code of Conduct
(https://funkwhale.audio/code-of-conduct/) and ensure your issue respect our guidelines.
To ensure we can deal with your idea or request, please try to stick
to the following structure. You can leave the parts text between `<!- ->`
markers untouched, they won't be displayed in your final message.
Please do not edit the following line, it's used for automatic classification
-->
## What is the problem you are facing?
Using funkwhale is great, mostly when we are multiple people using the instance to find new music. Sadly, there is no way to find what other members recently uploaded. Maybe having a radio or a playlist that feeds itself with new content regularly could be a good idea.
We could do a "added last week/month" radio that would automatically removed the most ancien songs after some times.
## What are the possible drawbacks or issues with the requested changes?
Multiple drawbacks I see :
* I'm no UX expert, but the home page might get a bit more congested/complex , depending on where we want to display this "recently added" playlist / radio.
* I don't know the quantity of work frontend + backend that will be needed about this.
* Could it be too much of computation for big funkwhale instances ?
Thanks in advance for your feedbacks.petitminionpetitminionhttps://dev.funkwhale.audio/funkwhale/funkwhale/-/issues/1025Implement new library card component2021-06-17T13:06:53ZAgateImplement new library card component@mjourdan has submitted a new mockup for the library cards.
- Full card redesign available here: https://dev.funkwhale.audio/mjourdan/funkwhale-design/blob/master/browser/cards.png
- See below
![image](/uploads/846255506c3d5f8d9c109a0...@mjourdan has submitted a new mockup for the library cards.
- Full card redesign available here: https://dev.funkwhale.audio/mjourdan/funkwhale-design/blob/master/browser/cards.png
- See below
![image](/uploads/846255506c3d5f8d9c109a040d4f52d7/image.png)
This is what the cards currently look like:
## Other user library (as seen in artists/albums/tracks pages)
Vue Component: `front/src/views/content/remote/Card.vue`
![image](/uploads/e869db46cd2970ef33dab90a5ee925b1/image.png)
## Owned library (as seen in `/content/libraries/`)
Vue Component: `front/src/views/content/libraries/Card.vue`
![image](/uploads/c734dc891fa5e74edb10b82dd5b44fb9/image.png)https://dev.funkwhale.audio/funkwhale/funkwhale/-/issues/1002Queue stores direct urls of album artwork in object storage/s32022-08-26T07:15:01ZDag StenstadQueue stores direct urls of album artwork in object storage/s3Unlike streamed media files wich are (can be?) proxied, URLs for album artwork are stored as direct object storage urls in the local queue of the user. As these urls have an expiry time (configured by env variable AWS_QUERYSTRING_EXPIRE ...Unlike streamed media files wich are (can be?) proxied, URLs for album artwork are stored as direct object storage urls in the local queue of the user. As these urls have an expiry time (configured by env variable AWS_QUERYSTRING_EXPIRE in the documentation) they will after a while start to fail, as they are no longer authenticated.
I propose that Funkwhale instead proxy+cache the media, so that images in playlist will never fail.
The current behaviour is documented here: https://docs.funkwhale.audio/admin/external-storages.htmlhttps://dev.funkwhale.audio/funkwhale/funkwhale/-/issues/984Add ability to move tracks between libraries2023-12-12T21:40:39ZThibGAdd ability to move tracks between librariesHaving added legally-sharable tracks to a private library, I would like to move them to a different, public library, without having to remove the tracks and re-import them.Having added legally-sharable tracks to a private library, I would like to move them to a different, public library, without having to remove the tracks and re-import them.https://dev.funkwhale.audio/funkwhale/funkwhale/-/issues/920Use uuid instead of username for actor fid2021-06-17T13:06:58ZAgateUse uuid instead of username for actor fidthis will make username change possible and easier ;)
cf https://matrix.to/#/!SwDTmjceJHHroSEIPP:matrix.org/$1568879698187722JWlCo:matrix.org?via=matrix.org&via=disroot.org&via=matrix.taboulisme.comthis will make username change possible and easier ;)
cf https://matrix.to/#/!SwDTmjceJHHroSEIPP:matrix.org/$1568879698187722JWlCo:matrix.org?via=matrix.org&via=disroot.org&via=matrix.taboulisme.comhttps://dev.funkwhale.audio/funkwhale/funkwhale/-/issues/900Clone playlists2022-11-09T21:01:20ZnoutsClone playlistsAs describe there : https://governance.funkwhale.audio/d/npXlaBWx/collaborative-playlists/9
I would like to be able to clone an existing playlist (mine or someone else). It creates a completely new and independent playlist I own.As describe there : https://governance.funkwhale.audio/d/npXlaBWx/collaborative-playlists/9
I would like to be able to clone an existing playlist (mine or someone else). It creates a completely new and independent playlist I own.https://dev.funkwhale.audio/funkwhale/funkwhale/-/issues/811Include results matching 'and' when '&' is in search query, and vice-versa2021-10-18T20:26:18ZQasim AliInclude results matching 'and' when '&' is in search query, and vice-versa<!--
Hi there! You are about to share feature request or an idea, and we want to thank you!
If it's the first time you post here, please take a moment to read our Code of Conduct
(https://funkwhale.audio/code-of-conduct/) and ensure yo...<!--
Hi there! You are about to share feature request or an idea, and we want to thank you!
If it's the first time you post here, please take a moment to read our Code of Conduct
(https://funkwhale.audio/code-of-conduct/) and ensure your issue respect our guidelines.
To ensure we can deal with your idea or request, please try to stick
to the following structure. You can leave the parts text between `<!- ->`
markers untouched, they won't be displayed in your final message.
Please do not edit the following line, it's used for automatic classification
-->
## What is the problem you are facing?
When the search query includes `and` or `&`, I would like to have the returned results also search for tracks/artists matching **both** `and` and `&`. For example, suppose that the tracks `Red & Black` and `Red and Black` are present on the server. Then searching for either `Red and Black` or `Red & Black` would return both.
Mockup:
![Screenshot_20190428_114322](/uploads/969159072613bfa9d8e02787845f3fcc/Screenshot_20190428_114322.png)
## What are the possible drawbacks or issues with the requested changes?
Search results involving `and` or `&` would no longer be literal. However (I believe) funkwhale already does fuzzy matching, so this isn't a huge concern.
## Context
Many online media services including Spotify already perform this sort of substitution.https://dev.funkwhale.audio/funkwhale/funkwhale/-/issues/762I-3 Global federation aka Domain follow2024-01-23T15:35:49ZgordonI-3 Global federation aka Domain follow- forum : https://forum.funkwhale.audio/d/209-domain-follow-feature/6
- spec : https://pad.funkwhale.audio/0ov04mrqRj2JEv6p16bHuw- forum : https://forum.funkwhale.audio/d/209-domain-follow-feature/6
- spec : https://pad.funkwhale.audio/0ov04mrqRj2JEv6p16bHuwCiarán Ainsworthsporiff@funkwhale.audiopetitminionCiarán Ainsworthsporiff@funkwhale.audiohttps://dev.funkwhale.audio/funkwhale/funkwhale/-/issues/753Add Two-Factor Authentication methods2021-06-17T13:06:54ZgordonAdd Two-Factor Authentication methodsNowadays, two-factor auth (2FA) is quite important for web services. It can prevent a large number of privacy violations (unauthorized access to an account), elevation of privileges, and attacks like phishing, or even Person in the Middl...Nowadays, two-factor auth (2FA) is quite important for web services. It can prevent a large number of privacy violations (unauthorized access to an account), elevation of privileges, and attacks like phishing, or even Person in the Middle with more advanced methods like [U2F](https://www.yubico.com/solutions/fido-u2f/). Funkwhale is using Django, which is a common web framework with very active community. So, I think adding 2FA methods should be an important improvement, and should also be easy to do, since there is community-driven Django 2FA apps.
So, I suggest to add support for the [Django-2FA](https://github.com/Bouke/django-two-factor-auth) project in Funkwhale. But doing so would break current authentication schemes, for example if there is Funkwhale clients (as far as i know, there is only the [Funkwhale CLI client](https://dev.funkwhale.audio/funkwhale/cli) and the [Modipy plugin](https://dev.funkwhale.audio/funkwhale/mopidy)), setting hardcoded login+password in the configuration file will not be enough (it’s also a bad security practice to write down a user password), so implementing a client authentication method should be a dependency of this feature (#752).
Steps:
- [ ] Implement Django-2FA in backend
- [ ] Implement REST endpoints if Django-2FA does not provide them (GET 2FA state for an user, for example)
- [ ] Create VueJS views for 2FA (second authentication step, 2FA methods management pages)https://dev.funkwhale.audio/funkwhale/funkwhale/-/issues/699Timer for stopping music2022-11-09T20:34:22ZOniriCorpeTimer for stopping music## What is the problem you are facing?
Hi, it could be nice tout add a timer to stop music streaming when it finished
This could be useful for :
- playing music before sleeping
- to help mesure time dedicated to a task (for example ...## What is the problem you are facing?
Hi, it could be nice tout add a timer to stop music streaming when it finished
This could be useful for :
- playing music before sleeping
- to help mesure time dedicated to a task (for example : work on a thing whithout an alarm clock)
- avoid pointless server load in general
## What are the possible drawbacks or issues with the requested changes?
None that I see ^^`
<!--
Altering the system behaviour is not always a free action, and it can impact
user experience, performance, introduce bugs or complexity, etc..
If you think about anything we should keep in mind while
examining your request, please describe it in this section.
-->https://dev.funkwhale.audio/funkwhale/funkwhale/-/issues/639II-3 Link to tracks that aren’t on Funkwhale2024-02-26T12:47:23ZIcaria36II-3 Link to tracks that aren’t on Funkwhale- mb endpoint for release : https://musicbrainz.org/ws/2/release/0141c67c-fa0f-4da7-8925-1ff093c4887e?inc=url-rels
- Forum : https://forum.funkwhale.audio/d/355-link-to-tracks-that-arent-on-funkwhale
- Draft specification: https://pad.fu...- mb endpoint for release : https://musicbrainz.org/ws/2/release/0141c67c-fa0f-4da7-8925-1ff093c4887e?inc=url-rels
- Forum : https://forum.funkwhale.audio/d/355-link-to-tracks-that-arent-on-funkwhale
- Draft specification: https://pad.funkwhale.audio/VyGBZFb4T5G57q-99pApGQ
Create a song page with external audio file embedded or URL pointing to external page
## What is the problem you are facing?
Due to copyright restrictions, there are tons of audio files that cannot be uploaded. However, quite often these audios can be listened legally in some URL, and sometimes it can be even embedded in a website. Just like Mastodon etc benefit from the possibility to link to articles, songs, videos... Funkwhale could become a lot more social if music with All Rights Reserved could be at least found, and maybe played through an embedded player.
## What are the possible drawbacks or issues with the requested changes?
<!--
Altering the system behaviour is not always a free action, and it can impact
user experience, performance, introduce bugs or complexity, etc..
If you think about anything we should keep in mind while
examining your request, please describe it in this section.
-->
These songs linked/embedded cannot be eligible for playlists, most likely. This is fine. The use case here is to enable people to share and discover more music in the wider sense. Once a user has discovered a song they like, they can figure out how to obtain it and/or keep listening it.
There is a risk of spam or other forms of abuse coming from this direction. File uploads also have risks (uploading copyrighted content, garbage, files with messed tags...) but typing a URL is even easier. Ways to reduce this risk could include
* having this feature disabled by default (admins would enable it manually)
* allow admins to limit URLs to certain services (Soundcloud, Bandcamp...)
* grant this feature to users manually or after they meet some trust criteria
## Context
<!--
If relevant, share additional context here like:
- Links to existing implementations or examples of the requested feature
- Screenshots
-->
This request goes in the lines of #498 Community features. Just like we are seeing specialized Mastodon instances, Funkwhale has the potential of building communities specializing in certain types of music or audio files. Imagine clubs of fans or music collectors. In these cases, compiling - sharing - discovering - rating - commenting music might become a reason for an instance to exist almost as important as actually listening that music.
A real use case: we are a group of freedom lovers thinking that one way of fighting fascism is to promote groups and songs with different values. I guess I'm not the only one that has been influenced by music when growing up and forming your own ideas. Anyway, we could set up a wiki or something along these lines, but I see a lot of potential in Funkwhale, allowing to collect and share together, adding the concept of Federation to the mix and the future possibility of interacting with other ActivityPub services. If the user experience is good (and I bet Funkwhale has the potential) I think we could attract many people who don't consider themselves free software or Fediverse enthusiasts. And we want to promote music with open licenses, but if we are restricted to these licenses... most of the songs that made us love freedom and hate fascism will be probably out of the plate.Ciarán Ainsworthsporiff@funkwhale.audiopetitminionCiarán Ainsworthsporiff@funkwhale.audiohttps://dev.funkwhale.audio/funkwhale/funkwhale/-/issues/596Possibility to "mute" users2021-10-18T18:23:47ZVonPossibility to "mute" users<!--
Hi there! You are about to share feature request or an idea, and we want to thank you!
To ensure we can deal with your idea or request, please try to stick
to the following structure. You can leave the parts text between `<!- ->`
m...<!--
Hi there! You are about to share feature request or an idea, and we want to thank you!
To ensure we can deal with your idea or request, please try to stick
to the following structure. You can leave the parts text between `<!- ->`
markers untouched, they won't be displayed in your final message.
Please do not edit the following line, it's used for automatic classification
-->
## What is the problem you are facing?
I see a person listening to the same artist everyday, I see almost only that artist all day looping in the "Recently listened" panel.
I'd like the possibility to not show the activity of that user, and extend it to the possibility to mute some specific users.
There's also the possibility to make a panel that shows only the users wanted. So the solution would be a blacklist or a whitelist.
<!--
Describe the problem you'd like to solve, and why we need to add or
improve something in the current system to solve that problem.
Be as specific as possible.
-->
## What are the possible drawbacks or issues with the requested changes?
<!--
Altering the system behaviour is not always a free action, and it can impact
user experience, performance, introduce bugs or complexity, etc..
If you think about anything we should keep in mind while
examining your request, please describe it in this section.
-->
## Context
I know it's really selfish, but maybe I'm not the only one so just in case I put it here. It's not urgent and I won't talk about it again as it's just a suggestion for something that bothers me :)
<!--
If relevant, share additional context here like:
- Links to existing implementations or examples of the requested feature
- Screenshots
-->https://dev.funkwhale.audio/funkwhale/funkwhale/-/issues/532Deduplication of uploaded files2022-10-14T21:43:52ZAgateDeduplication of uploaded filesAfter discussion and thinking, here are my findings about the subject:
- Deduplication would be good for storage
- Deduplication would require a model change : store the audio files themselves in a separate table, bind this one to a tra...After discussion and thinking, here are my findings about the subject:
- Deduplication would be good for storage
- Deduplication would require a model change : store the audio files themselves in a separate table, bind this one to a track, and store the user_library <-> audiofile relation
- Deduplication would impact quota management
- Proper deduplication cannot rely on file hash only, because any change in audio headers would result in a different hash, while the audio remains the same
- We can have a better hashing by stripping all metadata from file, and compute a hash on the remaining part. This way, if headers change but not the audio stream, the hash remain the same
- If we plan to have a client that can skip uploading for already available files, but still mark a track available in a user library, we have to be careful:
- Here again, we cannot rely on file hash only: someone could share publicly a list of popular hash, and a user could simulate an upload with those hash, without having access to the actual files
- Using our better hash only is not enough, for the same reason
- We need to think about this in term of challenge: we need a way to prove that the user has access to the file before uploading. I think this is doable by computing a challenge result on server side, for instance store a hash of (better_hash + instance_name), and ensure the client can submit the same thing.
## PoC using ffmpeg-python
There are non-technical considerations as well in next parts, don't hesitate to skip this.
```python
import ffmpeg
import subprocess
audio_file_path = 'test.mp3'
with open(audio_file_path, 'rb') as f:
original_file_content = f.read()
# Based on https://github.com/kkroening/ffmpeg-python/issues/49 to avoid loading file in memory
strip_tags_command = ffmpeg.input('pipe:').output('pipe:', **{'map_metadata': '-1', 'vcodec': 'copy', 'acodec': 'copy', 'format': 'mp3'})
strip_tag_process = subprocess.Popen(['ffmpeg'] + strip_tags_command.get_args(), stdin=subprocess.PIPE, stdout=subprocess.PIPE)
no_tag_file_content, err = strip_tag_process.communicate(input=original_file_content)
# tags were stripped, so the content do not match
assert no_tag_file_content != original_file_content
# now, open the original file in Musicbrainz Picard, add a new tag like "Hello world", then load its content again
with open(audio_file_path, 'rb') as f:
new_tag_file_content = f.read()
# new tag was added, so the new and original content do not match
assert original_file_content != new_tag_file_content
# we strip the headers on the new file content, and ensure the result match our previous attempt
strip_tag_process = subprocess.Popen(['ffmpeg'] + strip_tags_command.get_args(), stdin=subprocess.PIPE, stdout=subprocess.PIPE)
out, err = strip_tag_process.communicate(input=new_tag_file_content)
assert out == no_tag_file_content
```
Based on the previous snippet, we can see we can reliably strip headers from audio file using ffmpeg and use the result to store a more reliable hash of uploaded files. On the server side, we could store this hash, and also a challenge based on this hash:
```python
import hashlib
instance_name = 'demo.funkwhale.audio'
uploaded_file_hash = hashlib.sha1(no_tag_file_content).hexdigest()
challenge_body = no_tag_file_content + instance_name
challenge = hashlib.sha1(challenge_body).hexdigest()
```
This challenge is simple to compute for clients (because using the instance name, or other public, instance specific info), and do not require extra resources on the server (we compute it once and we're good). It makes it difficult to exploit a list of music files hash found in the wild, it would require having a hash for each instance of Funkwhale. However, it's not perfect nor future proof.
A more involved approach could use a dynamic part as the challenge body. Instead of sending a hash with the instance name appended to the file content as a challenge, the client would do something like that:
```python
import datetime
now = datetime.datetime.now().isoformat()
local_file_hash = hashlib.sha1(no_tag_file_content).hexdigest()
challenge_body = no_tag_file_content + now
challenge = hashlib.sha1(challenge_body).hexdigest()
requests.post('/upload/check/', {'challenge_input': now, 'challenge_result': challenge, 'file_hash': local_file_hash})
```
When receiving that, the server could do the same steps and match the result with the client specified value. However, this would require reding the file for each challenge, resulting in heavier resource usage.
## Quota and deduplication
One thing to consider when we deduplicate is how we handle user quota. At the moment, it's pretty simple: if user has a 1Gb quota and uploads 100Mb of file, their remaining quota is 900Mb. This makes less sense with deduplication, since the same file, uploaded in ten libraries, would result in only one file being actually stored on the server. How do we manage this case when we compute users quota? I can see at least three possibilities:
1. We don't do anything: if 10 users upload the same 10Mb track, each one will have 10Mb used in their quota. Pros: simple to manage. Cons: less storage available to users, a bit akward since it's not matching our underlying storage model
2. The first uploader "wins": if 10 users upload the same 10Mb track, only the first one will see its quota occupied. Pros: more storage available for the other users. Cons: not really nice for the first user, what happens if the first user delete the track in their library?
3. Divide usage: if 10 users upload the same 10Mb track, each one will have 1Mb used in their quota. Pros: more storage available for everyone. Cons: more intensive on database size, less visibility for user on their own quota, since it can go up and down based on other users activity
A like 3. the most, even if it's more intensive, because it's fair for everyone. New users can upload existing files and only use a fraction of the required quota, and users uploading popular tracks will get their quota back, allowing them to upload more stuff. It's more collaborative and less individualistic than other options.
## Conclusion
Let me know what you think about those ideas, and feel free to suggest new one and pinpoint errors :)
Regardless of all of that, I think deduplication should wait a bit before being implemented, as we'll probably a lot to work on once we get feedback about the 0.17 release, so we can take the time to figure this out properly.