[Meta] Designing the moderation andanti-harassment tools

I feel like we need input and feedback before working on that. Even if we already have #320 (closed) opened, I suggest we use this issue as a discussion starter around Funkwhale moderation and anti-harassment (AH) tools.

This issue is a draft of proposed tools / workflow, based on my reflexions and the subjects, and what is currently present and/or missing (at least IMHO) in the ecosystem. As such, it is probably biaised, incomplete, or worst and your input would be incredibly valuable to help with that.

Anything in this document is opened to discussion will be updated to reflect received feedback. What is not open to discussion, though, is that fact that we need moderation tools. If you don't need moderation tools on a personnal level, please don't make uncomfortable people who actually need them.

Why do we need moderation/AH tools?

Moderation may be helpful for:

Instance owners:

a) to ensure they are not legally liable (e.g. if they end up broadcasting copyrighted or illegal content), this is linked to #308 (closed), since storing the licences would makes things easier on this side
b) to avoid federating with bad or malicious actors (who broadcast low quality, harmful content, or who impersonate creators, etc.)
c) to fight against spam or unsollicited activity or content

End users:

a) to control who can access their content and/or activity (listening history, comments, etc.)
b) to avoid being exposed to harassment, unsollicited or unwanted content

Both personas may have similar and different concerns, and will sometimes operate at different levels, and the tools we'll design must take that in consideration. A concrete exemple: instance owners may want to moderate based on low-level information (such as IP address), while this usually makes less sense for end users.

A quick note about moderation/AH tools

Note: all of this is based on my personal analysis. If you have documentation and references about this, drop them in the comments!

Based on what exists in various networks and projects, we can see two kinds of tools:

Proactive tools: those tools help you prevent problems before they become problems. Typical examples include: locking your account on Mastodon so only your followers can access your content, restricting the audience of one of your toot on a per-post basis, blocking a list of actors or IPs known to be malicious, etc.
Reactive tools: those tools help you deal with problems when they occur. Typical examples include: reporting problematic content or actors, blocking or muting problematic actors and/or content

Ideally, we should have both kind implemented. Proactive tools are a first protection wall, and will reduce the scale of problems when they occur, but reactive tools will always be needed to deal with problems that are not stopped by our proactive tools.

Apart from that, I'd also like our moderation/AH tools to match those requirements:

Granularity: affect only the problematic actor/and or content. We do not want to block a 1000 users node if only one user is harassing someone (unless, of course, the node moderation team is unresponsive), or block a 10000 tracks library if only one track is problematic
Temporality: in the same vein, having a way to apply the moderation for a limited amount of time is also an interesting behaviour: if one node is subject to a wave of spam and the moderation team is overwhelmed, it makes sense to mute it for maybe a week and have the mute being unapplied automatically.
Security: this may be obvious, but those tools should improve the life of people using them, not make it worse. Typically, we should ensure that the act of moderation (such as a block) is not directly visible by the actor being blocked.
Reusability: as moderation require vast amounts of effort, I'd like to offer way to share this load under different actors / nodes. A common way to do that is to use block-lists, which could be federated, and a user could share their blocklist with trusted ones, or reuse external blocklists

Proposed tools

In the table below, you'll find the list of threats described earlier, and, for each one, the various tools that I suggest we implement to deal with it.

For instance owners

Threat	Proactive tools	Reactive tools
Being legally liable because of hosting / broadcasting content	Do not serve library publicly unless they are marked as trusted	DMCA / Takedown / Report form
	Shared blocklists of actors / nodes known to share illegal or copyrighted content	Blocking actors, nodes or specific content (done via !521 (merged))
	Content fingerprinting to detect copyrighted content upfront and switch containing library to private
Spam / Unsollicited content	Closed / invitation only instance (Done)	Report form to flag content, actors or nodes
	Shared blocklists of IPs / Email domains known for bad behaviour	Blocking / muting of nodes with spam activity (temporary or permanent)
	Email verification required before posting content	Mass deletion tools to purge spamming actors

For end users

Threat	Proactive tools	Reactive tools
Unauthorized access to content	Configurable visibility for content and activity occuring on the platform (private, followers, unlisted, etc.)	Content deletion
	Locked accounts with manual approve of followers to limit how activity is broadcasted	Manually revoking follows from specific actors or nodes
	Shared blocklists of follow bots or nodes known for their bad behaviour	Blocking of actors or nodes
Harassment, abuse, aggression	Shared blocklists of actors or nodes known for bad behaviour	Report form to flag content, actors or nodes
	Non-indexed content to limit the dicovery of targets by attackers using search	Blocking / muting of nodes with bad behaviour (temporary or permanent)
Unsolicited/unwanted content	Shared mute / filters lists of actors or nodes	Muting of nodes / actors (temporary or permanent)
		Content filters to hide content matching given criterias (partially done via !618 (merged))

As you can see, there is plenty of work to do, to figure out if those problems are real, well defined, if the proposed solutions are actually viable, and, finally, to actually do the implementation work!

Because of the insane amount of work this represent, we'll likely have to focus on some areas and release new features incrementally. As explained earlier, proactive tools are not enough by themselves, so maybe focusing on reactive tools first would be better.

In my opinion having block lists and a report/flagging system would be a good start, but I don't want to commit on anything before I hear what you have to say :)

GitLab