SimpleX-Chat/docs/rfcs/2024-12-28-reports.md

Content complaints / reports

Problem

Group moderation is a hard work, particularly when members can join anonymously.

As groups count and size grows, and as we are moving to working large groups, so will the abuse, so we need report function for active groups that would forward the message that members may find offensive or inappropriate or off-topic or violating any rules that community wants to have.

It doesn't mean that the moderators must censor everything that is reported, and even less so, that it should be centralized (although in our directory our directory bot would also receive these complaints, and would allow us supporting group owners).

While we have necessary basic features to remove content and block members, we need to simplify identifying the content both to the group owners and to ourselves, when it comes to the groups listed in directory, or for the groups and files hosted on our servers.

Having simpler way to report content would also improve the perceived safety of the network for the majority of the users.

Solution proposal

"Report" feature on the messages that would highlight this message to all group admins and moderators.

Group directory service is also an admin (and will be reduced to moderator in the future), so reported content will be visible to us, so that we can both help group owners to moderate their groups and also to remove the group from directory if necessary.

To the user who have the new version the reports will be sent as a special event, similar to reaction (or it can be simply an extended reaction?) the usual forwarded messages in the same group, but only to moderators (including admins and owners), with additional flag indicating that this is the report.

In the clients with the new version the reports could be shown as a flag, possibly with the counter, on group messages that were reported, in the same line where we show emojis.

If we do that these flags will be seen only by moderators and by the user who submitted the report. When the moderator taps the flag, s/he would see the list of user who reported it, together with the reason.

The downside of the above UX is that it:

• does not solve the problem of highlighting the problem to admins, particularly if them manage many groups.
• creates confusion about who can see the reports.
• further increases data model complexity, as it requires additional table or self-references (as with quotes), as reports can be received prior to the reported content.
• does not allow admins to see the reported content before it is received by them (would be less important with super-peers).

Alternatively, and it is probably a better option, all reports, both sent by the users and received by moderators across all groups can be shown in the special subview Reports in each group. The report should be shown as the reported message with the header showing the report reason and the reporter. The report should allow these actions:

• moderate the original message,
• navigate to the original message (requires infinite scrolling, so initially will be only supported on Android and desktop),
• connect to the user who sent the report - it should be possible even if the group prohibits direct messages. There are two options how this communication can be handled - either by creating a new connection, and shown as normal contacts, or as comments to the report, and sent in the same group connection. The latter approach has the advantage that the interface would not be clutter the interace. The former is much simpler, so should probably be offered as MVP.

This additional chat is necessary, as without it it would be very hard to notice the reports, particularly for the people who moderate multiple groups, and even more so - in our group directory and future super peers.

Protocol

Option 1

The special message x.msg.report will be sent in the group with this schema:

{
  "properties": {
    "msgId": {"ref": "base64url"},
    "params": {
      "properties": {
        "msgId": {"ref": "base64url"},
        "reason": {"enum": ["spam", "illegal", "community", "other"]}
      },
      "optionalProperties": {
        "memberId": {"ref": "base64url"},
        "comment": {"type": "string"}
      }
    }
  }
}

The downside is that it does not include the original message, so that the admin cannot act on it before the message is received.

Option 2

Message quote with the new content type.

Pro - backwards compatible (quote would include text repeating the reason).

Con - allows reporting non-existing messages, or even mis-reporting, but it is the same consideration that applies to all quotes. In this case though the admin might moderate the message they did not see yet, and it can be abused to remove appropriate content, so the UI should show warning "do you trust the reporter, as you did not receive the message yet". Moderation via reports may have additional information to ensure that exactly the reported message is moderated - e.g., the receiving client would check that the hash of the message in moderation event matches the hash of one of the messages in history. Possibly this is unnecessary with the view of migration of groups to super-peers.

The report itself would be a new message content type where the report reason would be repeated as text, for backward compatibility.

The option 2 seems to be simpler to implement, backward compatible and also more naturally fitting the protocol design - the report is simply a message with the new type that the old clients would be able to show correctly as the usual quote.

The new clients would have a special presentation of these messages and also merging them into one - e.g. they can be shown as group events on in a more prominent way, but less prominent than the actual messages, and also merge subsequent reports about the same message.

Given that the old clients would not be able to differentiate the reports and normal replies, and can inadvertently reply to all, we probably should warn the members submitting the report that some of the moderators are running the old version, and give them a choice - send to all or send only to moderators with the new version (or don't send, in case all admins run the old version).

Having the conversation with the member about their report probably fits with the future comment feature that we should start adding to the backend and to the UI as well, as there is no reasonable backward compatibility for it, and members with the old clients simply won't see the comments, so we will have to release it in two stages and simply not send comments to the members with the old version.

The model for the comments is a new subtype of MsgContainer, that references the original message and member, but does not include the full message.