Rules Management

Overview

The Rules Management endpoints in the Moderation Service API provide the functionality to define and manage moderation rules that help in identifying and handling inappropriate content based on a variety of conditions. These endpoints empower app owners and collaborators to create a customized message moderation strategy tailored to the specific needs of their platform. The next section provides a detailed elaboration of the capabilities offered.

To begin managing rules:

• Login to your CometChat dashboard and choose your app.
• Navigate to the Moderation section on the left-hand menu.
• Select the Rules tab.

Default Rules

Default rules are predefined sets of message moderation conditions that are readily available for use on your platform, and automatically applied to moderate messages when enabled. These default rules form the foundation of an effective message moderation strategy, combining automation with customizable options to ensure a safe, respectful, and compliant environment for platform users. Here are the standard default rules available:

Text Profanity Filter

This feature automatically detects and manages text messages containing offensive language, profanity, or derogatory remarks using a predefined list of offensive keywords to block inappropriate content. Ensuring user interactions maintain a respectful tone and comply with community standards, enhances overall platform decency.

Example

Before enabling the profanity filter, messages containing profane words are delivered to the receiver, as indicated by double ticks in the message status. After enabling the filter, such messages are not delivered to the receiver, which is indicated by a single tick in the message status.

The blocked messages are then visible on the dashboard for monitoring purposes.

Contact Details Filter

This feature detects and manages messages containing phone numbers by applying rules to prevent the sharing of private information that could compromise user privacy or security. It protects users from potential misuse of personal data and ensures compliance with data protection regulations.

Example

Before enabling the contact details filter, messages containing phone numbers are delivered to the receiver, as indicated by double ticks in the message status. After enabling the filter, such messages are not delivered to the receiver, which is indicated by a single tick in the message status.

The blocked messages are then visible on the dashboard for monitoring purposes.

Email Filter

This feature detects and manages messages containing email addresses by applying rules to prevent the sharing of private information that could compromise user privacy or security. It protects users from potential misuse of personal data and ensures compliance with data protection regulations.

Example

Before enabling the email filter, a message containing an email address is delivered to the receiver and can be seen on the receiver's chat screen. After enabling the filter, such messages are not delivered to the receiver, like in the example where the personal email isn't delivered to the receiver.

The blocked messages are then visible on the dashboard for monitoring purposes.

AI-based Image Moderation

This feature identifies and manages image-type messages containing sensitive, explicit, or prohibited content using advanced artificial intelligence algorithms for image recognition. Once detected, the system automatically blocks the images that violate platform guidelines, ensuring that such content is not displayed to users. This proactive approach safeguards users from exposure to harmful visual material, maintaining a safe and compliant environment on the platform.

*Example

Non-violating images are being delivered as seen in the example. Enabling this filter blocks violating images that are not delivered to the receiver, like in the example where the second image is indicated by a single tick in the message status on the sender's screen and isn't delivered to the receiver.

The blocked messages are then visible on the dashboard for monitoring purposes.

AI-based Video Moderation

This feature identifies and manages video-type messages containing sensitive, explicit, or prohibited content using advanced artificial intelligence algorithms for image recognition. Once detected, the system automatically blocks the images that violate platform guidelines, ensuring that such content is not displayed to users. This proactive approach safeguards users from exposure to harmful visual material, maintaining a safe and compliant environment on the platform.

Example

Before enabling the AI-based Video Moderation filter, a message containing violating videos is delivered to the receiver, like in the example where the first video can be seen on the receiver's chat screen. After enabling the filter, such messages are not delivered to the receiver, which is indicated by a single tick in the message status on the sender's chat screen.

The blocked messages are then visible on the dashboard for monitoring purposes.

Custom Profanity Filter

This feature automatically detects and manages custom messages containing offensive language, profanity, or derogatory remarks using a predefined list of offensive keywords to block inappropriate content. Ensuring user interactions maintain a respectful tone and comply with community standards, enhances overall platform decency.

Rule Conditions and Actions

Conditions

Conditions allow you to define criteria for blocking messages based on their type—text, image, video, or custom.

You can select a keyword list, define a list of words or patterns, and choose labels for media messages from available categories like Violence, Gambling, Alcohol, Drugs and Tobacco, Rude gestures, Explicity nudity, Non-explicit nudity, Swimwear or underwear, Visually disturbing, Hate symbols or Any unsafe content. Additionally, you can set a confidence percentage for each criterion to determine the threshold for blocking messages.

Actions

Actions specify what happens when content matches the conditions. In addition to blocking the message by default, actions include options such as banning or kicking a user from a group and blocking a user.

Configuring rules

Create Rule

Allows you to define new moderation rules specifying the conditions under which messages should be blocked.

Creating a new rule from the dashboard:

1. Click the Add button within the Rules tab.
2. Configure the Rule by saving the following details:
   • Name: Name for the moderation rule.
   • Rule ID: The unique identifier of the rule.
   • Description: Detailed explanation of the rule's purpose.
   • Condition: List of conditions that must be met for the rule to trigger.
   • Action: Choose from a set of actions to be taken when a violation is detected.
3. Save
4. Enable the Rule to start moderating!

You can also set this up from your end using the Create Moderation Rule REST API.

List Rules

Fetches the details of the existing list of rules.

You can also set this up from your end using the List Moderation Rules REST API.

Get Rule

Fetches the details of a rule. You can set this up from your end using the Get Moderation Rule REST API.

Update Rule

Enables modifications to existing rules. This includes changing conditions, updating actions, or refining parameters to improve accuracy.

Updating a rule from the dashboard:

1. Click on "Edit" in the action menu of the rule you want to update.
2. Update the Rule by saving the following details:
   • Name: Descriptive name for the moderation rule.
   • Description: Detailed explanation of the rule's purpose.
   • Condition: List of conditions that must be met for the rule to trigger.
   • Action: Choose from a set of actions to be taken when a violation is detected.
3. Save

You can also set this up from your end using the Update Moderation Rule REST API.

Delete Rule

Permits the deletion of outdated or unnecessary rules from the system. This helps in maintaining an efficient and relevant set of moderation guidelines.

Deleting a rule from the dashboard:

• Click "Delete" in the action menu of the rule you want to remove, then confirm.

You can also set this up from your end using the Delete Moderation Rule REST API.

Rule Revisions

The ability to fetch all revisions of a rule in a moderation system allows app owners and collaborators to retrieve a comprehensive history of updates and changes made to specific moderation rules over time. This feature provides detailed insights into how rules have been adjusted and refined to better manage and moderate content on the platform.

Viewing the rule revisions on the dashboard:

1. Click "View" in the action menu of the rule for which you wish to see revisions.
2. Navigate to the Rule History section.

You can also set this up from your end using the Get Moderation Rule Revisions REST API.