Koo’s AI-powered content moderation tackles nudity, pornography, and fake news

The social network’s platform seems to be identical to that of Twitter but the company claims that they stand out owing to their commitment to making Koo a safe and equitable space.

Indianexpress.com joined the team at Koo for an exclusive demonstration of the company’s latest update to its content moderation measures.

On nudity and pornography: If a user posts a nude picture on their Koo account, they will immediately get a notification that says- “This Koo has been deleted due to GRAPHIC, OBSCENE OR SEXUAL CONTENT.”

The process, the company says, is entirely automated and triggers within seconds after posting the picture. After the image is deleted, the user will get another notification stating why it was removed, prompting them to raise an appeal using the redressal form if they believe that it was an error. These notifications will appear in the preferred language set by the user.

“In countries like India and many others around the world, pornography is illegal. Someone posting nude pictures or pornography from an Indian IP is an illegal act, and platforms should take these off. But, with platforms from global companies, these things happen and we notice such content existing for a long time. With Koo, there is an intent behind introducing these features. We are a thoughts and opinions platform and we want people to come and engage with each other in a healthy way,” says Rajneesh Jaswal, Head of Legal & Policy at Koo.

Similarly, if a user posts a video containing nudity or pornography, it will be pulled in around five seconds, depending on the length of the video and the time it takes to process it. After deleting the video, Koo will send the user a notification. In case a user posts a nude photo as their display picture, Koo will delete it using a similar mechanism.

Koo says its nudity algorithms identify actual nude images leading to pornography, however, they exclude works of art.

Rahul Satyakam, Senior Manager, Operations, juxtaposed Koo with Twitter during the demonstration. Satyakam posted similar content on the Elon Musk-owned platform showing how it did not take any action against such posts. Similarly, Satyakam also showed an obscene post that he shared a few days ago on Twitter and how it continues to be visible to all.

On posts containing violence: Whenever a user shares an image that contains gore or graphic violence, Koo will allow the post but offer an additional layer of caution. A blurred image will appear, with the message, “This content may not be suitable for all users. A caution message has been placed in their interest.” Users will have the liberty to view the image, like, or comment.

Since some of these images may be related to a news development, Koo has adopted a nuanced approach and weeded out the blanket delete mechanism seen earlier in terms of obscene content.

Impersonation: The platform uses machine learning to detect instances of impersonation and the company has claimed that it is getting better at it. Impersonation detection is backed by AI, although, the actions taken are largely manual by human moderators. As part of the demo, Satyakam created an account with Shah Rukh Khan’s name and image.

The impersonation dashboard on the platform, strictly meant for company staff, shares crucial information about the user and the VIP who is being impersonated. One of its features — Soft Delete — removes all details that amount to impersonation such as name and display picture. “Even if a person is not on our platform and someone tries to impersonate them, we ensure that necessary action is taken,” says Jaswal.

After the removal of the impersonating content, the platform issues a notification, “Your profile details are removed – Your profile details are removed due to repeat violations of the Koo community guidelines or legal requirements.”

Fake news: The platform claims it runs a detection cycle every half an hour and can instantly take down fake news. When a user shares fake news, the dashboard detects it and offers information tracing the roots of the news, giving enough information to the moderator to take immediate action. Users get a notification layered on top of the fake news stating that it was an “Unverified or False Information: Reviews by a Fact Checker.” Users can also appeal for a review if they believe that their content is not fake.

Toxic comments and spam: As part of the demo, the user posted an abusive comment on a post. Koo identifies such posts and hides them effectively. Such posts will only be visible once users click on the Hidden Comments button. This feature works almost instantly. While banning comments would amount to restricting freedom of expression, according to the company, this is Koo’s way of allowing people to express their views.

Along with placing safety measures on content, Koo has also integrated ChatGPT for its select Yellow Tick users. With the AI chatbot, users can compose posts on anything with prompts.