Combating Visual Harassment and Bullying

The Growing Problem of Visual Harassment

Visual harassment has become one of the most pervasive threats to online safety. Unlike text-based harassment that's relatively easy to detect with keyword filters, visual harassment takes countless forms: threatening memes, manipulated photos intended to humiliate, screenshots containing personal information, images with harassing text overlays, and much more.

The impact is severe. Users who experience visual harassment often disengage from platforms entirely, taking their contributions and spending with them. For platforms, the cost extends beyond lost users - harassment incidents can trigger regulatory scrutiny, media coverage, and lasting reputation damage. In extreme cases, visual harassment has contributed to real-world harm and even loss of life.

Traditional moderation approaches fall short. Text filters miss image-based harassment entirely. Manual review cannot scale to the volume of modern platforms. And basic image detection focuses on explicit content while ignoring the nuanced forms harassment can take.

Multi-Modal Harassment Detection

Our approach to harassment detection combines multiple AI capabilities to catch the full spectrum of visual harassment content.

How It Works

Our API analyzes images across multiple dimensions simultaneously, returning detailed results about any harassment signals detected.

# Python - Detect harassment content in images
import requests

def check_for_harassment(image_url, api_key):
    response = requests.post(
        "https://api.imagemoderationapi.com/v1/moderate",
        headers={"Authorization": f"Bearer {api_key}"},
        json={
            "image_url": image_url,
            "models": ["hate", "violence", "ocr", "pii"]
        }
    )
    result = response.json()

    # Check for harassment indicators
    if result["moderation_classes"]["hate"]["harassment"] > 0.8:
        return {"action": "block", "reason": "harassment_detected"}

    # Check for doxxing content
    if result["pii_detected"]:
        return {"action": "review", "reason": "possible_doxxing"}

    # Check text in image for slurs/threats
    text_analysis = result.get("ocr_analysis", {})
    if text_analysis.get("contains_slurs") or text_analysis.get("contains_threats"):
        return {"action": "block", "reason": "text_harassment"}

    return {"action": "allow"}

Types of Visual Harassment We Detect

• Threatening Imagery: Weapons pointed at camera, violent gestures, imagery suggesting harm
• Harassing Memes: Images with slurs, targeted insults, or degrading text overlays
• Doxxing Content: Screenshots or images containing personal information like addresses, phone numbers, workplace info
• Hate Symbols: Extremist iconography, hate group logos, discriminatory symbols
• Manipulated Images: Doctored photos, face swaps, and composites intended to humiliate
• Pile-On Content: Screenshots of harassment used to invite further targeting
• Non-Consensual Imagery: Private photos shared without permission to embarrass or control

Protecting Vulnerable Users

Certain users face disproportionate harassment online. Our detection models are specifically trained to identify harassment targeting:

• Women and girls facing gender-based harassment
• LGBTQ+ individuals targeted with identity-based abuse
• Racial and ethnic minorities facing racist imagery
• Religious groups targeted with discriminatory content
• People with disabilities subjected to ableist harassment
• Public figures facing coordinated harassment campaigns