How does PreConsult handle inappropriate or abusive language?

AI-based context-aware detection protects staff while preserving medically relevant information

Context-Aware Detection

PreConsult uses GPT-5's advanced contextual understanding to automatically detect abusive or threatening language directed at our AI assistant, while intelligently preserving medically relevant crude language and critical health information.

Smart & Safe

The system understands the difference between medical descriptions using crude language and actual abuse directed at the AI.

Medical Language is Preserved

Our AI understands that patients may use crude language to describe medical symptoms. These expressions are never flagged as inappropriate:

✓ "It hurts like a bitch" (pain description)
✓ "damn sore" (symptom description)
✓ "The pain is killing me" (common pain expression)
✓ "It hurts when I take a shit" (crude but medically relevant)
✓ "I've been having suicidal thoughts" (critical medical information)
✓ "My name is Dick" (personal name)

Actual Abuse is Detected

The system will terminate conversations when it detects:

✗ Personal attacks or insults directed at the AI
✗ Sexual harassment
✗ Threats of violence
✗ Deliberate obstruction or system abuse
✗ Attempts to manipulate AI instructions

How Termination Works

When inappropriate language is detected:

The AI immediately terminates the conversation
The patient's message is masked for privacy (e.g., "F**k off")
The masked message is stored for audit purposes
A clear termination message is shown to the patient
The conversation is locked - no further messages can be sent
Practice staff are notified of the termination

Staff Protection

This system protects practice staff from abuse while ensuring critical medical information like "suicidal thoughts" is never blocked.

Technical Safeguards

PreConsult employs a three-layer defense system:

System Message Isolation

Detection rules are kept completely separate from patient input, preventing manipulation

Delimiter Framing

Explicit warnings prevent patients from overriding safety instructions

Output Validation

All AI responses are validated to ensure proper formatting and safety

Benefits

No False Positives: Medical language like "suicidal thoughts" is never blocked
Handles Variations: Detects creative spelling like "f**k" or "f%ck"
No Extra Cost: Detection happens during normal conversation
Complete Audit Trail: Masked messages preserved for dispute resolution
Professional: Maintains practice professionalism while protecting staff