Companion-chatbot safety and crisis-referral protocol
ai-companion-safety-protocolDomain: social-safetyType: processDescription
A published protocol and the implementing product behavior for an AI companion chatbot — the relationship-sustaining, socially-engaging class of assistant — covering self-harm and crisis handling, AI-identity disclosure, and minor-specific safeguards. California SB 243 is the first statute to put concrete operational requirements on this product category: the operator must maintain and publish a protocol that prevents the chatbot from producing suicidal-ideation, suicide, or self-harm content and that refers a user expressing suicidal ideation to crisis service providers; must disclose that the user is interacting with AI when a reasonable person would otherwise be misled; for users known to be minors must surface a break-reminder and AI-identity notice at least every three hours and prevent sexually explicit content; and must report referral counts and protocols to the California Office of Suicide Prevention annually beginning July 2027. The control is therefore part policy (the published protocol), part product (the crisis-detection-and-referral flow, the recurring minor reminders, the content guardrails), and part recordkeeping (the referral-notification counts feeding the annual report). The recurring mistake is treating a generic 'this is an AI' label as sufficient; the statute's minor-reminder and crisis-referral duties are affirmative product behaviors, not one-time disclosures.
Applicability
Applies when: features include ai-chat.
Required by (1 regulation)
- California SB 243
Operators of companion chatbots must publish a protocol to prevent self-harm content and refer users expressing suicidal ideation to crisis services, disclose AI identity when a reasonable person would be misled, provide minors a break-and-AI-identity reminder at least every three hours, prevent sexually explicit content to known minors, and report referral counts to the Office of Suicide Prevention annually from 2027-07-01.
California SB 243 (2025), Cal. Bus. & Prof. Code Division 8, Chapter 22.6, §§ 22601-22602; operative 2026-01-01
Evidence formats
- published companion-chatbot safety protocol covering self-harm content prevention and crisis-service referral
- AI-identity disclosure copy and the logic that triggers it
- minor-mode configuration showing the at-least-every-three-hours break reminder
- crisis-referral notification log feeding the annual Office of Suicide Prevention report