AI Safety Tools Fall Short for African Languages
Why this is here: Researchers gathered input from 155 domain experts to create culturally relevant safety policies for African languages, addressing gaps in existing AI safety benchmarks.
Researchers at an unspecified institution introduced UbuntuGuard, a new benchmark for testing AI safety in African languages. Current AI safety checks largely focus on Western cultures and major languages. This leaves AI systems vulnerable when used with African languages and local cultural contexts.
The team created UbuntuGuard using input from 155 experts. They developed safety policies and example responses tailored to specific risks and cultural norms. They then tested 15 AI models, including general-purpose and safety-focused systems, using this benchmark.
The tests showed existing English-based safety checks overestimate how well AI performs across languages. While some knowledge transfers between languages, it isn’t enough.
Dynamic AI models—those that can adjust based on new information—performed better but still struggled with local cultural nuances. This was a single study and doesn’t cover all African languages or cultural groups. Further research could improve AI safety and fairness for a wider range of users.
Surfaced by the Discovery lens — one of the vital signs ovr.news reads.
How we evaluated this
AI summary
read the original for the full story — Read on arxiv.org . How we work →