AI Evaluation and Safety Engineer
Build the test sets, graders and red-team checks that show our AI is safe and accurate, in Ghanaian English and Pidgin.
- Team
- Engineering
- Place
- Accra (Adentan)
- Work
- Hybrid
- Type
- Full time
How you'll work. Mostly one day a week from home, agreed for each role.
- You'll be employed by Nkonso Technologies Ltd.
About the role
Before AI helps keep Amee safe, we need to show that it works for the way people in Ghana actually write, in Ghanaian English and Pidgin, one category of harm at a time. Agoo Bot must be shown never to act beyond a person's permissions. You'll build the evaluations that prove it, and the gates that keep every model and prompt change honest.
What you'll do
- Build and maintain evaluation sets with our Trust and safety team, including Ghanaian English and Pidgin
- Write automated graders, and measure them against human judgement
- Report results on held-out cases for each category of harm, clearly and honestly
- Red-team our AI features, including attempts to make an assistant act beyond someone's permissions
- Build regression gates that run before every model or prompt change
- Turn moderators' and appeals decisions into well-labelled cases
- Write labelling guidelines, and guide the National Service assistants who help label
- Handle test data under our privacy rules
What you'll bring
- At least three years in machine learning, data science or software engineering, with hands-on evaluation of language models or classifiers
- Python and practical statistics: precision, recall, sampling and confidence
- Experience writing labelling guidelines and measuring agreement between labellers
- Experience with adversarial testing or red-teaming
- A good ear for Ghanaian English and Pidgin, or a habit of working closely with people who have one
- Clear writing for engineers, safety specialists and the founder alike
- If you're a Ghanaian graduate: National Service completed, or exempt
Nice to have
- Trust and safety or content moderation experience
- Using models as graders, and checking them
- Published or shared evaluation work
How you'll work
- Hybrid. You'll be based at our office at 10A Mega Street, Adentan, Accra, mostly with one day a week from home, agreed for the role.
- Hours. Standard office hours.
- This is a permanent, full-time role.
- Reporting. You'll report to the Head of Engineering, and work closely with the Head of Trust and Safety.
Looking after you. Some evaluation work means reading harmful text. We commit to limits on exposure, rotation between harder and lighter work, and access to confidential professional support. Child-safety material is never part of this role.
How to apply
Apply on this page with your details and a link to your CV. A short note on why the role interests you is welcome.
What happens next: we read every application. If there's a fit, we'll invite you to talk with us, then ask you to do a short task based on the work, and we'll give you an answer either way.
We may use AI to summarise applications and check for missing details. A person reads and decides every application.
We welcome applications from everyone. In line with the Labour Act, 2003 (Act 651), we don't discriminate on grounds of gender, race, colour, ethnic origin, religion, creed, social or economic status, disability or politics. Applicants with disabilities are warmly encouraged to apply: if you need an adjustment to apply or to take part in any stage, tell us what you need and we'll arrange it.
How we handle your details is set out in our applicant privacy notice.
Apply
Takes about five minutes. Every answer is needed unless it says optional.