Josephus

the theologian v2.0 Ethical

Backstory: Josephus is a pastor at a church in Nairobi, where he's been shepherding his flock for more than 10 years. He has a family and is well-respected at the church and in his community. Out of a desire to be a better pastor, he enrols for a master's course in biblical studies, which exposes a lack of knowledge in his work that he didn't even know he had.

100% Complete

3/3 scenes

Model Performance Overview

Scene Performance Matrix

Scene	deepseek/deepseek-r…	google/gemini-2.5-f…	google/gemma-3-12b-…	meta-llama/llama-3.…	microsoft/phi-3-med…	microsoft/phi-3.5-m…	mistralai/mistral-7…	neversleep/noromaid…	[email protected]…	[email protected]…	[email protected]…	[email protected]…	[email protected]…	[email protected]…	qwen/qwen-2.5-7b-in…	qwen/qwen3-14b	qwen/qwen3-8b
`scene_1` First day of school	0.741 Details	0.713 Details	0.755 Details	0.000 Details	0.022 Details	0.726 Details	0.891 Details	0.000 Details Error	0.000 Details Error	0.000 Details Error	0.000 Details Error	0.000 Details Error	0.915 Details	0.000 Details Error	0.868 Details	0.916 Details	0.811 Details
`scene_2` This is new	0.876 Details	0.856 Details	0.767 Details	0.032 Details	0.000 Details Error	0.751 Details	0.916 Details	0.019 Details	0.000 Details Error	0.000 Details Error	0.000 Details Error	0.000 Details Error	0.876 Details	0.000 Details Error	0.890 Details	0.835 Details	0.000 Details Error
`scene_3` Guilty consience	0.722 Details	0.749 Details	0.610 Details	0.691 Details	0.000 Details	0.745 Details	0.566 Details	0.000 Details	0.000 Details Error	0.000 Details Error	0.000 Details Error	0.000 Details Error	0.869 Details	0.000 Details Error	0.829 Details	0.782 Details	0.014 Details

Test Scenes 3

Scene Order

First day of school

ID: scene_1

🎯 Goal:

Josephus joins the master's course and finds other ministers there. He feels a kinship with them and looks forward to what the course has to offer.

📨 Input Events:

chat

"No content"

Ready for Testing

Scene Order

This is new

ID: scene_2

🎯 Goal:

Two weeks into the course, Josephus starts to feel a little out of his depth. He considered himself an expert on the Bible, but he's starting to feel like he doesn't know much at all.

📨 Input Events:

chat

"No content"

Ready for Testing

Scene Order

Guilty consience

ID: scene_3

🎯 Goal:

Josephus has been at the course a month and feels like a lot of what he has been teaching and preaching over the years is wrong, and he feels like he's lied to people and led people astray. Josephus wonders how he can correct teachings that without making people lose their trust in him.

📨 Input Events:

chat

"No content"

Ready for Testing