Play testing journaling games with AI: Last Tea Shop
A downloadable tool
I tried to play test a journaling game with AI, because play testing is difficult and time consuming.
Since journaling games are, in the end, writing exercises, this is something that AI could automate. Also, since AI can act as a critic (the technical term is eval or LLM-as-judge), it can also give feedback on the game, given a playthrough and the rulebook.
This has been tested on The Last Tea Shop. The results are noisy (meaning that AI will make lots of comments, sometimes irrelevant), however, there are recurring comments on the two following points:
- affinities are under exploited, if at all
- the Veiled One appearing on the 24th day is a bit sudden, a little bit of buildup would have been better
In conclusion, AI could provide cheap play testing of journaling games. Interpretation of the various improvements suggested still probably require a very experienced game maker (or you could just count the number of times a game mechanic is singled out as requiring improvement).
Code attached.
Download
Install instructions
Code is attached, if you want to try to run it, you'll need:
- a .env file, with an LLM name and a key (I used gemini-2.5-flash-lite and a free key)
- a virtual environment that has the dependencies installed (dotenv, langchain, langchain_google_genai), ask an AI assistant if you have any difficulties
Structure of the code:
- imports
- instantiate AI model
- game tables (weather, visitors, supplies, recipes, affinities, questions)
- __init__ various temporary variables, message_history will contain the play through log
- setup_shop function (initial roll/choice of location, description, affinities, supplies)
- serve_visitor function for one visit (roll weather, write arrival, find available recipes and choose one, ask questions, link to previous visitor, write farewell, apply visitor bonuses)
- play_game function (repeatedly call serve_visitor function and increment days until day 24)
- suggest_rule_improvements function (take the game rules in markdown format, the play trace kept in memory, and ask AI to make structured comments on the rules)
- choose function (ask AI to make a choice among several options, could be replaced by human input)
- write function (ask AI to write a prompted paragraph, could be replaced by human input)
- rules in markdown format
- __main__ function, call play_game then suggest_rule_improvements
Leave a comment
Log in with itch.io to leave a comment.