I built a customised AI chatbot using OpenAI’s API, which could run on GPT3.5 or GPT4 and used it to facilitate socratic questioning for students to arrive at the correct answer to a commonly misunderstood Physics concept. The process was not too complicated. One would need a OpenAI developer account to obtain the API key, and a Streamlit account to deploy the app. To input the API key, update the settings of the Streamlit app with this line: OPENAI_API_KEY=”{key}”. The codes can be forked from here.
This video shows a simulation of the interaction that I had with it.
I also stored the interactions that my students had with the bot in csv format, saved in an AWS S3 bucket.
Here are some samples, in which you can see the system prompt given: Speak like a teacher who assesses the response of the student based on clarity, precision, accuracy, logic, relevance and significance. Help the user get to the answer by asking guiding questions to scaffold the learning. The question is: Two balls are placed at the back of a truck that is moving at constant velocity. The blue ball is twice the mass of the red ball. The floor of the truck is perfectly smooth. Compare the movement of the two balls when the truck comes to an abrupt stop. The success criteria for the user is to be able to explain that both balls will move at the same speed once the truck comes to an abrupt stop, according to Newton’s first law, since there will be no net force acting on them since the floor of the truck is smooth and there is no friction.
It was rather entertaining for the students as some of them tried to trick the bot to giving them the answers. I did a pre-test for this question before the test and only 3-5% of the students in both classes got the answer correct but almost all of them (ascertained by an informal poll) knew the correct answer after going through the discussion with the bot.
I hope to continue experimenting with variations of the system prompts when I have time. However, it’s back to marking and preparing for staff PD now!