About the project
Getting feedback on how you teach usually takes a trained observer sitting in the room, so most teachers get very little, and what they get depends on who was watching. My PhD was about giving teachers that feedback from audio recordings of their own classes. In the systematic review that opened the thesis, only 11 of the 82 studies I read gave anything back to the teacher.
I wanted the work to be usable outside the papers, so I built MoviSound, where teachers could process their own recordings without going through me. Nine volunteer teachers from four faculties (Computer Science, Psychology, Veterinary Science and Law) recorded 287 classes between 2023 and 2026. For about two years I prepared reports of their classes by hand and asked which charts helped and which confused them. What they used most was simple: who talks when, and the questions their students asked, which usually get lost in a live class. One teacher found they had been underestimating how much they talked. In the last year the reports moved to the web app, built around those parts.
Eight of the nine teachers answered a survey. They rated the platform above 80% for usefulness, seven said they had changed their teaching strategy after the reports, and six of those saw the change in later reports. Everyone who answered preferred the web app to the PDF reports it replaced. The evidence is self-reported, by the same teachers who provided the recordings.
The platform is the applied part of my thesis, awarded Summa Cum Laude by the University of Murcia in 2026. It runs at movisound.inf.um.es behind university logins and its code is private, so the demo video below is the way to see it.
How it works
Each box in the diagram is its own Docker container. Teachers reach the Django web app through Caddy; accounts and reports live in PostgreSQL. When a recording is uploaded, a Celery worker copies it into MongoDB, which both halves of the system share, and starts a run on the FastAPI analysis service. That service transcribes the audio, separates the speakers, extracts the acoustic features and applies the models from the thesis. A GPU lock makes uploads take turns on the single GPU, and every step is published to Kafka so the web app shows progress while the analysis is still running. A language model served locally with Ollama turns the transcript into a summary of the session and a few suggestions. The results go back to MongoDB, and the web app copies them into PostgreSQL for the teacher to browse.
I built the first version in 2023 and 2024 with a colleague who did the frontend while I did the backend, over an API we agreed on. The current version, frontend included, I wrote on my own.
MoviSound's architecture
Platform demo
The walkthrough below covers the platform in two parts. First a tour of the architecture, service by service, and then the web interface itself: uploading a real classroom recording, watching the pipeline report its progress in real time, and going through the results, from participation metrics and the teaching methodology timeline to the diarized transcript and the summary generated by the language model.
The research behind it
The thesis is a compendium of three first-author papers. The first is a systematic review of how audio has been used in education research. The second classifies how a class is being taught (lecture, group work or Wooclap quizzes) from who speaks when, without looking at what is said, and held up on teachers and classrooms the model had not seen. The third fine-tunes a multilingual DeBERTa model to label what the teacher is doing each time they speak, combines it with features of the speech, and uses SHAP to show what drove each decision.