Generování a automatické vyhodnocování divadelních scénářů
| Thesis title in Czech: | Generování a automatické vyhodnocování divadelních scénářů |
|---|---|
| Thesis title in English: | Theatrical Script Generation and Automated Evaluation |
| Key words: | divadelní hry|generování přirozeného jazyka |
| English key words: | theatre plays|natural language generation |
| Academic year of topic announcement: | 2023/2024 |
| Thesis type: | diploma thesis |
| Thesis language: | |
| Department: | Institute of Formal and Applied Linguistics (32-UFAL) |
| Supervisor: | Mgr. Rudolf Rosa, Ph.D. |
| Author: | hidden - assigned and confirmed by the Study Dept. |
| Date of registration: | 30.09.2024 |
| Date of assignment: | 30.09.2024 |
| Confirmed by Study dept. on: | 30.09.2024 |
| Guidelines |
| The thesis aims to explore automatic generation of theatre scenarios and automatic verification whether they meet predetermined requirements using appropriate metrics.
The generation of scenarios will presumably rely on large language models (LLMs), using off-the-shelf and/or fine-tuned models. The metrics will focus on the evaluation of various formal aspects of the outputs (rather than the subjective evaluation of quality), such as consistency in the actions of the characters, adherence to the structure of the plot or genre coherence. The work will focus on the investigation of existing metrics that are used to evaluate text generative models and their adaptation or development of new metrics for the specific needs of theatre plays. Various settings of scenario generation will be experimented with, including open-ended assignments (allowing rather free creation) as well as more restricted assignments (e.g. defining genre, character characteristics, or thematic constraints). The optimal result of the work would be an experimental system using the metrics to generate a scenario in a controlled manner, thus ensuring that the result matches the required specifications. |
| References |
| Rudolf Rosa, Patrícia Schmidtová, Alisa Zakhtarenko, Ondrej Dusek, Tomáš Musil, David Mareček, Saad Ul Islam, Marie Novakova, Klara Vosecka, Daniel Hrbek, and David Kostak. 2022. THEaiTRobot: An Interactive Tool for Generating Theatre Play Scripts. In Proceedings of the 15th International Conference on Natural Language Generation: System Demonstrations, pages 10–13, Waterville, Maine, USA and virtual meeting. Association for Computational Linguistics.
Mirowski, Piotr, et al. "Co-writing screenplays and theatre scripts with language models: Evaluation by industry professionals." Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 2023. Zhong, Ming, et al. "Towards a unified multi-dimensional evaluator for text generation." arXiv preprint arXiv:2210.07197 (2022). Chen, Hong, et al. "StoryER: Automatic story evaluation via ranking, rating and reasoning." Journal of Natural Language Processing 30.1 (2023): 243-249. |