8th Dialogue System Technology Challenge
The goal of DSTC8 was to highlight the DST problem on unseen APIs given a schema of these target APIs, while supporting realistically many heterogeneous APIs with possibly overlapping functions.
TRACK: SCHEMA-GUIDED STATE TRACKING
Virtual assistants need to support an ever-increasing number of services and APIs. This track explores challenges associated with dialogue state tracking in such a setting. Each dialogue in the dataset was accompanied by schemas listing a set of user intents and slots, and their natural language description. The dialogue state needs to be predicted over these intents and slots. Submissions was judged based on the ability to generalize to new schemas which are possibly not present in the training set, both in single domain and multi domain dialogues.
OUR SOLUTION - GOLOMB
In this work, we proposed a GOaL-Oriented Multi-task BERT-based dialogue state tracker (GOLOMB) inspired by architectures for reading comprehension question answering systems. The model "queries" dialogue history with descriptions of slots and services as well as possible values of slots. This allowed to transfer slot values in multi-domain dialogues and had a capability to scale to unseen slot types. Our model achieved a joint goal accuracy of 53.97% on the SGD dataset, outperforming the baseline model.
Our team
- Pavel GulyaevAlumni
- Eugenia ElistratovaAlumni
- Vasily KonovalovResearcher at DeepPavlov
- Yury KuratovResearcher at DeepPavlov
- Leonid PugachevResearcher at DeepPavlov
- Mikhail BurtsevHead of DeepPavlov