Hello everyone and welcome!
Recently we shared with you our goals for 2020 - Q4 and 2021. In post you can find our directions and plans for updating DeepPavlov Library and other products. With this release, we’re happy to deliver some of our promised deliverables as well as share a couple of new exciting features that we didn’t initially announce in our Roadmap for Q4.
Library
New Features
Hugging Face 🤗 Datasets Support
Hugging Face library includes over 160 free and open NLP datasets. In this release we’re adding support for training DeepPavlov models with datasets made with HF format.Docs:
Examples:
Support for Metrics Tracking with Prometheus Middleware
Prometheus is an open-source toolkit for monitoring and alerting based on an embedded times-series database, a query dsl and various mechanics for scraping metrics data off endpoints. You can expand the REST API metrics with Prometheus metrics that you would like to expose:- total number of requests;
- latency percentiles in milliseconds;
- memory utilization.
Docs:
Improvements
KBQA, Entity Linking, Wikidata Parser
KBQA Online Bugfixes
In this update, we’ve focused on KBQA Online bug fixing. Prior to this update, if you used KBQA Online, your queries would go straight to Wikidata APIs but you’d face additional limitations imposed on your usage. This limitation is imposed when a particular library that uses Wikidata APIs doesn’t properly provide a header required by Wikidata. With this update, you won’t face those limitations (though there’s still a number of general limitations imposed by Wikidata APIs).Docs:
Examples:
Platform Updates
We’ve added smaller changes like .lock-files to synchronize the process of downloading components in an environment where multiple instances of the library are deployed.Dev Tools
Improvements
Go-Bot
In the previous big release of DeepPavlov Library, we’ve shipped an updated version of Go-Bot in which we’ve began adding support for using RASA DSLs as a semi-standard way to define the logic elements like intents, slots, responses, and stories for our Go-Bot. However, intent catcher and slot filler required original DSTC-2-based JSON config files for training.In this release, we’ve made further improvements to Go-Bot. With them, you’ll be able to use only RASA DSL files to train your Go-Bot-based goal-oriented skill.
In addition to that, we’ll publish a blog post alongside with the accompanying Demo repository showcasing how you can integrate your Go-bot-based goal-oriented skill into a DeepPavlov Dream-based multiskill AI Assistant. Stay tuned!
Docs:
Other Fixes
Small bugfixes in the documentation were added by one of the contributors.We encourage you to begin building your Conversational AI systems with our DeepPavlov Library on Github and let us know what you think! Feel free to test our BERT-based models by using our demo. And keep in mind that we have a dedicated forum, where any questions concerning the framework and the models are welcome.
Follow @deeppavlov on Twitter.