Announced at I/O 2018, ML Kit brings Google’s machine learning to mobile devices. Available for Android and iOS apps, the SDK now lets third-party developers add Smart Reply and other Natural Language Processing features to apps.
To date, existing ML Kit APIs are all focused on image/video processing. This allows for face and landmark detection, OCR, barcode scanning, and image labeling. Google is now expanding into Natural Language Processing to analyze and generate text.
A new Smart Reply API brings Google’s contextual and automated responses to other third-party messaging apps. Like in Gmail and Messages, this feature works completely on-device with message history not sent to the cloud to ensure user privacy.
The API provides suggestions based on the last 10 messages in a conversation, although it still works if only one previous message is available. It is a stateless API that fully runs on-device, so we don’t keep message history in memory nor send it to a server.
It’s built with TensorFlow Lite, and includes other advancements like a model to detect sensitive topics. Smart Reply will not make suggestions in response to profanity or in cases of personal tragedy/hardship. It is launching in English first, and will not provide suggestions for languages the core model is not trained on.
The second API today identifies the language of a piece of text. It is useful for apps that have functionality dependent on the language, like spell checking, text translation, or Smart Reply.
ML Kit recognizes text in 103 different languages and typically only requires a few words to make an accurate determination. It is fast as well, typically providing a response within 1 to 2 ms across iOS and Android phones.
Both of these features work fully on-device and are available on the latest version of the ML Kit SDK for Android (4.1 and higher) and iOS (9.0 and higher).