Deep Learning Weekly Issue #138
AI-powered scooters from Lime, Google's human-like chatbot, a new deep learning framework, and more
This week in deep learning we bring you personalization and recommendation at Taco Bell, Lime scooters that can predict if they are on sidewalks, a protein folding model from DeepMind, an AI that can play text-based games from Uber, and a Dataset search tool from Google.
You may also enjoy an incredible AR piano playing app, a philosophical discussion of GPT-2, a thorough review of GANs, an introduction to Bayesian neural networks, an overview of the Economics of AI, a new lightweight type-checked deep learning framework, and more.
As always, happy reading and hacking. If you have something you think should be in next week's issue, find us on Twitter: @dl_weekly.
Until next week!
Machine learning-based personalization and recommendation is becoming a requirement in the food industry.
After a year in public beta, you can now search through over 25 million indexed datasets.
DeepMind discusses their recent paper on protein folding.
Research out of Uber uses Seq2Seq models to generate policies agents use to play games.
Scooters use speedometer and accelerometer data to predict which surface they are riding on.
Mobile + Edge
Build a simple image classification iOS app that demonstrates how to integrate PyTorch Mobile with SwiftUI.
One of the most seamless and impressive AR demo’s I’ve seen.
Gary Marcus explores the philosophical implications of large transformer models.
A tutorial on deep learning for graphs that has something for beginners and experts.
An absurdly well referenced review of GANs.
An introduction to spiking neurons and how they relate to artificial neural networks.
A great review of how economists think about AI’s impact on markets and people.
Theory and examples behind Bayesian neural networks, implementations in JAX.
Libraries & Code
Kornia, a differentiable computer vision library for PyTorch has released a new version with GPU support for data augmentation.
Thinc is a type-checked functional deep learning library with support for layers defined in PyTorch, TensorFlow or MXNet.
Papers & Publications
Abstract: ….In this paper, we demonstrate the power of a simple combination of two common SSL methods: consistency regularization and pseudo-labeling. Our algorithm, FixMatch, first generates pseudo-labels using the model's predictions on weakly-augmented unlabeled images. For a given image, the pseudo-label is only retained if the model produces a high-confidence prediction. The model is then trained to predict the pseudo-label when fed a strongly-augmented version of the same image. Despite its simplicity, we show that FixMatch achieves state-of-the-art performance across a variety of standard semi-supervised learning benchmarks, including 94.93% accuracy on CIFAR-10 with 250 labels and 88.61% accuracy with 40 -- just 4 labels per class....
Abstract: We present Meena, a multi-turn open-domain chatbot trained end-to-end on data mined and filtered from public domain social media conversations. This 2.6B parameter neural network is trained to minimize perplexity, an automatic metric that we compare against human judgement of multi-turn conversation quality. To capture this judgement, we propose a human evaluation metric called Sensibleness and Specificity Average (SSA), which captures key elements of good conversation. Interestingly, our experiments show strong correlation between perplexity and SSA. The fact that the best perplexity end-to-end trained Meena scores high on SSA (72% on multi-turn evaluation) suggests that a human-level SSA of 86% is potentially within reach if we can better optimize perplexity. Additionally, the full version of Meena (with a filtering mechanism and tuned decoding) scores 79% SSA, 23% higher than the next highest scoring chatbot that we evaluated.