|November 18 · Issue #65 · View online |
Hey ho and welcome to another wild week in deep learning,
Happy reading and hacking!
If you like receiving this newsletter and would like to support our work, you can do so by sharing this issue with friends and colleagues who might find it interesting. Thanks!
| Fully-Parallel Text Generation for Neural Machine Translation |
Current text generation models based on neural networks and deep learning have had the same, surprisingly human, limitation: like us, they can only produce language word by word or even letter by letter. Today Salesforce is announcing a neural machine translation system that can overcome this limitation, producing translations an entire sentence at a time in a fully parallel way.
| Alphabet's Self-driving Cars Introduced to Public Roads in Phoenix |
For the first time, true driverless cars are introduced to public roads as Waymo (part of Alphabet) is rolling a limited experiment in Phoenix.
| CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning |
Stanford researcher develop an algorithm that can detect pneumonia from chest X-rays at a level exceeding practicing radiologists. The model, CheXNet, is a 121-layer convolutional neural network that inputs a chest X-ray image and outputs the probability of pneumonia along with a heatmap localizing the areas of the image most indicative of pneumonia.
| An On-device Deep Neural Network for Face Detection - Apple |
Apple shares details on their use of deep neural networks for face detection. Apple started using deep learning for face detection in iOS 10. This article describes the significant challenges they faced in developing the framework so that user privacy could be preserved while running efficiently on-device.
| The Seven Deadly Sins of Predicting the Future of AI |
Hardly a week goes by without the publication of some sensational, overblown AI prediction. This article provides a much needed reality check and sober assessment of what’s possible in the short and long term.
| Deep Learning is Eating Software or Software 2.0 |
Andrej Karpathy’s post on software 2.0
made a splash earlier this week contending that “a large portion of programmers of tomorrow do not maintain complex software repositories, write intricate programs, or analyze their running times. They collect, clean, manipulate, label, analyze and visualize data that feeds neural networks.”
Pete Warden expands on that point arguing that any traditional software project which after all is just data processing using explicit programming can be improved significantly by applying modern machine learning. A further interesting point is that single deep learning based model is far easier to improve than a set of deeply interconnected modules, and the maintenance becomes far easier.
| Feature Visualization: How Neural Networks Build up their Understanding of Images |
This article on feature visualization explores the intricacies, challenges and common approaches to visualizing features in neural networks. Remarkably, at the end, we find ourselves with relatively simple methods that are able to produce high-quality visualizations.
| Understanding LSTM and its Diagrams – ML Review – Medium |
Superb post explaining the inner workings of LSTMs with the help of clear and detailed diagrams.
| Google Developers Blog: Announcing TensorFlow Lite |
Developer preview of TensorFlow Lite, TensorFlow’s lightweight solution for mobile and embedded devices!
| End-to-end Computation Graphs for Reinforcement Learning |
‘This blogpost introduces version 0.3 of the TensorForce reinforcement learning (RL) library, and motivates major design changes. Development was guided by the aim to provide a better interface and implementation for optimization modules, and later execution. More generally, with version 0.3, we are taking a big step towards RL models as pure TensorFlow objects, including all control-flow.’
| Pytorch Implementations of Various Deep NLP Models in CS-224n |
Pytorch implementations of various Deep NLP models introduced in Stanford’s CS-224n.
| Globally and Locally Consistent Image Completion |
The authors introduce a novel approach for image completion that results in images that are both locally and globally consistent. A fully-convolutional neural network it is possible to complete images of arbitrary resolutions by filling-in missing regions of any shape. To train this image completion network to be consistent, global and local context discriminators that are trained to distinguish real images from completed ones are used.
| Bayesian GAN |
Generative adversarial networks (GANs) can implicitly learn rich distributions over images, audio, and data which are hard to model with an explicit likelihood. The authors present a practical Bayesian formulation for unsupervised and semi-supervised learning with GANs. Within this framework, they use stochastic gradient Hamiltonian Monte Carlo to marginalize the weights of the generator and discriminator networks.
| Data Augmentation Generative Adversarial Networks |
The potential of GANs do anonymize sensitive data (such as medical records) using the generator of a GAN trained on the raw data has been effectively demonstrated in the past, the authors of this paper take a similar approach to generate data for augmentation. The model, based on image conditional Generative Adversarial Networks, takes data from a source domain and learns to take any data item and generalize it to generate other within-class data items. As this generative process does not depend on the classes themselves, it can be applied to novel unseen classes of data.