|October 6 · Issue #60 · View online |
Welcome to a truly eventful week in deep learning. This week WaveNet launched in Google assistant, DeepMind launched a new ethics and society research initiative, we learn about the problem with GANs, how to setup a deep learning environment on macOS and that floating point arithmetic provide enough non-linearity for training neural networks.
As always happy hacking and reading.
If you’d like to help us celebrate the 60th issue of deep learning weekly, please share this newsletter with your friends and colleagues.
| Stupid Patents are Dragging down AI and Machine Learning |
It looks like the AI industry will be part of the ugly trend of frivolous patent lawsuits as the US Patent Office seems prepared to give out patents on “using machine learning in obvious and expected ways.”
| DeepMind Ethics & Society | DeepMind |
DeepMind launches Ethics and Society research unit So today we’re launching a new research unit to complement their work in AI science and application. It has a dual aim: ‘to help technologists put ethics into practice, and to help society anticipate and direct the impact of AI so that it works for the benefit of all.’
| Teachable Machine Explore Deep Learning in the Browser |
A fun gimmick by Google allowing you to visually train a neural network in the browser, impressively this was implemented entirely on the client using deeplearn.js
| Google AI Residency Program |
Google is expanding the scope of their ‘Brain Residency Program’ to a broader group of teams doing machine learning research within Google. Applications for the 2018 program are currently open!
| Samsung Opens AI Lab in Canada |
The Samsung Advanced Institute of Technology is opening an artificial intelligence lab in Montréal and joining the global academic movement that is turning the city into an international hub for deep learning. The research center will focus on developing core algorithms for use in robotics, autonomous driving, translation, voice and visual recognition.
| WaveNet Launches in the Google Assistant |
Twelve months ago DeepMind published details of WaveNet, a deep neural network for generating raw audio waveforms that was capable of producing more realistic-sounding speech than existing text-to-speech techniques. Now this new model is being used to generate the Google Assistant voices for US English and Japanese across all platforms.
| GANs are Broken in More than One Way: The Numerics of GANs |
A fascinating post exploring how GANs are broken at both the computational and algorithmic levels mainly stemming from the fact that simultaneous gradient descent leads to optimization in a non-conservative vector field.
| Is AI Riding a One-Trick Pony? |
Just about every AI advance you’ve heard of depends on a breakthrough that’s three decades old. Keeping up the pace of progress will require confronting AI’s serious limitations.
| The Hippocampus as a 'Predictive Map' |
This DeepMind post applies a neuroscience lens to a longstanding mathematical theory from machine learning to provide new insights into the nature of learning and memory. Specifically, the authors propose that the area of the brain known as the hippocampus offers a unique solution to this problem by compactly summarising future events using what we call a “predictive map.”
| Real-Time Recognition of Handwritten Chinese Characters Spanning a Large Inventory of 30,000 Characters - Apple |
The Apple team achieves real-time performance on iPhone, iPad, and Apple Watch (in Scribble mode). Their recognition system, based on deep learning, accurately handles a set of up to 30,000 characters.
| macOS for deep learning with Python, TensorFlow, and Keras - PyImageSearch |
Super helpful tutorial on how to configure your macOS machine for deep learning using Python, Keras, and TensorFlow.
| NIH Clinical Center Releases one of the Largest Publicly Available Chest X-Ray Datasets |
The NIH Clinical Center recently released over 100,000 anonymized chest x-ray images and their corresponding data. The dataset of scans is from more than 30,000 patients, including many with advanced lung disease.
| e-Lab Video Data Set |
A crowd-sourced data set for object recognition composed of 35 classes and a total of 2050 videos of roughly 10 seconds each. You can help by recording a 10-second videos of various household objects you have at home and submit them on the website.
| Variational Inference and Deep Learning; A New Synthesis |
Diederik Kingma’s PhD thesis which proposes novel solutions to the problems of variational (Bayesian) inference, generative modeling, representation learning, semi-supervised learning, and stochastic optimization.
| Continuous Learning Agents |
Great resource on the promising research area of continuous AI which studies “continuous learning” agents that construct a sophisticated understanding of the world from its own experience, through the autonomous incremental development of ever more complex skills and knowledge.
| But what *is* a Neural Network? | Deep learning, Part 1 |
Probably the best explanation of neural networks and representation learning we have come across supported by elaborate visualizations.
| Deep learning and Backprop in the Brain (Yoshua Bengio - CCN 2017) |
Great talk by Yoshua Bengio about the quest for biologically plausible backpropation and the current state of research.
| Nonlinear Computation in Deep Linear Networks |
It turns out floating point arithmetic is nonlinear enough to yield trainable deep networks.
| MILA ends Theano Development |
As one of the maintainers states in the google group: ‘After almost ten years of development, we have the regret to announce that we will put an end to our Theano development after the 1.0 release, which is due in the next few weeks. We will continue minimal maintenance to keep it working for one year, but we will stop actively implementing new features. Theano will continue to be available afterwards, as per our engagement towards open source software, but MILA does not commit to spend time on maintenance or support after that time frame.
| rubiks_cube_convnet: How to Train a ConvNet to Solve a Rubiks Cube |
| Variational Dropout Sparsifies DNN: Sparse Variational Dropout, ICML 2017 |
| Unsupervised Image-to-Image Translation Networks |
Unsupervised image-to-image translation aims at learning a joint distribution of images in different domains by using images from the marginal distributions in individual domains. Since there exists an infinite set of joint distributions that can arrive the given marginal distributions, one could infer nothing about the joint distribution from the marginal distributions without additional assumptions. To address the problem the authors make a shared-latent space assumption and propose an unsupervised image-to-image translation framework based on Coupled GANs.
| Generating Sentences by Editing Prototypes |
The authors propose a new generative model of sentences that first samples a prototype sentence from the training corpus and then edits it into a new sentence. Compared to traditional models that generate from scratch either left-to-right or by first sampling a latent sentence vector, the authors’ prototype-then-edit model improves perplexity on language modeling and generates higher quality outputs according to human evaluation.