|August 14 · Issue #52 · View online |
As always if you enjoy receiving this newsletter, you can help us by sharing it with friends and colleagues.
Happy reading and hacking!
| DeepMind and Blizzard open StarCraft II as an AI Research Environment |
DeepMind and Blizzard Entertainment announced the release of the Starcraft II Learning Environment (SC2LE), a suite of tools with the goal of accelerating AI research in the real-time strategy game, which is particularly interesting in the domain of reinforcement learning research.
| Google Chief Funds new Machine Learning Effort at Princeton's IAS |
The donation will launch new research at the Institute for Advanced Study (IAS) in Princeton to forge an understanding of how machine learning evolves. The three-year program will focus on the mathematical underpinnings of machine learning.
| Facebook Transitioning Entirely to Neural Machine Translation |
Facebook recently switched from using phrase-based machine translation models to neural networks to power their entire backend translation systems, which account for more than 2,000 translation directions and 4.5 billion translations each day. This post describes how they went about implementing the system including parameter tuning using FBLearner Flow
and scaling with Caffe2
| OpenAI Bot Beats Dota 2 Champion |
OpenAI made a splash this week by showcasing a bot that was able to beat the world’s top professionals at 1v1 matches of Dota 2 under standard tournament rules.
Denny Britz provides a sobering dissection
of which aspects of this achievement are novel and wich are overhyped.
| Deeplearning.ai: Announcing new Deep Learning courses on Coursera |
- Neural Networks and Deep Learning
- Improving Deep Neural Networks: Hyperparameter tuning, Regularization and Optimization
- Structuring Machine Learning Projects
- Convolutional Neural Networks
- Sequence Models
The first reviews
are already in and they highly positive. We’ll be reviewing these courses shortly and report back.
| Convolutional Neural Networks for Visual Recognition |
The lecture videos of Stanford’s popular CS231n class on computer vision with deep learning are now online.
| Superresolution with Semantic Guide |
A follow-up to last weeks post on experiments with highresolution GAN-generated faces exploring how go beyond 256x256 pixels to 768x768 images or larger.
| How to Plan and Run Machine Learning Experiments Systematically |
A great post that lays out a simple approach to planning and managing machine learning experiments, it includes a handy Google spreadsheet you can adapt for your own experiments.
| How to Make a Racist AI Without Really Trying |
TLDR: It is easier to make a racist classifier using standard ML techniques than a non-racist one.
| Deeplearn.js | Hardware Accelerated Machine Intelligence Library for the Web |
deeplearn.js is an open-source library that brings performant machine learning building blocks to the web, allowing you to train neural networks in a browser or run pre-trained models in inference mode. It comes with two APIs, an immediate execution model (think NumPy) and a deferred execution model mirroring the TensorFlow API.
| Deep Learning in OpenCV |
Since OpenCV 3.1 there is DNN module in the library that implements forward pass (inferencing) with deep networks, pre-trained using some popular deep learning frameworks, such as Caffe. In OpenCV 3.3 the module has been promoted to the main repository.
| Eyes Gaze Warping 5 |
| PyTorch Examples |
A great reference repository showcasing examples around PyTorch in Vision, Text, Reinforcement Learning.
| Audio Super Resolution | Code
for using deep convolutional neural networks to upsample audio signals such as speech or music. The approach is similar to image super resolution where individual audio samples replace pixels.
| StarData - Starcraft AI Research Dataset |
The largest StarCraft: Brood War replay dataset yet, with 65646 games. The full dataset after compression is 365 GB, 1535 million frames, and 496 million player actions.
| Unsupervised Representation Learning by Sorting Sequences |
A fascinating paper presenting an unsupervised representation learning approach using videos without semantic labels. The authors leverage the temporal coherence as a supervisory signal by formulating representation learning as a sequence sorting task. They take temporally shuffled frames (i.e., in non-chronological order) as inputs and train a convolutional neural network to sort the shuffled sequences.
| DeepMind papers at ICML 2017 |
The first of three blogs that give an overview of the papers DeepMind will be presenting at the ICML 2017 Conference in Sydney, Australia.
| Learning Language Representations for Typology Prediction |
An interesting investigation of neural machine translations and what it is that they learn. The authors find that NMT learns only syntactic but also phonological and phonetic features:
One central mystery of neural NLP is what neural models “know” about their subject matter. When a neural machine translation system learns to translate from one language to another, does it learn the syntax or semantics of the languages? Can this knowledge be extracted from the system to fill holes in human scientific knowledge? Existing typological databases contain relatively full feature specifications for only a few hundred languages. Exploiting the existence of parallel texts in more than a thousand languages, we build a massive many-to-one neural machine translation (NMT) system from 1017 languages into English, and use this to predict information missing from typological databases. Experiments show that the proposed method is able to infer not only syntactic, but also phonological and phonetic inventory features, and improves over a baseline that has access to information about the languages’ geographic and phylogenetic neighbors.
| DeepEyes: Progressive Visual Analytics for Designing Deep Neural Networks |
DeepEyes is a Progressive Visual Analytics system from TU Delft for the analysis of deep neural networks during training. The overview on the training is given by the commonly used loss- and accuracy-curves (a) and the Perplexity Histograms (b) a novel visualization that allows the detection of stable layers. A detailed analysis per layer is performed in three tightly linked visualizations. Degenerated filters are detected in the Activation Heatmap ©, and filter activations are visualized on the Input Map (d). Finally, in the Filter Map (e), relationships among the filters in a layer are visualized.
| Google Vizier: A Service for Black-Box Optimization |
This paper describes Google Vizier, a Google-internal service for performing black-box optimization that has become the de facto parameter tuning engine at Google.