|December 7 · Issue #67 · View online |
Hey and welcome to another week in deep learning!
Happy reading and hacking!
If you like receiving this newsletter and would like to support our work, you can do so by sharing this issue with friends and colleagues who might find it interesting. Thanks!
| AI Progress Measurement |
This project collects data and materials on different problems that are currently being solved using some sort of AI approach. The collection of achieved error rates and accuracies allows the comparison of different approaches and gives us the ability to see ‘where we are’ on a certain problem.
| The Impossibility of Intelligence Explosion |
Francois Chollet shares his thoughts on why there won’t be a ‘superintelligence’ that surpasses humans in everything. Written in great detail and containing many valuable insights, this is definitely worth a read.
| Artificial Intelligence Index Report |
The AI index report was created as a project of the One Hundred Year Study on AI at Stanford University (AI100)
, with the aim to facilitate an informed conversation about AI that is grounded in data. This inaugural annual report takes a look at the activity and progress in Artificial Intelligence through a range of perspectives.
| Optimization for Deep Learning Highlights in 2017 |
Sebastian Ruder takes a very deep look at the current state of optimization in deep learning. Starting with Adam, as the current ‘default’ optimizer, he covers a few optimizations and then moves on to learning rates, different variations of warm restarts and more. Great insights and definitely worth a look.
| A Year in Computer Vision |
Although this survey tackles the computer vision research in 2016, it still offers valuable insights into all the different subfields. Matching the theme there are lots of visualizations that allow quick skimming. Once you’ve decided on a topic you’re interested in, you get lots of details and references for further research. Really well done and incredibly handy!
| Population based training of neural networks |
DeepMind presented a very interesting technique for hyperparameter optimization. The algorithm starts with multiple randomly chosen hyperparameter configurations in parallel and then periodically updates the running experiments or spawns new ones to test different, more promising configurations. Although this requires a hefty amount of machines, DeepMind managed to find configurations that exceeded previous results. For a more in-depth look, you may want to check out the paper
| Sound Classification with TensorFlow |
A nice tutorial on how to do sound classification in TensorFlow. Very detailed and covers all the tricky parts, so you won’t give up to early. There’s even a web interface included.
| Sequence Modeling with CTC |
A visual guide to Connectionist Temporal Classification, an algorithm used to train deep neural networks in speech recognition, handwriting recognition and other sequence problems.
| Using Artificial Intelligence to Augment Human Intelligence |
Another interactive article on distill.pub, this time covering the use of artificial intelligence to help us humans create things. The example of exploring different fonts using a neural network that models the latent spaces is well explained and the interactive animations make reading a pleasure.
| Distributed TensorFlow: A Gentle Introduction |
The official documentation on Distributed TensorFlow
assumes a fair amount of background and can be hard to understand, start here for a more gentle introduction of how to harness these powerful TensorFlow feature.
| DeepSpeech: Mozilla’s Open Source Speech Recognition Model and Voice Dataset |
Mozilla has open sourced a speech recognition model approaching human level accuracy, this includes the world’s second-largest publicly available voice dataset, which was contributed to by nearly 20,000 people globally. The code is on Github.
| TopoSketch Web App: Generating Animations by Sketching in Conceptual Space |
TopoSketch is a new sketch based interface for generating animations that’s based on a neural network.
| StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation |
| AlphaZero |
AlphaZero is a generalization of AlphaZero Go that can achieve, tabula rasa, superhuman performance in many challenging domains. Starting from random play, and given no domain knowledge except the game rules, AlphaZero achieved within 24 hours a superhuman level of play in the games of chess and shogi (Japanese chess) as well as Go, and convincingly defeated a world-champion program in each case.
| Learning to Segment Every Thing |
The goal of this paper is to propose a new partially supervised training paradigm, together with a novel weight transfer function, that enables training instance segmentation models over a large set of categories for which all have box annotations, but only a small fraction have mask annotations.
| Distilling a Neural Network Into a Soft Decision Tree |
This paper tries to break down the complex internals of neural network by transforming it into a different representation. It describes a way of using a trained neural net to create a type of soft decision tree that generalizes better than one learned directly from the training data.