|March 12 · Issue #76 · View online |
Hey and welcome to another week in deep learning!
Happy reading and hacking!
If you like receiving this newsletter and would like to support our work, you can do so by sharing this issue with friends and colleagues who might find it interesting. Thanks!
| Microsoft wants to help developers bring their AI models to the desktop |
Along with all the major players, Microsoft keeps working on bringing AI to it’s products and system. To do so, they just announced tools to allow deployment of machine learning models in the next major version of Windows 10.
| Why Deep Learning Has Not Superseded Traditional Computer Vision |
Changing the perspective, this article explains, why deep learning may not always be the perfect tool of choice for tackling problems in computer vision and why one should still learn the ‘traditional’ methods. Main points are the data requirements, overly complex solutions and traditional skills as helpful fundamentals.
| AI Has a Hallucination Problem That's Proving Tough to Fix |
Nice overview on the latest developments in the field of adversarial attacks. The article tries to cover new work on depending against adversarial attacks, as well as work that ‘broke’ the new techniques, the different positions of the researchers involved and offers a rough glimpse at a possible future.
| The Building Blocks of Interpretability |
This article combines multiple interpretability techniques to understand what’s learned in different parts of GoogleNet and how everything works together to classify images. It offers great insights on the importance of floppy ears for image classification, gives a nice overview on the available techniques, and evaluates how much we can actually infer from the results. To top it off, everything is made clear using interactive visualizations and easily reproducible using Jupyter notebooks.
| OpenAI Scholars |
OpenAI is launching a remote program offering 6-10 stipends and mentorship to individuals from underrepresented groups. Although limited to folks located in US timezones, this sounds like an awesome opportunity and is a great move from OpenAI.
| Can increasing depth serve to accelerate optimization? |
A well written look at the role of depth for expressive power in neural network architecture. Based on his paper, the author explains why deeper networks may not always suffer from harder optimization and how more depth may actually accelerate optimization.
| Introduction to Visual Question Answering: Datasets, Approaches and Evaluation |
Another extensive introduction article from the fine folks at Tryolabs. This time, they take a deep look at visual question answering and cover some of the current datasets, approaches and evaluation metrics in the field. Finally they take a look at how to apply this task to real life use cases.
Sponsor Deep Learning Weekly and reach 7348 deep learning enthusiasts.
| Benchmarking Core ML — Estimating model runtimes on iOS |
Jameson Toole has done some extensive measurements of CoreML performance using real architectures and devices. He then used these to infer an estimated runtime of any architecture based on the number of operations used. These may come in handy if you need a rough estimation if your model will work on mobile and don’t want to go through the hassle of deploying your model using CoreML.
| Sequence modeling benchmarks and temporal convolutional networks |
This repository contains PyTorch code, implementing the experiments done in a recent paper on ‘An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling’.
| Emergence of grid-like representations by training recurrent neural networks to perform spatial localization |
A study showing how neural representations of space, including grid-like cells and borders cells as observed in the brain, could emerge from training a recurrent neural network to perform navigation tasks.
| Understanding Short-Horizon Bias in Stochastic Meta-Optimization |
This paper takes a look at meta-optimization techniques that try to tune hyperparameters prior to the actual training procedures. They found out that these techniques offen suffer from so called ‘short-horizon-bias’, which leads to way too small learning rates to be chosen by the meta-optimization techniques.
| Data Scientist |
Deep NLP and Vision for Business at Infinia ML.
| Data Scientist Intern |
Deep Learning for Business at Infinia ML.