|February 15 · Issue #74 · View online |
Hey and welcome to a new week in deep learning!
As always, we hope you’ll enjoy reading as much as we did and would appreciate you sharing this newsletter with friends and colleagues.
See you next week!
| Machine learning mega-benchmark: GPU providers |
In this long-awaited follow-up to his recent hardware
, Shiva Manne ran a large benchmark on all available cloud solutions for training deep learning models. Comparing cost, ease of use, stability, scalability, and performance, Paperspace scored the best results and using single low-end GPUs usually gets you the most bang for your buck.
| Cloud TPU machine learning accelerators now available in beta |
Although limited in quantity and only available after a review process, it looks like Google will finally make their TPUs available to developers. Clocking in at 6.50$/Hour/TPU, Google promises training large models or whole sets of models in record speeds.
| Quantum Algorithm Will Push AI 'Thinking' to New Heights |
Quantum computing will not only disrupt every industry but will also tackle problems that are intractable for the classic computers now. This article sheds some light on it’s role in artificial intelligence.
| Asking the Right Questions About AI |
An extensive and well-written look at all the discussions of how artificial intelligence will either save or destroy the world. How self-driving cars will keep us alive, the influence of social media bubbles and more…
| Luminovo.ai - Bespoke AI solutions |
Get to know luminovo.ai: A small team focusing on building tailored AI solutions. In our interview, Sebastian and Timon share how they got into deep learning, describe their business and talk about tech stacks, challenges, learned lessons, as well as their thoughts on the future of deep learning.
| Deep Reinforcement Learning Doesn't Work Yet |
A very extensive look a the current state of deep reinforcement learning, it’s main issues, which include sample inefficiency, superior methods, reward function design and more. Definitely worth a look, as it gives some valuable insights into the field.
| The Matrix Calculus You Need For Deep Learning |
This paper is an attempt to explain all the matrix calculus you need in order to understand the training of deep neural networks. It explains key concepts and links back to more detailed sources to help when you get stuck. There is an HTML version
available as well.
| Pruning Neural Networks: Two Recent Papers |
If you need to reduce the number of parameters in your network and want to find a compromise between computational requirements and performance, pruning is a common way to do so. This article explains three techniques and takes a look at two recent papers on the topic.
| How I Shipped a Neural Network on iOS with CoreML, PyTorch, and React Native |
This post covers the whole story of deploying a neural network for a specific problem, trained using PyTorch, in an iOS app, written using React Native. Especially the roadblocks and pitfalls described in the actual Deployment section should be read by every developer tackling such a task.
| Automatic Learning Rate Scheduling That Really Works |
Training deep learning models can be a pain. In particular, there is this perception that one of the reasons it’s a pain is because you have to fiddle with learning rates. This post describes the most common approach and shows how to let your computer do the tricky part for you.
| DensePose |
This project, created by a team at FAIR, achieved impressive results in dense human pose estimation, where human pixels in an image are mapped to a 3D surface of the human body. The team introduces a new dataset enhancing 50k COCO images with image-to-surface annotations and presents a new variant of Mask-RCNN.
| IMPALA: Scalable Distributed DeepRL in DMLab-30 |
After learning about the weaknesses and issues of deep reinforcement learning earlier, why not take a look at DeepMinds latest publication? IMPALA allows highly scalable training of agents on a new set of tasks using a new off-policy algorithm called V-trace.
| SMASH: One-Shot Model Architecture Search through HyperNetworks |
An interesting approach for optimizing network architecture search, utilizing an auxiliary HyperNet that generates the weights of a model, conditioned on that model’s architecture. This allows comparing the performance of different network architectures, without the need to actually train each variation.
| Face Destylization |
As conventional style transfer becomes more and more common, the focus already shifts to undoing such transfers. In this case, the authors propose a neural network architecture that’s able to recreate the photo-realistic faces from stylized ones.
| Efficient Neural Architecture Search via Parameters Sharing |
Efficient Neural Architecture Search via Parameters Sharing
This paper proposes another approach to improve neural network architecture exploration, but here the conventional training of multiple networks is optimized through parameter sharing.