Deep Learning Weekly Issue #155
Deep Learning Weekly Issue #155
A new neural machine translation toolkit, an AI logo designer, the computation limits of deep learning, and more
Hey folks,
This week in deep learning we bring you this application of a psychology concept to improve Reinforcement Learning, this AI logo designer, the short film “In Event of Moon Disaster” which serves to educate the public on the dangers of deepfakes, and this paper titled The Computational Limits of Deep Learning.
You may also enjoy this open-source neural machine translation toolkit, this article about how CMU and Facebook AI Research use machine learning to teach robots to navigate by recognizing objects, these new 4K spatial camera kits from OpenCV, and more!
As always, happy reading and hacking. If you have something you think should be in next week's issue, find us on Twitter: @dl_weekly.
Industry
Tackling the misinformation epidemic with “In Event of Moon Disaster”
New website from the MIT Center for Advanced Virtuality rewrites an important moment in history to educate the public on the dangers of deepfakes.
OpenAI’s new language generator GPT-3 is shockingly good—and completely mindless
GPT-3 is the largest language model ever created and can generate amazing human-like text on demand but won't bring us closer to true intelligence.
A concept in psychology is helping AI to better navigate our world
The theory of affordance states that when intelligent beings look at the world they perceive not simply objects and their relationships but also their possibilities. This can be applied to RL to enable agents to reach goals more efficiently.
Biases in algorithms hurt those looking for information on health
Anjana Susarla, Professor of Information Systems, Michigan State University, found that because the algorithms underlying recommendations on social media platforms are biased toward engagement and popularity, the most popular YouTube videos are the ones that tend to have easily understandable information but are not always medically valid.
Clients loved this designer’s work. Turns out, he was an AI
A Russian design firm passed off computer-generated work as human, and it’s a peek at things to come.
Mobile + Edge
OpenCV to launch budget-friendly 4K spatial camera kits for AI DIYers
OpenCV, an open source computer vision library, recently announced the crowdfunding launch of two OpenCV AI Kits (OAK-D and OAK-1).
Tiny computer vision for all embedded devices
Edge Impulse brings computer vision support to any device with at least a Cortex-M7 or equivalent microcontroller.
How BMW and Malong used edge AI and machine learning to streamline warehouse and checkout systems
During a panel at VentureBeat’s Transform 2020 conference, speakers including BMW Group’s Jimmy Nassif, Red Hat’s Jered Floyd, and Malong CEO Matt Scott discussed the challenges and opportunities in AI with respect to edge computing and IoT.
CMU and Facebook AI Research use machine learning to teach robots to navigate by recognizing objects
Researchers at CMU and Facebook AI designed a semantic navigation model that helps robots navigate around by recognizing familiar objects.
Silicon Valley execs and Pentagon AI chief talk AI at the edge
Discussion topics include public-private partnerships and edge security through encryption and federated learning.
Learning
Do we need deep graph neural networks?
This post explores whether or not depth in graph neural network architectures brings any advantage.
Everything you need to know about transfer learning in AI
This article gives a high-level overview of transfer learning.
Grounded Language Learning: A Look at the Paper ‘Understanding Early Word Learning in Situated Artificial Agents’
This post summarizes the paper which carefully engineers artificial-language learning experiments to replicate sources of information infants learn from and under what conditions.
High-performance self-supervised image classification with contrastive clustering
Facebook AI developed a new technique for self-supervised training of convolutional networks commonly used for image classification and other computer vision tasks.
Datasets
Source code for the ICML 2020 Workshop on Graph Representation Learning and Beyond (GRL+ 2020) paper “TUDataset: A collection of benchmark datasets for learning with graphs.” This repository contains graph kernel and GNN baseline implementations, data loaders, and evaluations scripts.
Libraries & Code
[GitHub] facebookresearch/pytorch3d
PyTorch3D is FAIR's library of reusable components for deep learning with 3D data.
[GitHub] THUNLP-MT/THUMT
An open-source neural machine translation toolkit developed by Tsinghua Natural Language Processing Group.
[GitHub] google-research/google-research/tree/master/experience_replay
This is the code for the paper Revisiting Fundamentals of Experience Replay (see below).
Papers & Publications
Revisiting Fundamentals of Experience Replay
Abstract: Experience replay is central to off-policy algorithms in deep reinforcement learning (RL), but there remain significant gaps in our understanding. We therefore present a systematic and extensive analysis of experience replay in Q-learning methods, focusing on two fundamental properties: the replay capacity and the ratio of learning updates to experience collected (replay ratio). Our additive and ablative studies upend conventional wisdom around experience replay -- greater capacity is found to substantially increase the performance of certain algorithms, while leaving others unaffected. Counterintuitively we show that theoretically ungrounded, uncorrected n-step returns are uniquely beneficial while other techniques confer limited benefit for sifting through larger memory. Separately, by directly controlling the replay ratio we contextualize previous observations in the literature and empirically measure its importance across a variety of deep RL algorithms. Finally, we conclude by testing a set of hypotheses on the nature of these performance benefits.
The Computational Limits of Deep Learning
Abstract: Deep learning's recent history has been one of achievement: from triumphing over humans in the game of Go to world-leading performance in image recognition, voice recognition, translation, and other tasks. But this progress has come with a voracious appetite for computing power. This article reports on the computational demands of Deep Learning applications in five prominent application areas and shows that progress in all five is strongly reliant on increases in computing power. Extrapolating forward this reliance reveals that progress along current lines is rapidly becoming economically, technically, and environmentally unsustainable. Thus, continued progress in these applications will require dramatically more computationally-efficient methods, which will either have to come from changes to deep learning or from moving to other machine learning methods.