|March 13 · Issue #100 · View online |
As you might have noticed, after 99 issues, Deep Learning Weekly has been on hiatus for the last few months.
Talk about a cliffhanger.
We’ve also got some news of our own. Deep Learning Weekly has joined Heartbeat, a like-minded community of ML enthusiasts that shares the same values and mission. You can read more about this new partnership here
. As you’ll see below, not much else is changing. We’ll still deliver all the latest and greatest deep learning news, industry updates, tutorials, code, and more each week.
As always, happy reading and hacking.
See you next week!
| TensorFlow Dev Summit |
The 2019 TensorFlow Dev Summit was held last week, with the release of TensorFlow 2.0 in Alpha making the biggest splash. But there was lots of other news, too. The next major version of the most popular deep learning framework brings Eager mode, first-class Keras integration, and support for more hardware like TPUs.
The broader TensorFlow ecosystem continues to expand, as well. TensorFlow Hub makes it easy to compose pipelines of pre-trained models, TensorFlow Datasets provides a simple API for loading and manipulating common deep learning datasets, and TensorFlow Federated marks the first open source framework for distributed, on-device model training.
There were also a host of updates to TensorFlow Lite, including support for Edge TPUs and even microcontrollers.
If you don’t have time to watch the 4 hour keynote, we’ve compiled summaries and clips on our TF 2.0
and mobile / embedded
| The AI-Art Gold Rush Is Here |
The first NYC gallery show featuring pieces generated by a deep learning model has come to an end. The Atlantic dives into the tensions brewing in the AI community and art world over what to do with works created by Generative Adversarial Networks.
| OpenAI Releases Neural MMO |
OpenAI has open sourced a new reinforcement learning tool that simulates multiple agents in a large environment to provide a scalable, persistent, reproducible benchmarking environment for researchers.
| Google Duplex is coming to 43 states and iPhones |
Google Duplex, the conversational AI that can schedule your appoints and reserve tables at restaurants, is expanding to 43 states (there are legal roadblocks in the other 7). The service, which for now is restricted to Pixel phones, will also open to iPhone users “in coming weeks”.
| Appen acquires Figure Eight for up to $300M, bringing two data annotation companies together – TechCrunch |
Appen has acquired Figure Eight (formerly CrowdFlower) for $300 million, effectively consolidating the two largest data labeling and annotation companies.
| How Transformers Work | Giuliano Giacaglia
has a fantastic write-up on all of the different Transformer-based neural network models that have been setting new benchmarks and making waves in the NLP community.
| Teaching an AI to Draft Magic: the Gathering – Towards Data Science | Zachary Witten
trains a model to select cards in a Magic: The Gathering draft. His initial approach is to simply train a model that predicts what a human would do. An interesting finding is that when the model disagrees with a human, the model is often making a better choice.
| Foundations Built for a General Theory of Neural Networks | Quanta Magazine |
Neural networks can be as unpredictable as they are powerful. Now mathematicians are beginning to reveal how a neural network’s form will influence its function.
| Best of Machine Learning |
The fine folks at remoteml.com have put together a curated list of machine learning courses and resources for those looking to get started or refine their skills.
| GitHub - zphang/bert_on_stilts: BERT on STILTS |
Want to beat Google’s state of the art performance using their own BERT model? Just train their model twice. Researchers at NYU have released an augmentation technique to BERT that runs two full training passes, with the first devoted to single-task pre-training instead of Google’s original multi-task training. It turns out that allowing the model to focus on one task at a time in pre-training boosts results on multiple tasks later on.
| GitHub - zjhuang22/maskscoring_rcnn: Codes for paper "Mask Scoring R-CNN". |
Codes for paper “Mask Scoring R-CNN”. A new and improved iteration of the powerful Mask R-CNN architecture for image segmentation.
| GitHub - switchablenorms/DeepFashion2 |
DeepFashion2 is a comprehensive fashion dataset. It contains 491K diverse images of 13 popular clothing categories from both commercial shopping stores and consumers. It [in total] has 801K clothing clothing items, where each item in an image is labeled with scale, occlusion, zoom-in, viewpoint, category, style, bounding box, dense landmarks, and per-pixel mask. There are also 873K Commercial-Consumer clothes pairs.
| [1903.01611] The Lottery Ticket Hypothesis at Scale |
MIT PhD student and Google Brain researcher Jonathan Frankle follows up his early work on neural network pruning and optimizing by testing the “Lottery Ticket Hypothesis at Scale”. Previously, Frankle et al showed that for relatively small neural networks performing simple tasks, the vast majority (over 99%) of trained network weights could be removed without significantly impacting accuracy. This paper tests the same idea on larger, more complex networks like ResNet50 with similar results.
| [1903.00374] Model-Based Reinforcement Learning for Atari |
A new model-based approach to reinforcement learning encourages agents to make predictions about future states alongside possible rewards. Most RL techniques focus solely on predicting rewards for certain actions. In this technique, models are encouraged to learn the dynamics of the game itself and use its understanding to improve play. Agents learn to reason about good moves in their minds and then apply their learnings to real-world tests, achieving convergence in far fewer iterations.