|October 3 · Issue #98 · View online |
As always happy reading and hacking. If you enjoy this newsletter please recommend us to your friends and colleagues.
Until next week!
| An Insider’s Guide to Keeping Up with the AI Experts |
This is an extensive list of recommended AI experts and machine learning researchers to follow online and on social media ranging from AI pioneers such as Andrew Ng over educators such as Chris Olah to luminaries in fields such as AI ethics.
| The Thriving AI Landscape In Israel And What It Means For Global AI Competition |
Fascinating article showing Israel’s rise as a global leader in AI, with more than 900 startups successfully deploying unique expertise in machine learning, deep learning, computer vision, NLP, robotics, and speech recognition. Israeli AI startups have raised close to $2 billion in 2017, an increase of 70% over 2016, and have already raised $1.5 billion this year.
| Unity and DeepMind to Advance AI Research Using Virtual Worlds |
Unity Technologies creator of the world’s leading real-time 3D development platform, announced its collaboration with DeepMind, the world leader in artificial intelligence (AI) research, that will enable the development of virtual environments and tasks in support of the company’s fundamental AI research program.
| An Online Hiring Marketplace Built For Developers (sponsored) |
Find a dev job through Vettery, it’s free for job seekers.
| Building Safe AI: Specification, Robustness, and Assurance |
The nascent field of technical AI safety is rapidly evolving and growing in importance as AI penetrates ever more industries and finds novel applications in security sensitive areas. In this blog post researchers at DeepMind introduce three areas of technical AI safety: specification, robustness, and assurance outlining categorizations for future research.
| Fast.ai IntroIntroduction to Machine Learning for Coders: Launch |
Following two courses on deep learning released to great acclaim Fast.ai follows up with and introductory course to machine learning. The course covers the most important practical foundations for modern machine learning. There are 12 lessons, each of which is around two hours long—a list of all the lessons along with a screenshot from each is at the end of this post.
| Inside NVIDIA’s AI infrastructure for Self-Driving Cars – @Scale |
In this talk, Clément Farabet, VP of AI Infrastructure at NVIDIA discusses NVIDIA’s production-level, end-to-end infrastructure and workflows to develop AI for self-driving cars.
He explores the platform that supports continuous data ingest from multiple cars (each producing TBs of data per hour) and enables autonomous AI designers to iterate training new neural network designs across thousands of GPU systems and validate their behavior over multi PB-scale data sets.
| Prometheus: Reproducible and fast DL/RL. |
High-level utils for PyTorch DL/RL research. It was developed with a focus on reproducibility, fast experimentation and code/ideas/models reusing. Being able to research/develop something new, rather then write another regular train loop. Best coding practices included.
| Facebook Accelerates AI Development with new Partners and Production Capabilities for PyTorch 1.0 |
This post introduces the PyTorch 1.0 prerelease. The latest additions to the framework include a new hybrid front end that enables tracing and scripting models from eager mode into graph mode for bridging the gap between exploration and production deployment, a revamped torch.distributed library that allows for faster training across Python and C++ environments, and an eager mode C++ interface (released in beta) for performance-critical research.
| Towards the First Adversarially Robust Neural Network Model on MNIST |
This exciting paper shows that even for handwritten digits no neural net is anywhere close as robust as human vision. The authors suggest a novel robust classification model that performs analysis by synthesis using learned class-conditional data distributions. Read the digest here
| Resilient Computing with Reinforcement Learning on a Dynamical System: Case Study in Sorting |
This paper formulates general computation as a feedback-control problem, which allows a robotic agent to autonomously overcome some limitations of standard procedural language programming: resilience to errors and early program termination. The authors demonstrate how this computation becomes a sequential decision making problem, solved with reinforcement learning (RL). They do so through a case study on a quintessential computer science problem, array sorting. Evaluations show that our RL sorting agent makes steady progress to an asymptotically stable goal, is resilient to faulty components, and performs less array manipulations than traditional Quicksort and Bubble sort.
| Cellular Automata as Convolutional Neural Networks |
Deep learning techniques have recently demonstrated broad success in predicting complex dynamical systems ranging from turbulence to human speech, motivating broader questions about how neural networks encode and represent dynamical rules. The authors explore this problem in the context of cellular automata (CA), simple dynamical systems that are intrinsically discrete and thus difficult to analyze using standard tools from dynamical systems theory. They show that any CA may readily be represented using a convolutional neural network with a network-in-network architecture.
| Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour |
Deep learning thrives with large neural networks and large datasets. However, larger networks and larger datasets result in longer training times that impede research and development progress. Distributed synchronous SGD offers a potential solution to this problem by dividing SGD minibatches over a pool of parallel workers. Yet to make this scheme efficient, the per-worker workload must be large, which implies nontrivial growth in the SGD minibatch size. In this paper, the authors empirically show that on the ImageNet dataset large minibatches cause optimization difficulties, but when these are addressed the trained networks exhibit good generalization.