Deep Learning Weekly: Issue #218
DeepRoute.ai, Argo, Matt Turck’s MAD landscape, Raspberry Pi’s AI kit, SpeechBrain, and more
Hey folks,
This week in deep learning, we bring you a Machine Learning, AI and Data landscape; Databrick’s huge fundraising round; DeepMind’s paper on precipitation forecasting; and the PASS dataset.
You may also enjoy a tutorial about training large models on GPUs, the SpeechBrain toolkit, music composition with deep learning, and more!
As always, happy reading and hacking. If you have something you think should be in next week's issue, find us on Twitter: @dl_weekly.
Until next week!
Industry
Red Hot: The 2021 Machine Learning, AI and Data (MAD) Landscape
Matt Turck, a venture capitalist specializing in Data & AI investments, publishes his eighth landscape on the AI ecosystem.
AI Can Write Code Like Humans—Bugs and All
A few months after GitHub released a beta version of a program that uses AI to assist programmers, researchers and practitioners discuss what kinds of errors the code written with the help of this program contains.
Alibaba leads $300 million investment into Chinese autonomous driving start-up DeepRoute.ai
Alibaba has led investments worth more than $300 million into Chinese autonomous driving start-up DeepRoute.ai. DeepRoute.ai makes self-driving systems for vehicles that involve both hardware and software.
AI Disruption: What VCs Are Betting On
The funding for AI deals continues its furious pace, with more than $30 billion invested in the latest quarter. This article lists the industries that are the most vulnerable to AI disruption.
How will artificial intelligence power the cities of tomorrow?
AI is becoming a critical feature of smart city applications, helping in the processing of data produced by sensors and IoT devices to provide insights that can lead to improvements in areas like disaster response, traffic management, or decarbonization.
What Databricks’s $1.6 billion funding round means for the enterprise AI market
There’s a lot of excitement around companies like Databricks, offering services for unifying, processing, and analyzing data stored in different sources and architectures. This article discusses what will come next in this field.
Mobile & Edge
Dutch AI Semiconductor Startup Axelera AI Launches With $12 Million Seed Round
Axelera AI is designing an advanced solution for AI at the edge, which will concentrate the computational power of an entire server into a single chip. It launches in early 2022 and will be integrated with the leading AI frameworks.
Raspberry Pi development kit opens up voice and AI in the IoT
Knowles has released a new Raspberry Pi Development Kit to support voice, audio edge processing and machine learning, including a library of onboard audio algorithms and AI/ML libraries.
Gentle introduction to GPUs inner workings
This article summarizes some low-level aspects of how a GPU executes computations. Having an understanding of this is needed to optimize the execution of your deep learning models on GPUs.
Learning
Accelerating Interpretable Machine Learning for Diversified Portfolio Construction
A team working for Munich Re Markets has developed an approach based on interpretable machine learning to construct robust investment strategies for the challenging goal of saving for old age.
Sourcing Training Data with Amazon’s Mechanical Turk
In this post, New York Times R&D explains how they experimented with Mechanical Turk to crowdsource high-quality training data for Switchboard, a Q&A tool that leverages NLP to cluster reader questions and match them with reporter-written answers.
Keeping an AI on MRIs: Industry Experts Share Latest on AI and Machine Learning in Medical Imaging
This post details a few applications of AI in healthcare: helping clinical teams automate parts of their radiology workflow, helping identify bone fractures, and helping detecting multiple diseases from a single image.
How to Train Really Large Models on Many GPUs?
This post reviews several training parallelism paradigms, as well as a variety of model architecture and memory saving designs to make it possible to train very large neural networks across a large number of GPUs.
Machine Learning Hyperparameter Optimization with Argo
This post presents a few methods for distributed hyperparameter optimization, an unavoidable problem when iterating on ML models at scale, with impressive results: this has decreased the optimization time of some models from a week to a little over a day.
Libraries & Code
SpeechBrain is a speech toolkit based on PyTorch that can be used to easily develop state-of-the-art speech technologies, including systems for speech recognition, speaker recognition, speech enhancement, multi-microphone signal processing and many others.
PASS (Pictures without humAns for Self-Supervision)
PASS is a large-scale image dataset, containing 1.4 million images, and notably does not include any humans, significantly reducing privacy concerns.
Papers & Publications
Skilful precipitation nowcasting using deep generative models of radar
Abstract:
Precipitation nowcasting, the high-resolution forecasting of precipitation up to two hours ahead, supports the real-world socio-economic needs of many sectors reliant on weather-dependent decision-making. State-of-the-art operational nowcasting methods typically advect precipitation fields with radar-based wind estimates, and struggle to capture important non-linear events such as convective initiations. Recently introduced deep learning methods use radar to directly predict future rain rates, free of physical constraints. While they accurately predict low-intensity rainfall, their operational utility is limited because their lack of constraints produces blurry nowcasts at longer lead times, yielding poor performance on more rare medium-to-heavy rain events. To address these challenges, we present a Deep Generative Model for the probabilistic nowcasting of precipitation from radar. Our model produces realistic and spatio-temporally consistent predictions over regions up to 1536 km x 1280 km and with lead times from 5-90 min ahead. In a systematic evaluation by more than fifty expert forecasters from the Met Office, our generative model ranked first for its accuracy and usefulness in 88% of cases against two competitive methods, demonstrating its decision-making value and ability to provide physical insight to real-world experts. When verified quantitatively, these nowcasts are skillful without resorting to blurring. We show that generative nowcasting can provide probabilistic predictions that improve forecast value and support operational utility, and at resolutions and lead times where alternative methods struggle.
The Values Encoded in Machine Learning Research
Abstract:
Machine learning (ML) currently exerts an outsized influence on the world, increasingly affecting communities and institutional practices. It is therefore critical that we question vague conceptions of the field as value-neutral or universally beneficial, and investigate what specific values the field is advancing. In this paper, we present a rigorous examination of the values of the field by quantitatively and qualitatively analyzing 100 highly cited ML papers published at premier ML conferences, ICML and NeurIPS. We annotate key features of papers which reveal their values: how they justify their choice of project, which aspects they uplift, their consideration of potential negative consequences, and their institutional affiliations and funding sources. We find that societal needs are typically very loosely connected to the choice of project, if mentioned at all, and that consideration of negative consequences is extremely rare. We identify 67 values that are uplifted in machine learning research, and, of these, we find that papers most frequently justify and assess themselves based on performance, generalization, efficiency, researcher understanding, novelty, and building on previous work. We present extensive textual evidence and analysis of how these values are operationalized. Notably, we find that each of these top values is currently being defined and applied with assumptions and implications generally supporting the centralization of power. Finally, we find increasingly close ties between these highly cited papers and tech companies and elite universities.
Music Composition with Deep Learning: A Review
Abstract:
Generating a complex work of art such as a musical composition requires exhibiting true creativity that depends on a variety of factors that are related to the hierarchy of musical language. Music generation have been faced with Algorithmic methods and recently, with Deep Learning models that are being used in other fields such as Computer Vision. In this paper we want to put into context the existing relationships between AI-based music composition models and human musical composition and creativity processes. We give an overview of the recent Deep Learning models for music composition and we compare these models to the music composition process from a theoretical point of view. We have tried to answer some of the most relevant open questions for this task by analyzing the ability of current Deep Learning models to generate music with creativity or the similarity between AI and human composition processes, among others.
 |
|