Deep Learning Weekly Issue #132
Graphcore coming to Azure, Nvidia's DeepStream, TF 2.0 hackathon, BodyPix for tf.js, and more...
This week in deep learning we bring you news of Graphcore accelerators in Azure, a TensorFlow 2.0 hackathon, Nvidia’s DeepStream SDK for edge IoT devices, and a neural network for foveated reconstruction in VR / AR from Facebook.
We also bring you a new body part estimation model from the TensorFlow team, a write-up of Kuzushiji recognition for ancient Japanese documents, a small course on exploiting and hacking neural networks, the official release of MobileNetV3, a new PyTorch library to accelerate 3D deep learning research from Nvidia, and recent work on few shot learning.
As always, happy reading and hacking. If you have something you think should be in next week's issue, find us on Twitter: @dl_weekly.
Until next week!
DeepMind explains how their LSTM model improves PlayStore rankings.
DeepStream, an SDK containing hardware-optimized DL frameworks and pre-trained models is now available on the Azure Edge IoT marketplace.
The first generation of neural network accelerators is scaling up.
TensorFlow 2.0 hackathon submissions are due by Dec. 31st.
A neat write-up on training models to perform Kuzushiji recognition for digitizing ancient Japanese documents and art.
A deep dive on distilling language models.
Google releases checkpoints for MobileNetV3, showing 15-30% speedups and increased accuracy compared to previous versions.
A small course on exploiting and defending neural networks.
A short course for getting started with deep learning for intracranial extracellular neurophysiology.
Libraries & Code
PyTorch library aimed at accelerating 3D deep learning research.
A neural network to predict whether your HN post will get up votes by the title.
Neural Reconstruction for Foveated Rendering and Video Compression using Learned Statistics of Natural Videos.
Human pose estimation and body part segmentation.
Papers & Publications
Abstract: ….[W]e present a Location-aware Upsampling (LaU) that adaptively refines the interpolating coordinates with trainable offsets. Then, location-aware losses are established by encouraging pixels to move towards well-classified locations. An LaU is offset prediction coupled with interpolation, which is trained end-to-end to generate confidence score at each position from coarse to fine. Guided by location-aware losses, the new module can replace its plain counterpart (e.g., bilinear upsampling) in a plug-and-play manner to further boost the leading encoder-decoder approaches.
Abstract: Few-shot learners aim to recognize new object classes based on a small number of labeled training examples. To prevent overfitting, state-of-the-art few-shot learners use meta-learning on convolutional-network features and perform classification using a nearest-neighbor classifier. This paper studies the accuracy of nearest-neighbor baselines without meta-learning. Surprisingly, we find simple feature transformations suffice to obtain competitive few-shot learning accuracies. For example, we find that a nearest-neighbor classifier used in combination with mean-subtraction and L2-normalization outperforms prior results in three out of five settings on the miniImageNet dataset.