2022-01-02 12:14:35
Digest 2021-12
# ML / Papers
Evaluating Syntactic Abilities of Language Models - https://ai.googleblog.com/2021/12/evaluating-syntactic-abilities-of.html
Efficiently and effectively scaling up language model pretraining for best language representation model on GLUE and SuperGLUE - https://www.microsoft.com/en-us/research/blog/efficiently-and-effectively-scaling-up-language-model-pretraining-for-best-language-representation-model-on-glue-and-superglue/
Improving Vision Transformer Efficiency and Accuracy by Learning to Tokenize - https://ai.googleblog.com/2021/12/improving-vision-transformer-efficiency.html
- TokenLearner is a learnable module that takes an image-like tensor (i.e., input) and generates a small set of tokens.
- Saves memory and computation by half or more w/o loss of accuracy
- Inserting TokenLearner after the initial quarter of the network (at 1/4) achieves almost identical accuracies as the baseline
General and Scalable Parallelization for Neural Networks - https://ai.googleblog.com/2021/12/general-and-scalable-parallelization.html
The Death of Feature Engineering is Greatly Exaggerated - https://petewarden.com/2021/12/11/the-death-of-feature-engineering-is-greatly-exaggerated/
A Fast WordPiece Tokenization System - https://ai.googleblog.com/2021/12/a-fast-wordpiece-tokenization-system.html - but why?
More Efficient In-Context Learning with GLaM - https://ai.googleblog.com/2021/12/more-efficient-in-context-learning-with.html - new 1T param MOE model
Interpretable Deep Learning for Time Series Forecasting - https://ai.googleblog.com/2021/12/interpretable-deep-learning-for-time.html
Why you should be using active learning to build ML - https://humanloop.com/blog/why-you-should-be-using-active-learning
Training Machine Learning Models More Efficiently with Dataset Distillation - https://ai.googleblog.com/2021/12/training-machine-learning-models-more.html
Farcical Self-Delusion - https://blog.piekniewski.info/2021/12/18/farcical-self-delusion/
How a Kalman filter works, in pictures - https://www.bzarg.com/p/how-a-kalman-filter-works-in-pictures/
AI and the Future of Work: What We Know Today - https://thegradient.pub/artificial-intelligence-and-work-two-perspectives/
WebGPT: Improving the factual accuracy of language models through web browsing - https://openai.com/blog/improving-factual-accuracy/#samples
Facebook AI’s WMT21 News Translation Task Submission - http://arxiv.org/abs/2108.03265
#digest
125 viewsAlexander, edited 09:14
S
