DAIR.AI @dair_ai:
Top ML Papers of the Week (Jan 30 - Feb 5):
- REPLUG
- SceneDreamer
- The FLAN collection
- Distractability of LLMs
- Blind navigation agents
- Mathematical capabilities of ChatGPT
...
1 of 11
DAIR.AI @dair_ai:
REPLUG - a retrieval-augmented LM framework that adapts a retriever to a large-scale, black-box LM like GPT-3.
2 of 11
twitter.com/WeijiaShi2/sta…
Weijia Shi @WeijiaShi2:
Enhancing GPT-3 with world knowledge
:
Introducing REPLUG: a retrieval-augmented LM framework that combines a frozen LM with a frozen/tunable retriever. Improving GPT-3 in language modeling & downstream tasks by prepending retrieved docs to LM inputs arxiv.org/abs/2301.12652
:
Introducing REPLUG: a retrieval-augmented LM framework that combines a frozen LM with a frozen/tunable retriever. Improving GPT-3 in language modeling & downstream tasks by prepending retrieved docs to LM inputs arxiv.org/abs/2301.12652
DAIR.AI @dair_ai:
Extracting Training Data from Diffusion Models - shows that diffusion-based generative models can memorize images from the training data and emit them at generation time.
3 of 11
twitter.com/Eric_Wallace_/…
Eric Wallace @Eric_Wallace_:
Models such as Stable Diffusion are trained on copyrighted, trademarked, private, and sensitive images.
Yet, our new paper shows that diffusion models memorize images from their training data and emit them at generation time.
Paper: arxiv.org/abs/2301.13188
[1/9]
[1/9]
DAIR.AI @dair_ai:
The FLAN Collection - release a more extensive publicly available collection of tasks, templates, and methods to advancing instruction-tuned models.
4 of 11
twitter.com/ShayneRedford/…
Shayne Longpre @ShayneRedford:
New PaperWhat’s the best completely public competitor to #ChatGPT?
Flan-T5 beats all public models we tested:
Flan-T5 3B T0++ 3B OPT-IML 175B GLM-130B Flan 2021 3B NIv2 3B
We release the @GoogleAI Flan Collectiondata + methods for Instruction Tuning!
1/
New PaperWhat’s the best completely public competitor to #ChatGPT?
Flan-T5 beats all public models we tested:
Flan-T5 3B T0++ 3B OPT-IML 175B GLM-130B Flan 2021 3B NIv2 3B
We release the @GoogleAI Flan Collectiondata + methods for Instruction Tuning!
1/
DAIR.AI @dair_ai:
Multimodal Chain-of-Though Reasoning - incorporates vision features to elicit chain-of-thought reasoning in multimodality, enabling the model to generate effective rationales that contribute to answer inference.
5 of 11
twitter.com/arankomatsuzak…
Aran Komatsuzaki @arankomatsuzaki:
Multimodal Chain-of-Thought Reasoning in Language Models
Multimodal-CoT outperforms GPT-3.5 by 16% (75.17% -> 91.68%) on ScienceQA and even surpasses human performance.
abs: arxiv.org/abs/2302.00923
repo: github.com/amazon-science…
DAIR.AI @dair_ai:
Dreamix - a diffusion model that performs text-based motion and appearance editing of general videos.
6 of 11
twitter.com/_akhaliq/statu…
AK @_akhaliq:
Dreamix: Video Diffusion Models are General Video Editors
abs: arxiv.org/abs/2302.01329
project page: dreamix-video-editing.github.io
present diffusion-based method that is able to perform text-based motion and appearance editing of general videos
Tianyi Zhang @Tianyi_Zh:
Have large language models solved news summarization? Almost there. Our new study shows that text-davinci-002 is comparable to freelance writers.
arxiv.org/abs/2301.13848
DAIR.AI @dair_ai:
Mathematical Capabilities of ChatGPT - investigates the mathematical capabilities of ChatGPT on a new holistic benchmark called GHOSTS.
8 of 11
twitter.com/omarsar0/statu…
elvis @omarsar0:
Mathematical Capabilities of ChatGPT
Was just thinking about a similar idea after the ChatGPT update yesterday.
Nice to see other researchers are also thinking about investigating this more in-depth.
arxiv.org/abs/2301.13867
DAIR.AI @dair_ai:
Training ‘Blind’ Agents - trains an AI agent to navigate purely by feeling its way around; no use of vision, audio, or any other sensing (as in animals).
9 of 11
twitter.com/DhruvBatraDB/s…
Dhruv Batra @DhruvBatraDB:
A thought-experiment to inspire scientists is to ask:
If you could write only 20 papers in your lifetime, would your current work be one of them?
This is one of my 20.
arxiv.org/abs/2301.13261
wijmans.xyz/publication/eo…
DAIR.AI @dair_ai:
SceneDreamer - a generative model that synthesizes large-scale 3D landscapes from random noises.
10 of 11
twitter.com/_akhaliq/statu…
AK @_akhaliq:
SceneDreamer: Unbounded 3D Scene Generation from 2D Image Collections
abs: arxiv.org/abs/2302.01330
project page: scene-dreamer.github.io
DAIR.AI @dair_ai:
LLMs and irrelevant context - finds that many prompting techniques fail when presented with irrelevant context for arithmetic reasoning.
11 of 11
twitter.com/johnjnay/statu…
John Nay @johnjnay:
LLMs Are Easily Distracted by Irrelevant Context
- Performance is dramatically worse when irrelevant info is included in prompt
- But adding *"Feel free to ignore irrelevant information given in the questions.”* consistently improves performance!
Paper: arxiv.org/abs/2302.00093