Salesforce's AI research team posts about its work on language models, code models, reinforcement learning, and the AI Economist.
HN links it for project details on CTRL, CodeT5, text-to-SQL, and the AI Economist; one commenter found the text-to-SQL results weaker than hoped.
That's what CTRL does:
Given a URL (or other prompt), generate some language.
From [1]:
> With CTRL, we can test which domain best explains a sequence. Note that this procedure is sensitive to subtle nuances in the query prompt. In the example below, "Global warming is a lie" differs from "Global warming is a lie." The latter is a simple declarative sentence as opposed to an open start to a sentence which may continue. Source attribution cannot be considered a measure of veracity, but only a measure of how much each domain token explains a given sequence.
Query Prompt Attributed Sources
Global warming is a lie. r/unpopularopinion, r/conspiracy, r/science
Global warming is a lie r/eli5, r/science, r/unpopularopinion
Global warming is a real phenomenon r/eli5, r/science, r/changemyview
Global warming is a real phenomenon. OpenWebText, r/changemyview, r/science
https://blog.einstein.ai/introducing-a-conditional-transformer-language-model-for-controllable-generation/
The article seems link free but the below seem to be related to the project.
https://blog.einstein.ai/the-ai-economist/
https://einstein.ai/the-ai-economist
https://github.com/salesforce/ai-economist
[edit: Sarcasm removed. I think the project is exciting. I don't think the stated value of data/ML being fairer than politics is very strong because (a.) politics is reflective of culture and represents a sophisticated method of developing consensus among many different ideas of what fair means, (b.) tax accounting work will just shift to GAN and the end result won't be much different just more complex.]
GAN https://en.wikipedia.org/wiki/Generative_adversarial_network
Here is an example for a very recent progress in language model which does not mention the language name at all:
"CTRL - A Conditional Transformer Language Model for Controllable Generation" -
https://blog.einstein.ai/introducing-a-conditional-transformer-language-model-for-controllable-generation/
As a researcher in a non-english, morphologically rich language, I completely agree with the article by Emily Bender. Everytime I read papers, when people ask questions or discuss on NLP in various forums such as reddit/stackoverlow, the language name is taken for granted as English.
AI researchers are important for this reason. Super intelligence, even if limited to simulations, can find results outside of the mainstream that are more optimal than the economic convention.[1]
[1] https://blog.einstein.ai/the-ai-economist/
Maybe works in 10% of cases.
I remember seeing a natural language to SQL engine which worked in 50% of cases so the expectation was higher. https://blog.einstein.ai/talk-to-your-data-one-model-any-database/
related links:
* (code is open-source) https://github.com/salesforce/CodeT5
* https://blog.einstein.ai/codet5/