Language models like ChatGPT have revolutionized the field of natural language processing, but they still struggle with some basic tasks like arithmetic and fact checking. Last Thursday, researchers at Meta unveiled his AI language model, which can self-learn to use external tools such as search engines, calculators, and calendars, without sacrificing core language modeling capabilities. Revealed Toolformer.
The key to Toolformer is the availability of an API (application programming interface). An API is a set of protocols that allow different applications to communicate with each other, often in a seamless and automated way. During training, the researcher provided her Toolformer with a small human-created sample set showing how each API would be used, and annotated a large language modeling dataset with potential API calls. allowed to attach This was done in a “self-managed” manner. This means that it can be learned without the need for explicit human guidance.
The model has learned to predict text-based API calls as if they were text in any other format. Calls can be inserted as needed during operations that generate text as a result of human input. Moreover, the tool former You can “decide” which tools to use in the right context and how to use them.
This API call capability allows Toolformer to use external software tools such as search engines, calculators, language translators, and fact references. For example, Large Language Models (LLM) are well known. not particularly good at mathToolformer can work around this limitation with a calculator program. Or, if someone wants her LLM-based assistant to add a date to her calendar, Toolformer can handle that task with an API link to her calendar app.
-
A diagram provided by meta researcher Timo Schick shows an example of Toolformer making an API call to a calendar app.
-
A diagram provided by meta researcher Timo Schick shows an example of Toolformer making an API call to a calculator app.
-
The diagram provided by meta researcher Timo Chic shows an example of Toolformer making API calls against external factual references.
Based on Toolformer Pre-trained GPT-J model 6.7 billion parametersExperiments conducted by researchers on tasks using various tools seem to indicate that Toolformer achieves much stronger performance than the much larger GPT-3 model containing 175 billion parameters. .
This isn’t the first time researchers have tried to compensate for the limitations of language models. In fact, his most recent Bing Chat model, which made headlines this week, can do his web searches on its own on demand, while others have tried to integrate with browsers, calculators, and search engines. According to researchers at Meta, most existing approaches for integrating tools into language models either relied on large amounts of human annotation or were limited to specific task-specific settings. In contrast, Toolformer can learn how to use different tools in a generalized way that does not require task-specific training.
We see a future where LLM enhanced with the ability to use external apps could (ostensibly) be a much more versatile and reliable assistant, with techniques like those seen in Toolformer. However, the ability to make API calls can allow LLMs to harm user data (within the app) or cause problems in the outside world (via web browsers and communication tools). .
