The first open-source equivalent of OpenAI’s ChatGPT is out, but run it on your laptop.
This week, Philip Wang, the developer responsible for reverse-engineering closed-source AI systems, including Meta’s Make-A-Video, released PaLM + RLHF, a text generation model that behaves similarly to ChatGPT. The system combines PaLM, Google’s large-scale language model, with a technique called reinforcement learning and human feedback (RLHF for short) to do almost everything ChatGPT can do, including drafting emails and suggesting computer code. create a system that can perform the tasks of
However, PaLM + RLHF were not pretrained. In other words, the system wasn’t trained on sample data from the web that it needed to work in the real world. Downloading PaLM + RLHF does not magically install a ChatGPT-like experience. To do that, you’ll need to compile several gigabytes of text that your model can learn from, and find hardware powerful enough to handle the training workload.
Similar to ChatGPT, PaLM+RLHF is basically a statistical tool for predicting words. Given a large number of examples from training data such as posts from Reddit, news articles, and ebooks, PaLM + RLHF can estimate how likely a word is to occur based on patterns such as the semantic context of the surrounding text. You will learn whether there is a gender.
ChatGPT and PaLM + RLHF share a special source of reinforcement learning with human feedback. This is a technique intended to better match the language model with what the user wants to achieve. RLHF includes training a language model (PaLM + her PaLM for RLHF) and prompts (such as “explain machine learning to her 6-year-old”) combined with what the human volunteer expects from the model. Includes fine-tuning on the containing dataset. say (e.g., “Machine learning is a form of AI…”). The aforementioned prompts are sent to the fine-tuned model, several responses are generated, and volunteers rank all responses from best to worst. Finally, we use the rankings to train a “reward model”. This model takes the original model’s responses, sorts them by preference, and filters the top responses for a given prompt.
Collecting training data is an expensive process. And the training itself is not cheap. PaLM has a size of 540 billion parameters, where “parameters” refer to the parts of the language model learned from the training data. A 2020 study puts the cost of developing a text generation model with just 1.5 billion parameters at a whopping $1.6 million. And training the open source model Bloom with 176 billion parameters took 3 months with 384 he Nvidia A100 GPUs. A single His A100 costs thousands of dollars.
Running a trained model of size PaLM + RLHF is also not trivial. Bloom requires a dedicated PC with about 8 A100 GPUs. Cloud alternatives are expensive, and back-of-the-envelope math puts the cost of running OpenAI’s text generation GPT-3 (with about 175 billion parameters) on a single Amazon Web Service at about $87,000 per year. I know there is.
In a LinkedIn post about PaLM + RLHF, AI researcher Sebastian Raschka points out that the necessary scale-up of development workflows can also be a challenge. “Even if someone gives his 500 GPUs to train this model, we have to have a software his framework that can deal with the infrastructure and handle it,” he said. . “It’s obviously possible, but it’s a big effort at the moment (yes, we’re developing frameworks to make it easier, but it’s still not easy).”
In other words, PaLM + RLHF will not currently replace ChatGPT. Unless a well-funded venture (or individual) takes the trouble to train and make it available to the public.
The good news is that several other efforts to replicate ChatGPT are moving rapidly, including one led by a research group called CarperAI. CarperAI plans to partner with open AI research organization EleutherAI and startups Scale AI and Hugging Face to release ready-to-run ChatGPT-like AI models trained on human feedback.
LAION, the nonprofit that provided the initial dataset used to train Stable Diffusion, is also spearheading a project to replicate ChatGPT using the latest machine learning techniques. LAION ambitiously aims to build the ‘Assistant of the Future’. It’s more than just writing emails and cover letters, it’s “an assistant that does meaningful work, uses APIs, dynamically researches information, and much more.” Early stage. However, his GitHub page with the project’s resources went live a few weeks ago.
