PaLM 2: Google's latest language model, designed to challenge OpenAI's GPT-4

Google has recently announced PaLM 2, its latest AI language model, and a competitor to rival systems like OpenAI's GPT-4. At the company's I/O conference, Google CEO Sundar Pichai revealed that PaLM 2 models are stronger in logic and reasoning, thanks to broad training in those areas. PaLM 2's training on multilingual text from over 100 languages has significantly enhanced its proficiency in various text-based tasks, including but not limited to reasoning, coding, and translation.

According to Google's senior research director Slav Petrov, PaLM 2 is significantly improved compared to PaLM 1, which was announced in April 2022. Petrov also showed how PaLM 2 understands idioms in different languages, giving the example of the German phrase "Ich verstehe nur Bahnhof," which is better understood as "I don't understand what you're saying" or "it's all Greek to me" in English.

According to a research paper by Google's engineers that details the capabilities of PaLM 2, the system's proficiency in a language is so high that it can be used to teach the language. The paper attributed this proficiency in part to the abundance of non-English texts included in the system's training data. PaLM 2 is available in four sizes named Gecko, Otter, Bison, and Unicorn, with different versions that will be deployed in consumer and enterprise settings.

Google has also adapted PaLM 2 to perform certain tasks for enterprise customers. There is a version of PaLM trained on health data (Med-PaLM 2) that can answer questions similar to those found on the US Medical Licensing Examination at an "expert" level. There is also another version trained on cybersecurity data (Sec-PaLM 2) that can explain the behaviour of potential malicious scripts and help detect threats in code. Initially, Google Cloud will offer access to both of these models to a limited set of customers.

PaLM 2 is already being used to power 25 features and products within Google's own domain, including Bard, the company's experimental chatbot. Bard offers users access to updated coding features and enhanced language support. Additionally, PaLM 2 is being leveraged to enhance the functionality of Google Workspace applications such as Docs, Slides, and Sheets.

According to Google, the most lightweight version of PaLM 2, called Gecko, is compact enough to function on mobile devices, processing approximately 20 tokens per second, or roughly equivalent to 16 or 17 words. Reducing the size of language models has notable implications, as running these models in the cloud can be costly, and having the ability to use them locally can enhance privacy and offer other advantages. Nonetheless, the predicament remains that smaller variants of language models are bound to be less proficient than their larger equivalents.

With PaLM 2, Google aims to close the "AI gap" between the company and competitors like Microsoft, which has been aggressively pushing AI language tools into its suite of Office software. While PaLM 2 marks a notable advancement for Google's AI language models, it still grapples with common issues and hurdles that pervade the field of natural language processing.

For example, some experts are beginning to question the legality of training data used to create language models, which is usually scraped from the internet and often includes copyright-protected text and pirated ebooks. Google has not provided specific details about the sources of PaLM 2's training corpus, noting only that it is comprised of "a diverse set of sources: web documents, books, code, mathematics, and conversational data."

Language models have inherent issues with their output, including the phenomenon of "hallucinations," where the systems tend to generate fabricated information without a basis in reality.

Also Read 

'Buying Netflix at $4 billion would've been better instead of...': Former Yahoo CEO Marissa Mayer

ChatGPT beats top investment funds in stock-picking experiment