Browse Definitions :
Definition

Pathways Language Model (PaLM)

What is Pathways Language Model (PaLM)?

The Pathways Language Model is the name of a family of AI large language models developed by Google. The effort gets its name from a Google Research initiative to create what researchers dubbed pathways, in an approach designed to build a single powerful model that could serve as a foundation for multiple use cases.

There are multiple versions of the Pathways Language Model (PaLM). Among the versions of PaLM 2 are Med-PaLM 2, which is fine-tuned for life sciences and medical information; and Sec-PaLM, which is focused for use in cybersecurity deployments to expedite threat analysis.

In May 2023, Google publicly stated that its Bard conversational AI technology is powered by PaLM 2. The PaLM 2 large language model (LLM) also enables generative AI capabilities in the Google Workspace suite of applications -- including Gmail and Docs -- as well as Google Cloud with a technology known as Duet AI.

What can PaLM do?

PaLM -- and more specifically PaLM 2 -- can provide many functions, including the following:

  • Text generation. PaLM 2 generates text on any topic a user requests using a text prompt.
  • Summarization. Another core capability that summarizes large volumes of content into a more compact form.
  • Content analysis. This feature helps users understand what's in a given block of content. This can include sentiment analysis to identify if the tone of the content is positive or negative.
  • Reasoning. An improved attribute of the PaLM 2 model is the ability to reason. PaLM 2 has a diverse data set that encompasses scientific papers and content with mathematical expressions. This data set enhances the model's proficiency in logic and common-sense reasoning about problem sets provided by users via a prompt.
  • Code generation. PaLM 2 generates computer programming code in 80 different languages, including Java, JavaScript and Python.
  • Code analysis. The model can look at a block of code and identify potential bugs or coding errors.
  • Text translation. PaLM is trained in multiple languages and can execute text translations.

How does PaLM work?

PaLM uses a transformer neural network-based model commonly referred to as a transformer. At a basic level, PaLM is similar to rival transformer-based models, including OpenAI's GPT-3 and GPT-4 models.

PaLM uses the Google-developed Pathways machine learning system to train a model across multiple pods of tensor processing units. The model uses a technique known as few-shot learning that lets it learn from a limited number of labeled examples -- or shots -- to help it quickly adapt and generalize new tasks or classes with minimal data labeling.

As a transformer network, PaLM understands and creates patterns across content, including text and code. The transformer model undergoes a comprehensive learning process, uncovering statistical patterns and connections that exist among words and phrases within content. This acquired knowledge empowers PaLM to generate responses that are both coherent and relevant in various contexts.

What are the limitations of PaLM?

While PaLM is powerful, it has the following limitations on use and capabilities, as well as other items of concern.

  • Use. PaLM is a Google-developed and published model. With the launch of PaLM 2, Google has opened some use for external developers via API, Firebase and on Colab; however, commercial terms of use aren't clear. External developers can't contribute new code or help in the development of PaLM as it's a proprietary model and not open source.
  • Images. PaLM 2 can bring in visual results as part of a query. What it can't do entirely on its own is generate new images. However, Google does let tools built with PaLM 2 -- including Bard -- be extended with support. For example, Bard can connect with Adobe Firefly to let users create AI-generated images.
  • Explainability. PaLM is a closed model and doesn't provide much -- if any -- details to support explainable AI, which is critical for users and organizations to understand how a model comes to a specific decision. Explainable AI is an issue of growing importance as it lets users and organizations understand models better, so they can be trusted more.
  • Toxic content. A key limitation of PaLM identified by Google researchers is the risk of toxic content -- content that can be construed as being biased, malicious or harmful to users.

"Prompted dialog systems built from PaLM 2 continue to produce toxic language harms, and to exhibit patterns of bias in how those harms vary by language and queries related to identity terms," Google researchers wrote in the "PaLM 2 Technical Report."

Differences between PaLM and GPT-3 and GPT-4

There are many similarities and differences between PaLM and OpenAI's GPT-3 -- and the more recent GPT-4 -- LLMs. Both sets of technologies are generally referred to as generative AI, and they both benefit from the use of a transformer model for deep learning. Both technologies can also create, summarize and understand text.

PaLM and PaLM 2 GPT-3 and GPT-4

Developer

Google DeepMind

OpenAI

Chatbot interface

Bard

ChatGPT

Code generation

Fully integrated model

Draws on data from OpenAI's Codex LLM

Multilingual capabilities

PaLM 2 currently supports more than 40 languages

GPT-4 currently supports 26 languages

History of PaLM

Google announced PaLM in April 2022, with the initial version of the language model providing 540 billion parameters.

Google researchers provided early metrics on PaLM performance in a research paper titled "PaLM: Scaling Language Modeling with Pathways," including full details explaining the innovations the model introduces. The model remained private until March 17, 2023, when Google provided an initial set of public APIs to let developers try the model. On May 10, 2023, PaLM 2 was publicly announced at the Google I/O conference.

At the time of the PaLM 2 launch, Google didn't publicly disclose the size of the model in terms of parameters, though it made many claims about the model being larger, more capable and overall providing better performance than its initial model.

Google did disclose that PaLM 2 is being used as the foundational AI behind many of its generative AI efforts, including Bard.

Looking forward, it's unclear if there will be a PaLM 3 model. At the Google I/O 2023 event, company executives said the next-generation LLM being developed by the organization is called Gemini. It's not clear what -- if any -- relationship Gemini has to the PaLM-based approach.

This was last updated in May 2023

Continue Reading About Pathways Language Model (PaLM)

Networking
  • SD-WAN security

    SD-WAN security refers to the practices, protocols and technologies protecting data and resources transmitted across ...

  • net neutrality

    Net neutrality is the concept of an open, equal internet for everyone, regardless of content consumed or the device, application ...

  • network scanning

    Network scanning is a procedure for identifying active devices on a network by employing a feature or features in the network ...

Security
CIO
  • strategic management

    Strategic management is the ongoing planning, monitoring, analysis and assessment of all necessities an organization needs to ...

  • IT budget

    IT budget is the amount of money spent on an organization's information technology systems and services. It includes compensation...

  • project scope

    Project scope is the part of project planning that involves determining and documenting a list of specific project goals, ...

HRSoftware
  • director of employee engagement

    Director of employee engagement is one of the job titles for a human resources (HR) manager who is responsible for an ...

  • digital HR

    Digital HR is the digital transformation of HR services and processes through the use of social, mobile, analytics and cloud (...

  • employee onboarding and offboarding

    Employee onboarding involves all the steps needed to get a new employee successfully deployed and productive, while offboarding ...

Customer Experience
  • chatbot

    A chatbot is a software or computer program that simulates human conversation or "chatter" through text or voice interactions.

  • martech (marketing technology)

    Martech (marketing technology) refers to the integration of software tools, platforms, and applications designed to streamline ...

  • transactional marketing

    Transactional marketing is a business strategy that focuses on single, point-of-sale transactions.

Close