Skip to main content

Gradient AI

LangChain.js supports integration with Gradient AI. Check out Gradient AI for a list of available models.

Setup

You'll need to install the official Gradient Node SDK as a peer dependency:

npm i @gradientai/nodejs-sdk

You will need to set the following environment variables for using the Gradient AI API.

  1. GRADIENT_ACCESS_TOKEN
  2. GRADIENT_WORKSPACE_ID

Alternatively, these can be set during the GradientAI Class instantiation as gradientAccessKey and workspaceId respectively. For example:

const model = new GradientLLM({
gradientAccessKey: "My secret Access Token"
workspaceId: "My secret workspace id"
});

Usage

npm install @langchain/community

Using Gradient's Base Models

import { GradientLLM } from "@langchain/community/llms/gradient_ai";

// Note that inferenceParameters are optional
const model = new GradientLLM({
modelSlug: "llama2-7b-chat",
inferenceParameters: {
maxGeneratedTokenCount: 20,
temperature: 0,
},
});
const res = await model.invoke(
"What would be a good company name for a company that makes colorful socks?"
);

console.log({ res });

API Reference:

Using your own fine-tuned Adapters

The use your own custom adapter simply set adapterId during setup.

import { GradientLLM } from "@langchain/community/llms/gradient_ai";

// Note that inferenceParameters are optional
const model = new GradientLLM({
adapterId: process.env.GRADIENT_ADAPTER_ID,
inferenceParameters: {
maxGeneratedTokenCount: 20,
temperature: 0,
},
});
const res = await model.invoke(
"What would be a good company name for a company that makes colorful socks?"
);

console.log({ res });

API Reference:


Help us out by providing feedback on this documentation page: