Skip to content
OrbContext
19 pages

Build

JavaScript SDK

The typed Node client: chat, documents and knowledge search.

@orbcontext/sdk is a typed client for the REST API. It has no runtime dependencies, supports Node 20.3+ and ESM, and handles streaming, concurrency limits and error typing so you do not have to.

Install

Shell
npm install @orbcontext/sdk

The package is pre-1.0: minor 0.x versions may change interfaces, patch releases stay compatible. CommonJS is not supported.

Create a client

TypeScript
import { OrbContext } from '@orbcontext/sdk';

const orb = new OrbContext({
  apiKey: process.env.ORBCONTEXT_API_KEY!,   // sk_live_…
  baseURL: process.env.ORBCONTEXT_API_URL!,  // https://api-staging.orbcontext.com while testing
});

Create one client per process and reuse it. It owns the rate-limit gate, concurrency queues and cache, which only work if every call goes through the same instance.

Chat

Get a complete answer

TypeScript
const a = await orb.chat.create({ botId, message: 'What is the refund policy?' });
console.log(a.text);          // the full answer
console.log(a.refused);       // true if the bot could not ground an answer
console.log(a.usage, a.latencyMs);

// Continue the same conversation.
const b = await orb.chat.create({
  botId,
  message: 'And for sale items?',
  conversationId: a.conversationId,
});

create resolves only after the server sends done. A dropped connection, an error frame or a malformed stream throws StreamError. Partial text is never returned as if it were an answer.

Stream tokens

TypeScript
for await (const event of orb.chat.stream({ botId, message: 'Hi' })) {
  switch (event.type) {
    case 'started': /* event.conversationId, event.messageId */ break;
    case 'token':   process.stdout.write(event.value); break;
    case 'done':    console.log('\nrefused:', event.refused); break;
  }
}

Breaking out of the loop cancels the connection and frees its concurrency slot immediately.

Documents

TypeScript
const doc = await orb.documents.upload(botId, file);   // File | Blob | { data, filename }
await orb.documents.waitUntilReady(doc.id, { timeoutMs: 120_000 });

const page = await orb.documents.list(botId, { limit: 50 });
for await (const d of orb.documents.listAll(botId)) console.log(d.name, d.status);

await orb.documents.refresh(websiteDocId);   // website documents only, once an hour; the old version keeps answering
await orb.documents.delete(doc.id);

waitUntilReady polls with a deadline and throws DocumentProcessingError if the document fails.

Needs a key with the knowledge:read scope.

TypeScript
const { hits } = await orb.knowledge.search({
  botId,
  query: 'What is the refund policy?',
  limit: 5,
  // audience: 'staff',   // only if this key is allowed to request it
});
for (const hit of hits) console.log(hit.text, hit.documentId, hit.page);

Search returns passages with no generated answer and no conversation. Each successful search, including one with no match, costs 0.25 credit (a provisional price). Failed searches are refunded. Search is never retried, cached or coalesced.

Bots

TypeScript
const bots = await orb.bots.list();
const bot = await orb.bots.get(botId);

Cancellation and timeouts

Every method takes { signal, timeoutMs } as its last argument.

TypeScript
const controller = new AbortController();
setTimeout(() => controller.abort(), 5_000);
await orb.chat.create({ botId, message: 'Hi' }, { signal: controller.signal });

Errors

Everything the SDK throws extends OrbContextError. Messages never contain your key.

ClassWhen
APIErrorThe API answered with an error: status, code, requestId, issues, retryAfterMs.
ConnectionErrorThe server could not be reached.
TimeoutErrorA deadline passed.
AbortErrorYou aborted the call.
StreamErrorA stream ended without done, sent an error frame or was malformed.
QueueFullErrorThe local wait queue is full (backpressure).
InvalidArgumentErrorThe call arguments were invalid before any request was made.
DocumentProcessingErrorA document failed or did not become ready in time.
TypeScript
import { APIError, StreamError } from '@orbcontext/sdk';

try {
  await orb.chat.create({ botId, message });
} catch (err) {
  if (err instanceof APIError && err.status === 429) {
    // err.retryAfterMs tells you when to try again.
  } else if (err instanceof StreamError) {
    // The answer was cut off. Nothing partial was returned.
  } else {
    throw err;
  }
}

Load management

MechanismBehaviour
RetriesIdempotent GETs only: 2 retries with jittered backoff, honouring Retry-After up to 30s. Uploads, refresh, delete, chat, search and streams are never retried, because a retry could duplicate work or a charge.
429 gateAfter any 429 every call from the client waits out Retry-After instead of rediscovering the limit.
ConcurrencymaxConcurrentRequests (16), maxConcurrentStreams (3), maxConcurrentSearches (8). Extra calls queue in order.
BackpressureThe queue is bounded (maxQueueSize, 256); beyond it calls fail fast with QueueFullError.
CoalescingIdentical concurrent GETs share one request.
CachingBot reads are cached for 30s. Document status and paid search are never cached. cache: false disables it.
Deadlines30s per request, 120s for uploads, 130s total plus 60s idle for streams.

orb.stats exposes active and queued counts if you want to export metrics.