Skip to content
OrbContext
19 pages

Guide

NestJS

A typed module, provider and controller around the SDK.

In NestJS, wrap the client in a provider so it is created once, injected where needed and easy to replace in tests. The controller streams answers in the same NDJSON format the React hook expects.

  1. Install

    Shell
    npm install @orbcontext/sdk

    Nest's own @nestjs/config works well for the three variables ORBCONTEXT_API_URL, ORBCONTEXT_API_KEY and ORBCONTEXT_BOT_ID.

  2. Provide the client

    orbcontext/orbcontext.module.ts
    import { Module } from '@nestjs/common';
    import { ConfigModule, ConfigService } from '@nestjs/config';
    import { OrbContext } from '@orbcontext/sdk';
    import { ChatController } from './chat.controller';
    
    export const ORBCONTEXT = Symbol('ORBCONTEXT');
    export const ORBCONTEXT_BOT = Symbol('ORBCONTEXT_BOT');
    
    @Module({
      imports: [ConfigModule],
      controllers: [ChatController],
      providers: [
        {
          provide: ORBCONTEXT,
          inject: [ConfigService],
          useFactory: (config: ConfigService) =>
            new OrbContext({
              apiKey: config.getOrThrow('ORBCONTEXT_API_KEY'),
              baseURL: config.getOrThrow('ORBCONTEXT_API_URL'),
            }),
        },
        {
          provide: ORBCONTEXT_BOT,
          inject: [ConfigService],
          useFactory: (config: ConfigService) => config.getOrThrow<string>('ORBCONTEXT_BOT_ID'),
        },
      ],
      exports: [ORBCONTEXT],
    })
    export class OrbContextModule {}
  3. Validate input

    Use whichever validator your project already has. With class-validator:

    orbcontext/chat.dto.ts
    import { IsOptional, IsString, Length } from 'class-validator';
    
    export class ChatDto {
      @IsString()
      @Length(1, 4000)
      message!: string;
    
      @IsOptional()
      @IsString()
      conversationId?: string;
    }
  4. Stream from a controller

    orbcontext/chat.controller.ts
    import { Body, Controller, Inject, Post, Req, Res } from '@nestjs/common';
    import { APIError, type OrbContext } from '@orbcontext/sdk';
    import type { Request, Response } from 'express';
    import { ChatDto } from './chat.dto';
    import { ORBCONTEXT, ORBCONTEXT_BOT } from './orbcontext.module';
    
    @Controller('api/chat')
    export class ChatController {
      constructor(
        @Inject(ORBCONTEXT) private readonly orb: OrbContext,
        @Inject(ORBCONTEXT_BOT) private readonly botId: string,
      ) {}
    
      @Post()
      async chat(@Body() dto: ChatDto, @Req() req: Request, @Res() res: Response) {
        // Cancel generation when the client disconnects.
        const abort = new AbortController();
        res.on('close', () => abort.abort());
    
        res.setHeader('Content-Type', 'application/x-ndjson; charset=utf-8');
        res.setHeader('Cache-Control', 'no-store');
        res.flushHeaders();
    
        try {
          for await (const event of this.orb.chat.stream(
            { botId: this.botId, message: dto.message, conversationId: dto.conversationId },
            { signal: abort.signal },
          )) {
            res.write(JSON.stringify(event) + '\n');
          }
        } catch (err) {
          if (!abort.signal.aborted) {
            const busy = err instanceof APIError && err.status === 429;
            res.write(
              JSON.stringify({ type: 'error', message: busy ? 'The service is busy. Try again shortly.' : 'The answer failed.' }) + '\n',
            );
          }
        } finally {
          res.end();
        }
      }
    }

    Returning from the handler after taking @Res() means Nest no longer manages the response, which is what you want for a stream. Enable a global ValidationPipe in main.ts so ChatDto is enforced.

Use it from other providers

Inject ORBCONTEXT anywhere, for example to keep a document in sync after your own CMS publishes a page, or to search before running a workflow:

orbcontext/knowledge.service.ts
import { Inject, Injectable } from '@nestjs/common';
import type { OrbContext } from '@orbcontext/sdk';
import { ORBCONTEXT, ORBCONTEXT_BOT } from './orbcontext.module';

@Injectable()
export class KnowledgeService {
  constructor(
    @Inject(ORBCONTEXT) private readonly orb: OrbContext,
    @Inject(ORBCONTEXT_BOT) private readonly botId: string,
  ) {}

  /** Passages only: no answer is generated. Needs the knowledge:read scope. */
  async find(query: string) {
    const { hits } = await this.orb.knowledge.search({ botId: this.botId, query, limit: 5 });
    return hits.map((h) => ({ text: h.text, documentId: h.documentId, page: h.page }));
  }
}

In tests, provide a fake object for ORBCONTEXT instead of mocking the network.