This tutorial walks you through building an AI agent that can answer questions about your codebase using RAG (Retrieval Augmented Generation). By the end, you will have a fully deployed agent running on FunctionFly that can search your repository, pull relevant context, and generate accurate answers grounded in your actual code.

Unlike a generic chatbot, a RAG agent retrieves real documents before generating a response. This dramatically reduces hallucinations and gives your users answers that cite specific files, functions, and commits.

Prerequisites

Before you start, make sure you have the following ready:

  • A FunctionFly account with billing enabled (the free tier covers this tutorial)
  • The FunctionFly CLI installed: npm install -g @functionfly/cli
  • A GitHub repository you want the agent to search
  • Node.js 20+ or Bun 1.1+ on your machine
  • Basic familiarity with TypeScript and async/await patterns

If you have not installed the CLI yet, run ff auth login after installation to authenticate with your account.

Step 1: Create the Agent Function

Start by scaffolding a new agent function using the CLI:

bash 
ff init my-code-agent --template rag-agent
cd my-code-agent

This creates a project with the following structure:

text 
my-code-agent/
├── src/
│   ├── index.ts          # Agent entry point
│   ├── tools/
│   │   ├── search.ts     # Code search tool
│   │   └── context.ts    # Context assembly tool
│   └── prompts/
│       └── system.ts     # System prompt definition
├── fly.toml              # Deployment configuration
├── package.json
└── tsconfig.json

Open src/index.ts. This is the main agent definition file. It exports a handler that receives a request and returns a streaming response:

typescript 
import { defineAgent, type AgentContext } from "@functionfly/sdk";
import { searchCode } from "./tools/search";
import { assembleContext } from "./tools/context";
import { SYSTEM_PROMPT } from "./prompts/system";

export default defineAgent({
  name: "code-assistant",
  model: "anthropic/claude-sonnet-4-20250514",
  systemPrompt: SYSTEM_PROMPT,
  tools: [searchCode, assembleContext],
  maxToolRounds: 5,
  async handler(ctx: AgentContext) {
    const { messages, stream } = ctx;
    const lastMessage = messages[messages.length - 1];

    // The agent loop handles tool calls automatically.
    // You only need custom logic if you want to post-process
    // the final response.
    const response = await ctx.runAgent(messages);
    return stream(response);
  },
});

The defineAgent function wires up the model, tools, and system prompt. FunctionFly handles the conversation loop, tool execution, and streaming automatically.

Step 2: Add the Code Search Tool

Tools extend what your agent can do. The code search tool queries a vector index of your repository. Open src/tools/search.ts:

typescript 
import { defineTool, type ToolContext } from "@functionfly/sdk";
import { embed } from "@functionfly/ai";
import { queryVectors } from "@functionfly/vectors";

export const searchCode = defineTool({
  name: "search_code",
  description: "Search the codebase for relevant code snippets, files, and documentation. Use this when the user asks about how something works, where a function is defined, or what a piece of code does.",
  parameters: {
    type: "object",
    properties: {
      query: {
        type: "string",
        description: "Natural language search query describing what you are looking for",
      },
      language: {
        type: "string",
        description: "Optional filter by programming language (e.g. typescript, python, go)",
      },
      maxResults: {
        type: "number",
        description: "Maximum number of results to return (default: 5)",
      },
    },
    required: ["query"],
  },
  async execute(params: any, ctx: ToolContext) {
    const { query, language, maxResults = 5 } = params;

    // Generate an embedding for the search query
    const embedding = await embed(query, { model: "text-embedding-3-small" });

    // Query the vector index
    const results = await queryVectors({
      index: ctx.env.CODE_INDEX,
      vector: embedding,
      topK: maxResults,
      filter: language ? { language } : undefined,
    });

    // Format results for the LLM
    return results.map((r: any) => ({
      file: r.metadata.filePath,
      line: r.metadata.lineRange,
      language: r.metadata.language,
      code: r.content,
      relevance: r.score.toFixed(3),
    }));
  },
});

This tool takes a natural language query, generates an embedding, and searches the vector index. The agent decides when to call it based on the user question and the tool description.

Step 3: Configure the System Prompt

A good system prompt makes or breaks a RAG agent. Open src/prompts/system.ts:

typescript 
export const SYSTEM_PROMPT = `You are a code assistant that helps developers understand a codebase.

You have access to a code search tool. ALWAYS search the codebase before answering questions about code. Never guess or make up function names, file paths, or implementation details.

When answering:
1. Search for relevant code using the search_code tool
2. Read the retrieved snippets carefully
3. Explain what the code does in plain language
4. Cite specific files and line numbers
5. If the search returns nothing relevant, say so honestly

Format code references as: \`src/path/file.ts:42\`
Keep answers concise but thorough. Use markdown formatting for readability.`;

The key instruction is "ALWAYS search the codebase before answering." Without this, the model will often skip the tool and hallucinate.

Step 4: Deploy and Test

Deploy your agent with a single command:

bash 
ff deploy

FunctionFly builds the function, provisions a vector index, and returns a URL. The first deployment takes about 90 seconds because it indexes your repository.

Test it in the playground:

bash 
fly playground --agent my-code-agent

Try these questions to verify it works:

  • "How does authentication work in this project?"
  • "Where is the database connection pool configured?"
  • "What does the HandleListPosts function do?"

Each answer should cite specific files and include code snippets from your repository.

Step 5: Add a Web Search Tool

To make the agent more useful, add a web search tool for questions that go beyond your codebase. Create src/tools/websearch.ts:

typescript 
import { defineTool } from "@functionfly/sdk";
import { webSearch } from "@functionfly/search";

export const searchWeb = defineTool({
  name: "search_web",
  description: "Search the web for documentation, blog posts, and answers to general programming questions. Use this when the question is not about the local codebase.",
  parameters: {
    type: "object",
    properties: {
      query: { type: "string", description: "Search query" },
    },
    required: ["query"],
  },
  async execute(params: any) {
    const results = await webSearch(params.query, { maxResults: 5 });
    return results.map((r: any) => ({
      title: r.title,
      url: r.url,
      snippet: r.snippet,
    }));
  },
});

Register it in your agent definition by adding searchWeb to the tools array.

Step 6: Monitor and Iterate

After deployment, monitor your agent in the FunctionFly dashboard. The Observability tab shows:

  • Tool call frequency: Which tools the agent uses most
  • Latency breakdown: Time spent in embedding, vector search, and LLM generation
  • Error rate: Failed tool calls and their causes
  • Token usage: Cost per conversation

Use this data to tune your system prompt, adjust tool descriptions, and optimize the vector index. If the agent over-uses web search for code questions, strengthen the instruction to search the codebase first.

Next Steps

Now that your agent is live, consider these enhancements:

  • Add memory: Use State Fabric to persist conversation context across sessions
  • Multi-repo support: Index multiple repositories and let the agent search across them
  • CI integration: Trigger the agent on pull requests to provide automated code reviews
  • Access control: Use the Trust Layer to restrict which files the agent can access

Each of these builds on the foundation you created in this tutorial. The FunctionFly SDK provides first-class APIs for all of them.

For the complete API reference, visit the FunctionFly documentation.