omi-backend-patterns — community omi-backend-patterns, community, ide skills, Claude Code, Cursor, Windsurf

v1.0.0
GitHub

About this Skill

Ideal for Conversation Analysis Agents requiring advanced backend integration with LangGraph, Firestore, Pinecone, or Redis. AI wearables. Put it on, speak, transcribe, automatically

BasedHardware BasedHardware
[0]
[0]
Updated: 3/5/2026

Agent Capability Analysis

The omi-backend-patterns skill by BasedHardware is an open-source community AI agent skill for Claude Code and other IDE workflows, helping agents execute tasks with better context, repeatability, and domain-specific guidance.

Ideal Agent Persona

Ideal for Conversation Analysis Agents requiring advanced backend integration with LangGraph, Firestore, Pinecone, or Redis.

Core Value

Empowers agents to process conversations, extract memories, and integrate with the LangGraph chat system using Python, while providing robust backend patterns for API endpoint implementation and data storage with Firestore, Pinecone, or Redis.

↓ Capabilities Granted for omi-backend-patterns

Implementing new API endpoints for conversation processing
Extracting memories from user interactions
Integrating LangGraph with existing chat systems
Optimizing backend code in the `backend/` directory

! Prerequisites & Limits

  • Requires Python backend environment
  • Specific to Omi backend and LangGraph integration
  • Needs access to Firestore, Pinecone, or Redis for data storage
Labs Demo

Browser Sandbox Environment

⚡️ Ready to unleash?

Experience this Agent in a zero-setup browser environment powered by WebContainers. No installation required.

Boot Container Sandbox

omi-backend-patterns

Install omi-backend-patterns, an AI agent skill for AI agent workflows and automation. Works with Claude Code, Cursor, and Windsurf with one-command setup.

SKILL.md
Readonly

Omi Backend Patterns Skill

This skill provides guidance for working with the Omi backend, including conversation processing, memory extraction, chat system, and LangGraph integration.

When to Use

Use this skill when:

  • Working on backend Python code in backend/
  • Implementing new API endpoints
  • Processing conversations or extracting memories
  • Working with the LangGraph chat system
  • Integrating with Firestore, Pinecone, or Redis

Key Patterns

Conversation Processing

The conversation processing pipeline follows this flow:

  1. Audio arrives via WebSocket (/v4/listen)
  2. Transcription via Deepgram/Soniox/Speechmatics
  3. Conversation creation in Firestore (status: "in_progress")
  4. Processing trigger via POST /v1/conversations or timeout
  5. LLM extraction of structured data:
    • Title and overview
    • Action items
    • Calendar events
    • Memories (user facts)
  6. Storage in Firestore and Pinecone

Key Function: utils/conversations/process_conversation.py::process_conversation()

Memory Extraction

Memories are extracted from conversations using LLM:

python
1from utils.llm.conversation_processing import _extract_memories
2
3memories = await _extract_memories(
4    transcript=transcript,
5    existing_memories=existing_memories,
6)

Categories: personal, health, work, relationships, preferences

Chat System Architecture

The chat system uses LangGraph for routing:

  1. Classification: requires_context() determines path
  2. Simple Path: Direct LLM response (no context needed)
  3. Agentic Path: Full tool access with LangGraph ReAct agent
  4. Persona Path: Persona app responses

Key File: utils/retrieval/graph.py

Module Hierarchy

CRITICAL: Always follow the import hierarchy:

  1. database/ - Data access (lowest)
  2. utils/ - Business logic
  3. routers/ - API endpoints
  4. main.py - Application entry

Never import from higher levels in lower levels!

Database Patterns

  • Firestore: Primary database for conversations, memories, users
  • Pinecone: Vector embeddings for semantic search
  • Redis: Caching (speech profiles, enabled apps, user names)
  • GCS: Binary files (audio, photos, speech profiles)

API Endpoint Patterns

  • Use FastAPI routers in routers/
  • Keep routers thin - business logic in utils/
  • Use dependency injection for auth
  • Return consistent error formats

Common Tasks

Adding a New API Endpoint

  1. Create router function in appropriate routers/*.py
  2. Add business logic in utils/
  3. Use database functions from database/
  4. Follow error handling patterns
  5. Add to router in main.py

Processing Conversations

  1. Use process_conversation() from utils/conversations/process_conversation.py
  2. Handle extraction results
  3. Store in Firestore and Pinecone
  4. Trigger app webhooks if needed

Adding a Chat Tool

  1. Create tool function in utils/retrieval/tools/
  2. Use @tool decorator from LangChain
  3. Add to tool loading in utils/retrieval/tools/app_tools.py
  4. Tool will be available in agentic chat path

The docs/ folder is the single source of truth for all user-facing documentation, deployed at docs.omi.me.

  • Backend Deep Dive: docs/doc/developer/backend/backend_deepdive.mdx - View online
  • Chat System: docs/doc/developer/backend/chat_system.mdx - View online
  • Data Storage: docs/doc/developer/backend/StoringConversations.mdx - View online
  • Transcription: docs/doc/developer/backend/transcription.mdx - View online
  • Backend Setup: docs/doc/developer/backend/Backend_Setup.mdx - View online
  • Backend Architecture: .cursor/rules/backend-architecture.mdc

Rules

  • .cursor/rules/backend-architecture.mdc - System architecture and module hierarchy
  • .cursor/rules/backend-api-patterns.mdc - FastAPI router patterns
  • .cursor/rules/backend-database-patterns.mdc - Database storage patterns
  • .cursor/rules/backend-llm-patterns.mdc - LLM integration patterns
  • .cursor/rules/backend-testing.mdc - Testing patterns
  • .cursor/rules/backend-imports.mdc - Import rules
  • .cursor/rules/memory-management.mdc - Memory management

Subagents

  • .cursor/agents/backend-api-developer/ - Uses this skill for API development
  • .cursor/agents/backend-llm-engineer/ - Uses this skill for LLM integration
  • .cursor/agents/backend-database-engineer/ - Uses this skill for database work

Commands

  • /backend-setup - Uses this skill for setup guidance
  • /backend-test - Uses this skill for testing patterns
  • /backend-deploy - Uses this skill for deployment patterns

FAQ & Installation Steps

These questions and steps mirror the structured data on this page for better search understanding.

? Frequently Asked Questions

What is omi-backend-patterns?

Ideal for Conversation Analysis Agents requiring advanced backend integration with LangGraph, Firestore, Pinecone, or Redis. AI wearables. Put it on, speak, transcribe, automatically

How do I install omi-backend-patterns?

Run the command: npx killer-skills add BasedHardware/omi. It works with Cursor, Windsurf, VS Code, Claude Code, and 19+ other IDEs.

What are the use cases for omi-backend-patterns?

Key use cases include: Implementing new API endpoints for conversation processing, Extracting memories from user interactions, Integrating LangGraph with existing chat systems, Optimizing backend code in the `backend/` directory.

Which IDEs are compatible with omi-backend-patterns?

This skill is compatible with Cursor, Windsurf, VS Code, Trae, Claude Code, OpenClaw, Aider, Codex, OpenCode, Goose, Cline, Roo Code, Kiro, Augment Code, Continue, GitHub Copilot, Sourcegraph Cody, and Amazon Q Developer. Use the Killer-Skills CLI for universal one-command installation.

Are there any limitations for omi-backend-patterns?

Requires Python backend environment. Specific to Omi backend and LangGraph integration. Needs access to Firestore, Pinecone, or Redis for data storage.

↓ How To Install

  1. 1. Open your terminal

    Open the terminal or command line in your project directory.

  2. 2. Run the install command

    Run: npx killer-skills add BasedHardware/omi. The CLI will automatically detect your IDE or AI agent and configure the skill.

  3. 3. Start using the skill

    The skill is now active. Your AI agent can use omi-backend-patterns immediately in the current project.

Related Skills

Looking for an alternative to omi-backend-patterns or another community skill for your workflow? Explore these related open-source skills.

View All

widget-generator

Logo of f
f

f.k.a. Awesome ChatGPT Prompts. Share, discover, and collect prompts from the community. Free and open source — self-host for your organization with complete privacy.

★ 149.6k
⑂ 0
AI

flags

Logo of vercel
vercel

flags is a Next.js feature management skill that enables developers to efficiently add or modify framework feature flags, streamlining React application development.

★ 138.4k
⑂ 0
Browser

zustand

Logo of lobehub
lobehub

The ultimate space for work and life — to find, build, and collaborate with agent teammates that grow with you. We are taking agent harness to the next level — enabling multi-agent collaboration, effortless agent team design, and introducing agents as the unit of work interaction.

★ 72.8k
⑂ 0
AI

data-fetching

Logo of lobehub
lobehub

The ultimate space for work and life — to find, build, and collaborate with agent teammates that grow with you. We are taking agent harness to the next level — enabling multi-agent collaboration, effortless agent team design, and introducing agents as the unit of work interaction.

★ 72.8k
⑂ 0
AI