Skip to content

Storage System

The storage system in LLMling Agent provides flexible logging and history management through multiple storage providers. It allows tracking conversations, tool usage, and commands while supporting different storage backends and filtering options.

Overview

The storage system serves multiple purposes:

  • Recording conversation history for context and analysis
  • Logging tool usage and command execution
  • Enabling conversation recovery and continuation
  • Supporting debugging and monitoring
  • Allowing to pretty-print and output conversations in a customizable way

Multiple providers can be active simultaneously, each handling different aspects of storage:

  • Some providers focus on full history tracking
  • Others specialize in output logging or monitoring
  • Providers can be filtered to handle specific agents

Settings can be configured at two levels:

  • Global settings affecting all providers
  • Provider-specific settings for fine-grained control

Storage Providers

SQL Storage

The SQL provider offers full history tracking with database persistence: 

storage:
  providers:
    - type: sql
      url: sqlite:///history.db  # Default location
      pool_size: 5              # Connection pool size
      auto_migration: true      # Add missing columns automatically
Features:

  • Complete conversation history
  • Efficient querying and filtering
  • Support for SQLite and other SQL databases
  • Automatic schema management
  • Default provider if none specified

File Storage

File-based storage using YAML or JSON formats:

storage:
  providers:
    - type: file
      path: history.yml        # Or .json
      format: auto            # Or yaml/json
      encoding: utf-8
Features:

  • Human-readable storage format
  • Easy version control integration
  • Flexible file organization
  • Support for multiple file formats

Text Log Provider

Simple text-based logging with customizable formats:

storage:
  providers:
    - type: text_file
      path: agent_logs.txt
      format: chronological     # Or conversations
      template: custom_template.txt  # Optional
Features:

  • Simple text file output
  • Customizable log formats using jinja2-templating
  • Streaming-friendly
  • Output-only (no history queries)

Memory Storage

In-memory storage for testing and development:

storage:
  providers:
    - type: memory
Provides temporary storage without persistence, ideal for testing and development.

Mem0 Storage

The mem0 provider offers semantic-powered conversation history with cloud persistence:

storage:
  providers:
    - type: mem0
      api_key: "your-mem0-api-key"  # Required if no MEM0_API_KEY env var set
      page_size: 100                # Results per page for queries
      output_format: "v1.1"         # API version format
Features:

  • Semantic search capabilities
  • Long-term memory persistence
  • Cloud-hosted (no local setup)
  • Automatic context management
  • Advanced filtering options
  • API version selection (v1.0/v1.1)

Provider Capabilities

History Providers (SQL, File):

  • Full conversation history
  • Message querying and filtering
  • Session recovery
  • Statistics and analytics

Output Providers (Text Log):

  • Message logging
  • Format customization
  • Real-time monitoring
  • No query capabilities

Storable Content:

  • Messages (user/assistant/system)
  • Conversation metadata
  • Tool call details
  • Command execution logs
  • Context additions

Configuration

Global Settings: 

storage:
  # Global filters
  agents: [agent1, agent2]     # Only log these agents
  log_messages: true           # Enable message logging
  log_conversations: true      # Enable conversation tracking
  log_tool_calls: true        # Enable tool call logging
  log_commands: true          # Enable command logging
  log_context: true           # Enable context logging

  # Provider handling
  filter_mode: and            # How to combine filters (and/override)
  default_provider: sql       # Provider for history queries

Provider Filtering:

  • Global agent filters
  • Provider-specific filters
  • Combination modes:
  • and: Both global and provider filters must match
  • override: Provider filter takes precedence

Multiple Providers Example: 

storage:
  providers:
    - type: sql               # Main history storage
      url: sqlite:///history.db

    - type: text_file         # Additional logging
      path: logs/output.txt
      agents: [assistant]     # Only log specific agent

Usage Examples

Basic Setup: 

storage:
  providers:
    - type: sql              # Single provider

Multiple Providers: 

storage:
  providers:
    - type: sql             # Main storage
      url: postgresql://...

    - type: text_file      # Debug logging
      path: debug.log

    - type: file           # Backup
      path: backup.yml

History Queries: 

# Get recent messages
messages = await storage.filter_messages(
    SessionQuery(
        name="session_1",
        since="1h",
        roles={"assistant"}
    )
)

# Get statistics
stats = await storage.get_conversation_stats(
    filters=StatsFilters(
        cutoff=datetime.now() - timedelta(days=1),
        group_by="agent"
    )
)

Custom Filtering: 

storage:
  # Global filter
  agents: [assistant, researcher]

  providers:
    - type: sql
      agents: [assistant]     # Only assistant

    - type: text_file
      agents: [researcher]    # Only researcher