Vectors

Vectors are vector stores used for RAG (Retrieval-Augmented Generation) with AI agents. They enable semantic search and similarity matching by storing embeddings of your data, allowing AI agents to retrieve relevant information based on meaning rather than exact keyword matches.

What are Vectors?

Vectors enable you to:

• Store Embeddings: Convert data into vector embeddings using AI models
• Semantic Search: Search data by meaning and context, not just keywords
• AI Agent Integration: Provide context to AI agents for better responses
• Knowledge Graphs: Define relationships between data sources
• Multi-Source Data: Include data from Objects, MFlows, MReports, and Files

Key Concepts

Vector Stores

Vector stores are databases optimized for storing and searching vector embeddings:

• Embeddings: Numerical representations of data that capture semantic meaning
• Similarity Search: Find similar data based on vector distance
• Efficient Retrieval: Fast search across large amounts of data
• PostgreSQL pgvector: Uses PostgreSQL with pgvector extension for storage

Embedding Models

Embedding models convert text/data into vector embeddings:

• Model Providers: OpenAI, Ollama, VertexAI
• Model Selection: Choose appropriate embedding model for your use case
• Model Configuration: Configure model settings (temperature, topK, etc.)
• Consistent Embeddings: Same model produces consistent embeddings

Knowledge Graphs

Knowledge graphs define relationships between data sources:

• Nodes: Represent data sources (Objects, MFlows, MReports)
• Edges: Represent relationships between data sources
• Visual Representation: Graph view showing connections
• Relationship Labels: Custom labels for relationships

Data Sources

Vectors can include data from multiple sources:

• Objects: Object records and their data
• MFlows: Workflow definitions and configurations
• MReports: Report definitions and configurations
• Files: Uploaded files (documents, PDFs, etc.)

How Vectors Work

Embedding Process

Data is converted into embeddings:

1. Data Selection: Select data sources (Objects, MFlows, MReports, Files)
2. Text Extraction: Extract text content from selected sources
3. Embedding Generation: Use embedding model to generate vector embeddings
4. Storage: Store embeddings in vector store with metadata

Vector Search

When searching vectors:

1. Query Embedding: Convert search query into embedding using same model
2. Similarity Calculation: Calculate similarity between query and stored embeddings
3. Top-K Results: Retrieve top-K most similar results
4. Context Retrieval: Return relevant data for AI agent context

AI Agent Integration

Vectors are used with AI agents:

• Context Retrieval: Agents retrieve relevant context from vectors
• RAG Pattern: Retrieval-Augmented Generation for better responses
• Semantic Understanding: Agents understand meaning, not just keywords
• Dynamic Context: Context adapts to user queries

Vector Features

Multi-Source Data

Include data from various sources:

• Objects: Object records and schema
• MFlows: Workflow definitions and logic
• MReports: Report configurations and queries
• Files: Document content and metadata

Knowledge Graph

Define relationships between data sources:

• Visual Graph: Drag-and-drop interface for creating knowledge graphs
• Node Types: Different node types for different data sources
• Relationship Mapping: Define how data sources relate to each other
• Column Selection: Select specific columns from Objects to include

Embedding Configuration

Configure embedding generation:

• Model Provider: Choose provider (OpenAI, Ollama, VertexAI)
• Embedding Model: Select specific embedding model
• Model Settings: Configure temperature, topK, and other parameters
• Consistent Models: Use same model for consistent embeddings

Vector Search

Search vectors for relevant information:

• Semantic Search: Search by meaning and context
• Similarity Threshold: Set minimum similarity threshold
• Top-K Results: Configure number of results to return
• Filtered Search: Filter results by metadata

Using Vectors

Vectors can be:

• Created: Create new vector stores with data sources and knowledge graphs
• Edited: Modify vector configurations, data sources, and knowledge graphs
• Searched: Search vectors for relevant information
• Used with AI Agents: Integrate vectors with AI agents for RAG
• Managed: View, update, and delete vector stores

Getting Started

1. Creating Vectors: Learn how to create vector stores with data sources and knowledge graphs
2. Searching Vectors: Learn how to search vectors for semantic similarity
3. Using Vectors with AI Agents: Understand how to configure vectors for AI agents
4. Configure Data Sources: Select Objects, MFlows, MReports, and Files
5. Build Knowledge Graph: Define relationships between data sources

Related Introduction

• Objects Introduction - Understand Objects used as vector data sources
• Workflows Introduction - Learn about workflows used as vector data sources
• Reports Introduction - Understand reports used as vector data sources