SKVector Architecture

Vector Pipeline

Text Input → Embedding Model → Vector (384/1536 dims) → Qdrant Collection → Search Results

Components

Embedding Layer

Converts text into dense vector representations:

• Default: sentence-transformers all-MiniLM-L6-v2 (384 dimensions)
• High quality: OpenAI text-embedding-3-small (1536 dimensions)
• Local/private: Ollama with any embedding model

Storage Layer (Qdrant)

• Collections — Organized by memory tier (short/mid/long) or custom
• Payload filtering — Combine vector similarity with metadata filters
• Quantization — Reduce memory usage for large collections
• Persistence — WAL-based durability

Search Layer

Hybrid search combining:

1. Vector similarity — cosine distance in embedding space

2. Payload filters — date ranges, tags, importance thresholds

3. Re-ranking — optional cross-encoder re-ranking for precision

Indexing Strategy

Automatic Indexing

When integrated with SKMemory, every snapshot is automatically:

1. Embedded using the configured model

2. Stored in the appropriate Qdrant collection

3. Tagged with metadata (emotions, importance, quadrant)

Batch Indexing

For importing existing memories:

from skvector import VectorStore

vs = VectorStore(url="http://localhost:6333")
vs.batch_index(memories, batch_size=100)

Performance

• Indexing: ~1000 memories/second (local embeddings)
• Search: < 10ms for collections under 1M vectors
• Memory: ~1KB per vector (384 dims, float32)