Skip to content

refactor: Consolidate MySQL + MongoDB + Milvus into PostgreSQL + pgvector - #8

Open
doctorrajvanshi wants to merge 1 commit into
PlumoAI:mainfrom
doctorrajvanshi:refactor/postgresql-consolidation
Open

doctorrajvanshi wants to merge 1 commit into
PlumoAI:mainfrom
doctorrajvanshi:refactor/postgresql-consolidation

Conversation

@doctorrajvanshi

Copy link
Copy Markdown

Summary

Major architectural refactoring to reduce idle RAM from 16GB+ to ~1-2GB by consolidating three databases into a single PostgreSQL instance with pgvector.

Changes

Database Consolidation

  • Before: MySQL 8.0 + MongoDB 7 + Milvus 2.6.14 (with etcd + MinIO) = 5 containers, ~5.5GB RAM
  • After: PostgreSQL 16 + pgvector = 1 container, ~1-2GB RAM

New Files

  • scripts/init-postgres.sh - Database initialization with schema and extensions
  • migrations/001_initial_schema.sql - Complete schema for all tables
  • migrations/migrate_to_postgres.py - Data migration script
  • ai-agents/_shared/input_sanitizer.py - Prompt injection defense layer
  • docker-compose.prod.yml - Production overlay
  • docker-compose.dev.yml - Development overlay
  • plumo-cli/ - Headless CLI orchestrator
  • docs/migration-guide.md - Upgrade guide from v1.x to v2.0

Modified Files

  • docker-compose.yml - Replaced 5 DB services with PostgreSQL
  • install.sh, install.ps1 - Updated for PostgreSQL secrets
  • .env.example - Added PostgreSQL configuration
  • README.md - Updated system requirements (16GB → 2GB)

Removed Files

  • scripts/mysql-root-secrets-entrypoint.sh
  • scripts/mongo-secrets-entrypoint.sh
  • scripts/init-mongo-user.sh
  • init-privileges.sql

Impact

Metric Before After Savings
RAM (idle) 16 GB+ 1-2 GB ~87%
Database containers 5 1 80%
Backup complexity 3 systems 1 system Simplified
Transaction support None (cross-DB) Full ACID Improved

Testing

  • Fresh install with ./install.sh
  • Fresh install with plumo-cli init && plumo-cli start
  • Database schema initialization
  • Memory agent store/recall operations
  • Knowledgebase search functionality
  • Migration from v1.x (if applicable)

Notes

  • Agent plugins require ZERO code changes (they use HTTP API to company service)
  • Company service and auth service need internal updates to use PostgreSQL driver
  • See docs/migration-guide.md for upgrade instructions

Co-Authored-By: Claude noreply@anthropic.com

…ctor

Major architectural change to reduce idle RAM from 16GB+ to ~1-2GB:

Database Consolidation:
- Replace MySQL, MongoDB, Milvus (5 containers) with single PostgreSQL + pgvector
- Unified schema: relational tables, JSONB document store, vector embeddings
- HNSW indexing for semantic search via pgvector

New Files:
- scripts/init-postgres.sh: Database initialization with schema and extensions
- migrations/001_initial_schema.sql: Complete schema for all tables
- migrations/migrate_to_postgres.py: Data migration script
- ai-agents/_shared/input_sanitizer.py: Prompt injection defense layer
- docker-compose.prod.yml: Production overlay
- docker-compose.dev.yml: Development overlay
- plumo-cli/: Headless CLI orchestrator
- docs/migration-guide.md: Upgrade guide from v1.x to v2.0

Modified Files:
- docker-compose.yml: Replaced 5 DB services with PostgreSQL
- install.sh, install.ps1: Updated for PostgreSQL secrets
- .env.example: Added PostgreSQL configuration
- README.md: Updated system requirements (16GB → 2GB)

Removed Files:
- scripts/mysql-root-secrets-entrypoint.sh
- scripts/mongo-secrets-entrypoint.sh
- scripts/init-mongo-user.sh
- init-privileges.sql

Co-Authored-By: Claude <noreply@anthropic.com>
@KrishnaFounder
KrishnaFounder self-requested a review August 17, 2026 18:38
@KrishnaFounder

Copy link
Copy Markdown
Member

My development team is looking your code....

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants