pgpipe moves your data with cursor-based pagination, smart column mapping, resumable state, and a beautiful TUI — or run headless from your scripts.
Features
Built for real migrations: large tables, mismatched schemas, interrupted runs, and automated pipelines.
Built with Bubble Tea. Navigate tables, columns, and settings with keyboard shortcuts. Search with /. No YAML required to get started.
Run pgpipe run --config=file.yaml from scripts, cron, or CI pipelines. No TUI overhead. Parallel table migrations with separate state files.
State saved after every batch. Interrupted? Resume from the exact cursor position. SHA-256 config hashing detects changes and resets automatically.
No OFFSET degradation. Each batch queries by primary key for constant performance across billions of rows.
Auto-matches by name. Auto-detects transforms for type mismatches: TEXT→JSONB, TINYINT→BOOLEAN, CHAR→UUID. Full editor for custom mappings.
pgpipe generate-configs introspects both schemas and writes one YAML per table. Migrate everything in one parallel loop.
Live updates every 500ms: batches, rows, speed (rows/sec), skipped count. Terminal-native. No web dashboards, no agents.
Invalid JSON, bad UUIDs, insert failures — skipped and logged to a timestamped JSONL file. Parse with jq. Migration never stops.
Drop a .env in your working directory. pgpipe loads it automatically. No extra tooling. Override any value from the shell.
Installation
Pre-built binaries for Linux and macOS. Or install via go install.
# Download v0.1.2 binary curl -L https://github.com/RobertoGongora/pgpipe/releases/download/v0.1.2/pgpipe-linux-amd64 \ -o pgpipe chmod +x pgpipe ./pgpipe
# Download v0.1.2 binary (Apple Silicon) curl -L https://github.com/RobertoGongora/pgpipe/releases/download/v0.1.2/pgpipe-darwin-arm64 \ -o pgpipe chmod +x pgpipe ./pgpipe
# Download v0.1.2 binary (Intel Mac) curl -L https://github.com/RobertoGongora/pgpipe/releases/download/v0.1.2/pgpipe-darwin-amd64 \ -o pgpipe chmod +x pgpipe ./pgpipe
# Requires Go 1.24+ go install github.com/RobertoGongora/pgpipe/cmd/pgpipe@latest
git clone https://github.com/RobertoGongora/pgpipe.git cd pgpipe make build
v0.1.2 release binaries
1. Create a .env file
# MySQL source MYSQL_HOST=localhost MYSQL_PORT=3306 MYSQL_USER=root MYSQL_PASSWORD=secret MYSQL_DATABASE=source_db # PostgreSQL target PGSQL_HOST=localhost PGSQL_PORT=5432 PGSQL_USER=postgres PGSQL_PASSWORD=secret PGSQL_DATABASE=target_db
2. Launch and follow the wizard
./pgpipe # interactive TUI # — or headless — ./pgpipe run \ --config=.pgpipe/config.yaml # — or generate configs for all tables — ./pgpipe generate-configs \ --output-dir=./configs # — parallel migration — for f in ./configs/*.yaml; do pgpipe run --config="$f" & done && wait
Auto-Transforms
pgpipe detects incompatible column types and applies the right transform — in the TUI wizard and in generated configs.
| Transform | MySQL Source Types | PostgreSQL Target | Behavior |
|---|---|---|---|
| text_to_jsonb | TEXT, VARCHAR, LONGTEXT, JSON | JSON, JSONB | Validates JSON. Invalid rows skipped + logged. |
| int_to_bool | TINYINT, SMALLINT, INT, BIGINT | BOOLEAN | 0 → false, non-zero → true, NULL → NULL |
| string_to_uuid | CHAR(36), VARCHAR | UUID | Passes through. PostgreSQL validates format. |
Transforms are set automatically when you run pgpipe generate-configs or step through the TUI wizard.
You can override them manually in YAML config files for custom mappings.
Additional transforms (dates, enums, custom functions) are planned for v2. Open an issue to propose one.