Skip to main content

​
Ibex Data Platform

The ibex-data-platform serves as the centralized configuration manager and the backbone of the control plane for data ingestion.

​
Architectural Role

Written in Go and running on port 8080, this service handles all the logic presented by the Config UI (ibex-platform-ui). It handles integrations to external sources, pipeline setup, and secret management. It is broadly exposed via Traefik as the catch-all for /api/* and /configs/* paths.

​
High-Level Working

​
Registry & Source Management

The platform registers and tracks all overarching data sources (MySQL, PostgreSQL, REST APIs, Iceberg). When modified, it broadcasts these updates so downstream consumers—specifically the ibex-analytics-service—can live-attach them as DuckDB catalogs.

​
Security via HashiCorp Vault

It acts as the intermediary between static passwords/API keys and the rest of the application. At startup, it reads credentials from the vault-secrets container (using a root token file mounted at /vault/keys/root_token) and uses these to establish connections.

​
CDC & Pipelines

The ibex-data-platform is responsible for provisioning and maintaining Change Data Capture (CDC) pipelines. It automatically pushes pipeline events and data ingestion requests to redpanda, maintaining state tracking within the central postgres-metadata database via the business_pipelines tables.