Connectors

A connector is the bridge between DagFlux and one external system: a database, a vector store, a cloud bucket, a spreadsheet service, a REST API, a messaging platform, or a local file format.

What connectors provide

Each connector tells DagFlux:

How to test that the connection works.
How to list the things you can read from (tables, collections, buckets, indexes, channels).
How to list the columns or fields of one of those things.
How to run a query, fetch records, upload an object, or whatever the underlying system supports.
Optionally, how to write the AI prompt that produces transformations for that specific system, so the AI uses the dialect's idioms.

Categories

Relational databases (SQL)

Generated transformations produce SQL. Joins produce SQL with temporary tables when needed. Each connector implements its dialect's identifier quoting, preview syntax, and supported types.

Connector	Notes
PostgreSQL	Multi-schema, SSL support, system schemas hidden
MySQL	Standard pooling, dialect-specific batch inserts
MSSQL (SQL Server)	Named instances, default dbo schema
BigQuery	Google data warehouse, dataset.table naming, partitioning
Snowflake	Cloud warehouse, account identifier, warehouse + role context
Redshift	AWS warehouse, JDBC URL parsing

Document databases

Operations are object-shaped (collections, filters, aggregation pipelines). Generated transformations may use filters, updates, deletes, aggregation, or custom JavaScript.

Connector	Notes
MongoDB	Aggregation pipelines, batched JS transformations
NeDB (local)	Local document store for imported JSON or generated documents

Vector databases

Special paths in joins, outputs, and the AI assistant handle embeddings and similarity search.

Connector	Notes
Pinecone	Cloud vector index, namespace-as-table model
LanceDB	Local vector database — available everywhere out of the box

Cloud storage

File picker UX and bucket / object listing screens.

Connector	Notes
Amazon S3	AWS keys, S3-compatible endpoints (MinIO, etc.)
Google Cloud Storage	Service account JSON or key file
Azure Blob Storage	Connection string or account key
Google Drive	OAuth, file IDs, format export
OneDrive	OAuth, Microsoft Graph, multi-tenant
SharePoint	OAuth, Microsoft Graph, site / drive model
Dropbox	OAuth, team-space detection

Spreadsheet services

Connector	Notes
Google Sheets	OAuth, spreadsheetId + sheetName as table identifier

REST API & Messaging

Connector	Notes
REST API	Generic HTTP integration with four auth schemes; per-data-source fetch logic generated by the AI
Slack	OAuth, channels treated as tables, long-lived tokens

Local file formats

These don't need a connector — they're imported into the local workspace by the Data Source node. Supported formats: CSV, Excel, Parquet, Avro, JSON (into a local document store), SQLite, NeDB, LanceDB.