Providers

Fluid Forge uses a provider system to deploy your data contracts to different platforms. The same YAML contract works everywhere — locally with DuckDB, on GCP with BigQuery, on AWS with S3+Athena, or on Snowflake.

How Providers Work

Each provider implements two methods: plan() (pure planning, no side effects) and apply() (execute against the target). See Architecture for the full design.

Provider Status

Provider	Plan / Apply	Airflow Gen	Dagster / Prefect	Status
GCP	✅	✅	✅	Production
AWS	✅	✅	✅	Production
Snowflake	✅	✅	✅	Production
Local	✅	—	—	Production
Azure	🔜 Q3 2026	🔜	🔜	Planned

Data Catalog & Standards

Export Target	Purpose	Status
ODPS	Linux Foundation Open Data Product Spec v4.1	✅ Production
ODCS	Open Data Contract Standard v3.1 (Bitol.io)	✅ Production
OPDS	Open Data Product Specification	✅ Production
Datamesh Manager	Publish to datamesh-manager.com	✅ Production

Export Metadata Mapping

When you export to ODPS or publish to DataMesh Manager, Fluid Forge derives catalog metadata from your contract with a few explicit fallback rules.

Output field	Source fields	Notes
Output port provider / type	`binding.platform`, then legacy `provider` where supported	Exports prefer the FLUID 0.7.1 `binding.platform` field.
Expose kind	`expose.kind`, then `expose.type`	Use `kind` in new contracts; `type` is kept as a fallback.
ODPS product type	`metadata.product_type`, `metadata.type`, top-level `product_type`, top-level `type`	The first populated value is exported as the ODPS `type` custom property.
ODPS tags	`metadata.tags`, then top-level `tags`	ODPS uses one tag source, with metadata taking precedence.
DataMesh Manager tags	top-level `tags` merged with `metadata.tags`	Metadata tags are appended without duplicating existing top-level tags.

If multiple candidate fields are present, keep binding.platform, kind, and metadata.* values aligned so exported catalogs stay predictable across providers.

Quick Start by Provider

GCP (BigQuery)

gcloud auth application-default login
gcloud config set project YOUR_PROJECT_ID

fluid apply contract.fluid.yaml --provider gcp --yes

Full GCP Guide →

AWS (S3 + Athena + Glue)

aws configure

fluid apply contract.fluid.yaml --provider aws --yes

Full AWS Guide →

Snowflake

export SNOWFLAKE_ACCOUNT=your_account
export SNOWFLAKE_USER=your_user

fluid apply contract.fluid.yaml --provider snowflake --yes

Full Snowflake Guide →

Local (DuckDB)

fluid init my-project --quickstart
fluid apply contract.fluid.yaml --yes

Full Local Guide →

Feature Matrix

Feature	GCP	AWS	Snowflake	Local
Plan & apply	✅	✅	✅	✅
IAM / RBAC compilation	✅	✅	✅	—
Airflow DAG generation	✅	✅	✅	—
Dagster pipeline export	✅	✅	✅	—
Prefect flow export	✅	✅	✅	—
Sovereignty validation	✅	✅	✅	—
ODPS / ODCS export	✅	✅	✅	—

Key Services per Provider

Capability	GCP	AWS	Snowflake
Data warehouse	BigQuery	Athena + Redshift	Native tables
Object storage	GCS	S3	Stages
IAM	Cloud IAM	IAM policies	RBAC grants
Orchestration	Cloud Composer	MWAA	Snowflake Tasks
Streaming	Pub/Sub	Kinesis	Snowpipe

Choosing a Provider

If you...	Choose
Are developing or testing locally	Local
Use Google Cloud / BigQuery	GCP
Are on AWS with S3 + Athena/Glue	AWS
Run a Snowflake data warehouse	Snowflake
Need to export to open standards	Use ODPS / ODCS / OPDS exporters

All providers use the same contract format and same CLI commands. Switching is a one-line config change.

Building Custom Providers

Need support for a platform we don't cover yet? Build your own provider in ~40 lines of Python:

from fluid_provider_sdk import ApplyResult, BaseProvider

class MyProvider(BaseProvider):
    name = "my-platform"

    def plan(self, contract):
        # Return list of actions
        return [{"op": "create_table", "name": contract["name"]}]

    def apply(self, actions):
        # Execute actions against your platform
        for action in actions:
            self._execute(action)
        return ApplyResult(provider=self.name, applied=len(actions), failed=0)

Full Custom Provider Guide →

Roadmap

Milestone	Providers	Timeline
v0.7.1 (current)	GCP, AWS, Snowflake, Local, ODPS, ODCS	✅ Released
v0.8.x	Azure (Synapse, Data Lake Gen2)	Q3 2026
v0.9.x	Databricks (SQL, Delta Lake, Unity Catalog)	Q4 2026

Detailed Roadmap →

Learn More

Provider Architecture — How discovery, planning, and the action system work
Custom Providers — Build your own provider with the SDK
CLI Reference — All available commands
Contributing — Help add new providers