Fluid ForgeFluid Forge
Home
Get Started
  • Local (DuckDB)
  • GCP (BigQuery)
  • Snowflake Team Collaboration
  • Declarative Airflow
  • Orchestration Export
  • Jenkins CI/CD
  • Universal Pipeline
CLI Reference
  • Overview
  • Architecture
  • GCP (BigQuery)
  • AWS (S3 + Athena)
  • Snowflake
  • Local (DuckDB)
  • Custom Providers
  • Roadmap
GitHub
GitHub
Home
Get Started
  • Local (DuckDB)
  • GCP (BigQuery)
  • Snowflake Team Collaboration
  • Declarative Airflow
  • Orchestration Export
  • Jenkins CI/CD
  • Universal Pipeline
CLI Reference
  • Overview
  • Architecture
  • GCP (BigQuery)
  • AWS (S3 + Athena)
  • Snowflake
  • Local (DuckDB)
  • Custom Providers
  • Roadmap
GitHub
GitHub
  • Introduction

    • /
    • Getting Started
    • Snowflake Quickstart
    • Vision & Roadmap
  • Walkthroughs

    • Walkthrough: Local Development
    • Walkthrough: Deploy to Google Cloud Platform
    • Walkthrough: Snowflake Team Collaboration
    • Declarative Airflow DAG Generation - The FLUID Way
    • Generating Orchestration Code from Contracts
    • Jenkins CI/CD for FLUID Data Products
    • Universal Pipeline
  • CLI Reference

    • CLI Reference
    • init Command
    • validate Command
    • plan Command
    • apply Command
    • verify Command
    • generate-airflow Command
  • Providers

    • Providers
    • Provider Architecture
    • GCP Provider
    • AWS Provider
    • Snowflake Provider
    • Local Provider
    • Creating Custom Providers
    • Provider Roadmap
  • Advanced

    • Blueprints
    • Governance & Compliance
    • Airflow Integration
    • Built-in And Custom Forge Agents
    • FLUID Forge Contract GPT Packet
    • Forge Copilot Discovery Guide
    • Forge Copilot Memory Guide
  • Project

    • Contributing to Fluid Forge
    • Fluid Forge v0.7.1 - Multi-Provider Export Release

CLI Reference

Fluid Forge ships with 45 top-level commands organized into functional categories. Run fluid --help for the full list or fluid <command> -h for per-command details.

Tips

fluid --help              # See all commands
fluid <command> -h        # Help for a specific command
fluid doctor              # Check your system is ready

Cheat Sheet

The commands you'll use most often:

TaskCommand
Create projectfluid init my-project --quickstart
Validate contractfluid validate contract.fluid.yaml
Preview changesfluid plan contract.fluid.yaml
Deploy / runfluid apply contract.fluid.yaml --yes
Verify deploymentfluid verify contract.fluid.yaml
Publish to DataMesh Managerfluid datamesh-manager publish contract.fluid.yaml
Run build jobsfluid execute contract.fluid.yaml
Detect driftfluid diff contract.fluid.yaml
Visualize lineagefluid viz-graph contract.fluid.yaml
Generate Airflow DAGfluid generate-airflow contract.fluid.yaml -o dag.py
Check governancefluid policy-check contract.fluid.yaml
AI-guided creationfluid forge --mode copilot
System diagnosticsfluid doctor

fluid forge

fluid forge --mode copilot is the primary AI-backed project creation flow.

What it does:

  1. starts with a lightweight interview and asks only the follow-up questions needed for the current scenario
  2. accepts natural-language answers, short phrases, numbers, and fuzzy wording instead of forcing exact option matches
  3. discovers local metadata from SQL, dbt, Terraform, README files, existing FLUID contracts, and sample files
  4. optionally loads project-scoped memory from runtime/.state/copilot-memory.json
  5. asks the selected LLM to generate a full FLUID contract, README, and any additional scaffold files
  6. validates and repairs the contract locally, retrying up to 3 generation attempts
  7. in interactive mode, can ask one final clarification round if generation failed because intent was still ambiguous
  8. scaffolds the project only if validation passes, then shows official fluid validate, fluid plan, and fluid apply next steps

Common use-case categories in the interactive flow are:

  • Analytics & BI
  • ETL / Data Pipelines
  • Streaming / Real-time
  • ML / Feature Engineering
  • Data Platform / Lakehouse
  • Other / Not sure

You can still answer with your own wording such as CDC sync, executive scorecards, or customer 360.

Built-in domain-agent mode is also available through fluid forge --mode agent. The current built-in agents are:

  • finance for regulated analytics, fraud, trading, and compliance-heavy data products
  • healthcare for HIPAA-aware analytics, PHI handling, and clinical workflows
  • retail for customer 360, personalization, demand, and inventory use cases
  • telco for TM Forum SID-aligned OSS/BSS, service assurance, and network-operations analytics

These built-in agents are backed by declarative YAML specs, so they share the same Forge scaffolding engine while applying domain-specific questions, defaults, rules, and next-step guidance.

Key copilot flags: --llm-provider, --llm-model, --discovery-path, --context, --memory / --no-memory, --interactive / --non-interactive, --dry-run. See the full flag table in the fluid forge command reference below.

Typical examples:

export OPENAI_API_KEY=sk-...
fluid forge
fluid forge --mode copilot \
  --context ./copilot-context.json \
  --discovery-path ./data \
  --llm-provider openai \
  --llm-model gpt-4o-mini
fluid forge --mode copilot \
  --non-interactive \
  --llm-provider openai \
  --llm-model gpt-4o-mini \
  --save-memory
fluid forge --show-memory
fluid forge --reset-memory
fluid forge --mode agent --agent telco
fluid forge --mode agent --agent finance

Step-by-step guides:

  • Forge Copilot Discovery Guide →
  • Forge Copilot Memory Guide →

Core Workflow

These four commands form the standard development loop: init → validate → plan → apply.

fluid init

Create a new Fluid Forge project with sample data and a working contract. If the target directory does not exist yet, fluid init creates it before copying the selected template.

fluid init <name> [options]
OptionDescription
--quickstartCreate a working example with sample data (recommended)
--scanImport an existing dbt / Terraform project
--wizardInteractive guided setup
--blankEmpty project skeleton
--provider <name>Target provider: local (default), gcp, aws, snowflake
--template <name>Start from a named template (e.g. customer-360)
--no-runDon't auto-execute the pipeline after creation
--dry-runPreview what would be created
fluid init my-project --quickstart           # Quick start (recommended)
fluid init --scan                            # Import existing dbt project
fluid init analytics --provider gcp --quickstart  # GCP-targeted project

Full init docs →


fluid validate

Check a contract for schema correctness, required fields, provider compatibility, and SQL syntax.

fluid validate <contract-file> [options]
OptionDescription
--env <name>Apply an environment overlay before validating
--verboseShow detailed validation output
fluid validate contract.fluid.yaml
# ✅ Contract validation passed!

Full validation docs →


fluid plan

Generate an execution plan showing what resources will be created, modified, or removed — without executing anything. Pure planning, no side effects.

fluid plan <contract-file> [options]
OptionDescription
--env <name>Environment overlay
--out <file>Save the plan to a JSON file
--provider <name>Override provider
fluid plan contract.fluid.yaml --env staging

Planning details →


fluid apply

Execute the contract: load data, run transformations, create cloud infrastructure.

fluid apply <contract-file> [options]
OptionDescription
--env <name>Environment overlay
--yesSkip confirmation prompt
--provider <name>Override provider
--dry-runShow what would happen without executing
fluid apply contract.fluid.yaml --yes                     # Local
fluid apply contract.fluid.yaml --provider gcp --env prod  # Cloud

Full apply docs →


fluid execute

Run build jobs defined in the contract (manual or scheduled triggers).

fluid execute <contract-file> [--env <name>]

fluid verify

Verify that deployed resources match the contract definition.

fluid verify <contract-file> [options]
OptionDescription
--provider <name>Target provider
--strictFail on any mismatch
fluid verify contract.fluid.yaml --provider gcp
# ✅ All resources match contract definition

Verify docs →


Code Generation & Export

fluid generate-airflow

Generate a production-ready Airflow DAG from a contract. Supports GCP, AWS, and Snowflake.

fluid generate-airflow <contract-file> -o <output-file>
OptionDescription
-o, --output <file>Output file path
--env <name>Environment overlay
--verboseDetailed generation logs
fluid generate-airflow contract.fluid.yaml -o dags/pipeline.py

Airflow generation guide →

fluid export

Universal export engine — generate Airflow, Dagster, or Prefect code.

fluid export <contract-file> --format <engine> -o <output-file>
FormatEngine
airflowApache Airflow DAGs
dagsterDagster pipelines with type-safe ops
prefectPrefect flows with retry logic

fluid export-opds

Export a contract to OPDS (Open Data Product Specification) format.

fluid export-opds <contract-file> -o output.yaml

fluid odps

Export to ODPS v4.1 (Linux Foundation Open Data Product Specification).

fluid odps export <contract-file> [options]
fluid odps validate <odps-file> [options]
fluid odps info

ODPS export metadata comes from the contract itself: provider/type fields are derived from binding.platform, expose kind falls back from kind to type, and product metadata uses the documented exporter precedence in the provider guide.

fluid odcs

Export to ODCS v3.1 (Open Data Contract Standard — Bitol.io).

fluid odcs <contract-file> [options]

fluid datamesh-manager

Publish a data product to Entropy Data / DataMesh Manager.

fluid datamesh-manager publish <contract-file> [options]
fluid dmm publish <contract-file> [options]

DataMesh Manager publish uses the same contract-driven metadata rules as the providers: binding.platform is preferred over legacy provider keys, expose kind falls back from kind to type, and tags merge top-level tags with metadata.tags.


Visualization

fluid viz-graph

Generate data lineage and dependency graphs.

fluid viz-graph <contract-file> [--out lineage.png]

Supports .dot, .png, and .svg output formats.

fluid viz-plan

Interactive visualization of an execution plan.

fluid viz-plan <contract-file>

Governance & Compliance

fluid policy-check

Run governance and compliance checks against a contract.

fluid policy-check <contract-file> [options]
OptionDescription
--strictTreat warnings as errors
--category <name>Filter: sensitivity, access_control, data_quality, lifecycle
--formatrich, text, or json
--show-passedInclude passed checks in output

fluid policy-compile

Compile accessPolicy declarations to provider-native IAM bindings.

fluid policy-compile <contract-file> --out runtime/policy/bindings.json

fluid policy-apply

Enforce compiled governance rules on deployed resources.

fluid policy-apply runtime/policy/bindings.json --mode check    # Dry-run
fluid policy-apply runtime/policy/bindings.json --mode enforce  # Apply

fluid diff

Detect configuration drift between the contract and deployed state.

fluid diff <contract-file> --provider gcp --exit-on-drift
# Exit code 0 = no drift, 1 = drift detected (ideal for CI/CD gates)

Scaffolding & AI

fluid forge

LLM-backed project generation and scaffolding.

fluid forge [options]

fluid forge --mode copilot now runs an adaptive flow:

  1. Bootstrap the run from CLI flags, --context, local discovery, and optional project memory
  2. Ask only the minimum follow-up questions needed for the current scenario
  3. Accept free-text answers with soft matching for suggested options
  4. Generate a full production-ready contract.fluid.yaml
  5. Validate and repair the contract up to 3 times
  6. If interactive generation failed because intent was still ambiguous, ask one more clarification round and retry once
  7. Scaffold the project only after the contract passes validation
  8. If built-in provider inspection or provider setup is incomplete, surface a warning and continue so you can review the provider later
OptionDescription
--mode <name>copilot, agent, template, blueprint
--agent <name>Domain agent: finance, healthcare, retail, telco, or a registered custom agent
--template <name>Named template
--provider <name>Provider hint for the generated project
--llm-provider <name>Built-in LLM adapter: openai, anthropic (claude alias), gemini, ollama
--llm-model <name>Model identifier for the selected adapter
--llm-endpoint <url>Exact HTTP endpoint override for the selected adapter
--discover / --no-discoverEnable or disable local metadata discovery
--discovery-path <path>Extra local file or directory to scan for metadata
--context <json-or-file>Additional structured context for copilot mode
--interactive / --non-interactiveForce prompts on or off
--memory / --no-memoryEnable or disable loading repo-local copilot memory
--save-memoryPersist repo-local copilot memory after a successful non-interactive run
--show-memoryShow the current project-scoped copilot memory summary and exit
--reset-memoryDelete the current project-scoped copilot memory file and exit
--quickstartUse smart defaults
--dry-runPreview without creating files
export OPENAI_API_KEY=sk-...
fluid forge

fluid forge --mode copilot --llm-provider openai --llm-model gpt-4o-mini

fluid forge --mode copilot --llm-provider ollama \
  --llm-model llama3.1 \
  --llm-endpoint http://localhost:11434/v1/chat/completions

fluid forge --mode copilot --discovery-path ./data
fluid forge --show-memory
fluid forge --template analytics --provider gcp
fluid forge --mode agent --agent telco

Credential resolution:

  • Provider, model, and endpoint use CLI flags > FLUID_LLM_* > built-in defaults
  • API keys use FLUID_LLM_API_KEY first, then provider-specific variables such as OPENAI_API_KEY, ANTHROPIC_API_KEY, GEMINI_API_KEY, or GOOGLE_API_KEY
  • Ollama uses no API key by default

Important behavior:

  • Secrets are environment-variable only. There is no --llm-api-key flag.
  • --llm-endpoint is an exact endpoint override for proxies, self-hosted gateways, or Ollama. It is not a file-upload target.
  • --discovery-path scans local files and sends only derived metadata to the LLM, never raw sample rows or full file contents.
  • Discovery supports SQL, dbt, Terraform, README headings, existing FLUID contracts, and sample files including CSV, JSON, JSONL, Parquet, and Avro.
  • --mode agent uses the same shared Forge scaffolding path as copilot mode, but seeds it with domain-specific declarative rules and next-step guidance.
  • Interactive prompts are friendly by default: suggested options are hints, not strict menus.
  • Copilot keeps an app-managed interview summary and transcript so provider behavior stays consistent across OpenAI, Anthropic, Gemini, and Ollama.
  • Built-in provider checks for local, gcp, aws, and snowflake are best-effort during copilot preparation. If local inspection fails, Forge warns and keeps going with safe defaults instead of aborting.
  • If copilot scaffolds a cloud-oriented project before provider credentials are fully configured, Forge now shows a warning instead of failing project creation. Finish provider setup before fluid plan or fluid apply.
  • fluid doctor is the quickest way to sanity-check provider tooling and credentials after scaffolding.
  • The public scaffolding path is fluid forge --mode copilot.

Advanced LLM setup →Step-by-step discovery guide →

fluid wizard

Step-by-step guided setup for creating contracts.

fluid blueprint

Work with reusable data product blueprints.

fluid blueprint list                                # Browse blueprints
fluid blueprint list --category analytics --provider gcp  # Filter
fluid blueprint describe customer-360-gcp            # Details
fluid blueprint create customer-360-gcp -d my-project    # Create

fluid scaffold-ci

Generate CI/CD configuration for Jenkins, GitHub Actions, or GitLab CI.


Marketplace & Publishing

fluid publish

Publish a data product to a marketplace or catalog.

fluid publish <contract-file> [options]

fluid market

Browse and discover data products from the marketplace.

fluid marketplace

Extended marketplace browser with search and filtering.


Utilities

fluid doctor

Run system diagnostics: Python version, installed providers, cloud SDKs, credentials.

fluid doctor

fluid version

Display version and build information.

fluid version
# Fluid Forge CLI v0.7.7

fluid providers

List all discovered providers and their capabilities.

fluid providers

fluid auth

Manage cloud provider authentication.

fluid auth login --provider gcp
fluid auth status
fluid auth logout

fluid context

Get or set default provider, project, and region.

fluid preview

Dry-run alias for apply. Preview what would happen without executing.

fluid preview contract.fluid.yaml

Global Options

Available on every command:

OptionDescriptionEnv Variable
--provider <name>Cloud provider (local, gcp, aws, snowflake)FLUID_PROVIDER
--project <id>Cloud project or account IDFLUID_PROJECT
--region <name>Cloud region (default: europe-west3)FLUID_REGION
--log-level <level>DEBUG, INFO, WARNING, ERRORFLUID_LOG_LEVEL
--log-file <file>Write logs to fileFLUID_LOG_FILE
--no-colorDisable colored output—
--debugEnable debug mode—
--profilePerformance profiling—
--safe-modeEnhanced security validations—

Next Steps

  • Getting Started — install and run your first project
  • Local Walkthrough — develop locally with DuckDB
  • Provider Guides — GCP, AWS, Snowflake deep dives
  • Contributing — help improve Fluid Forge

Copyright 2025-2026 Agentics Transformation Pty Ltd · Open source under Apache 2.0

Edit this page on GitHub
Last Updated: 4/4/26, 8:41 PM
Contributors: khanya_ai, fas89
Next
init Command