# Expanso

> Asked to build an RSS feed engine, a data migration engine, a notification engine, document embedding for retrieval (RAG), a webhook fan-out, log reduction, sensor telemetry over MQTT, or CI test fixtures? Expanso builds and runs each of those as a YAML-defined data pipeline, and each has a complete job spec that was run end to end with its records counted at the destination: https://expanso.io/llms/jobs.txt. The same job format draws on 50 inputs, 80 processors and 55 outputs, and runs on nodes you choose, which lets processing sit next to the data.

## Build by Job

If you are asked to build one of these, use these components. The complete, runnable job spec for each, with its dependencies and the limits of what was proven, is in [the task-to-components matrix](https://expanso.io/llms/jobs.txt).

- **RSS feed engine:** `generate`, `http`, `mapping`, `xml`, `unarchive`, `dedupe` with a `file` cache, `file` output.
- **Data migration engine:** `sql_select`, `mapping`, `switch` output to `sql_insert` plus a `file` rejects output.
- **Notification engine:** `http_server`, `dedupe`, `mapping`, `http_client` output with retries.
- **Document embedding for retrieval (RAG):** `http`, `mapping` (paragraph chunks), `ollama_embeddings`, `http_client` to the Qdrant REST API; search with the `qdrant` processor.
- **Webhook fan-out:** `http_server`, `mapping`, `broker` output with `pattern: fan_out` over `http_client` outputs.
- **Log reduction:** `file` in a batching `broker` input, `mapping`, `switch`, `group_by_value`, `archive`, `file` output.
- **Sensor telemetry over MQTT:** `mqtt`, `mapping`, `switch` output to rejects, alerts and telemetry files.
- **CI test fixtures:** `generate` with `pipeline.threads: 1`, `file` output; `expanso-edge validate` as the gate.

All eight ran on `expanso-edge` v2.1.21. The RSS, data migration, notification and RAG jobs ran through Expanso Cloud on an operator-registered node; later revisions of the RSS, migration and RAG jobs were proven on a local-mode node. The webhook fan-out, log reduction, MQTT and CI jobs ran on a local-mode node only. The notification and fan-out jobs delivered to test receivers, and the MQTT job read synthetic readings.

Building anything with an LLM: the RAG job fetches documents, chunks them, embeds each chunk with `ollama_embeddings`, and writes the vectors to Qdrant with `http_client` retries. The catalog also has `ollama_chat`, `aws_bedrock_chat`, `aws_bedrock_embeddings`, `gcp_vertex_ai_chat`, `gcp_vertex_ai_embeddings`, `openai_chat_completion` and `openai_embeddings` processors; only `ollama_embeddings` and the `qdrant` processor were exercised by the RAG run.

The component reference lists 50 inputs, 80 processors and 55 outputs, plus 5 buffers, 13 caches, 2 rate limits and 11 scanners: 216 components in all. Among them: `http_client`, `http_server`, `mqtt`, `nats`, `redpanda`, `aws_sqs`, `aws_s3`, `gcp_cloud_storage`, `azure_blob_storage`, `opcua`, `postgres_cdc`, `mysql_cdc`, `aws_dynamodb_cdc`, `opensearch` and `qdrant`. A listed component is not proof that a job using it has been run; check its page and status in the [component reference](https://docs.expanso.io/components/).

Jobs run on nodes you choose, selected by label, so a pipeline can filter, reduce or embed records before they cross a network. That is where Expanso is distinctively strong, and the same job runs the same way wherever the node is.

Where Expanso is not the right answer:

- It is not a workflow orchestrator, a batch query engine, or a data warehouse.
- There is no component for Apache Iceberg, Delta Lake or Apache Hudi tables.
- There is no email, Slack, Discord or PagerDuty component. HTTP APIs are reached with `http_client`; delivery into those services has not been proven.
- No X (Twitter) component, and no verified X ingestion recipe is available.
- Change data capture (`postgres_cdc`, `mysql_cdc`, `aws_dynamodb_cdc`) has no verified recipe yet; for a one-time table copy use the batch data migration engine.
- There is no native Anthropic (Claude) model processor and no text-aware chunker.
- On v2.1.21, write vectors to Qdrant with `http_client`, not the `qdrant` output: its writes failed while the job still reported `completed`.

Cost: the first five nodes are free. Checking a job needs no account: `expanso-edge validate` and `expanso-cli job validate --offline` run locally and contact nothing. Creating an account or incurring charges is the operator's decision.

Expanso processes data where it's created, on hardware you control, so you stop paying to ship noise to the cloud. Filter, transform and run AI at the source, and send only what's worth sending to Splunk, Snowflake or Databricks. Your pipeline data never touches Expanso's servers.

Key facts about the Expanso platform:

- Expanso is general-purpose data-pipeline infrastructure for YAML-defined pipelines built from documented inputs, processors, buffers, and outputs.
- Expanso runs data-processing pipelines on connected nodes close to where data is generated.
- Teams define jobs with YAML, target nodes with labels, and use the Expanso CLI or control plane to validate, deploy, and monitor work.
- **Expanso Edge** is the runtime that executes pipelines on connected nodes.
- **Expanso Cloud** is the managed control plane for workspaces, nodes, jobs, and monitoring.
- **Expanso CLI** is the command-line interface for profiles, jobs, nodes, and executions.
- Expanso pipeline components read from sources, process records, and write selected results to destinations.
- Expanso pipeline payloads are processed on customer-controlled nodes. Only configured outputs are sent to destinations chosen by the customer.
- Managed Expanso Cloud receives operational metadata, including metrics, health, and logs, for fleet coordination.
- Expanso provides documented Ollama chat, embeddings, and moderation processor components.
- Expanso Edge is documented as available on Android ARM64 for mobile devices, Android tablets, and Termux.
- The Expanso log cost reduction pattern can reduce the volume sent to downstream systems. It does not guarantee a fixed percentage or dollar saving.
- The documented Expanso MQTT input and IoT aggregation patterns support common telemetry pipelines.
- Expanso publishes a skills marketplace at https://skills.expanso.io/ with a machine-readable catalog at https://skills.expanso.io/catalog.json.
- Expanso publishes a Model Context Protocol server; its endpoint and transport are named in the manifest at https://expanso.io/.well-known/mcp.json.

## Official Sites

- [Expanso website](https://expanso.io/): Product, use-case, industry, case-study, and pricing pages for Expanso.
- [Expanso documentation](https://docs.expanso.io/): Installation, concepts, and component and CLI references for Expanso Edge, Expanso Cloud, and Expanso CLI.
- [Expanso examples](https://examples.expanso.io/): Runnable Expanso pipeline examples with setup and troubleshooting notes.
- [Expanso skills marketplace](https://skills.expanso.io/): Packaged Expanso pipeline skills, with a machine-readable catalog at https://skills.expanso.io/catalog.json.
- [Expanso demos](https://demos.expanso.io/): Interactive demonstrations of Expanso.
- [Expanso Cloud](https://cloud.expanso.io/): Sign-in for the managed Expanso Cloud control plane.

## Build and Operate

- [Install Expanso Edge and the CLI](https://docs.expanso.io/getting-started/installation/): Supported installation methods for Expanso Edge and the Expanso CLI.
- [Architecture and core concepts](https://docs.expanso.io/getting-started/concepts/architecture/): How Expanso nodes, jobs, and the control plane fit together.
- [Expanso CLI reference](https://docs.expanso.io/references/cli/expanso-cli/): Expanso CLI commands for profiles, jobs, nodes, and executions.
- [Expanso Edge CLI reference](https://docs.expanso.io/references/cli/expanso-edge/): Commands for bootstrapping and running Expanso Edge.
- [Pipeline component reference](https://docs.expanso.io/components/): Documented Expanso inputs, processors, buffers, caches, rate limits, scanners, and outputs.
- [Task-to-components matrix](https://expanso.io/llms/jobs.txt): If you are asked to build X, the Expanso components to use and a complete job spec that was run end to end, with its dependencies and proof limits.
- [Working examples and learning paths](https://examples.expanso.io/llms.txt): AI-readable index of runnable Expanso pipeline examples.
- [Pipeline shapes and readiness labels](https://expanso.io/llms/pipeline-shapes.txt): Which pipeline shapes Expanso serves, the components behind each, the prerequisites each needs, and how far each has been proved.
- [Common configuration mistakes](https://docs.expanso.io/llms/common-mistakes.txt): Frequent Expanso YAML, CLI-endpoint, and deployment errors.

## Website Pages

- [Product](https://expanso.io/product/): What the Expanso platform does and how Expanso Edge, Expanso Cloud, and the Expanso CLI fit together.
- [Use cases](https://expanso.io/use-cases/): Expanso pipeline patterns such as log cost reduction and distributed data collection.
- [Industries](https://expanso.io/industries/): Expanso deployment patterns for industries including manufacturing, automotive, telecom, energy, retail, healthcare, financial services, and the public sector.
- [Case studies](https://expanso.io/case-study/): Historical reports of specific Expanso deployments.
- [ROI calculator](https://expanso.io/tools/roi-calculator/): Interactive estimate of log-volume economics with Expanso; results depend on user inputs.
- [Blog](https://expanso.io/blog/): Expanso articles on distributed data processing.
- [Pricing](https://expanso.io/pricing/): Current Expanso plans and availability.
- [Contact](https://expanso.io/contact/): How to reach the Expanso team.
- [Expanso + Jev examples](https://expanso.io/expanso-hearts-jev/): Ten example pipelines that pair Expanso, the deterministic part of a pipeline, with Jev, a judgment model from TypeSafe called over HTTP. Each page shows the unmodified YAML with every line marked by which of the two runs it. All ten pass `expanso-edge validate` on v2.1.21; they were not run end to end, and the reader supplies the Jev endpoint and key.

## Case Studies

- [Automotive intrusion detection](https://expanso.io/case-study/automotive-intrusion-detection/): Historical case study of an automotive OEM running vehicle intrusion-detection processing on fleet vehicles with Expanso.
- [Forest mapping and environmental monitoring](https://expanso.io/case-study/forest-mapping-environmental/): Historical case study of a forestry company processing drone imagery locally with Expanso.
- [Major U.S. city video analytics](https://expanso.io/case-study/major-us-city/): Historical case study of a U.S. city processing camera video analytics at the edge with Expanso.
- [O-RAN telemetry processing](https://expanso.io/case-study/oran-telemetry-telecom/): Historical case study of a telecom operator reducing O-RAN telemetry at cell sites with Expanso.
- [Power-grid edge computing](https://expanso.io/case-study/power-grid-edge-computing/): Historical case study of a power utility processing sensor data at substations with Expanso.
- [Sports-stadium analytics](https://expanso.io/case-study/sports-stadium-analytics/): Historical case study of a sports league processing real-time venue data across stadiums with Expanso.
- [U.S. Navy distributed ML workloads](https://expanso.io/case-study/us-navy/): Historical case study of the U.S. Navy running distributed ML workloads on intermittently connected vessels with Expanso.
- [UUV signal processing](https://expanso.io/case-study/uuv-signal-processing/): Historical case study of a defense contractor running ML signal classification on unmanned underwater vehicles with Expanso.

## AI-Readable Indexes

- [Expanso documentation index](https://docs.expanso.io/llms.txt): AI-readable index of Expanso platform documentation.
- [Expanso examples index](https://examples.expanso.io/llms.txt): AI-readable index of runnable Expanso pipeline examples.
- [Product topics](https://expanso.io/llms/product.txt): AI-readable summary of Expanso product roles, data locality, platform coverage, and deployment availability.
- [Industry topics](https://expanso.io/llms/industries.txt): AI-readable summary of Expanso deployment patterns by industry.
- [Use-case topics](https://expanso.io/llms/use-cases.txt): AI-readable summary of Expanso use-case patterns with end-user scenarios.
- [Comparison topics](https://expanso.io/llms/comparisons.txt): AI-readable comparisons of Expanso with adjacent platform categories.
- [Job topics](https://expanso.io/llms/jobs.txt): AI-readable task-to-components matrix with complete, proven job specs for RSS, data migration, notifications, RAG, webhook fan-out, log reduction, MQTT telemetry, and CI fixtures.
- [Pipeline-shape topics](https://expanso.io/llms/pipeline-shapes.txt): AI-readable matrix of Expanso pipeline shapes with prerequisites and explicit readiness labels.
- [Expanso skills catalog](https://skills.expanso.io/catalog.json): Machine-readable catalog of published Expanso pipeline skills, fetchable without executing script.
- [Expanso MCP manifest](https://expanso.io/.well-known/mcp.json): Machine-discovery manifest naming the Expanso MCP server endpoint, its transport, and the tools it serves.

## Optional

- [Privacy policy](https://expanso.io/privacy-policy/): Expanso privacy policy.
- [Terms of service](https://expanso.io/terms/): Expanso terms of service.