Architecture & Core Concepts
This chapter explains the fundamental architecture of cdktr and the core concepts you need to understand to effectively use the system.
Overview
cdktr is a distributed workflow orchestration system built on a principal-agent architecture. It is designed to be lightweight and scalable, using source-controllable YAML files for workflow definitions, ZeroMQ for fast, reliable communication between components, and DuckDB for storing logs and execution history with powerful query capabilities.
What's Covered
This section explores the key components and patterns that make cdktr work:
- System Overview: The big picture—how components fit together in a distributed pull-based architecture
- Principal: The central coordinator managing workflows, scheduling, work distribution, and persistent state
- Agents: Execution nodes that run tasks in parallel using DAG-based topological sorting
- Event Listeners & Scheduler: The event-driven architecture powering cron scheduling and custom event sources
- Workflows & Tasks: YAML workflow definitions, task types (Subprocess and UvPython), and dependency execution
- Logs & Database: Real-time ZeroMQ log streaming and DuckDB persistence with insert-only audit trails
- Examples: Practical workflow examples demonstrating common patterns
Key Design Principles
- Simplicity: Single binary deployment with no external dependencies
- Performance: Built in Rust for speed and safety
- Distributed: Scale horizontally by adding more agents
- Observable: TUI for real-time monitoring without a web UI
- Flexible: ZeroMQ API allows integration with any language
Let's dive into each concept in detail.