lukas.pupkalipinski/UltimateAgents
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs

Browse Source
This commit is contained in:
Lukas
2026-04-16 20:50:38 +02:00
commit 4b8aed50d5
6 changed files with 1591 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

577
docs/Architecture.md Normal file
View File

@@ -0,0 +1,577 @@
# Architecture.md — Orchestration Service
**Version:** 1.0
**Date:** 2026-04-07
**Author:** Senior Software Architect
**Stack:** C# / .NET 9 / ASP.NET Core
**Eventing:** In-process C# events (no external message broker)
---
## 1. Overview
The **Orchestration Service** is a distributed, event-driven system that accepts task definitions, schedules and dispatches AI agents into isolated container runtimes, tracks execution progress, enforces complexity rules, and escalates on failure. It is the central nervous system for multi-agent workflows.
```
┌──────────────────────────────────────────────────────────┐
│ CLI Frontend (C#) │
│ orchestrate task create / start / status │
└─────────────────────────┬────────────────────────────────┘
│ HTTP/REST + gRPC
┌─────────────────────────▼────────────────────────────────┐
│ Orchestration API (ASP.NET Core) │
│ TaskController │ AgentController │ ContainerController │
└──────┬───────────┬──────────────────────┬────────────────┘
│ │ │
┌──────▼──┐ ┌─────▼──────┐ ┌───────────▼──────────────┐
│Scheduler│ │ Dispatcher │ │ Container Manager │
│ Service │ │ Service │ │ Isolated runtime adapter │
└──────┬──┘ └─────┬──────┘ └───────────┬──────────────┘
│ │ │
┌──────▼───────────▼──────────────────────▼──────────────┐
│ Domain Layer (C# Domain Models) │
│ Task │ Agent │ Schedule │ Complexity │ Escalation │
└────────────────────────┬───────────────────────────────┘
│
┌────────────────────────▼───────────────────────────────┐
│ Persistence Layer │
│ SQLite (Dapper) │ No external cache │
└────────────────────────────────────────────────────────┘
```
---
## 2. Solution Structure
```
UltimateAgents/
├── src/
│ ├── UltimateAgents.Api/ # ASP.NET Core Web API host
│ │ ├── Controllers/
│ │ │ ├── TasksController.cs
│ │ │ ├── AgentsController.cs
│ │ │ ├── ContainersController.cs
│ │ │ ├── ToolsController.cs
│ │ │ └── SchedulesController.cs
│ │ ├── Program.cs
│ │ └── appsettings.json
│ │
│ ├── UltimateAgents.Domain/ # Pure domain models, no framework deps
│ │ ├── Entities/
│ │ │ ├── AgentTask.cs # base task
│ │ │ ├── ForeachTask.cs # specialised: foreach iteration
│ │ │ ├── GotoTask.cs # specialised: jump with merged prompt
│ │ │ ├── ConditionTask.cs # specialised: if/else branch
│ │ │ ├── TaskGroup.cs # sequential or parallel group of tasks
│ │ │ ├── Agent.cs
│ │ │ ├── RunningContainer.cs
│ │ │ ├── Tool.cs
│ │ │ └── Schedule.cs
│ │ ├── Enums/
│ │ │ ├── ComplexityLevel.cs
│ │ │ ├── TaskTag.cs
│ │ │ ├── TaskType.cs # Standard | Foreach | Goto | Condition
│ │ │ ├── GroupType.cs # Sequential | Parallel
│ │ │ └── ScheduleType.cs
│ │ ├── ValueObjects/
│ │ │ ├── Escalation.cs
│ │ │ └── RetryPolicy.cs
│ │ └── Exceptions/
│ │ ├── DependencyCycleException.cs
│ │ └── TaskNotFoundException.cs
│ │
│ ├── UltimateAgents.Application/ # Use cases / application services
│ │ ├── Tasks/
│ │ │ ├── CreateTaskCommand.cs
│ │ │ ├── StartTaskCommand.cs
│ │ │ ├── GetTaskStatusQuery.cs
│ │ │ └── TaskComplexityEvaluator.cs
│ │ ├── Agents/
│ │ │ ├── AssignAgentCommand.cs
│ │ │ └── AgentMatchingService.cs
│ │ ├── Scheduling/
│ │ │ ├── SchedulerService.cs
│ │ ├── Events/
│ │ │ ├── IDomainEvent.cs
│ │ │ ├── TaskCreatedEvent.cs
│ │ │ ├── TaskStartedEvent.cs
│ │ │ ├── TaskCompletedEvent.cs
│ │ │ ├── TaskFailedEvent.cs
│ │ │ ├── EscalationRaisedEvent.cs
│ │ │ └── AllAgentsFinishedEvent.cs
│ │ └── Orchestration/
│ │ ├── OrchestratorEngine.cs
│ │ │ ├── DependencyGraph.cs
│ │ │ └── EscalationHandler.cs
│ │ └── Interfaces/
│ │ ├── ITaskRepository.cs
│ │ ├── IAgentRepository.cs
│ │ ├── IContainerRuntime.cs
│ │ └── IEventDispatcher.cs
│ │
│ ├── UltimateAgents.Infrastructure/ # Dapper, runtime adapter, events
│ │ ├── Persistence/
│ │ │ ├── DatabaseSchema.sql
│ │ │ ├── TaskRepository.cs
│ │ │ ├── TaskGroupRepository.cs
│ │ │ └── AgentRepository.cs
│ │ ├── Runtime/
│ │ │ ├── LocalRuntimeAdapter.cs
│ │ │ └── RuntimeAdapter.cs
│ │ └── Events/
│ │ └── DomainEventDispatcher.cs # in-process C# event dispatcher
│ │
│ └── UltimateAgents.Cli/ # CLI frontend (System.CommandLine)
│ ├── Commands/
│ │ ├── TaskCommands.cs
│ │ ├── AgentCommands.cs
│ │ └── ContainerCommands.cs
│ └── Program.cs
│
└── tests/
├── UltimateAgents.Domain.Tests/
├── UltimateAgents.Application.Tests/
└── UltimateAgents.Api.Tests/
```
---
## 3. Domain Model
### 3.1 Core Entities
#### AgentTask (base)
```csharp
public sealed class AgentTask
{
public Guid Id { get; private set; }
public string Name { get; private set; }
public string Prompt { get; private set; }
public string ContainerImage { get; private set; }
public int? ResolveTimeEstimate { get; private set; } // minutes; null = ExtremHard
public ComplexityLevel ComplexityLevel { get; private set; }
public IReadOnlyList<TaskTag> TaskTags { get; private set; }
public Schedule Schedule { get; private set; }
public Escalation? Escalation { get; private set; }
public Dictionary<string, string> Metadata { get; private set; }
public RetryPolicy RetryPolicy { get; private set; }
public TaskType TaskType { get; private set; } // discriminator
// Chain links
public Guid? NextTaskId { get; private set; }
public Guid? PreviousTaskId { get; private set; }
}
```
#### ForeachTask
```csharp
public sealed class ForeachTask : AgentTask
{
public IReadOnlyList<string> ForeachItems { get; private set; }
public Guid? ForeachTemplateTaskId { get; private set; }
}
```
#### GotoTask
```csharp
public sealed class GotoTask : AgentTask
{
public Guid GotoTargetTaskId { get; private set; }
public string? GotoPrompt { get; private set; }
}
```
#### ConditionTask
```csharp
public sealed class ConditionTask : AgentTask
{
public string Condition { get; private set; }
public Guid? TrueNextTaskId { get; private set; }
public Guid? FalseNextTaskId { get; private set; }
}
```
#### TaskGroup
```csharp
/// <summary>A chain participant that owns children (tasks or nested groups) and
/// completes when all its children finish, then activates <see cref="Next"/>.</summary>
public sealed class TaskGroup
{
public Guid Id { get; private set; }
public string Name { get; private set; }
public GroupType Type { get; private set; } // Sequential | Parallel
public IReadOnlyList<GroupChildRef> Children { get; private set; } // ordered
public GroupChildRef? Next { get; private set; } // task or group to activate after completion
public GroupChildRef? Previous { get; private set; } // predecessor in the chain
}
```
#### Agent
```csharp
public sealed class Agent
{
public Guid Id { get; private set; }
public IReadOnlyList<string> Skills { get; private set; }
public IReadOnlyList<TaskTag> TaskTags { get; private set; }
public AgentMatchRule ChooseRule { get; private set; }
public AgentCapabilities Capabilities { get; private set; }
}
```
### 3.2 Enumerations
```csharp
public enum ComplexityLevel { Low, Middle, Hard, ExtremHard }
public enum TaskTag { DataIngestion, Nlp, ImageProcessing,
Deployment, Security, Testing, Documentation }
public enum TaskType { Standard, Foreach, Goto, Condition }
public enum GroupType { Sequential, Parallel }
public enum GroupChildKind { Task, Group } // discriminates GroupChildRef
public enum ScheduleType { Immediate, TimeBased, DatetimeEvent }
```
### 3.3 Value Objects
```csharp
/// <summary>Points to either a task or a nested group inside a TaskGroup.</summary>
public sealed record GroupChildRef(Guid ChildId, GroupChildKind Kind);
public sealed record Escalation(string Reason, string Prompt);
public sealed record RetryPolicy(int Attempts, BackoffStrategy Backoff, int MaxRetries);
```
### 3.4 Complexity Rules (Domain Service)
```csharp
public static class ComplexityEvaluator
{
public static ComplexityLevel Evaluate(int? resolveTimeEstimate) =>
resolveTimeEstimate switch
{
null => ComplexityLevel.ExtremHard,
> 60 => ComplexityLevel.Hard,
> 30 => ComplexityLevel.Middle,
_ => ComplexityLevel.Low
};
}
```
---
## 4. Application Layer
### 4.1 Orchestrator Engine
The `OrchestratorEngine` is the central coordinator. It operates as a hosted background service (`IHostedService`) and drives the task lifecycle.
```
Task Created
│
▼
[ValidateDependencyGraph] ──cycle?──► Reject (DependencyCycleException)
│
▼
[EvaluateComplexity]
│
▼
[SchedulerService] ──── immediate ──► [Dispatcher]
──── time/event ──► enqueue in scheduler
──── event ──► process with event handlers
│
▼
[Dispatcher] ── match Agent via AgentMatchingService
│
▼
[ContainerRuntime.StartAsync(task, agent)]
│
├── success ──► advance to NextTask or GotoTask
│
└── failure ──► [EscalationHandler]
│
├── retries remaining ──► re-dispatch
└── exhausted ──────────► escalate to operator
```
### 4.2 Chain Validation
Both tasks and groups participate in one unified directed graph. Cycle detection runs at task/group creation time over this unified graph. Any cycle results in a `DependencyCycleException`.
```csharp
public sealed class DependencyGraph
{
// Register a node (task OR group) with its optional successor.
// nextRef == null means the node has no successor.
public void AddNode(Guid id, GroupChildRef? nextRef) { ... }
// Register that a group contains a child (task or nested group).
public void AddChild(Guid groupId, GroupChildRef child) { ... }
public bool HasCycle(out IReadOnlyList<Guid> cyclicIds) { ... }
}
```
### 4.3 Agent Matching Service
```csharp
public interface IAgentMatchingService
{
Agent? FindBestMatch(IEnumerable<Agent> agents, Task task);
}
```
Matching strategies (Strategy pattern):
- `ExactSingleTagStrategy` — agent must have exactly the one task tag.
- `SupersetTagStrategy` — agent's tags must be a superset of task tags.
- `AllRequiredTagsStrategy` — agent must cover every tag in the task.
### 4.4 Scheduler Service
```csharp
public class SchedulerService : BackgroundService
{
// Polls DatetimeEvent and TimeBased tasks
// Dispatches regular task events when schedules are met
}
```
Event types include:
- `AllAgentsFinishedSuccessfully`
- `AllAgentsFinished`
- `OneOrMoreAgentsFailed`
---
## 5. API Layer (ASP.NET Core)
### 5.1 REST Endpoints
| Method | Route | Description |
|--------|--------------------------------|-------------------------------------|
| POST | /api/tasks | Create a task |
| GET | /api/tasks/{id} | Get task details |
| GET | /api/tasks/{id}/status | Get execution status + logs |
| POST | /api/tasks/{id}/start | Manually trigger a task |
| DELETE | /api/tasks/{id} | Delete a task |
| POST | /api/agents | Register an agent |
| GET | /api/agents | List all agents |
| POST | /api/containers | Define a container spec |
| POST | /api/groups | Create a task group |
| GET | /api/groups/{id} | Get group details |
| DELETE | /api/groups/{id} | Delete a group |
| POST | /api/events | Fire a named event |
### 5.2 Request / Response Models (DTOs)
All DTOs are located in `UltimateAgents.Api/Models/`. Input models are validated with **FluentValidation**.
```csharp
public sealed record CreateTaskRequest(
string Name,
string Prompt,
string ContainerImage,
int? ResolveTimeEstimate,
List<string> TaskTags,
ScheduleRequest Schedule,
string TaskType, // "Standard" | "Foreach" | "Goto" | "Condition"
EscalationRequest? Escalation,
Dictionary<string, string>? Metadata,
// ForeachTask fields (when TaskType = "Foreach")
List<string>? ForeachItems,
Guid? ForeachTemplateTaskId,
// GotoTask fields (when TaskType = "Goto")
Guid? GotoTargetTaskId,
string? GotoPrompt,
// ConditionTask fields (when TaskType = "Condition")
string? Condition,
Guid? TrueNextTaskId,
Guid? FalseNextTaskId);
```
### 5.3 Error Handling
Global exception middleware maps domain exceptions to HTTP status codes:
| Exception | HTTP Status |
|-----------------------------|-------------|
| `TaskNotFoundException` | 404 |
| `DependencyCycleException` | 422 |
| `ValidationException` | 400 |
| Unhandled | 500 |
---
## 6. Infrastructure Layer
### 6.1 Persistence — Dapper + SQLite
- `DatabaseSchema.sql` defines the SQLite schema for tasks, agents, containers, tools, and schedules.
- `Dapper` is used for lightweight mapping and raw SQL queries.
- The database is a local file store (`Data Source=ultimateagents.db`) and is the source of truth for task state.
- Schema updates are managed via versioned SQL scripts rather than EF Core migrations.
### 6.2 Runtime Adapter Abstraction
```csharp
public interface IContainerRuntime
{
Task<ContainerHandle> StartAsync(ContainerSpec spec, CancellationToken ct);
Task StopAsync(ContainerHandle handle, CancellationToken ct);
Task<ContainerStatus> GetStatusAsync(ContainerHandle handle, CancellationToken ct);
}
```
Implementations:
- `LocalRuntimeAdapter` — executes tasks in a local isolated runtime sandbox.
- `RuntimeAdapter` — generic runtime adapter interface for future platform-specific implementations.
Runtime is selected via `appsettings.json` → `"RuntimeAdapter": "LocalSandbox"` and registered with DI.
### 6.3 Events
All eventing is in-process using C# delegates. No external message broker is required.
```csharp
/// <summary>Marker interface for all domain events.</summary>
public interface IDomainEvent { }
/// <summary>In-process dispatcher backed by a thread-safe delegate registry.</summary>
public interface IEventDispatcher
{
/// <summary>Raise an event synchronously to all registered handlers.</summary>
void Raise<TEvent>(TEvent @event) where TEvent : IDomainEvent;
/// <summary>Register a handler for a specific event type.</summary>
void Subscribe<TEvent>(Action<TEvent> handler) where TEvent : IDomainEvent;
/// <summary>Remove a previously registered handler.</summary>
void Unsubscribe<TEvent>(Action<TEvent> handler) where TEvent : IDomainEvent;
}
```
`IEventDispatcher` is defined in `Application/Interfaces/`. The sole implementation, `DomainEventDispatcher`, lives in `Infrastructure/Events/` and uses a `ConcurrentDictionary<Type, List<Delegate>>` as its handler registry.
**Domain event types** (in `Application/Events/`):
| Event | Raised when |
|--------------------------|--------------------------------------------------|
| `TaskCreatedEvent` | A new task is persisted. |
| `TaskStartedEvent` | A container runtime starts the task. |
| `TaskCompletedEvent` | An agent reports success. |
| `TaskFailedEvent` | An agent reports failure. |
| `EscalationRaisedEvent` | Retry limit is exhausted; operator must act. |
| `AllAgentsFinishedEvent` | Every agent in a group has finished. |
---
## 7. CLI Frontend
Built with **System.CommandLine** (.NET).
```
orchestrate
├── task
│ ├── create --name --prompt-file --container --tags --schedule
│ ├── start --id
│ ├── status --id [--follow]
│ ├── list
│ └── delete --id
├── agent
│ ├── create --name --task-tags --skills --match-rule
│ ├── list
│ └── delete --id
├── container
│ ├── define --image --tools --resources
│ └── list
└── schedule
└── trigger --event-name
```
The CLI calls the Orchestration REST API. API base URL is configured via `~/.orchestrate/config.json` or `--api-url` flag.
---
## 8. Cross-Cutting Concerns
### 8.1 Logging
- **Serilog** with structured JSON output.
- All task state transitions, agent assignments, escalations, and container events are logged at `Information` or `Warning` level.
- Required fields per log entry: `timestamp`, `taskId`, `agentId`, `containerId`, `event`, `errorContext`.
### 8.2 Security
- API secured with **JWT Bearer** tokens.
- Runtime isolation is enforced by the local sandbox adapter.
- Agents are strictly isolated; no cross-runtime resource access.
- Input validation via FluentValidation on all API request models.
- SQL injection prevented by Dapper parameterized queries.
### 8.3 Observability
- `/health` endpoint via `AspNetCore.HealthChecks`.
- Basic operational metrics are emitted through structured logs.
- No Prometheus or OpenTelemetry integration is required for this version.
### 8.4 Configuration
```json
{
"Database": { "ConnectionString": "Data Source=ultimateagents.db" },
"RuntimeAdapter": "LocalSandbox",
"Jwt": { "Authority": "...", "Audience": "orchestration-api" }
}
```
---
## 9. Deployment
```
┌─────────────┐ ┌─────────────────┐ ┌────────────┐
│ CLI Client │──►│ Orchestration │──►│ SQLite │
└─────────────┘ │ API (Pod/s) │ └────────────┘
└────────┬────────┘
│
▼
┌───────────┐
│ Local │
│ Runtime │
└───────────┘
```
- The application runs as a local service with a SQLite file database.
- Horizontal scaling is not required for this version; task coordination is managed via the local runtime and database.
- Secrets are managed using environment variables or local configuration.
---
## 10. Technology Stack Summary
| Concern | Technology |
|----------------------|-------------------------------------|
| Language | C# 13 / .NET 9 |
| Web Framework | ASP.NET Core 9 (Minimal API + MVC) |
| ORM | Dapper |
| Database | SQLite |
| Cache / State | None |
| Messaging | In-process C# events (no broker) |
| Container Runtime | Local runtime adapter |
| CLI | System.CommandLine |
| Logging | Serilog |
| Validation | FluentValidation |
| Auth | ASP.NET Core JWT Bearer |
| Observability | Structured logs + health checks |
| Testing | xUnit + Moq |
---
## 11. Key Design Decisions
| Decision | Rationale |
|---|---|
| Specialised task types (ForeachTask, GotoTask, ConditionTask) are distinct entities | Each type has different data; discriminated storage keeps the schema clean and the domain explicit |
| `TaskGroup` is a chain participant with typed children (`GroupChildRef`) | Children can be any mix of tasks and nested groups; the group waits for all children then advances via its own `Next` pointer, enabling arbitrary nesting without a separate chaining model |
| `GroupChildKind` discriminator on `GroupChildRef` | Allows a single `next_id` pointer to resolve to either a task or a group at the DB level without ambiguity |
| Unified `DependencyGraph` over tasks and groups | Both tasks and groups are nodes; `AddNode` + `AddChild` captures all edges; one cycle-detection pass covers the whole execution graph |
| Complexity evaluated automatically from `resolve_time_estimate` | Consistent rule enforcement; no manual classification errors |
| Container runtime behind interface | Portable between local runtime adapter implementations without external container dependencies |
| In-process C# events via `IEventDispatcher` | Zero external dependencies; handlers are registered at startup and invoked synchronously in-process; simple to test with mock subscriptions |
| Serilog structured logging with mandatory correlation fields | Enables log aggregation and alerting per task/agent/container |

381
docs/Backend-Developer.md Normal file
View File

@@ -0,0 +1,381 @@
# Backend-Developer.md — Developer Notes
**Project:** UltimateAgents — Orchestration Service
**Stack:** C# 13 / .NET 9 / ASP.NET Core / SQLite / Dapper
**Last Updated:** 2026-04-07
> These notes are for developers working on the backend. Read [Architecture.md](Architecture.md) first for the full system design. This document focuses on practical day-to-day guidance: how to set up, where things live, what rules to follow, and common pitfalls.
---
## Table of Contents
1. [Getting Started](#1-getting-started)
2. [Project Layout Cheat Sheet](#2-project-layout-cheat-sheet)
3. [Domain Rules — Must Know](#3-domain-rules--must-know)
4. [Adding a New Feature — Step-by-Step](#4-adding-a-new-feature--step-by-step)
5. [Database & Migrations](#5-database--migrations)
6. [Working with the Orchestrator Engine](#6-working-with-the-orchestrator-engine)
7. [Agent Matching](#7-agent-matching)
8. [Scheduling & Events](#8-scheduling--events)
9. [Container Runtime](#9-container-runtime)
10. [Error Handling & Escalation](#10-error-handling--escalation)
11. [Testing Guidelines](#11-testing-guidelines)
12. [Logging Conventions](#12-logging-conventions)
13. [Common Pitfalls](#13-common-pitfalls)
14. [Environment Variables & Config](#14-environment-variables--config)
15. [Useful Commands](#15-useful-commands)
---
## 1. Getting Started
### Prerequisites
- .NET 9 SDK (`dotnet --version` should show `9.x.x`)
- `dotnet restore`
```bash
# Restore packages
dotnet restore
# Run the API
dotnet run --project src/UltimateAgents.Api
# Run the CLI
dotnet run --project src/UltimateAgents.Cli -- task list
```
The service uses a local SQLite database and in-process C# events. No external message broker or Docker is required for local development.
---
## 2. Project Layout Cheat Sheet
```
src/
├── UltimateAgents.Domain/ ← Pure C# models, no framework dependencies
├── UltimateAgents.Application/ ← Use cases, commands, queries, interfaces
├── UltimateAgents.Infrastructure/← Dapper, runtime adapter, RabbitMQ
├── UltimateAgents.Api/ ← ASP.NET Core host, controllers, DTOs
└── UltimateAgents.Cli/ ← System.CommandLine frontend
```
**Dependency direction (strict — never reverse):**
```
Api → Application → Domain
Infrastructure → Application → Domain
Cli → Api (via HTTP client)
```
- `Domain` has **zero** NuGet dependencies.
- `Application` depends only on `Domain` and its own interfaces (defined in `Application/Interfaces/`).
- `Infrastructure` implements those interfaces. It is the only project that touches Dapper, the runtime adapter, or RabbitMQ.
- Controllers in `Api` call Application services only — never repositories directly.
---
## 3. Domain Rules — Must Know
These rules are enforced in the domain layer. Do not bypass them in controllers or services.
### Complexity is always auto-evaluated
```csharp
// Always use the evaluator — never set ComplexityLevel manually in external code
var level = ComplexityEvaluator.Evaluate(task.ResolveTimeEstimate);
```
| `ResolveTimeEstimate` | Result |
|-----------------------|---------------|
| `null` | `ExtremHard` |
| `> 60` | `Hard` |
| `> 30` | `Middle` |
| `≤ 30` | `Low` |
### Tasks are always flat — no parent/subtask nesting
Use `next_task` and `TaskGroup` to compose tasks. Never introduce a “parent task” concept.
- `next_task` / `previous_task` express ordering within or across groups.
- `ForeachTask` expansion creates independent task instances linked by `next_task` / `previous_task`, placed into a sequential group.
- `GotoTask` jumps to an arbitrary target (task or group) and merges an additional prompt into it.
- `ConditionTask` evaluates a boolean expression and routes to a true or false branch.
### Chain cycles are rejected at write time
`DependencyGraph.HasCycle()` is called inside `CreateTaskCommand` and `CreateGroupCommand` **before** the task or group is persisted. It validates the entire chain — task `NextTaskId` links, group `Next` pointers, and parent‚child relationships — as one unified directed graph. If a cycle is detected, throw `DependencyCycleException` — the API maps this to HTTP 422.
### Agent isolation is absolute
Agents run inside their assigned container. The domain model enforces that an agent cannot reference tools or volumes outside its declared container spec. Never pass cross-container paths in task prompts.
---
## 4. Adding a New Feature — Step-by-Step
Follow this checklist to keep the architecture clean.
1. **Domain first.** Does the feature need a new entity, value object, or enum? Add it to `UltimateAgents.Domain`. Write unit tests for any domain logic.
2. **Define the interface.** If the feature needs infrastructure (DB query, external call), add an interface to `Application/Interfaces/`.
3. **Write the use case.** Add a command or query class in `Application/`. Use a plain service class to implement the handler. No Dapper here.
4. **Implement infrastructure.** Implement the new interface in `Infrastructure/`. Use Dapper and schema scripts for persistence.
5. **Expose via API.** Add or extend a controller in `Api/Controllers/`. Add a DTO and a FluentValidation validator.
6. **Wire up DI.** Register the new service/repository in `Program.cs` or a dedicated `ServiceCollectionExtensions` class.
7. **Add CLI command** (if user-facing). Add a subcommand to the appropriate file in `Cli/Commands/`.
8. **Write tests.** Unit-test domain + application layers; integration-test the API endpoint with a local SQLite database.
---
## 5. Database & Migrations
### Database schema
- The SQLite schema is defined in `Infrastructure/Persistence/DatabaseSchema.sql`.
- Use Dapper for lightweight mapping and parameterized SQL queries.
- Many-to-many relationships are represented using explicit join tables.
- Schema updates are managed via versioned SQL scripts, not EF Core migrations.
---
## 6. Working with the Orchestrator Engine
`OrchestratorEngine` runs as an `IHostedService`. It watches for eligible tasks and drives them through their lifecycle.
### Task lifecycle states
```
Created → Queued → Running → Succeeded
↘ Failed → Retrying → Succeeded
↘ Escalated
```
### Rules for state transitions
- Only `OrchestratorEngine` is allowed to write task execution state. Controllers must use the `StartTaskCommand` application service, which queues the intent — never set state directly from a controller.
- State is persisted in SQLite after every transition for status queries. The local database is the source of truth.
- Goto task: when a task completes and `GotoTaskId` is set, the engine merges `GotoPrompt` into the target task's `Prompt` (append by default) before transitioning. Never overwrite the original prompt.
### Group execution
Tasks are organised into `TaskGroup` entities. A group has a `GroupType` and a `Children` list of `GroupChildRef`. Children can be tasks (any type) or nested `TaskGroup` instances.
- `Sequential` — the engine activates the first child; when it completes, activates the next sibling child in order. After the last child completes, the group itself is marked complete; the engine then activates `group.Next` (if set).
- `Parallel` — all children are dispatched simultaneously. When every child is complete, the group is marked complete and `group.Next` is activated.
- Nested groups follow the same rules recursively — a nested group must itself be fully complete before its parent considers it done.
- `group.Next` is a `GroupChildRef` and can point to a task or another group.
- The engine uses local coordination and database transactions to prevent the same group from being activated twice.
---
## 7. Agent Matching
Agent selection uses the Strategy pattern. The matching strategy is resolved from the agent's `ChooseRule` field.
```csharp
// Registered in DI as keyed services
services.AddKeyedScoped<IAgentMatchStrategy, ExactSingleTagStrategy>(AgentMatchRule.ExactSingle);
services.AddKeyedScoped<IAgentMatchStrategy, SupersetTagStrategy>(AgentMatchRule.Superset);
services.AddKeyedScoped<IAgentMatchStrategy, AllRequiredTagsStrategy>(AgentMatchRule.AllRequired);
```
**Order of precedence when multiple agents match:** prefer agents with the fewest extra skills (most specific match). If equal, pick the agent with the lowest current load.
**No match found:** the dispatcher retries matching after the configured `backoff` period. After `max_retries`, the task is escalated.
---
## 8. Scheduling & Events
### Schedule types
| Type | Behavior |
|------------------|-----------------------------------------------------------------------|
| `Immediate` | Queued for dispatch as soon as the task is created. |
| `TimeBased` | Cron or ISO 8601 repeat expression. Evaluated by `SchedulerService`. |
| `DatetimeEvent` | Fires once at the specified UTC datetime. Use ISO 8601 format. |
This version does not use signal-based schedule triggers. Tasks are scheduled using immediate, time-based, or datetime-event rules. Internal coordination is handled through in-process C# events dispatched via `IEventDispatcher`.
---
## 9. Container Runtime
The `IContainerRuntime` interface abstracts the local isolated runtime. Switch the implementation via config:
```json
"RuntimeAdapter": "LocalSandbox"
```
### Local development
Use the local runtime adapter. No Docker or Kubernetes runtime is required.
### Runtime spec validation
Before `StartAsync` is called, the `ContainerSpec` is validated:
- All declared tools must have a `name` and `version`.
- `resources.cpu` and `resources.memory` must be positive values.
- `network_policy` must be explicitly declared (empty is allowed, but `null` is rejected).
> **Never pass user-controlled strings directly into container exec calls.** Always use the tool definition's pre-declared `usage_guidelines` to build the command. This prevents command injection.
---
## 10. Error Handling & Escalation
### Controller layer
All controllers are wrapped by the global exception middleware in `Api/Middleware/ExceptionMiddleware.cs`. Do **not** add try/catch blocks in controllers for domain exceptions — let the middleware handle them.
```csharp
// Good
var result = await _createTaskService.ExecuteAsync(command, ct);
return Ok(result);
// Bad — don't do this
try { ... } catch (TaskNotFoundException) { return NotFound(); }
```
### Escalation flow
1. Agent reports failure with `Escalation.Reason` and `Escalation.Prompt`.
2. `EscalationHandler` checks the task's `RetryPolicy`.
3. If retries remain: re-queue the task with incremented attempt counter and apply backoff.
4. If retries exhausted: publish an `EscalationEvent` and set task state to `Escalated`.
5. The human operator receives the escalation via the configured notification channel (webhook or log alert — configured externally).
**Always populate `Escalation` on failure.** A missing escalation object on a failed task is treated as a bug and logged at `Error` level.
---
## 11. Testing Guidelines
### Unit tests (`UltimateAgents.Domain.Tests`, `UltimateAgents.Application.Tests`)
- Test domain rules in isolation — no database, no Docker.
- Mock `ITaskRepository`, `IContainerRuntime`, and `IEventDispatcher` with Moq.
- Cover every `ComplexityEvaluator.Evaluate()` branch.
- Cover cycle detection: test graphs with cycles, linear chains, and parallel groups.
```bash
dotnet test tests/UltimateAgents.Domain.Tests
dotnet test tests/UltimateAgents.Application.Tests
```
### Integration tests (`UltimateAgents.Api.Tests`)
- Use local SQLite or file-backed database instances for integration testing.
- Test the full HTTP request → DB → response cycle.
- Seed data via the configured persistence layer in the test fixture, not via the API.
```bash
dotnet test tests/UltimateAgents.Api.Tests
```
### Coverage target
- Domain layer: **≥ 90%** line coverage.
- Application layer: **≥ 80%** line coverage.
- API controllers: covered by integration tests, not unit tests.
---
## 12. Logging Conventions
Use **Serilog** with structured properties. Always include correlation context.
```csharp
_logger.LogInformation(
"Task {TaskId} transitioned to {NewState} by agent {AgentId} in container {ContainerId}",
task.Id, newState, agent.Id, container.Id);
```
**Mandatory fields for task/agent log entries:**
| Field | Description |
|---------------|------------------------------------------|
| `TaskId` | Guid of the task |
| `AgentId` | Guid of the acting agent (if applicable) |
| `ContainerId` | Runtime container ID |
| `Event` | Short event name (e.g. `TaskStarted`) |
| `ErrorContext`| Exception message / stack trace on error |
**Log levels:**
| Level | When to use |
|-------------|----------------------------------------------------------|
| `Debug` | Detailed internal flow (disabled in production) |
| `Information` | State transitions, task/agent lifecycle events |
| `Warning` | Retries, unexpected but recoverable situations |
| `Error` | Escalations, unhandled exceptions, data integrity issues |
Never log task `Prompt` content at `Information` or above — it may contain sensitive instructions. Use `Debug` only.
---
## 13. Common Pitfalls
| Pitfall | Why it happens | Fix |
|---|---|---|
| `DependencyCycleException` at runtime | A `next_task` or `next_group` chain forms a cycle that wasn’t caught at write-time | Always call `DependencyGraph.HasCycle()` in `CreateTaskCommand` and `CreateGroupCommand` before saving |
| Agent never matched | Agent’s `TaskTags` don’t intersect with task’s `TaskTags` | Check `ChooseRule` on the agent; verify tags are using the same enum string values |
| Cache state out of sync with DB | The local state cache or temporary store is stale compared to the database | Write to SQLite first in a transaction, then update any local cache. Never update cache before the DB commit. |
| Container start fails silently | `IContainerRuntime.StartAsync` threw but the exception was swallowed | Always `await` container calls; propagate exceptions to `EscalationHandler` |
| Foreach tasks run in wrong order | `ForeachTask.ForeachTemplateTaskId` set but `next_task`/`previous_task` links not generated | `ForeachExpander` in the application layer must set ordering links on generated task instances |
| Double-scheduling in multi-instance deploy | Two API instances pick up the same eligible task | Use local database locking and coordination; ensure task state transitions are serialized through SQLite. |
| User input injected into container commands | Prompt strings passed directly to `ContainerRuntime.ExecAsync` | Only use pre-declared tool `usage_guidelines` to construct exec commands; validate/sanitize any user-supplied values |
---
## 14. Environment Variables & Config
All settings follow the ASP.NET Core configuration hierarchy: environment variables override `appsettings.json`.
| Environment Variable | Default | Purpose |
|------------------------------------------|----------------------------|--------------------------------------|
| `Database__ConnectionString` | (required) | SQLite connection string or file path |
| `ContainerRuntime` | `LocalSandbox` | Local runtime adapter name |
| `Jwt__Authority` | (required in production) | OIDC authority URL |
| `Jwt__Audience` | `orchestration-api` | JWT audience claim |
| `Orchestrator__RetryPolicy__MaxRetries` | `3` | Global default max retries |
| `Orchestrator__RetryPolicy__Backoff` | `exponential` | `fixed` or `exponential` |
For local development, copy `appsettings.Development.json.example` to `appsettings.Development.json` and fill in values. This file is `.gitignore`d.
---
## 15. Useful Commands
```bash
# Run all tests
dotnet test
# Run only unit tests
dotnet test --filter "Category=Unit"
# Add a new schema migration
# (Use SQL scripts or schema versioning tools as appropriate.)
# Roll back last schema change (dev only)
# (Use SQL script rollback or schema versioning practices.)
# Watch-mode for API development
dotnet watch run --project src/UltimateAgents.Api
# Build the CLI as a self-contained binary
dotnet publish src/UltimateAgents.Cli -c Release -r linux-x64 --self-contained
# Check for vulnerable NuGet packages
dotnet list package --vulnerable --include-transitive
# Format code
dotnet format
# View structured logs (if using Seq locally)
open http://localhost:5341
```
---
> **Questions?** Check [Architecture.md](Architecture.md) for design decisions, or [features.md](features.md) for the full feature specification.

294
docs/Instructions.md Normal file
View File

@@ -0,0 +1,294 @@
# AI Agent — General Instructions
You are an autonomous coding agent working on **UltimateAgents**, an Orchestration Service that assigns and coordinates AI agents to complete tasks inside isolated running containers. This document defines how you operate, what rules you follow, and which workflow you repeat for every task.
Read this document completely before starting any work.
---
## 1. Project Context
UltimateAgents is a C# / .NET 9 backend system. There is no separate frontend application — the user-facing surface is a CLI tool (`UltimateAgents.Cli`).
| Layer | Stack |
|---|---|
| **API** | ASP.NET Core 9, minimal API + controllers, FluentValidation, JWT Bearer |
| **Application** | C# use cases, plain service handlers (commands/queries), hosted background services |
| **Domain** | Pure C# 13 entities, value objects, enums — zero framework dependencies |
| **Infrastructure** | Dapper, SQLite, in-process C# event dispatcher, local runtime adapter |
| **CLI** | System.CommandLine, calls the API over HTTP |
Supporting documentation you must know and respect:
| Document | Purpose |
|---|---|
| [features.md](features.md) | Complete feature list, data models, and expected behaviour |
| [Architecture.md](Architecture.md) | System architecture, component relationships, data flow, tech stack decisions |
| [Backend-Developer.md](Backend-Developer.md) | Developer guidelines, conventions, pitfalls, useful commands |
**Always** consult the relevant document before writing code. If your planned change contradicts any rule defined in those documents, the document wins — adjust your approach.
---
## 2. General Rules
### 2.1 Follow the Docs
- Every coding convention, naming rule, project structure decision, and library choice is defined in the docs above. Do not deviate.
- Architecture decisions follow [Architecture.md](Architecture.md) — layer boundaries, dependency direction, interface abstractions, deployment model.
- Developer conventions follow [Backend-Developer.md](Backend-Developer.md) — Dapper and SQLite patterns, DI wiring, logging fields, environment config, testing setup.
- Feature behaviour follows [features.md](features.md) — task model, specialised task types, group execution, scheduling types, complexity rules, agent matching, escalation flow.
### 2.2 Write Production-Quality Code
- Write code as if it ships today. No TODOs, no placeholders, no half-implementations.
- Every public method has explicit C# XML doc comments (`/// <summary>`).
- No `object` or untyped generics where a proper type can be defined.
- No magic numbers or strings — use named constants or enum values.
- No dead code, no commented-out blocks, no unused `using` directives.
- Prefer `record` for value objects and DTOs. Prefer `sealed class` for domain entities.
- All async methods must be `async Task` or `async Task<T>` — never `async void`.
### 2.3 Keep It Small and Focused
- One class per file.
- One use case per command/query handler.
- One service per domain concept.
- If a file grows beyond ~200 lines, split it.
- Controllers must contain no business logic — delegate immediately to Application services.
### 2.4 Respect the Dependency Direction (Strict)
```
Api → Application → Domain
Infrastructure → Application → Domain
Cli → Api (via HTTP client only)
```
- `Domain` has zero NuGet dependencies.
- `Application` depends only on `Domain` and its own interfaces (`Application/Interfaces/`).
- `Infrastructure` implements those interfaces — it is the **only** project that references Dapper, the runtime adapter, or the event dispatcher implementation.
- Never reference `Infrastructure` from `Api` directly for business operations — always go through `Application` interfaces.
### 2.5 Never Break Existing Code
- Before changing any file, understand what it does and who depends on it.
- Run the existing test suite before and after your changes. If tests fail after your change, fix them before moving on.
- Do not remove or rename public types, methods, or API endpoints without updating all callers and contracts.
### 2.6 Think Before You Code
- Read the task description carefully. If it is ambiguous, check [features.md](features.md) and [Architecture.md](Architecture.md) for clarification.
- Plan your changes before writing code. Identify which files are affected, which layers are involved, and what tests are needed.
- Prefer the simplest correct solution. Do not over-engineer.
---
## 3. Task Workflow
Repeat the following cycle for every task. Do not skip steps.
### Step 1 — Pick a Task
- Open `tasks.md` and pick the next unfinished task (highest priority first).
- Mark the task as **in progress**.
- Read the task description thoroughly. Understand the expected outcome before proceeding.
### Step 2 — Plan Your Steps
- Break the task into concrete implementation steps.
- Identify which files need to be created, modified, or deleted.
- Identify which layers are affected: Domain entity, value object, enum → Application command/query/interface → Infrastructure implementation → API controller/DTO → CLI command.
- Identify edge cases and error scenarios.
- Write down your plan before touching any code.
### Step 3 — Write Code
Follow the checklist for each layer touched:
**Domain layer**
- Add or update entities, value objects, enums, or domain services in `UltimateAgents.Domain/`.
- No framework dependencies — pure C# only.
- Enforce invariants in constructors and factory methods. Never expose setters for invariant-protected state.
**Application layer**
- Add a command or query class and its handler in `UltimateAgents.Application/`.
- Define any new repository or runtime interfaces in `Application/Interfaces/`.
- Apply `ComplexityEvaluator.Evaluate()` whenever a task is created or its estimate changes.
- Call `DependencyGraph.HasCycle()` in `CreateTaskCommand` before persisting any task with a `NextTaskId`.
- Call `DependencyGraph.HasCycle()` in `CreateGroupCommand` before persisting any group with a `NextGroupId`.
**Infrastructure layer**
- Implement new interfaces in `UltimateAgents.Infrastructure/`.
- Add Dapper persistence and SQL schema scripts if the schema changed.
- Include `task_type` discriminator column for specialised task types when adding new persistence.
- Never pass user-supplied strings directly to `IContainerRuntime` exec calls.
**API layer**
- Add or extend controllers in `UltimateAgents.Api/Controllers/`.
- Add request/response DTOs in `Api/Models/` and a FluentValidation validator.
- Never add try/catch for domain exceptions in controllers — the global `ExceptionMiddleware` handles them.
**CLI layer**
- Add subcommands in `UltimateAgents.Cli/Commands/` using `System.CommandLine`.
- CLI calls the REST API over HTTP — no direct Application or Infrastructure references.
### Step 4 — Add Logging
- Use **Serilog** with structured properties. Never use `Console.Write` or `Debug.Write`.
- Log every task state transition, agent assignment, container start/stop, escalation, and retry.
- Every log entry for a task/agent event must include: `TaskId`, `AgentId`, `ContainerId`, `Event`, and `ErrorContext` (on error).
- Log levels:
- `Information` — state transitions, lifecycle events.
- `Warning` — retries, recoverable anomalies.
- `Error` — escalations, unhandled exceptions, data integrity issues.
- `Debug` — detailed internal flow (disabled in production).
- **Never** log task `Prompt` content at `Information` or above — it may be sensitive.
### Step 5 — Write Tests
- Write tests for every new or changed piece of functionality.
- **Domain / Application (unit tests):** xUnit + Moq. No database, no Docker. Mock all interfaces.
- **API (integration tests):** xUnit with a local SQLite test database. Test the full HTTP → DB → response cycle.
- Test the happy path **and** error/edge cases.
- Follow the naming pattern: `MethodName_Scenario_ExpectedResult`.
- Coverage targets:
- Domain layer: **≥ 90 %** line coverage.
- Application layer: **≥ 80 %** line coverage.
- API controllers: covered by integration tests.
```bash
dotnet test tests/UltimateAgents.Domain.Tests
dotnet test tests/UltimateAgents.Application.Tests
dotnet test tests/UltimateAgents.Api.Tests
```
### Step 6 — Review Your Code
Run a thorough self-review before considering the task done.
#### 6.1 — Build and Static Analysis
```bash
dotnet build --no-incremental -warnaserror
dotnet format --verify-no-changes
```
- Zero build warnings, zero errors.
- Code is correctly formatted (`dotnet format`).
#### 6.2 — Test Coverage
- Run `dotnet test` with coverage and confirm thresholds are met.
- If coverage is below the threshold, write additional tests before proceeding.
#### 6.3 — Coding Principles
- [ ] **Clean Code** — Meaningful names, small methods, no magic values, guard clauses over deep nesting.
- [ ] **Separation of Concerns** — Each class has a single, well-defined responsibility. Layers are not mixed.
- [ ] **Single Responsibility Principle** — Every class and method has one reason to change.
- [ ] **DRY** — No duplicated logic. Shared behaviour is extracted into domain services or helpers.
- [ ] **KISS** — The simplest correct solution is used. No over-engineering.
- [ ] **Type Safety** — All types are explicit. No `object` casts without justification. No `#pragma warning disable` without a comment.
#### 6.4 — Architecture Compliance
Verify against [Architecture.md](Architecture.md):
- [ ] Files are in the correct project and directory (entities in `Domain/Entities/`, handlers in `Application/`, etc.).
- [ ] Dependency direction is respected — no `Infrastructure` or `Api` references leaking into `Domain`.
- [ ] No circular project references.
- [ ] No business logic in controllers or CLI commands.
- [ ] No Dapper or runtime adapter code outside `Infrastructure`.
- [ ] New API endpoints are documented in the route table in `Architecture.md § 5.1` (or update it).
- [ ] DTOs use `task_type` discriminator for specialised task fields — no polymorphic base DTO.
- [ ] DTOs are separate from domain entities.
- [ ] New interfaces are defined in `Application/Interfaces/`, not in `Infrastructure`.
#### 6.5 — Security Checklist
- [ ] No raw SQL with user input (use Dapper parameterized queries only).
- [ ] No user-supplied strings passed to container exec calls.
- [ ] No secrets, passwords, or tokens in code or config files.
- [ ] All new API endpoints are protected by JWT Bearer authentication.
- [ ] All new request models have FluentValidation validators.
### Step 7 — Update Documentation
If your change introduces new features, new endpoints, new services, or changes existing behaviour, update the relevant docs:
- [features.md](features.md) — if feature behaviour or data model changed.
- [Architecture.md](Architecture.md) — if new modules, services, interfaces, or data flows were added.
- [Backend-Developer.md](Backend-Developer.md) — if new conventions, pitfalls, or commands are relevant.
Keep documentation accurate and in sync with the code. Outdated docs are worse than no docs.
### Step 8 — Mark Task Complete
- Open `tasks.md` and mark the task as **done**.
- Add a brief summary of what was implemented or changed.
### Step 9 — Commit
- Stage all changed files.
- Write a commit message in **imperative tense**, max 72 characters for the subject line.
- Good: `Add goto task prompt merge in OrchestratorEngine`
- Bad: `added stuff` / `WIP` / `fix`
- If the change is large, include a body explaining **why**, not just **what**.
- Branch naming: `feature/<short-description>`, `fix/<short-description>`, `chore/<short-description>`.
- Ensure the commit passes: build, formatter check, all tests.
- **Never** push directly to `main` — always use a feature branch and open a pull request.
### Step 10 — Next Task
- Return to **Step 1** and pick the next task.
---
## 4. Workflow Summary
```
┌─────────────────────────────────────────────────┐
│ 1. Pick task from tasks.md │
│ 2. Plan your steps │
│ 3. Write code (Domain → App → Infra → API) │
│ 4. Add structured Serilog logging │
│ 5. Write xUnit tests │
│ 6. Review your code │
│ ├── 6.1 Build -warnaserror / format │
│ ├── 6.2 Test coverage thresholds │
│ ├── 6.3 Coding principles │
│ ├── 6.4 Architecture compliance │
│ └── 6.5 Security checklist │
│ 7. Update documentation if needed │
│ 8. Mark task complete in tasks.md │
│ 9. Git commit on feature branch │
│ 10. Pick next task ──────────── loop ──────┐ │
│ ▲ │ │
│ └──────────────────────────────────┘ │
└─────────────────────────────────────────────────┘
```
---
## 5. When You Are Stuck
- Re-read the task description and the relevant docs.
- Search the existing codebase for similar patterns — follow established conventions before inventing new ones.
- Check [Backend-Developer.md § 13](Backend-Developer.md) (Common Pitfalls) — your problem may already be documented.
- If a decision is genuinely ambiguous and no document covers it, choose the simplest option that is consistent with existing code and document your reasoning in an XML doc comment.
---
## 6. What You Must Never Do
- **Never** commit code that does not build or has compiler errors.
- **Never** commit code without tests.
- **Never** use libraries not already in the solution or explicitly approved — check existing `.csproj` files first.
- **Never** suppress build warnings with `#pragma warning disable` without a justification comment.
- **Never** hard-code secrets, connection strings, or tokens — use environment variables or `appsettings.json` (see [Backend-Developer.md § 14](Backend-Developer.md)).
- **Never** write to `Console` or `Debug` for operational logging — use Serilog.
- **Never** push directly to `main` — always use feature branches.
- **Never** skip the review step — sloppy code compounds over time.
- **Never** leave a task half-done — finish it or revert it.
- **Never** pass user-controlled input directly into container exec calls or raw SQL.

256
docs/features.md Normal file
View File

@@ -0,0 +1,256 @@
# FEATURE.md — Orchestration Service
## Purpose
Define features, data models, behaviors, and operational rules for an Orchestration Service that assigns and coordinates agents to complete tasks inside isolated running containers.
---
## Service Overview
- Name — Orchestration Service for agent-based task execution.
- Goal — Accept task definitions, schedule and dispatch agents, manage execution environments, track progress and complexity, and escalate on failures.
- Primary actors — Task, Agent, Running Container, Tool, Scheduler, Operator.
---
## Task Model
### Core Fields
- id — Unique identifier.
- name — Human-readable title.
- prompt — Primary instruction text prepended to the task before execution.
- container — Reference to the required running container.
- resolve_time_estimate — Estimated completion time in minutes.
- task_tags — Array of TaskTag values describing domain(s).
- complexity_level — One of: Low, Middle, Hard, ExtremHard.
- escalation — Optional object with reason and prompt.
- metadata — Free-form key/value map.
### Task Flow Control
A task is always a first-class entity. There is no parent/subtask concept. Tasks chain to each other via `next_task`. Tasks are organised into groups using a `TaskGroup` entity that defines sequential or parallel execution.
- schedule — Schedule definition.
- type: immediate, time-based, datetime-event.
- datetime-event is used for the schedule task feature (starts on a specific date/time).
- next_task — Optional ID of the next task in the chain (within or across groups).
- previous_task — Optional ID of the previous task in the chain.
Specialised control-flow behaviours — iteration, conditional branching, and jump-with-prompt — are expressed as distinct task types (see **Specialised Task Types** below). Each specialised task type is a separate entity and a separate database row with a `task_type` discriminator.
---
## Task Groups
### Purpose
A `TaskGroup` is a first-class chain participant, just like a task. It owns a set of children, waits for all of them to complete, and then advances to its `next` item — which can itself be either a task or another group.
Children of a group can be any mix of:
- `AgentTask` (standard or any specialised type: `ForeachTask`, `GotoTask`, `ConditionTask`)
- Nested `TaskGroup` instances (groups can be nested arbitrarily deep)
### TaskGroup Fields
- id — Unique identifier.
- name — Human-readable label.
- type — `Sequential` or `Parallel`.
- `Sequential` — Children run one at a time in the declared order. When one child completes, the next child is activated.
- `Parallel` — All children are dispatched simultaneously. The group completes when every child has finished.
- children — Ordered list of `GroupChildRef` entries. Each entry identifies a child by its ID and kind (`Task` or `Group`).
- next — Optional `GroupChildRef` identifying the item to activate after this group completes. It can point to a task or another group.
- previous — Optional `GroupChildRef` identifying the item that precedes this group in the chain.
### GroupChildRef
A `GroupChildRef` is a value object that points to either a task or a group:
- child_id — Guid of the child.
- kind — `Task` or `Group`.
### Orchestrator Behavior
- A group begins execution when it is activated (either by the root scheduler or by the `next` pointer of a predecessor).
- **Sequential**: the engine activates the first child; when that child completes, it activates the second child, and so on. After the last child completes, the group itself is marked complete and activates its `next`.
- **Parallel**: the engine activates all children simultaneously. After all children complete, the group is marked complete and activates its `next`.
- If a child is itself a `TaskGroup`, the same rules apply recursively — the nested group must complete before it counts as done for its parent.
- A child belongs to at most one group.
- Cycles in the chain (via `next` or through children) must be rejected at validation time (`DependencyCycleException`).
---
## Specialised Task Types
Each specialised control-flow behaviour is a distinct domain entity. Each type is stored as a separate database row with a `task_type` discriminator column. All specialised types inherit the core task fields and `next_task` / `previous_task` chaining.
### ForeachTask
Iterates over a list of items and spawns one task instance per item.
- foreach_items — Ordered list of string values to iterate over.
- foreach_template_task_id — ID of the task template to clone per item.
- Generated task instances are linked via `next_task` / `previous_task` in creation order and can be placed into a group.
### GotoTask
On completion, merges additional context into a target task and activates it — regardless of where the target sits in the chain.
- goto_target_task_id — ID of the task to activate.
- goto_prompt — Additional prompt text merged (appended) into the target task's `Prompt` before execution.
### ConditionTask (If-Task)
Evaluates a boolean expression against the task result and routes to one of two branches.
- condition — A boolean expression evaluated against the task result.
- true_next_task_id — ID of the task to activate when the condition evaluates to true.
- false_next_task_id — ID of the task to activate when the condition evaluates to false.
---
## Schedule
### Types
- time-based — Cron or ISO schedule expression.
- datetime-event — Start at a fixed date/time instant or range (e.g. `2026-04-01T09:00:00Z`). This enables schedule tasks.
- immediate — Start as soon as created.
The system uses normal event publishing and handling for task coordination; it does not use dedicated signal-based schedules.
### Goto Tasks
- Use a `GotoTask` (see **Specialised Task Types**) to jump to an arbitrary target task.
- The target task receives the additional `goto_prompt` text merged (appended) into its `Prompt` and activates immediately when the `GotoTask` completes.
### Retry Policy
- attempts — integer.
- backoff — fixed or exponential.
- max_retries — integer.
---
## Complexity Level
### Definition
Defines how complex a task is and when it must be split.
| Level | Assignment rule | Characteristics |
|-------------|-------------------------------------------|----------------|
| Low | resolve_time_estimate ≤ 30 | Simple logic; single topic. |
| Middle | 30 < resolve_time_estimate ≤ 60 | Moderate complexity; single topic. |
| Hard | resolve_time_estimate > 60 | Multi-topic; requires subtasks. |
| ExtremHard | resolve_time_estimate unknown/unbounded | Multi-topic; unknown duration. |
### Automatic Rules
- resolve_time_estimate > 60 → Hard.
- Missing or unbounded estimate → ExtremHard.
- Parent prompt must be prepended to all subtasks.
---
## TaskTag
### Purpose
Enum identifying task working areas and agent capabilities.
### Example Values
- DATA_INGESTION
- NLP
- IMAGE_PROCESSING
- DEPLOYMENT
- SECURITY
- TESTING
- DOCUMENTATION
---
## Agent
### Core Fields
- id — Unique identifier.
- skills — List of assigned skills.
- task_tags — List of TaskTag values the agent can handle.
- choose_rules — Matching rules for accepting tasks.
- capabilities — wait_for_subagents, call_subagent, return_status, escalation.
### Matching Rules Examples
- Accept if task has a single TaskTag equal to one of agent's TaskTags.
- Accept if agent supports a superset of task tags.
- Accept only when agent has all TaskTags required by the task.
### Execution Behavior
- Agent executes inside its container and returns success or failure.
- On failure, escalation.reason and escalation.prompt are required.
- Agents may spawn sub-agents and wait for them.
---
## Running Container
### Definition
Isolated runtime environment required for a task.
### Attributes
- image — Container image reference.
- tools — List of required tools and binaries.
- resources — CPU, memory, disk limits.
- network_policy — Allowed network rules.
- volumes — Mounts and persistence rules.
### Constraints
- Agents cannot access resources outside their container.
- Containers must declare all tools at startup.
---
## Tool
### Definition
Runnable binary or library available inside a container.
### Attributes
- name — Tool name.
- version — Required version.
- purpose — Description of tool’s role.
- usage_guidelines — Allowed and forbidden operations.
- install_instructions — Installation steps.
---
## Failure and Escalation Flow
### Agent Failure
- Agent returns failure with escalation.reason and escalation.prompt.
- Orchestrator evaluates retry policy.
- If retries exhausted → escalate to parent agent or human operator.
### Logging
- All failures and escalations must be logged with timestamps, container id, agent id, and error context.
---
## Frontend CLI
### Capabilities
- Manage Agents, Tasks, Tools, Containers, Complexity Levels, TaskTags, Schedules.
- Create, update, delete, list resources.
- Start tasks manually and view logs.
- Trigger event-driven execution.
### Example Commands
- orchestrate agent create --name <name> --task-tags <tags> --skills <skills>
- orchestrate task create --name <name> --prompt-file <file> --container <image> --schedule <spec>
- orchestrate container define --image <image> --tools <list> --resources <spec>
- orchestrate task start --id <task-id>
- orchestrate task status --id <task-id> --follow
---
## Example Task Definition (YAML)
```yaml
id: task-200
name: "Transform raw data"
prompt: "Clean and normalize raw input data."
container: "data-worker:v1.0"
resolve_time_estimate: 20
task_tags:
- DATA_INGESTION
complexity_level: Low
schedule:
type: immediate
group_id: "group-preprocessing"
tools:
- python:3.11
- pip:latest
escalation:
reason: "Validation failed"
prompt: "Please inspect input data and retry."

82
docs/prosafeature.md Normal file
View File

@@ -0,0 +1,82 @@
# Service
Service to orchestration for agent for a given task
## Task
- name
- tasks promt
- defined running container
- resolve time
- one or multable TaskTag
- complexty level
--- complex level is defined by rules like: "resolve time" > 60 then complex level = extrem hard
--- and add Promt at beeginning for that tasks
- Schedule
--- define when and how the tasks is scheduled
## Schedule
- Time based
- immediately
- datetime-event (schedule task starts at a specific date/time)
- Event-driven triggers via normal published events
- agent or agents finished events
--- all agents are finished successfully
--- all agents are finished
--- one or more agents are failed
## Task Flow Control (no super tasks)
- Every task is first-class. subtasks are normal tasks and use the same fields.
- next_task / previous_task for explicit linear task order.
- condition for if-style branching (task result chooses next path).
- foreach_items and foreach_task_template for item expansion flows.
- goto_task with optional additional prompt appended/merged into target task.
## complexty level
define how complex a tasks is.
Low, middle, Hard, extrem hard.
low: is easy to resolve. has easy logic. only one topic area. resolve within max 30 min.
middle: is modreate to resolve. has easy logic. only one topic area. resolve within max 60 min.
Hard: need to split in sub tasks. not easy logic. multiable topic areas. resolve time is greater 60 min
extram hard: need to split in sub tasks. not easy logic. multiable topic areas. resolve time is unkown.
## TaskTag
Define a enum Tag that define a Tasks working area.
This is needed to define what a agant is able to do.
A Tasks can have multiable TaskTag what would identfy that a Tasks is to complex and needs to be splitted.
## Agent
A Agent has multipale assigned skills
A Agent has a TaskTag
has choose rules. like: Task has only one TaskTag equals my TaskTag
Agent can wait for sub agents
Agent can call a sub agent
Agent return a success status
--- on not success agent escalate. but need to give a escalation promt / escalation reason
## running container
define the needed running enviroment.
a container is fully isolated except root container
a contaienr has a set of needed tools
a agent can not access outside a container
## Tool
define the needed binary and runable application
like shell, python, pip, pip libs, etc.
also contains a instructuons how to use and not use a tool
define the perpose of a tool
# Frontend cli
cli tool to manage Agents, Tasks, Tools, container, complexty level, TaskTag, Schedule
cli tool to define a Tasks and also start them.

1
docs/tasks.md Normal file
View File

@@ -0,0 +1 @@
# Task List