UltimateAgents/docs/Architecture.md

Architecture.md — Orchestration Service

Version: 1.0
Date: 2026-04-07
Author: Senior Software Architect
Stack: C# / .NET 9 / ASP.NET Core
Eventing: In-process C# events (no external message broker)

---

1. Overview

The Orchestration Service is a distributed, event-driven system that accepts task definitions, schedules and dispatches AI agents into isolated container runtimes, tracks execution progress, enforces complexity rules, and escalates on failure. It is the central nervous system for multi-agent workflows.

┌──────────────────────────────────────────────────────────┐
│                    CLI Frontend (C#)                     │
│         orchestrate task create / start / status        │
└─────────────────────────┬────────────────────────────────┘
                          │ HTTP/REST + gRPC
┌─────────────────────────▼────────────────────────────────┐
│               Orchestration API  (ASP.NET Core)          │
│   TaskController │ AgentController │ ContainerController │
└──────┬───────────┬──────────────────────┬────────────────┘
       │           │                      │
┌──────▼──┐  ┌─────▼──────┐  ┌───────────▼──────────────┐
│Scheduler│  │ Dispatcher │  │   Container Manager      │
│ Service │  │  Service   │  │  Isolated runtime adapter │
└──────┬──┘  └─────┬──────┘  └───────────┬──────────────┘
       │           │                      │
┌──────▼───────────▼──────────────────────▼──────────────┐
│                  Domain Layer (C# Domain Models)        │
│  Task │ Agent │ Schedule │ Complexity │ Escalation       │
└────────────────────────┬───────────────────────────────┘
                         │
┌────────────────────────▼───────────────────────────────┐
│               Persistence Layer                        │
│          SQLite (Dapper)  │  No external cache     │
└────────────────────────────────────────────────────────┘

---

2. Solution Structure

UltimateAgents/
├── src/
│   ├── UltimateAgents.Api/               # ASP.NET Core Web API host
│   │   ├── Controllers/
│   │   │   ├── TasksController.cs
│   │   │   ├── AgentsController.cs
│   │   │   ├── ContainersController.cs
│   │   │   ├── ToolsController.cs
│   │   │   └── SchedulesController.cs
│   │   ├── Program.cs
│   │   └── appsettings.json
│   │
│   ├── UltimateAgents.Domain/            # Pure domain models, no framework deps
│   │   ├── Entities/
│   │   │   ├── AgentTask.cs              # base task
│   │   │   ├── ForeachTask.cs            # specialised: foreach iteration
│   │   │   ├── GotoTask.cs               # specialised: jump with merged prompt
│   │   │   ├── ConditionTask.cs          # specialised: if/else branch
│   │   │   ├── TaskGroup.cs              # sequential or parallel group of tasks
│   │   │   ├── Agent.cs
│   │   │   ├── RunningContainer.cs
│   │   │   ├── Tool.cs
│   │   │   └── Schedule.cs
│   │   ├── Enums/
│   │   │   ├── ComplexityLevel.cs
│   │   │   ├── TaskTag.cs
│   │   │   ├── TaskType.cs               # Standard | Foreach | Goto | Condition
│   │   │   ├── GroupType.cs              # Sequential | Parallel
│   │   │   └── ScheduleType.cs
│   │   ├── ValueObjects/
│   │   │   ├── Escalation.cs
│   │   │   └── RetryPolicy.cs
│   │   └── Exceptions/
│   │       ├── DependencyCycleException.cs
│   │       └── TaskNotFoundException.cs
│   │
│   ├── UltimateAgents.Application/       # Use cases / application services
│   │   ├── Tasks/
│   │   │   ├── CreateTaskCommand.cs
│   │   │   ├── StartTaskCommand.cs
│   │   │   ├── GetTaskStatusQuery.cs
│   │   │   └── TaskComplexityEvaluator.cs
│   │   ├── Agents/
│   │   │   ├── AssignAgentCommand.cs
│   │   │   └── AgentMatchingService.cs
│   │   ├── Scheduling/
│   │   │   ├── SchedulerService.cs
│   │   ├── Events/
│   │   │   ├── IDomainEvent.cs
│   │   │   ├── TaskCreatedEvent.cs
│   │   │   ├── TaskStartedEvent.cs
│   │   │   ├── TaskCompletedEvent.cs
│   │   │   ├── TaskFailedEvent.cs
│   │   │   ├── EscalationRaisedEvent.cs
│   │   │   └── AllAgentsFinishedEvent.cs
│   │   └── Orchestration/
│   │       ├── OrchestratorEngine.cs
│   │   │   ├── DependencyGraph.cs
│   │   │   └── EscalationHandler.cs
│   │   └── Interfaces/
│   │       ├── ITaskRepository.cs
│   │       ├── IAgentRepository.cs
│   │       ├── IContainerRuntime.cs
│   │       └── IEventDispatcher.cs
│   │
│   ├── UltimateAgents.Infrastructure/   # Dapper, runtime adapter, events
│   │   ├── Persistence/
│   │   │   ├── DatabaseSchema.sql
│   │   │   ├── TaskRepository.cs
│   │   │   ├── TaskGroupRepository.cs
│   │   │   └── AgentRepository.cs
│   │   ├── Runtime/
│   │   │   ├── LocalRuntimeAdapter.cs
│   │   │   └── RuntimeAdapter.cs
│   │   └── Events/
│   │       └── DomainEventDispatcher.cs   # in-process C# event dispatcher
│   │
│   └── UltimateAgents.Cli/              # CLI frontend (System.CommandLine)
│       ├── Commands/
│       │   ├── TaskCommands.cs
│       │   ├── AgentCommands.cs
│       │   └── ContainerCommands.cs
│       └── Program.cs
│
└── tests/
    ├── UltimateAgents.Domain.Tests/
    ├── UltimateAgents.Application.Tests/
    └── UltimateAgents.Api.Tests/

---

3. Domain Model

3.1 Core Entities

AgentTask (base)

public sealed class AgentTask
{
    public Guid Id { get; private set; }
    public string Name { get; private set; }
    public string Prompt { get; private set; }
    public string ContainerImage { get; private set; }
    public int? ResolveTimeEstimate { get; private set; }   // minutes; null = ExtremHard
    public ComplexityLevel ComplexityLevel { get; private set; }
    public IReadOnlyList<TaskTag> TaskTags { get; private set; }
    public Schedule Schedule { get; private set; }
    public Escalation? Escalation { get; private set; }
    public Dictionary<string, string> Metadata { get; private set; }
    public RetryPolicy RetryPolicy { get; private set; }
    public TaskType TaskType { get; private set; }          // discriminator

    // Chain links
    public Guid? NextTaskId { get; private set; }
    public Guid? PreviousTaskId { get; private set; }
}

ForeachTask

public sealed class ForeachTask : AgentTask
{
    public IReadOnlyList<string> ForeachItems { get; private set; }
    public Guid? ForeachTemplateTaskId { get; private set; }
}

GotoTask

public sealed class GotoTask : AgentTask
{
    public Guid GotoTargetTaskId { get; private set; }
    public string? GotoPrompt { get; private set; }
}

ConditionTask

public sealed class ConditionTask : AgentTask
{
    public string Condition { get; private set; }
    public Guid? TrueNextTaskId { get; private set; }
    public Guid? FalseNextTaskId { get; private set; }
}

TaskGroup

/// <summary>A chain participant that owns children (tasks or nested groups) and
/// completes when all its children finish, then activates <see cref="Next"/>.</summary>
public sealed class TaskGroup
{
    public Guid Id { get; private set; }
    public string Name { get; private set; }
    public GroupType Type { get; private set; }                   // Sequential | Parallel
    public IReadOnlyList<GroupChildRef> Children { get; private set; } // ordered
    public GroupChildRef? Next { get; private set; }              // task or group to activate after completion
    public GroupChildRef? Previous { get; private set; }          // predecessor in the chain
}

Agent

public sealed class Agent
{
    public Guid Id { get; private set; }
    public IReadOnlyList<string> Skills { get; private set; }
    public IReadOnlyList<TaskTag> TaskTags { get; private set; }
    public AgentMatchRule ChooseRule { get; private set; }
    public AgentCapabilities Capabilities { get; private set; }
}

3.2 Enumerations

public enum ComplexityLevel  { Low, Middle, Hard, ExtremHard }
public enum TaskTag          { DataIngestion, Nlp, ImageProcessing,
                               Deployment, Security, Testing, Documentation }
public enum TaskType         { Standard, Foreach, Goto, Condition }
public enum GroupType        { Sequential, Parallel }
public enum GroupChildKind   { Task, Group }             // discriminates GroupChildRef
public enum ScheduleType     { Immediate, TimeBased, DatetimeEvent }

3.3 Value Objects

/// <summary>Points to either a task or a nested group inside a TaskGroup.</summary>
public sealed record GroupChildRef(Guid ChildId, GroupChildKind Kind);

public sealed record Escalation(string Reason, string Prompt);

public sealed record RetryPolicy(int Attempts, BackoffStrategy Backoff, int MaxRetries);

3.4 Complexity Rules (Domain Service)

public static class ComplexityEvaluator
{
    public static ComplexityLevel Evaluate(int? resolveTimeEstimate) =>
        resolveTimeEstimate switch
        {
            null              => ComplexityLevel.ExtremHard,
            > 60              => ComplexityLevel.Hard,
            > 30              => ComplexityLevel.Middle,
            _                 => ComplexityLevel.Low
        };
}

---

4. Application Layer

4.1 Orchestrator Engine

The OrchestratorEngine is the central coordinator. It operates as a hosted background service (IHostedService) and drives the task lifecycle.

Task Created
     │
     ▼
[ValidateDependencyGraph]  ──cycle?──► Reject (DependencyCycleException)
     │
     ▼
[EvaluateComplexity]
     │
     ▼
[SchedulerService] ──── immediate ──► [Dispatcher]
                   ──── time/event ──► enqueue in scheduler
                   ──── event    ──► process with event handlers
     │
     ▼
[Dispatcher] ── match Agent via AgentMatchingService
     │
     ▼
[ContainerRuntime.StartAsync(task, agent)]
     │
     ├── success ──► advance to NextTask or GotoTask
     │
     └── failure ──► [EscalationHandler]
                          │
                          ├── retries remaining ──► re-dispatch
                          └── exhausted ──────────► escalate to operator

4.2 Chain Validation

Both tasks and groups participate in one unified directed graph. Cycle detection runs at task/group creation time over this unified graph. Any cycle results in a DependencyCycleException.

public sealed class DependencyGraph
{
    // Register a node (task OR group) with its optional successor.
    // nextRef == null means the node has no successor.
    public void AddNode(Guid id, GroupChildRef? nextRef) { ... }

    // Register that a group contains a child (task or nested group).
    public void AddChild(Guid groupId, GroupChildRef child) { ... }

    public bool HasCycle(out IReadOnlyList<Guid> cyclicIds) { ... }
}

4.3 Agent Matching Service

public interface IAgentMatchingService
{
    Agent? FindBestMatch(IEnumerable<Agent> agents, Task task);
}

Matching strategies (Strategy pattern):

• ExactSingleTagStrategy — agent must have exactly the one task tag.
• SupersetTagStrategy — agent's tags must be a superset of task tags.
• AllRequiredTagsStrategy — agent must cover every tag in the task.

4.4 Scheduler Service

public class SchedulerService : BackgroundService
{
    // Polls DatetimeEvent and TimeBased tasks
    // Dispatches regular task events when schedules are met
}

Event types include:

• AllAgentsFinishedSuccessfully
• AllAgentsFinished
• OneOrMoreAgentsFailed

---

5. API Layer (ASP.NET Core)

5.1 REST Endpoints

Method	Route	Description
POST	/api/tasks	Create a task
GET	/api/tasks/{id}	Get task details
GET	/api/tasks/{id}/status	Get execution status + logs
POST	/api/tasks/{id}/start	Manually trigger a task
DELETE	/api/tasks/{id}	Delete a task
POST	/api/agents	Register an agent
GET	/api/agents	List all agents
POST	/api/containers	Define a container spec
POST	/api/groups	Create a task group
GET	/api/groups/{id}	Get group details
DELETE	/api/groups/{id}	Delete a group
POST	/api/events	Fire a named event

5.2 Request / Response Models (DTOs)

All DTOs are located in UltimateAgents.Api/Models/. Input models are validated with FluentValidation.

public sealed record CreateTaskRequest(
    string Name,
    string Prompt,
    string ContainerImage,
    int? ResolveTimeEstimate,
    List<string> TaskTags,
    ScheduleRequest Schedule,
    string TaskType,               // "Standard" | "Foreach" | "Goto" | "Condition"
    EscalationRequest? Escalation,
    Dictionary<string, string>? Metadata,
    // ForeachTask fields (when TaskType = "Foreach")
    List<string>? ForeachItems,
    Guid? ForeachTemplateTaskId,
    // GotoTask fields (when TaskType = "Goto")
    Guid? GotoTargetTaskId,
    string? GotoPrompt,
    // ConditionTask fields (when TaskType = "Condition")
    string? Condition,
    Guid? TrueNextTaskId,
    Guid? FalseNextTaskId);

5.3 Error Handling

Global exception middleware maps domain exceptions to HTTP status codes:

Exception	HTTP Status
TaskNotFoundException	404
DependencyCycleException	422
ValidationException	400
Unhandled	500

---

6. Infrastructure Layer

6.1 Persistence — Dapper + SQLite

• DatabaseSchema.sql defines the SQLite schema for tasks, agents, containers, tools, and schedules.
• Dapper is used for lightweight mapping and raw SQL queries.
• The database is a local file store (Data Source=ultimateagents.db) and is the source of truth for task state.
• Schema updates are managed via versioned SQL scripts rather than EF Core migrations.

6.2 Runtime Adapter Abstraction

public interface IContainerRuntime
{
    Task<ContainerHandle> StartAsync(ContainerSpec spec, CancellationToken ct);
    Task StopAsync(ContainerHandle handle, CancellationToken ct);
    Task<ContainerStatus> GetStatusAsync(ContainerHandle handle, CancellationToken ct);
}

Implementations:

• LocalRuntimeAdapter — executes tasks in a local isolated runtime sandbox.
• RuntimeAdapter — generic runtime adapter interface for future platform-specific implementations.

Runtime is selected via appsettings.json → "RuntimeAdapter": "LocalSandbox" and registered with DI.

6.3 Events

All eventing is in-process using C# delegates. No external message broker is required.

/// <summary>Marker interface for all domain events.</summary>
public interface IDomainEvent { }

/// <summary>In-process dispatcher backed by a thread-safe delegate registry.</summary>
public interface IEventDispatcher
{
    /// <summary>Raise an event synchronously to all registered handlers.</summary>
    void Raise<TEvent>(TEvent @event) where TEvent : IDomainEvent;

    /// <summary>Register a handler for a specific event type.</summary>
    void Subscribe<TEvent>(Action<TEvent> handler) where TEvent : IDomainEvent;

    /// <summary>Remove a previously registered handler.</summary>
    void Unsubscribe<TEvent>(Action<TEvent> handler) where TEvent : IDomainEvent;
}

IEventDispatcher is defined in Application/Interfaces/. The sole implementation, DomainEventDispatcher, lives in Infrastructure/Events/ and uses a ConcurrentDictionary<Type, List<Delegate>> as its handler registry.

Domain event types (in Application/Events/):

Event	Raised when
TaskCreatedEvent	A new task is persisted.
TaskStartedEvent	A container runtime starts the task.
TaskCompletedEvent	An agent reports success.
TaskFailedEvent	An agent reports failure.
EscalationRaisedEvent	Retry limit is exhausted; operator must act.
AllAgentsFinishedEvent	Every agent in a group has finished.

---

7. CLI Frontend

Built with System.CommandLine (.NET).

orchestrate
├── task
│   ├── create   --name --prompt-file --container --tags --schedule
│   ├── start    --id
│   ├── status   --id [--follow]
│   ├── list
│   └── delete   --id
├── agent
│   ├── create   --name --task-tags --skills --match-rule
│   ├── list
│   └── delete   --id
├── container
│   ├── define   --image --tools --resources
│   └── list
└── schedule
    └── trigger   --event-name

The CLI calls the Orchestration REST API. API base URL is configured via ~/.orchestrate/config.json or --api-url flag.

---

8. Cross-Cutting Concerns

8.1 Logging

• Serilog with structured JSON output.
• All task state transitions, agent assignments, escalations, and container events are logged at Information or Warning level.
• Required fields per log entry: timestamp, taskId, agentId, containerId, event, errorContext.

8.2 Security

• API secured with JWT Bearer tokens.
• Runtime isolation is enforced by the local sandbox adapter.
• Agents are strictly isolated; no cross-runtime resource access.
• Input validation via FluentValidation on all API request models.
• SQL injection prevented by Dapper parameterized queries.

8.3 Observability

• /health endpoint via AspNetCore.HealthChecks.
• Basic operational metrics are emitted through structured logs.
• No Prometheus or OpenTelemetry integration is required for this version.

8.4 Configuration

{
  "Database": { "ConnectionString": "Data Source=ultimateagents.db" },
  "RuntimeAdapter": "LocalSandbox",
  "Jwt": { "Authority": "...", "Audience": "orchestration-api" }
}

---

9. Deployment

┌─────────────┐   ┌─────────────────┐   ┌────────────┐
│  CLI Client │──►│  Orchestration  │──►│ SQLite      │
└─────────────┘   │  API (Pod/s)    │   └────────────┘
                  └────────┬────────┘
                           │
                           ▼
                     ┌───────────┐
                     │  Local    │
                     │  Runtime  │
                     └───────────┘

• The application runs as a local service with a SQLite file database.
• Horizontal scaling is not required for this version; task coordination is managed via the local runtime and database.
• Secrets are managed using environment variables or local configuration.

---

10. Technology Stack Summary

Concern	Technology
Language	C# 13 / .NET 9
Web Framework	ASP.NET Core 9 (Minimal API + MVC)
ORM	Dapper
Database	SQLite
Cache / State	None
Messaging	In-process C# events (no broker)
Container Runtime	Local runtime adapter
CLI	System.CommandLine
Logging	Serilog
Validation	FluentValidation
Auth	ASP.NET Core JWT Bearer
Observability	Structured logs + health checks
Testing	xUnit + Moq

---

11. Key Design Decisions

Decision	Rationale
Specialised task types (ForeachTask, GotoTask, ConditionTask) are distinct entities	Each type has different data; discriminated storage keeps the schema clean and the domain explicit
TaskGroup is a chain participant with typed children (GroupChildRef)	Children can be any mix of tasks and nested groups; the group waits for all children then advances via its own Next pointer, enabling arbitrary nesting without a separate chaining model
GroupChildKind discriminator on GroupChildRef	Allows a single next_id pointer to resolve to either a task or a group at the DB level without ambiguity
Unified DependencyGraph over tasks and groups	Both tasks and groups are nodes; AddNode + AddChild captures all edges; one cycle-detection pass covers the whole execution graph
Complexity evaluated automatically from resolve_time_estimate	Consistent rule enforcement; no manual classification errors
Container runtime behind interface	Portable between local runtime adapter implementations without external container dependencies
In-process C# events via IEventDispatcher	Zero external dependencies; handlers are registered at startup and invoked synchronously in-process; simple to test with mock subscriptions
Serilog structured logging with mandatory correlation fields	Enables log aggregation and alerting per task/agent/container