Dynamic inbound A2A skills: Learn and Propose trust levels

[rockfordlhotka]

Context

Inbound A2A skills are currently hardcoded in RockBotTaskHandler.HandleTaskAsync via a switch statement. The trust model defines four levels (Observe → Learn → Propose → Act), but only Observe (level 1) and Act (level 4) are implemented. Levels 2 and 3 would enable agents to learn and create new inbound skills dynamically based on observed user behavior.

Current state

• Observe (1): Read-only LLM pass, stores summary in working memory, notifies user. Always returns Completed. ✅ Implemented
• Learn (2): Same as Observe, but system observes user responses and drafts candidate skills. ❌ Not implemented
• Propose (3): System has candidate skills and asks user to approve them. ❌ Not implemented
• Act (4): Approved skills execute autonomously via hardcoded switch (notify-user, query-availability, negotiate-meeting). ✅ Implemented (but only for built-in skills)

Proposed approach

Phase 1: LLM-driven Act-level execution for non-built-in skills

When a trusted caller invokes an approved skill that isn't in the hardcoded switch, instead of falling through to the Observe path, run the LLM with the full tool set (not the restricted Observe set) and the skill's stored instructions as the system prompt.

This enables dynamically registered skills to actually execute at Act level without code changes.

Changes:

• Add an InboundSkillStore (or reuse ISkillStore) for skill definitions with execution instructions
• In the Act-level dispatch default case, look up the skill in the store and run the LLM with the skill's instructions + full tools
• Skill definitions include: system prompt, allowed tools, whether to return InputRequired for multi-turn

Phase 2: Learn (level 2) — observation and skill drafting

When a caller is at Learn level:

1. Run the Observe path as today (summary + notification)
2. After the user responds to the notification (via the Blazor UI), capture the user's response pattern
3. Over multiple interactions, identify recurring patterns (e.g., "user always checks calendar when Agent X asks about meetings")
4. Draft a candidate skill definition with:
   • Skill ID and description
   • System prompt instructions (derived from observed user behavior)
   • Required tools
5. Store the draft in a CandidateSkillStore

Phase 3: Propose (level 3) — user approval flow

When the system has a candidate skill with sufficient confidence:

1. Present the skill to the user: "I noticed you always check the calendar when Agent X asks about meetings. Want me to do that automatically?"
2. User can approve, modify, or reject
3. On approval: add the skill to the agent's inbound skill store and the caller's ApprovedSkills list
4. The skill is now available at Act level for that caller

Phase 4: Self-improvement loop

Once a dynamically created skill is running at Act level:

• Track success/failure rates
• Capture user corrections ("that's not what I wanted")
• Refine the skill's instructions via the dream/feedback system
• Allow the user to revoke approval (skill drops back to Observe)

Key design decisions needed

1. Skill definition format — what does a stored inbound A2A skill look like? System prompt + tool allowlist + InputRequired capability?
2. Observation mechanism — how does the agent connect a user's response to a specific Observe-level notification? The notification queue + Blazor UI flow would need a feedback loop.
3. Confidence threshold — how many observations before proposing a skill? Too few = premature automation, too many = user frustration.
4. Skill scope — are learned skills per-caller or global? Per-caller is safer (Agent X's pattern may not apply to Agent Y).

Files likely affected

• src/RockBot.Agent/A2A/RockBotTaskHandler.cs — Act-level default case for dynamic skills
• src/RockBot.A2A.Abstractions/AgentTrustLevel.cs — already defines Learn/Propose
• src/RockBot.A2A.Abstractions/AgentTrustEntry.cs — may need candidate skill tracking
• New: InboundSkillStore or extension of ISkillStore
• New: CandidateSkillStore for drafted skills pending approval
• src/RockBot.Agent/A2A/InboundA2AToolSet.cs — full tool set variant for Act-level LLM execution

References

• Existing skill implementation pattern: docs/a2a.md → "Implementing inbound A2A skills"
• Reference implementation: HandleNegotiateMeetingAsync in RockBotTaskHandler.cs
• Trust model: AgentTrustLevel enum (Observe=1, Learn=2, Propose=3, Act=4)
• Outcome persistence pattern: a2a-outcomes/{skill}/{contextId} with category "a2a-outcome"