ATOM Documentation

Training Workflows User Guide

Table of Contents

1. Introduction to Agent Training
2. Understanding Agent Maturity Levels
3. STUDENT Agent Training Workflow
4. INTERN Agent Proposal Workflow
5. SUPERVISED Agent Monitoring Workflow
6. Best Practices for Training Success
7. Troubleshooting Common Issues
8. FAQ

---

1. Introduction to Agent Training

What is Agent Training?

Agent training is a structured process for developing AI agents from STUDENT level (read-only) to AUTONOMOUS level (full automation). The training system uses human-in-the-loop workflows to ensure agents develop the skills and confidence needed for autonomous operation.

Why Train Agents?

• **Safety**: Agents learn to perform tasks safely before gaining autonomy
• **Quality**: Human supervision ensures high-quality output during learning
• **Trust**: Gradual progression builds trust in agent capabilities
• **Efficiency**: Well-trained agents reduce human intervention over time

Training Overview

STUDENT → Training → INTERN → Proposals → SUPERVISED → Supervision → AUTONOMOUS
(<0.5)                (0.5-0.7)              (0.7-0.9)               (>0.9)

---

2. Understanding Agent Maturity Levels

STUDENT (Confidence < 0.5)

**Capabilities:**

• Read-only access (view data, generate charts)
• No write operations (no state changes)
• Blocked from all automated triggers

**Training Requirements:**

• 10 training sessions minimum
• 50% intervention rate maximum
• 0.70 confidence score target

**Typical Duration:** 5-7 days

INTERN (Confidence 0.5-0.7)

**Capabilities:**

• Proposal-based execution (requires human approval)
• Streaming responses
• Form presentations

**Training Requirements:**

• 25 training sessions minimum
• 20% intervention rate maximum
• 0.85 confidence score target

**Typical Duration:** 7-14 days

SUPERVISED (Confidence 0.7-0.9)

**Capabilities:**

• Execute with real-time supervision
• Form submissions
• State changes

**Training Requirements:**

• 50 training sessions minimum
• 0% intervention rate target
• 0.95 confidence score target

**Typical Duration:** 14-30 days

AUTONOMOUS (Confidence > 0.9)

**Capabilities:**

• Full automation
• No supervision required
• Can supervise other agents

**Maintenance:**

• Periodic performance reviews
• Episodic memory tracking
• Continuous learning

---

3. STUDENT Agent Training Workflow

3.1 When Training is Triggered

**Automatic Triggers:**

• STUDENT agents blocked from automated triggers (AI Coordinator, Workflow Engine, Data Sync)
• System generates training proposal automatically

**Manual Triggers:**

• Administrator creates training proposal manually
• User requests training for specific capability gap

3.2 Reviewing Training Proposals

**Step 1: Access Proposals**

1. Navigate to /training/proposals
2. Filter by status: "Pending"
3. Review proposal details

**Step 2: Evaluate Proposal**

• **Agent Context:** Review agent's current confidence score, intervention rate, episode count
• **Learning Objectives:** Understand what capabilities will be developed
• **Capability Gaps:** Identify specific skills the agent needs to learn
• **Duration Estimate:** Review AI-estimated training duration (can override)

**Step 3: Make Decision**

**Approve if:**

• Capability gaps are clearly defined
• Training objectives align with business needs
• Duration estimate is reasonable
• Agent has completed previous training successfully

**Reject if:**

• Agent is not ready for training (too recent, incomplete prerequisites)
• Capability gaps are not specific enough
• Training objectives don't align with current priorities
• Duration estimate is unrealistic

3.3 Conducting Training Sessions

**Step 1: Start Training Session**

1. After approval, training session is created automatically
2. Navigate to /training/sessions/{id}
3. Review session details and agent context

**Step 2: Complete Training Steps**

**Step 1: Introduction**

• Review training objectives with agent
• Ensure agent understands learning goals
• Set expectations for performance

**Step 2: Capability Gap Training**

• Address each capability gap with targeted exercises
• Use real-world scenarios relevant to agent's role
• Provide examples and demonstrations

**Step 3: Practical Exercises**

• Guide agent through practice scenarios
• Provide feedback on performance
• Correct mistakes and reinforce learning

**Step 4: Assessment**

• Evaluate agent performance (1-5 scale)
• Identify capabilities developed
• Document remaining capability gaps
• Provide feedback for improvement

**Step 3: Complete Session**

1. Fill out assessment form with:

• Performance score (1-5)
• Capabilities developed (select from list)
• Capability gaps remaining (select from list)
• Supervisor feedback (free text)

1. Click "Complete Training"
2. System updates agent's confidence score automatically

3.4 Tracking Progress

**View Agent Training History:**

1. Navigate to /agents/{id}
2. Click "Training History" tab
3. View all completed training sessions
4. Track confidence score progression over time

**Promotion Criteria:**

• 10 training sessions completed
• 50% intervention rate or lower
• 0.70 confidence score achieved
• All primary capability gaps addressed

---

4. INTERN Agent Proposal Workflow

4.1 Understanding Action Proposals

**What are Action Proposals?**

INTERN agents cannot execute actions autonomously. Instead, they generate proposals for human review and approval.

**Proposal Components:**

• **Proposed Action:** What the agent wants to do
• **Reasoning:** Why the agent believes this action is appropriate
• **Expected Outcome:** What result the agent expects
• **Risk Assessment:** Potential risks and mitigation strategies

4.2 Reviewing Action Proposals

**Step 1: Access Proposals**

1. Navigate to /training/proposals
2. Filter by proposal type: "Action"
3. Filter by status: "Pending"

**Step 2: Evaluate Proposal**

**Checklist:**

• [ ] Proposed action aligns with business goals
• [ ] Reasoning is sound and well-supported
• [ ] Expected outcome is realistic
• [ ] Risks are identified and acceptable
• [ ] Agent has appropriate maturity for this action

**Step 3: Make Decision**

**Approve if:**

• Action is safe and appropriate
• Reasoning demonstrates understanding
• Benefits outweigh risks
• Agent has capability to execute successfully

**Reject if:**

• Action is too risky or complex
• Reasoning is flawed or incomplete
• Agent lacks necessary capabilities
• Better alternative exists

**Modify if:**

• Minor adjustments needed to action
• Additional context or constraints required
• Risk mitigation strategies needed

4.3 After Approval

**System Actions:**

• Agent executes approved action automatically
• Execution result recorded in agent history
• Agent's confidence score updated based on outcome

**Feedback Loop:**

• Successful execution → Confidence increase
• Failed execution → Confidence decrease, additional training

---

5. SUPERVISED Agent Monitoring Workflow

5.1 Starting Supervision Sessions

**Automatic Supervision:**

• SUPERVISED agents automatically enter supervision when triggered
• Real-time monitoring session created
• Supervisor notified via in-app notification

**Manual Supervision:**

1. Navigate to /training/supervision
2. Select agent from active sessions list
3. Click "Start Supervision"

5.2 Monitoring Dashboard

**Dashboard Components:**

**Agent Information:**

• Current maturity level
• Confidence score
• Session duration
• Intervention count

**Live Action Feed:**

• Real-time agent actions
• Action timestamps
• Action metadata (inputs, outputs, errors)

**Intervention History:**

• Previous interventions (pause, correct, terminate)
• Intervention timestamps
• Correction messages

**Performance Metrics:**

• Actions completed count
• Success rate percentage
• Average response time

5.3 Interventions

**Types of Interventions:**

**Pause (Temporary Stop)**

• Use when: Agent is heading in wrong direction but can recover
• Effect: Agent pauses execution, waits for guidance
• After: Provide correction or guidance, then resume

**Correct (Provide Guidance)**

• Use when: Agent makes mistake or needs improvement
• Effect: Correction message sent to agent
• After: Agent adjusts behavior based on feedback

**Terminate (Stop Execution)**

• Use when: Agent cannot recover or action is too risky
• Effect: Session ends immediately
• After: Review why termination was needed, adjust training

**When to Intervene:**

• Agent attempts unsafe action
• Agent demonstrates misunderstanding
• Agent exceeds acceptable error rate
• Agent requests help (via proposal)

5.4 Completing Supervision

**Step 1: End Session**

1. Wait for agent to complete task or terminate manually
2. Review session summary (actions, interventions, performance)
3. Provide supervisor rating (1-5 scale)
4. Add feedback comments

**Step 2: System Updates**

• Agent's confidence score updated based on:
• Supervisor rating
• Intervention count (more interventions = lower boost)
• Performance metrics
• Promotion to AUTONOMOUS considered if:
• 0.95 confidence score achieved
• 0% intervention rate
• 50+ supervision sessions completed

---

6. Best Practices for Training Success

6.1 STUDENT Training Best Practices

**Do:**

• Start with simple, well-defined tasks
• Provide clear, specific learning objectives
• Use real-world examples from your domain
• Be patient - learning takes time
• Give constructive feedback

**Don't:**

• Rush through training steps
• Overwhelm agent with too many new capabilities
• Skip assessment step
• Ignore capability gaps
• Train for too long without breaks

6.2 INTERN Proposal Best Practices

**Review Tips:**

• Read agent's reasoning carefully
• Consider agent's perspective
• Provide feedback on rejected proposals
• Approve proposals that stretch agent's capabilities slightly

**Approval Criteria:**

• Action is within agent's current capabilities
• Risks are manageable
• Agent demonstrates understanding of context
• Expected outcome is valuable

6.3 SUPERVISED Monitoring Best Practices

**Monitoring Tips:**

• Check dashboard regularly but don't micromanage
• Intervene only when necessary
• Provide constructive corrections
• Track patterns in agent mistakes
• Celebrate agent successes

**Intervention Guidelines:**

• Pause before terminate (give agent chance to recover)
• Provide specific, actionable corrections
• Explain why intervention was needed
• Document intervention reasons for future reference

6.4 General Best Practices

**Training Cadence:**

• STUDENT: Daily training sessions (30-60 minutes)
• INTERN: Review proposals within 1 hour
• SUPERVISED: Monitor sessions actively (check every 5-10 minutes)

**Feedback Quality:**

• Be specific about what went well or poorly
• Provide examples when giving corrections
• Focus on behavior, not agent's "personality"
• Balance positive and negative feedback

**Progress Tracking:**

• Monitor confidence score trends (should increase over time)
• Track intervention rates (should decrease over time)
• Review training history regularly
• Adjust training approach based on progress

---

7. Troubleshooting Common Issues

7.1 Agent Not Progressing

**Symptom:** Agent stuck at same maturity level for weeks

**Possible Causes:**

1. Training sessions too infrequent
2. Training quality poor (low scores, high interventions)
3. Capability gaps not properly addressed
4. Agent not ready for promotion

**Solutions:**

1. Increase training frequency (daily sessions)
2. Improve training quality (better preparation, clearer objectives)
3. Reassess capability gaps (are they the right ones?)
4. Check promotion criteria (has agent met all requirements?)

7.2 High Intervention Rate

**Symptom:** SUPERVISED agent requires constant intervention

**Possible Causes:**

1. Agent promoted too early (not ready for SUPERVISED)
2. Tasks too complex for agent's current abilities
3. Training didn't cover current scenarios
4. Agent confidence score inflated

**Solutions:**

1. Consider demoting agent to INTERN for more training
2. Simplify tasks or provide more guidance
3. Add training scenarios for current situations
4. Recalibrate confidence score based on actual performance

7.3 Proposals Always Rejected

**Symptom:** INTERN agent's proposals consistently rejected

**Possible Causes:**

1. Agent doesn't understand business context
2. Agent's reasoning is flawed
3. Proposal quality is poor
4. Approval criteria unclear

**Solutions:**

1. Provide more business context in training
2. Review agent's reasoning patterns, identify flaws
3. Train agent on proposal writing (examples, templates)
4. Document approval criteria clearly, share with agent

7.4 Training Sessions Failing

**Symptom:** Training sessions not completing successfully

**Possible Causes:**

1. Technical issues (WebSocket disconnect, API errors)
2. Agent crashes during training
3. Training data corrupted
4. Supervisor errors (incorrect data entry)

**Solutions:**

1. Check technical issues (browser console, network tab)
2. Review agent logs for crash reasons
3. Verify training data integrity
4. Double-check supervisor data entry before submission

---

8. FAQ

**Q: How long does it take to train an agent from STUDENT to AUTONOMOUS?**

A: Typical timeline is 4-6 weeks:

• STUDENT → INTERN: 5-7 days (10 training sessions)
• INTERN → SUPERVISED: 7-14 days (25 training sessions)
• SUPERVISED → AUTONOMOUS: 14-30 days (50 training sessions)

**Q: Can I skip training levels?**

A: No. Each maturity level must be achieved sequentially. Skipping levels risks agent safety and performance.

**Q: What if an agent never reaches AUTONOMOUS?**

A: Some agents may plateau at INTERN or SUPERVISED. This is acceptable if the agent performs well at that level. Not all agents need to be AUTONOMOUS.

**Q: How do I know if my agent is ready for promotion?**

A: Check the promotion criteria in the agent's training history. The system will automatically suggest promotion when criteria are met.

**Q: Can I train multiple agents at once?**

A: Yes, but quality may suffer. Focus on one agent at a time for best results.

**Q: What happens if I reject a training proposal?**

A: The agent remains at current maturity level. You can create a new proposal with different objectives later.

**Q: How do I correct a mistake in a completed training session?**

A: You cannot modify completed sessions. However, you can create a new training session to address gaps.

**Q: Can I export training data for external analysis?**

A: Yes, use the Export button on the analytics dashboard to download CSV or Excel files.

**Q: Who can approve training proposals?**

A: Users with ADMIN or SUPER_ADMIN role can approve proposals. Regular users cannot.

**Q: Is training mandatory?**

A: Training is mandatory for STUDENT agents blocked from automated triggers. For other levels, it's optional but recommended.