Repository navigation
One-command workflow orchestration (msp-readiness run) #44
Description
Problem
Currently achieving MSP readiness requires multiple manual steps:
# 1. Generate playbooks
msp-readiness generate
# 2. Review playbooks (manual)
cd playbooks/ && ls
# 3. Collect evidence
msp-readiness collect-evidence
# 4. Assess completeness
msp-readiness assess --self
# 5. Review failures (manual)
cat assessment-report-fipco.md
# 6. Fix issues (manual)
# ... research, AWS CLI commands, code changes ...
# 7. Re-assess
msp-readiness assess --self
# 8. Approve playbooks (manual)
msp-readiness approve OPS-004,OPS-005,SEC-003,...
# 9. Generate dashboard
msp-readiness dashboard --workspaceThis defeats the automation goal. Users want:
msp-readiness run
# → 80%+ complete in one commandSolution
Create an orchestration command that runs the entire workflow:
msp-readiness run [options]What it does:
- ✅ Generate AWS-aware playbooks (Issue AWS-aware playbook generation (as-built documentation) #42)
- ✅ Collect evidence for all requirements (Issue Complete evidence collection for all 19 MSP requirements #40)
- ✅ Validate evidence against playbooks (Issue Add evidence validation to verify playbook compliance #39)
- ✅ Auto-approve validated playbooks (Issue Auto-approval logic for validated playbooks #41)
- ✅ Show remediation for failures (Issue Remediation engine with actionable fix guidance #43)
- ✅ Generate assessment report and dashboard
- ✅ Print summary with next steps
Orchestration Flow
┌─────────────────────────────────────────────────────────────┐
│ msp-readiness run --auto-approve │
└─────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ 1. Generate AWS-aware playbooks │
│ → 19 playbooks generated with actual AWS resource details│
└─────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ 2. Collect evidence from AWS │
│ → 19 evidence files collected │
│ → Handles missing permissions gracefully │
└─────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ 3. Validate evidence │
│ → Run 150+ validation checks │
│ → 12 requirements pass, 5 fail, 2 need manual work │
└─────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ 4. Auto-approve validated playbooks │
│ → 12 playbooks auto-approved │
│ → 5 marked needs-remediation │
│ → 2 marked needs-manual-input │
└─────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ 5. Generate reports │
│ → assessment-report-fipco.md/json │
│ → dashboard-workspace.md │
└─────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ 6. Display summary │
│ → Overall: 63% complete (12/19 requirements) │
│ → Next: Fix 5 validation failures (45 min estimated) │
│ → Show remediation commands │
└─────────────────────────────────────────────────────────────┘
Implementation Plan
1. Create Orchestrator (8h)
// src/orchestrator/run-workflow.ts
export interface RunOptions {
autoApprove: boolean;
awsAware: boolean;
collectEvidence: boolean;
validate: boolean;
generateReports: boolean;
showRemediation: boolean;
dryRun: boolean;
}
export async function runWorkflow(config: Config, options: RunOptions) {
const results = {
playbooks: { generated: 0, updated: 0, errors: 0 },
evidence: { collected: 0, failed: 0 },
validation: { passed: 0, failed: 0 },
approval: { autoApproved: 0, needsRemediation: 0, needsManual: 0 }
};
console.log('🚀 Starting MSP readiness workflow...\n');
// Step 1: Generate playbooks
console.log('📝 Generating AWS-aware playbooks...');
if (options.awsAware) {
results.playbooks = await generateAWSAwarePlaybooks(config);
} else {
results.playbooks = await generatePlaybooks(config);
}
console.log(` ✓ ${results.playbooks.generated} playbooks generated\n`);
// Step 2: Collect evidence
console.log('🔍 Collecting evidence from AWS...');
if (options.collectEvidence) {
results.evidence = await collectAllEvidence(config);
}
console.log(` ✓ ${results.evidence.collected} evidence files collected\n`);
// Step 3: Validate evidence
console.log('✅ Validating evidence...');
if (options.validate) {
const validationResults = await validateAllEvidence(config);
results.validation = summarizeValidation(validationResults);
}
console.log(` ✓ ${results.validation.passed} requirements validated\n`);
// Step 4: Auto-approve
console.log('🎯 Auto-approving validated playbooks...');
if (options.autoApprove) {
results.approval = await autoApprovePlaybooks(config);
}
console.log(` ✓ ${results.approval.autoApproved} playbooks auto-approved\n`);
// Step 5: Generate reports
console.log('📊 Generating reports...');
if (options.generateReports) {
await generateAssessmentReport(config);
await generateDashboard(config);
}
console.log(` ✓ Reports generated\n`);
// Step 6: Show summary
printWorkflowSummary(results);
if (options.showRemediation && results.approval.needsRemediation > 0) {
printRemediationGuidance(config);
}
return results;
}2. CLI Command (4h)
// src/cli.ts
program
.command('run')
.description('Run complete MSP readiness workflow')
.option('--no-auto-approve', 'Disable auto-approval of validated playbooks')
.option('--no-aws-aware', 'Generate generic playbooks instead of AWS-aware')
.option('--no-collect-evidence', 'Skip evidence collection')
.option('--no-validate', 'Skip evidence validation')
.option('--no-reports', 'Skip report generation')
.option('--show-remediation', 'Show detailed remediation guidance', true)
.option('--dry-run', 'Show what would be done without executing')
.action(async (options) => {
const config = loadConfig();
const runOptions: RunOptions = {
autoApprove: options.autoApprove !== false,
awsAware: options.awsAware !== false,
collectEvidence: options.collectEvidence !== false,
validate: options.validate !== false,
generateReports: options.reports !== false,
showRemediation: options.showRemediation,
dryRun: options.dryRun || false
};
await runWorkflow(config, runOptions);
});3. Progress Display (4h)
Real-time progress with spinners and status updates:
import ora from 'ora';
const spinner = ora('Collecting CloudTrail evidence...').start();
// ... collect evidence
spinner.succeed('CloudTrail evidence collected');
spinner.start('Validating backup configuration...');
// ... validate
if (validationPassed) {
spinner.succeed('Backup configuration validated');
} else {
spinner.fail('Backup configuration validation failed');
}4. Summary Report (4h)
Comprehensive end-of-run summary:
╔══════════════════════════════════════════════════════════════╗
║ MSP Readiness Workflow Complete ║
╚══════════════════════════════════════════════════════════════╝
📊 Overall Results:
✓ 12 requirements complete (63%)
⚠ 5 requirements need remediation (26%)
ℹ 2 requirements need manual input (11%)
📝 Playbooks:
✓ 19 generated
✓ 12 auto-approved
⚠ 5 need remediation
ℹ 2 need manual input
🔍 Evidence:
✓ 17 collected successfully
⚠ 2 collection failed (permissions)
✅ Validation:
✓ 12 requirements passed all checks
⚠ 5 requirements failed validation
⏩ 2 requirements skipped (no evidence)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
⚠️ Requirements Needing Remediation (5):
1. OPS-003: Monitoring and Alerting
Issue: Missing CloudWatch alarms for critical metrics
Time: 15 minutes
Fix: msp-readiness remediate OPS-003
2. OPS-004: Logging
Issue: CloudTrail retention < 90 days
Time: 10 minutes
Fix: msp-readiness remediate OPS-004
3. SEC-004: Identity and Access Management
Issue: 3 users without MFA enabled
Time: 20 minutes
Fix: msp-readiness remediate SEC-004
4. SEC-009: Data Protection
Issue: 2 S3 buckets without encryption
Time: 10 minutes
Fix: msp-readiness remediate SEC-009
5. SECP-002: Public Resources Detection
Issue: 1 S3 bucket publicly accessible
Time: 5 minutes
Fix: msp-readiness remediate SECP-002
Total estimated time: 60 minutes
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
ℹ️ Requirements Needing Manual Input (2):
1. OPSP-001: Incident Management
Action: Review and customize incident runbooks
Location: playbooks/incident-response.md
2. OPSP-002: Problem Management
Action: Add post-mortem template and examples
Location: playbooks/problem-management.md
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
📁 Generated Files:
assessment-report-fipco.md
assessment-report-fipco.json
dashboard-workspace.md
playbooks/ (19 files)
Next Steps:
1. Fix 5 validation failures: msp-readiness remediate --all
2. Review manual input requirements
3. Re-run workflow: msp-readiness run
4. Target: 95%+ completion after remediation
Benefits
- ✅ One command - From 0% to 80%+ in one run
- ✅ Zero manual steps - Fully automated workflow
- ✅ Clear next steps - Shows exactly what needs manual intervention
- ✅ Time estimates - Plan remediation work
- ✅ Progress visibility - Real-time status updates
- ✅ Comprehensive report - Detailed summary of all actions
- ✅ Flexible - Disable steps with flags for testing/debugging
Acceptance Criteria
-
runcommand orchestrates full workflow - Real-time progress display with spinners
- Comprehensive end-of-run summary
- Shows remediation guidance for failures
- Handles errors gracefully (continues workflow)
-
--dry-runflag shows what would happen - Flags to enable/disable individual steps
- Tests verify orchestration logic
- Documentation with examples
Estimated Effort
20 hours
- Orchestrator logic: 8h
- CLI command: 4h
- Progress display: 4h
- Summary report: 4h
Dependencies
- Issue Add evidence validation to verify playbook compliance #39: Evidence validation
- Issue Complete evidence collection for all 19 MSP requirements #40: Complete evidence collection
- Issue Auto-approval logic for validated playbooks #41: Auto-approval logic
- Issue AWS-aware playbook generation (as-built documentation) #42: AWS-aware playbook generation
- Issue Remediation engine with actionable fix guidance #43: Remediation engine
This is the culmination issue - brings everything together into one command.
Related Issues
- Add evidence validation to verify playbook compliance #39, Complete evidence collection for all 19 MSP requirements #40, Auto-approval logic for validated playbooks #41, AWS-aware playbook generation (as-built documentation) #42, Remediation engine with actionable fix guidance #43 (depends on all)
🤖 Generated with Claude Code
Activity