In today’s complex vendor landscape, organizations need smarter ways to evaluate third-party risk. Vanguard-MAS (Multi-Agent System) transforms vendor risk assessments from manual questionnaires into intelligent, automated investigations powered by specialized AI agents working in parallel.
Background
Developed to operationalize the University’s Vendor Risk Management Policy smoothly and efficiently, Vanguard-MAS addresses the challenge of scaling vendor assessments while maintaining rigorous security standards. Human review remains an integral part of the process.
What is Vanguard-MAS?
Vanguard-MAS is a provider-agnostic multi-agent system that transforms user-defined risk criteria into structured investigations. Using specialized AI agents that work in parallel and sequence, it produces vetted, multi-perspective vendor recommendations with verified citations.
Key Features
- Parallel Investigation: Multiple agents research different risk domains simultaneously for faster assessments
- Hybrid Search Strategy: Combines site-scoped searches with general web searches for comprehensive vendor intelligence
- URL Validation: All findings are validated for accessibility (HTTP 200) before inclusion
- Adversarial Analysis: A Devil’s Advocate agent challenges findings to identify hidden risks
- Lawsuit Detection: Searches for active litigation with case details and verified sources
- Breach Mitigation Analysis: Evaluates control adequacy when breaches are found
- Provider-Agnostic: Supports Anthropic Claude, OpenAI, DeepSeek, Perplexity, and OpenAI Search
- Web Interface: React-based UI with real-time progress tracking
Web Interface
Vanguard-MAS has a simple web interface you can open in a browser to run assessments and see progress update in real time.
Interface Screenshots
Assessment Approach: PVE + Assessment Modes
Vanguard-MAS uses PVE classification to determine the appropriate depth of vendor assessment, followed by one of two assessment modes based on your vendor relationship:
PVE Classification (Product Vulnerability Exposure)
What is PVE?
Product Vulnerability Exposure (PVE) measures the potential impact on an organization if an ICT product fails or is compromised—whether due to technical weaknesses in the product itself or the vendor’s inability to manage security or operational risks.
PVE focuses on the consequences of failure or security incidents, not their likelihood. It assesses how severe the impact would be on operations, data, recovery capability, and resilience if an incident were to occur.
PVE Classification Criteria:
| Classification | Description |
|---|---|
| Low Impact | Failures have little to no organizational impact, do not materially disrupt operations, and can be handled using existing incident response capabilities |
| Medium Impact | Failures cause moderate impact that may disrupt services or expose data, but the organization can fully recover within established service restoration timeframes |
| High Impact | Failures have severe impact that may exceed recovery thresholds or be partially/fully irrecoverable, leading to significant operational, financial, legal, or reputational harm |
How PVE Is Used:
- Standalone triage: Run PVE alone to quickly classify business impact before committing resources to a full assessment
- With Assess Mode: After PVE classification, run targeted Assess mode focused on specific capabilities relevant to the PVE level
- With VRM Mode: PVE is integrated as Phase I of the VRM framework for comprehensive SaaS evaluations
PVE-Calibrated Assessment Scope:
| PVE Level | Data Sensitivity | Assessment Scope |
|---|---|---|
| LOW | Public / Internal | Light check: Security documentation, basic certifications, incident response policies |
| MEDIUM | Some PII / Location data | Standard check: SOC 2/ISO 27001 verification, data handling practices, breach history |
| HIGH | Credentials / PHI / Financial | Deep dive: Full security questionnaire, penetration testing reports, subcontractor audits, regulatory compliance validation |
Key Clarifications:
- Product-centric: PVE evaluates the ICT product’s exposure, not vendor maturity (assessed separately under Vendor Capability)
- Cost and size irrelevant: Even low-cost or widely used tools can have high PVE if their failure would significantly impact operations
- Feeds overall risk: PVE combines with Vendor Capability to determine final risk exposure and required mitigation measures
PVE helps organizations calibrate their assessment depth based on actual business risk—faster onboarding for low-risk vendors while maintaining rigor for high-risk relationships.
1. Assess Mode
What is Assess Mode?
Assess Mode uses a custom risk criteria approach with a dedicated verification phase. It provides flexibility to define your own risk domains and requirements, making it ideal for targeted assessments of specific vendor capabilities.
When to Use Assess Mode:
- After PVE classification for Low/Medium impact vendors requiring focused capability review
- Infrastructure/platform vendors (AWS, GCP, Cloudflare) where compliance certifications and incident response matter more than data handling
- Custom risk domains outside the standard VRM framework
Assess Mode Pipeline:
A six-phase pipeline that transforms custom risk criteria into actionable recommendations:
- Phase I: Orchestrator decomposes criteria into tasks
- Phase II: Research — Parallel investigation across domains
- Phase III: Verify — URL validation and fact-checking
- Phase IV: Analyze — Map findings to requirements
- Phase V: Challenge — Adversarial analysis for counter-arguments
- Phase VI: Synthesize — Final Go/No-Go/Conditional recommendation
2. VRM Mode
What is VRM Mode?
The VRM (Vendor Risk Management) mode provides a structured framework assessment with integrated PVE classification and risk matrix analysis. It is ideal for SaaS vendors handling sensitive data, offering a comprehensive evaluation across four maturity domains.
When to Use VRM Mode:
- SaaS vendors storing or processing sensitive data (PII, credentials, PHI)
- High PVE vendors requiring full framework assessment
- Regulatory requirements (SOC 2, ISO 27001, HIPAA) that demand comprehensive evaluation
VRM Mode Pipeline:
A comprehensive seven-phase framework:
- Phase I: PVE Classification — Business impact scoring (Low/Medium/High)
- Phase II: Vendor Capability — Parallel research across 4 maturity domains
- Phase III: Risk Matrix — Initial inherent risk (VC × PVE)
- Phase IV: Controls — Identify mitigating security controls
- Phase V: Challenge — Adversarial analysis for hidden risks
- Phase VI: Residual Risk — Final risk after controls
- Phase VII: Conclusion — Engage/Conditional/Do Not Engage
Vendor Capability (VC) Maturity Domains:
- A. Operational Maturity: Data retention, disposal, anonymization, data residency & sovereignty
- B. Technical Maturity: Integration security, data integrity, encryption (at rest & in transit), access management, privacy compliance
- C. Engineering Maturity: Code vulnerability management, secure development practices, ethical AI (if applicable)
- D. Risk Management Maturity: Subcontractor oversight, third-party risk management, independent security assurance (SOC 2, ISO 27001)
Risk Matrix Decision Logic:
| Vendor Capability | PVE: Low | PVE: Medium | PVE: High |
|---|---|---|---|
| VC: Low | ✅ Low Risk | ❌ High Risk | ❌ High Risk |
| VC: Medium | ✅ Low Risk | ⚠️ Medium Risk | ❌ High Risk |
| VC: High | ✅ Low Risk | ✅ Low Risk | ⚠️ Medium Risk |
Multi-Agent Architecture
Vanguard-MAS leverages specialized AI agents that collaborate in parallel and sequence to transform vendor risk assessments into intelligent investigations. Each agent has a defined role:
| Agent | Role | Modes |
|---|---|---|
| Orchestrator | Manages assess mode workflow, decomposes criteria into tasks | Assess |
| VRMOrchestrator | Manages VRM framework workflow, maps findings to VRM schema | VRM |
| InvestigatorAgent | Executes web searches, gathers findings from multiple sources | Both |
| AuditorAgent | URL validation, fact-checking, confidence filtering | Assess |
| RiskAnalystAgent | Maps findings against user requirements | Assess |
| DevilsAdvocateAgent | Finds counter-arguments and weak signals (lawsuits, litigation, CVEs) | Both |
| URLValidator | Validates URL accessibility (HTTP 200) before inclusion | Both |
How Agents Work Together:
In Assess mode, the pipeline follows a six-phase flow:
- Orchestrator decomposes risk criteria into domain-specific tasks
- InvestigatorAgent executes parallel web searches (site-scoped + general) for each domain
- AuditorAgent validates URLs and fact-checks findings
- RiskAnalystAgent maps verified findings to requirements
- DevilsAdvocateAgent challenges findings with topic-specific negative queries
- Final report synthesizes all agent outputs into a Go/No-Go/Conditional recommendation
In VRM mode, the VRMOrchestrator manages a structured seven-phase framework where the InvestigatorAgent and DevilsAdvocateAgent work with maturity domain scoring, and URLValidator ensures continuous verification.
*For detailed technical documentation on agent roles, pipelines, and configuration, see the project documentation. Code will be available soon on GitHub.
Why It Matters
Traditional VRM assessments apply the same detailed security questionnaire regardless of vendor risk profile. Vanguard-MAS introduces calibrated assessment—focusing due diligence efforts proportionally to actual business impact.
The framework distinguishes between inherent risk (before controls) and residual risk (after mitigating controls), enabling organizations to:
- Apply Standard InfoSec for low-risk engagements
- Require Tailored InfoSec for medium-risk scenarios
- Reject high-risk vendors before investing in lengthy assessments
The result: Faster vendor onboarding without compromising security scrutiny.
Future Directions
Vanguard-MAS continues to evolve with planned enhancements across several areas:
- Integration & API: RESTful API for programmatic access and integration with procurement workflows
- Continuous Monitoring: Automated re-assessment of vendors based on triggers (time-based, breach news, certification expiry)
- Multi-Vendor Comparison: Side-by-side analysis and scoring for vendor selection decisions
- Enhanced Storage: Database backend for scalable assessment management and advanced querying
- Collaborative Review: Team workflow for assessment review, comments, and approval chains
- Benchmarking: Industry-wide risk scoring and peer vendor comparisons
- Custom Frameworks: Configurable risk frameworks supporting organization-specific assessment criteria beyond VRM
Resources:
- Project Documentation
- GitHub Repository (code coming soon)