CROAK Overview
Computer Recognition Orchestration Agent Kit
CROAK is an open-source agentic framework that guides you through the complete lifecycle of building and deploying object detection models. It operates as a specialized "team" of AI agents callable from modern coding assistants like Claude Code.
When your model croaks, CROAK helps you figure out why.
Installation
Option 1: npm CLI (Recommended)
Requires Node.js 18+.
# Initialize a new CROAK project with interactive setup
npx croak-cv init
# Or install globally for repeated use
npm install -g croak-cv
croak init
Option 2: pip (Python Package)
pip install croak-cv
croak init
Option 3: From Source
git clone https://github.com/vfrog-ai/croak.git
cd croak
./install.sh # Unix/macOS
# or
./install.ps1 # Windows PowerShell
Quick Start
# 1. Initialize a new project
croak init
# 2. Check your environment (Python, GPU, vfrog CLI, etc.)
croak doctor
# 3. Add images to data/raw/ and scan them
croak scan
# 4. Follow the guided workflow
croak annotate # Annotate via vfrog SSAT or import from external tools
croak train # Train locally, on Modal.com, or on vfrog platform
croak evaluate # Evaluate model performance & diagnostics
croak deploy # Deploy to vfrog inference, Modal, or edge
How It Works
CROAK provides structured workflows through five specialist agents:
| Agent | Role | What It Does |
|---|---|---|
| Dispatcher | Pipeline Coordinator | Routes requests to specialists, tracks pipeline state |
| Scout | Data Engineer | Validates datasets, manages annotations (vfrog SSAT or classic) |
| Coach | Training Specialist | Configures and executes training across local GPU, Modal, or vfrog |
| Judge | Evaluation Specialist | Analyzes model performance with actionable diagnostics |
| Shipper | Deployment Specialist | Deploys to vfrog inference API, Modal serverless, or edge devices |
Each agent has guardrails to prevent common mistakes, a knowledge base for domain expertise, and handoff contracts for passing context between pipeline stages.
Annotation Paths
CROAK supports two annotation workflows. You are never locked into one path.
vfrog SSAT (Recommended)
Iterative auto-annotation powered by the vfrog CLI. Upload dataset images, create a reference object, run SSAT iterations, review labels in HALO, and train on vfrog's managed platform.
croak vfrog setup # Login and select organisation/project
croak annotate # Guided SSAT workflow
croak train --provider vfrog
Classic (Full Control)
Import annotations from external tools (CVAT, Label Studio, Roboflow, etc.) in YOLO, COCO, or VOC format. Train on your own GPU or on Modal.com.
croak annotate --method classic --format yolo --path ./annotations
croak train --provider local # or --provider modal
Comparison
| vfrog SSAT | Classic | |
|---|---|---|
| Annotation | Auto-annotation + HALO review | External tools (CVAT, Label Studio, etc.) |
| Training | vfrog managed platform | Local GPU or Modal.com |
| Deployment | vfrog inference API | Edge (ONNX, TensorRT) or Modal |
| Setup | croak vfrog setup | Bring your own annotations |
| Best for | Getting started quickly | Full control over pipeline |
Training Providers
| Provider | Command | Description |
|---|---|---|
| Local | croak train --provider local | Train on your own NVIDIA GPU |
| Modal | croak train --provider modal | Serverless GPU via Modal.com |
| vfrog | croak train --provider vfrog | Managed training on vfrog platform (requires vfrog annotations) |
Deployment Targets
| Target | Command | Description |
|---|---|---|
| vfrog | croak deploy vfrog | Managed inference API with auto-scaling |
| Edge | croak deploy edge | Export to ONNX, TensorRT, CoreML, or TFLite |
| Modal | croak deploy modal | Serverless inference via Modal.com |
Requirements
- Node.js 18.0.0+ (for CLI installer)
- Python 3.10+ (for training and evaluation)
- Git (recommended)
- vfrog CLI (optional but recommended for SSAT annotation and vfrog deployment)
- vfrog.ai account (for vfrog SSAT and inference)
- NVIDIA GPU (optional — can use Modal.com for cloud GPU)
Environment Variables
| Variable | Description | Required |
|---|---|---|
VFROG_API_KEY | vfrog.ai API key for inference | Only for croak deploy vfrog |
MODAL_TOKEN_ID | Modal.com token (via modal setup) | For cloud GPU training |
VFROG_API_KEY is only needed for inference. Annotation, training, and other vfrog operations use CLI authentication (croak vfrog setup).
Project Structure
After running croak init, your project will have:
your-project/
├── .claude/ # Claude Code integration
│ └── skills/
│ ├── croak-router/ # /croak-router skill
│ ├── croak-data/ # /croak-data skill
│ └── ... # Other agent skills
├── .croak/ # CROAK configuration
│ ├── config.yaml # Project configuration
│ ├── pipeline-state.yaml # Pipeline progress tracking
│ ├── agents/ # Agent YAML definitions
│ ├── workflows/ # Workflow specifications
│ ├── knowledge/ # Knowledge base
│ └── contracts/ # Handoff contracts
├── CLAUDE.md # Project context for Claude Code
├── data/
│ ├── raw/ # Raw images
│ └── processed/ # Processed datasets
├── training/
│ ├── configs/ # Training configurations
│ ├── scripts/ # Training scripts
│ └── experiments/ # Experiment outputs
├── evaluation/
│ └── reports/ # Evaluation reports
└── deployment/
└── edge/ # Edge deployment packages
Next Steps
- Command Reference — Complete list of CROAK commands
- Claude Code Integration — Use CROAK with slash commands in Claude Code
- CLI Overview — vfrog CLI setup and usage