Production Ready v0.6.2 • 20 Core MCP Tools

Visual Memory MCP

Name: Visual Memory MCP
Author: PuterVision LLC

A persistent visual cache and perceptual hashing server for developer agents. Caches webpage and app screenshots locally to optimize context windows, accelerate execution speed, and eliminate redundant LLM vision costs.

Get Started IDE Configs Calculate ROI

RESEARCH PAPER GROUNDED

Theoretical Foundations, Multimodal Memory Architecture & Capabilities

📦 What Visual Memory MCP Provides

A persistent visual state cache, perceptual hashing server, and visual baseline verifier for multimodal developer AI agents. Caches webpage and application UI screenshots locally to optimize context windows, accelerate execution speed, and eliminate redundant LLM vision costs.

⚡ First-Hop Visual Determinism (90% Token Savings)

Grounded in First-Hop Visual Determinism principles, vision-memory-mcp externalizes visual UI state into persistent local memory resources over MCP. Bypasses redundant 4K screenshot re-sending on every turn, keeping active multimodal context focused strictly on current UI changes.

🔍 Sub-5ms 4-Tier Perceptual Caching Pipeline

Operates a 4-tier caching architecture: L1 In-Memory Cache (<1ms), L2 dHash/aHash Perceptual Matching (<5ms), L3 On-Device CLIP ViT-B/32 Vector Search over LanceDB, and L4 Vision LLM Fallback. Recognizes identical or near-duplicate UI screens instantly.

📐 Visual Spec-Driven Development & Layout Diffs

Registers UI design mockups as visual contract baselines (`set_visual_spec`) and verifies live UI execution against baselines (`verify_visual_spec`, `get_visual_diff`). Calculates layout region deltas and generates visual blockers when UI design drift occurs.

🚀 Subterranean Trajectory Fine-Tuning

Exposes `export_visual_trajectories` to export multimodal UI state transition paths. Allows developers to fine-tune local open-source models (3B–8B parameters) to execute complex screen workflows implicitly without explicit prompt flowcharts.

📂 Multi-Database Sub-Directory Monorepo Discovery

Automatically scans workspace sub-directories, submodules, and sub-packages (`.vision-memory-mcp` or `.vision-memory`), federating multi-database visual memory queries across complex enterprise repositories.

Interactive Visual Graph Map Visualizer vision-memory-mcp view

💡 Tiered Visual Verification & Tooling

Persists layout states locally. Eliminates redundant screenshot ingestion overhead for developers and agent blackboards.

👁️ L1/L2 Exact Cache Hit

Instantly matches screenshots using exact perceptual Difference Hashing (dhash) in less than 5ms, avoiding API calls entirely.

🧠 L3 Local CLIP Vector Search

Runs local vector similarity search against CLIP image embeddings using LanceDB, identifying layout templates even under minor CSS changes.

🌳 AX-Tree Validation

Cross-references accessibility tree strings to detect structural shifts (e.g. content changes in forms), ensuring zero cache-hit false positives.

🩺 System Health Checks

Use vision-memory-mcp doctor to diagnose LanceDB write permissions, sharp image binary support, git environment, and Node runtime.

📉 Token & Cost Savings

Screenshots consume thousands of context tokens. Recovers 95%+ input token space by replacing raw images with cached text state descriptions.

📸 Visual Checkpoints & Diffing

Save snapshots of memory state and diff checkpoints using snapshot save and snapshot diff to track layout drift.

💻 Powerful CLI Suite

Inspect states, compute ROI, run health checks, and manage visual memory graph checkpoints.

🩺 doctor

Run environment health checks verifying LanceDB storage writability, Sharp bindings, sub-directory Git repos, and Node runtime.

vision-memory-mcp doctor

🔍 audit

Audit sub-directory Git repos, submodules, sub-directory memory database locations, and total visual states.

vision-memory-mcp audit

📊 metrics

Calculate cache hit rate, token savings, estimated cost reduction, and database size.

vision-memory-mcp metrics

🌐 view

Launch an interactive HTML force-directed graph view of the memory in your browser.

vision-memory-mcp view

📸 snapshot

Save memory state checkpoints and diff two snapshots to catch visual drift and layout shifts.

vision-memory-mcp snapshot save baseline

🔌 Full MCP Specification & Tool Reference

Equipped with complete spec features including annotations, resources, and all 20 core tools.

FULL 20 MCP TOOL SUITE

Complete Tool Definitions & Function Signatures

🖼️ Visual Cache & Ingestion (3 Tools)

analyze_screenshot, recall_memory, batch_analyze_screenshots.

How to Leverage: Call analyze_screenshot before sending any image to an external visual LLM. If is_known: true, reuse the cached layout description instantly in <5ms, bypassing redundant 1,400+ vision token API calls!

⚡ View JSON Payload Example

// Request
{ "screenshot": "/path/to/login.png", "route": "/login" }

// Response
{ "is_known": true, "state_id": "state_login_01", "similarity": 0.992 }

🧭 Trajectory & State Navigation (4 Tools)

record_outcome, get_navigation_paths, get_session_context, predict_next_action.

How to Leverage: Call record_outcome after every UI action (click/type). When navigating complex UI flows, call get_navigation_paths to retrieve optimal BFS click pathways to reach target screens.

⚡ View JSON Payload Example

// Request
{ "from_state_id": "login", "to_state_id": "dashboard", "action": "click('#submit-btn')" }

// Response
{ "recorded": true, "success_rate": 1.0 }

📐 Layout Diffs & Spec Compliance (4 Tools)

compare_states, get_visual_diff, set_visual_spec, verify_visual_spec.

How to Leverage: Use set_visual_spec to register Figma design contract baselines. Call verify_visual_spec in Playwright/Cypress pipelines to catch layout drift and visual regressions automatically.

⚡ View JSON Payload Example

// Request
{ "route": "/checkout", "live_screenshot": "/path/to/live.png" }

// Response
{ "compliant": true, "structural_diff_pct": 0.8 }

📸 Checkpoints, Privacy & Fine-Tuning (9 Tools)

save_visual_snapshot, diff_visual_snapshots, undo_last_visual_mutation, create_visual_blocker, forget_state, export_visual_trajectories, get_metrics, export_snapshot, restore_snapshot.

How to Leverage: Scrub sensitive memory via forget_state, call export_snapshot / restore_snapshot for standalone backups, query real-time performance via get_metrics, or call export_visual_trajectories to export UI interaction paths in JSONL format to fine-tune local models.

⚡ View JSON Payload Example

// Request
{ "format": "jsonl", "output_path": "./visual_trajectories.jsonl" }

// Response
{ "exported_count": 28, "status": "success" }

📁 Exposed Visual Resources

Provides read-only visual contexts using RFC 6570 templates under the vision-memory:/// URI scheme:

vision-memory:///{project}/summary

Visual cache statistics, total states, hit rates, and disk usage metrics.
vision-memory:///{project}/states

Recent visual state list with thumbnails and perceptual dHash records.
vision-memory:///{project}/transitions

Graph of visual UI transitions and navigation paths between states.
vision-memory:///{project}/snapshots

Historical visual checkpoints for visual layout diffing and regression auditing.

💬 Reusable Prompts

Template instructions that automatically fetch visual state data to bootstrap vision workflows:

analyze-ui-state

Analyzes UI screen layout, input fields, interactive controls, and active alerts for a state.
diagnose-visual-regression

Compares baseline vs current snapshot checkpoints to diagnose visual layout drift.
navigate-to-goal

Formulates step-by-step navigation paths from current state to reach a goal state.

📘 Formal API Reference

Complete OpenAPI & JSON-RPC 2.0 schema specifications, input parameter types, and example payloads for all 19 vision-memory-mcp tools.

View Formal API Reference →

🔒

Local-First Privacy & Zero Telemetry

100% On-Device Data Sovereignty

All screenshots, perceptual hashes, vector embeddings, and transition graphs are stored locally on your machine inside .vision-memory-mcp/ using embedded LanceDB. Zero telemetry, tracking, or external API calls are made unless you explicitly configure L4 LLM Vision fallback endpoints.

📊 Visual Cache Value & ROI Estimator

Estimate the API cost and developer time saved by caching visual states rather than re-evaluating raw images repeatedly.

Cached Screenshots (Avoided Requests) 150

Avg Vision LLM Prompt Size (Tokens per Image) 1,400

Average Layout Cache-Hit Ratio 75%

Target Visual LLM Pricing

Estimated API Cost Saved

$0.47

LLM Context Tokens Saved

157,500 tokens

Visual Wait Time Saved

7.5 minutes

Calculating ROI...

* Disclaimer & Cost Risk Notice: All ROI calculations, token savings estimates (e.g. up to 90%), and latency metrics are illustrative benchmark estimates based on selected visual LLM token rates ($3.00/M tokens for Claude 3.5 Sonnet) and average perceptual cache-hit ratios. Actual token savings, execution speedups, and financial drops vary depending on workflow screen repetition, image resolution, model provider pricing, and prompt structure. PuterVision LLC does not guarantee specific monetary savings or API throughput performance.

🚀 Quickstart

Get vision-memory-mcp configured globally in 10 seconds.

1. Install @putervision/vision-memory-mcp globally

npm install -g @putervision/vision-memory-mcp

2. Initialize inside your project repository

vision-memory-mcp init --yes

3. Run environment health checks

vision-memory-mcp doctor

4. Display ROI and cache metrics from CLI

vision-memory-mcp metrics

🔌 Client & IDE Integrations

Zero manual effort: vision-memory-mcp init automatically scaffolds config files for all supported IDEs.

⚡ Recommended: 1-Step Auto-Scaffolding

Run vision-memory-mcp init --yes inside your repository. It automatically generates and updates configuration files across all your installed agent environments, including workspace agent rules (.agents/AGENTS.md) and database paths.

vision-memory-mcp init --yes

👽 Google Antigravity

Global: ~/.gemini/config/mcp_config.json
Workspace: .agents/settings.json

{
  "mcpServers": {
    "vision-memory-mcp": {
      "command": "vision-memory-mcp",
      "args": ["run"]
    }
  }
}

🤖 Claude Code & Desktop

One-line CLI: claude mcp add vision-memory-mcp -- vision-memory-mcp run
Desktop config: claude_desktop_config.json

{
  "mcpServers": {
    "vision-memory-mcp": {
      "command": "vision-memory-mcp",
      "args": ["run"]
    }
  }
}

🪄 Cursor & VS Code

Cursor: .cursor/mcp.json
VS Code: .vscode/mcp.json

{
  "mcpServers": {
    "vision-memory-mcp": {
      "command": "vision-memory-mcp",
      "args": ["run"]
    }
  }
}

🌊 Windsurf & Roo / Cline

Windsurf: ~/.codeium/windsurf/mcp_config.json
Roo Code / Cline: .cline/mcp.json

{
  "mcpServers": {
    "vision-memory-mcp": {
      "command": "vision-memory-mcp",
      "args": ["run"]
    }
  }
}

🤖 Agent System Rules Prompt

Add this rule to AGENTS.md, .cursorrules, or .windsurfrules to instruct your AI agent to check the visual cache automatically:


# Visual Memory Rules
This project uses vision-memory-mcp to cache visual UI states.

1. Call analyze_screenshot with base64 screenshots before visual checks.
2. On Cache Hit (is_known: true): Reuse cached description; skip external vision model.
3. On Cache Miss (is_known: false): Query vision model, then call analyze_screenshot to seed cache.
4. Call record_outcome after click/type/navigation actions.

Visual Memory MCP

Theoretical Foundations, Multimodal Memory Architecture & Capabilities

📦 What Visual Memory MCP Provides

⚡ First-Hop Visual Determinism (90% Token Savings)

🔍 Sub-5ms 4-Tier Perceptual Caching Pipeline

📐 Visual Spec-Driven Development & Layout Diffs

🚀 Subterranean Trajectory Fine-Tuning

📂 Multi-Database Sub-Directory Monorepo Discovery

💡 Tiered Visual Verification & Tooling

👁️ L1/L2 Exact Cache Hit

🧠 L3 Local CLIP Vector Search

🌳 AX-Tree Validation

🩺 System Health Checks

📉 Token & Cost Savings

📸 Visual Checkpoints & Diffing

💻 Powerful CLI Suite

🩺 doctor

🔍 audit

📊 metrics

🌐 view

📸 snapshot

🔌 Full MCP Specification & Tool Reference

Complete Tool Definitions & Function Signatures

🖼️ Visual Cache & Ingestion (3 Tools)

🧭 Trajectory & State Navigation (4 Tools)

📐 Layout Diffs & Spec Compliance (4 Tools)

📸 Checkpoints, Privacy & Fine-Tuning (9 Tools)

📁 Exposed Visual Resources

vision-memory:///{project}/summary

vision-memory:///{project}/states

vision-memory:///{project}/transitions

vision-memory:///{project}/snapshots

💬 Reusable Prompts

analyze-ui-state

diagnose-visual-regression

navigate-to-goal

📘 Formal API Reference

Local-First Privacy & Zero Telemetry

📊 Visual Cache Value & ROI Estimator

🚀 Quickstart

1. Install @putervision/vision-memory-mcp globally

2. Initialize inside your project repository

3. Run environment health checks

4. Display ROI and cache metrics from CLI

🔌 Client & IDE Integrations

⚡ Recommended: 1-Step Auto-Scaffolding

👽 Google Antigravity

🤖 Claude Code & Desktop

🪄 Cursor & VS Code

🌊 Windsurf & Roo / Cline

🤖 Agent System Rules Prompt