Interactive benchmarking tool leveraging Model Context Protocol to evaluate vLLM models' performance by running customizable multi-iteration tests and comparing results for optimized model assessment.
Unlock the full potential of MCP vLLM Benchmarking Tool through LangDB's AI Gateway. Get enterprise-grade security, analytics, and seamless integration with zero configuration.
Free tier available • No credit card required
Discover shared experiences
Shared threads will appear here, showcasing real-world applications and insights from the community. Check back soon for updates!
This is proof of concept on how to use MCP to interactively benchmark vLLM.
We are not new to benchmarking, read our blog:
This is just an exploration of possibilities with MCP.
{
"mcpServers": {
"mcp-vllm": {
"command": "uv",
"args": [
"run",
"/Path/TO/mcp-vllm-benchmarking-tool/server.py"
]
}
}
}
Then you can prompt for example like this:
Do a vllm benchmark for this endpoint: http://10.0.101.39:8888
benchmark the following model: deepseek-ai/DeepSeek-R1-Distill-Llama-8B
run the benchmark 3 times with each 32 num prompts, then compare the results, but ignore the first iteration as that is just a warmup.
Enables AI assistants to interact with Kubernetes clusters via natural language using the Model Cont...
Advanced Model Context Protocol server enabling headless web browsing with multi-tab support, dynami...
A framework leveraging the Model Context Protocol for creating multi-agent AI systems with coordinat...
Provides Model Context Protocol functionality to retrieve token total supply by address and chain ID...
A Model Context Protocol server enabling AI models to execute SQL queries, manage SQLite database sc...