mcp-server-webcrawl
Bridge web crawl data and AI language models using Model Context Protocol with full-text boolean search, multi-crawler support, content filtering, and seamless integration for building customizable website knowledgebases.
Supercharge Your AI with mcp-server-webcrawl
Unlock the full potential of mcp-server-webcrawl through LangDB's AI Gateway. Get enterprise-grade security, analytics, and seamless integration with zero configuration.
Free tier available • No credit card required
The path to the data source, varies by crawler type
The type of crawler to use (wget, warc, interrobot, katana, siteone)
Security Notice
Your environment variables and credentials are securely stored and encrypted. LangDB never shares these configuration values with third parties.
Model Context Protocol server enabling advanced web scraping, intelligent content search, site crawling, site mapping, and structured data extraction with customizable options, multi-language support, and API integration for efficient data retrieval and analysis.
5 toolsAdded May 30, 2025
Model Context Protocol server enabling efficient local search of AWS Lambda Powertools documentation across Python, TypeScript, Java, and .NET runtimes using lunr.js indexes for seamless integration with Large Language Models.
2 toolsAdded May 30, 2025
Model Context Protocol server providing REST API access to multi-agent AI tools for web search, data analysis, and custom task collaboration, enabling seamless integration with large language models.
Added May 30, 2025
Enables seamless integration of powerful language models via Model Context Protocol, offering anonymous API access, multi-turn conversation support, automatic model fallback, and advanced configuration controls for optimized AI interactions.
2 toolsAdded May 29, 2025
Provides real-time stock news search with customizable queries, domain-specific filtering, and seamless integration across MCP-compatible AI clients using the Model Context Protocol.
1 toolsAdded May 29, 2025