Provides Model Context Protocol tools enabling large language models to interact with web content by scraping URLs and returning markdown-formatted data for seamless integration and content retrieval.
Unlock the full potential of MCP Web Tools Server through LangDB's AI Gateway. Get enterprise-grade security, analytics, and seamless integration with zero configuration.
Free tier available • No credit card required
A Model Context Protocol (MCP) server that provides tools for web-related operations. This server allows LLMs to interact with web content through standardized tools.
Clone this repository:
git clone cd MCP
Install the required dependencies:
pip install -r requirements.txt
Alternatively, you can use uv for faster installation:
uv pip install -r requirements.txt
This repository includes convenient scripts to run either the MCP server or the Streamlit UI.
On macOS/Linux:
# Run the server with stdio transport (default) ./run.sh server # Run the server with SSE transport ./run.sh server --transport sse --host localhost --port 5000 # Run the Streamlit UI ./run.sh ui
On Windows:
# Run the server with stdio transport (default) run.bat server # Run the server with SSE transport run.bat server --transport sse --host localhost --port 5000 # Run the Streamlit UI run.bat ui
Alternatively, you can run the server directly:
python server.py
python server.py --transport sse --host localhost --port 5000
This will start an HTTP server on localhost:5000
that accepts MCP connections.
And to run the Streamlit UI manually:
streamlit run streamlit_app.py
The MCP Inspector is a tool for testing and debugging MCP servers. You can use it to interact with your server:
Install the MCP Inspector:
npm install -g @modelcontextprotocol/inspector
Run the Inspector with your server:
npx @modelcontextprotocol/inspector python server.py
Use the Inspector interface to test the web_scrape
tool by providing a URL like example.com
and viewing the returned markdown content.
To use this server with Claude for Desktop:
Make sure you have Claude for Desktop installed.
Open the Claude for Desktop configuration file:
~/Library/Application Support/Claude/claude_desktop_config.json
%APPDATA%\Claude\claude_desktop_config.json
Add the following configuration (adjust the path as needed):
{ "mcpServers": { "web-tools": { "command": "python", "args": [ "/absolute/path/to/MCP/server.py" ] } } }
Restart Claude for Desktop.
You should now see the web_scrape tool available in Claude's interface. You can ask Claude to fetch content from a website, and it will use the tool.
Once integrated with Claude, you can ask questions like:
Claude will use the web_scrape tool to fetch the content and provide it in its response.
To add more tools to this server:
Create a new Python file in the tools/
directory, e.g., tools/new_tool.py
.
Implement your tool function, following a similar pattern to the existing tools.
Import your tool in server.py
and register it with the MCP server:
# Import your new tool from tools.new_tool import new_tool_function # Register the tool with the MCP server @mcp.tool() async def new_tool(param1: str, param2: int) -> str: """ Description of what your tool does. Args: param1: Description of param1 param2: Description of param2 Returns: Description of return value """ return await new_tool_function(param1, param2)
This repository includes a Streamlit application that allows you to connect to and test all your MCP servers configured in Claude for Desktop.
streamlit run streamlit_app.py
This will start the Streamlit server and open a web browser with the UI.
requirements.txt
are installed.This project is available under the MIT License. See the LICENSE file for more details.
Discover shared experiences
Shared threads will appear here, showcasing real-world applications and insights from the community. Check back soon for updates!