Provides Model Context Protocol tools enabling large language models to interact with web content by scraping URLs and returning markdown-formatted data for seamless integration and content retrieval.
Unlock the full potential of MCP Web Tools Server through LangDB's AI Gateway. Get enterprise-grade security, analytics, and seamless integration with zero configuration.
Free tier available • No credit card required
Discover shared experiences
A Model Context Protocol (MCP) server that provides tools for web-related operations. This server allows LLMs to interact with web content through standardized tools.
Clone this repository:
git clone cd MCP
Install the required dependencies:
pip install -r requirements.txt
Alternatively, you can use uv for faster installation:
uv pip install -r requirements.txt
This repository includes convenient scripts to run either the MCP server or the Streamlit UI.
On macOS/Linux:
# Run the server with stdio transport (default) ./run.sh server # Run the server with SSE transport ./run.sh server --transport sse --host localhost --port 5000 # Run the Streamlit UI ./run.sh ui
On Windows:
# Run the server with stdio transport (default) run.bat server # Run the server with SSE transport run.bat server --transport sse --host localhost --port 5000 # Run the Streamlit UI run.bat ui
Alternatively, you can run the server directly:
python server.py
python server.py --transport sse --host localhost --port 5000
This will start an HTTP server on localhost:5000
that accepts MCP connections.
And to run the Streamlit UI manually:
streamlit run streamlit_app.py
The MCP Inspector is a tool for testing and debugging MCP servers. You can use it to interact with your server:
Install the MCP Inspector:
npm install -g @modelcontextprotocol/inspector
Run the Inspector with your server:
npx @modelcontextprotocol/inspector python server.py
Use the Inspector interface to test the web_scrape
tool by providing a URL like example.com
and viewing the returned markdown content.
To use this server with Claude for Desktop:
Make sure you have Claude for Desktop installed.
Open the Claude for Desktop configuration file:
~/Library/Application Support/Claude/claude_desktop_config.json
%APPDATA%\Claude\claude_desktop_config.json
Add the following configuration (adjust the path as needed):
{ "mcpServers": { "web-tools": { "command": "python", "args": [ "/absolute/path/to/MCP/server.py" ] } } }
Restart Claude for Desktop.
You should now see the web_scrape tool available in Claude's interface. You can ask Claude to fetch content from a website, and it will use the tool.
Once integrated with Claude, you can ask questions like:
Claude will use the web_scrape tool to fetch the content and provide it in its response.
To add more tools to this server:
Create a new Python file in the tools/
directory, e.g., tools/new_tool.py
.
Implement your tool function, following a similar pattern to the existing tools.
Import your tool in server.py
and register it with the MCP server:
# Import your new tool from tools.new_tool import new_tool_function # Register the tool with the MCP server @mcp.tool() async def new_tool(param1: str, param2: int) -> str: """ Description of what your tool does. Args: param1: Description of param1 param2: Description of param2 Returns: Description of return value """ return await new_tool_function(param1, param2)
This repository includes a Streamlit application that allows you to connect to and test all your MCP servers configured in Claude for Desktop.
streamlit run streamlit_app.py
This will start the Streamlit server and open a web browser with the UI.
requirements.txt
are installed.This project is available under the MIT License. See the LICENSE file for more details.
Shared threads will appear here, showcasing real-world applications and insights from the community. Check back soon for updates!