mendableai_firecrawl_mcp_server
by mendableaiFirecrawl MCP Server for LLM Integration
Overview
The Firecrawl MCP Server is a powerful tool designed to integrate web scraping capabilities with Large Language Models (LLMs) like Claude, Cursor, and others. It provides advanced features such as web scraping with JavaScript rendering, URL discovery, batch processing, and deep research capabilities. This server is part of the Firecrawl ecosystem, enabling seamless interaction with web content for LLM applications.
Features
- Web Scraping with JS Rendering: Extract content from dynamic websites.
- Batch Processing: Efficiently scrape multiple URLs with built-in rate limiting.
- Deep Research: Conduct comprehensive web research using intelligent crawling and LLM analysis.
- Content Extraction: Extract structured data from web pages using LLMs.
- Rate Limiting & Retries: Automatic retries with exponential backoff for rate-limited requests.
- Credit Monitoring: Track API credit usage to avoid service interruptions.
- Mobile/Desktop Viewport Support: Scrape content tailored for different devices.
- Smart Content Filtering: Include or exclude specific HTML tags during extraction.
Installation
Running with npx
env FIRECRAWL_API_KEY=fc-YOUR_API_KEY npx -y firecrawl-mcp
Manual Installation
npm install -g firecrawl-mcp
Running on Cursor
- Open Cursor Settings.
- Go to Features > MCP Servers.
- Add a new MCP Server with the command:
shell env FIRECRAWL_API_KEY=your-api-key npx -y firecrawl-mcp
Running on Windsurf
Add the following to ./codeium/windsurf/model_config.json:
{
"mcpServers": {
"mcp-server-firecrawl": {
"command": "npx",
"args": ["-y", "firecrawl-mcp"],
"env": {
"FIRECRAWL_API_KEY": "YOUR_API_KEY_HERE"
}
}
}
}
Configuration
Environment Variables
- Required for Cloud API:
FIRECRAWL_API_KEY: Your Firecrawl API key.-
FIRECRAWL_API_URL: Custom API endpoint for self-hosted instances. -
Optional Configuration:
- Retry settings:
FIRECRAWL_RETRY_MAX_ATTEMPTS,FIRECRAWL_RETRY_INITIAL_DELAY, etc. - Credit monitoring:
FIRECRAWL_CREDIT_WARNING_THRESHOLD,FIRECRAWL_CREDIT_CRITICAL_THRESHOLD.
Usage
Available Tools
- Scrape Tool (
firecrawl_scrape): Scrape content from a single URL. - Batch Scrape Tool (
firecrawl_batch_scrape): Scrape multiple URLs efficiently. - Search Tool (
firecrawl_search): Search the web and extract content. - Crawl Tool (
firecrawl_crawl): Start an asynchronous crawl. - Extract Tool (
firecrawl_extract): Extract structured data using LLMs. - Deep Research Tool (
firecrawl_deep_research): Conduct deep web research. - Generate LLMs.txt Tool (
firecrawl_generate_llmstxt): Generate standardizedllms.txtfiles for websites.
Logging and Error Handling
- Comprehensive Logging: Tracks operation status, performance metrics, and credit usage.
- Error Handling: Automatic retries for transient errors and detailed error messages.
Development
# Install dependencies
npm install
# Build
npm run build
# Run tests
npm test
License
This project is licensed under the MIT License. See the LICENSE file for details.
About
The Firecrawl MCP Server is developed by MendableAI and is part of the Firecrawl ecosystem. It enhances LLM applications with powerful web scraping and content extraction capabilities.
Invite Gmail Contacts
