Dify

Official Dify Plugin: marketplace.dify.ai/plugins/langgenius/firecrawlOfficial plugin by Dify team • 170,000+ installs • Chatflow & Agent apps • Free to use

Dify Integration Overview

Dify is an open-source LLM app development platform. The official Firecrawl plugin enables web crawling and scraping directly in your AI workflows.

Chatflow & Workflow Apps

Build visual pipelines with Firecrawl nodes for data extraction

Agent Applications

Give AI agents the power to scrape live web data on demand

Firecrawl Tools in Dify

The plugin ships seven actions.

Search the web and optionally scrape the returned results, so you get fresh result metadata or full page content in one step.Use Cases: Research assistants, competitor discovery, grounding answers in live sources.

Scrape

Convert any URL into clean, structured data. Transform raw HTML into actionable insights.Use Cases: Extract product data, scrape article content, get structured data with JSON mode.

Crawl

Perform recursive crawls of websites and subdomains to gather extensive content.Use Cases: Full site content extraction, documentation scraping, multi-page data collection.

Map

Generate a complete map of all URLs present on a website.Use Cases: Site structure analysis, SEO auditing, URL discovery for batch scraping.

Crawl Job

Retrieve scraping results based on a Job ID or cancel ongoing tasks.Use Cases: Monitor long-running crawls, manage async scraping workflows, cancel operations when needed.

Create Monitor

Create a scheduled monitor that re-checks a target on a recurring schedule.Use Cases: Keeping ingested data fresh, changelog watching, competitor tracking.

Monitor Checks

Retrieve a monitor’s details and its check results, so you can act only on the pages that changed.Use Cases: Incremental knowledge base updates, change alerts, downstream triggers.

Getting Started

Install Firecrawl Plugin

Access the Dify Plugin Marketplace and install the Firecrawl tool

Get Firecrawl API Key

Visit Firecrawl API Keys and create a new API key

Authorize in Dify

Navigate to Plugins > Firecrawl > To Authorize and input your API key

Add to Your Workflow

Drag Firecrawl tools into your Chatflow, Workflow, or Agent application

Configure & Test

Set up parameters and test your workflow

Usage Patterns

Chatflow Apps
Workflow Apps
Agent Apps

Visual Pipeline Integration

Add Firecrawl node to your pipeline
Select action (Map, Crawl, Scrape)
Define input variables
Execute pipeline sequentially

Example Flow:

User Input → Firecrawl (Scrape) → LLM Processing → Response

Automated Data ProcessingBuild multi-step workflows with:

Scheduled scraping
Data transformation
Database storage
Notifications

Example Flow:

Schedule Trigger → Firecrawl (Crawl) → Data Processing → Storage

Common Use Cases

AI Chatbot with Live Data

Build RAG-powered chatbots that scrape and reference live website content

Content Analysis Agent

Agents that research topics by scraping and analyzing multiple sources

Competitor Monitoring

Automated workflows that track competitor websites and alert on changes

Data Enrichment Pipeline

Extract and enrich data from websites into structured databases

Firecrawl Actions

Tool	Description	Best For
Search	Web search with optional page content	Grounding answers in live sources
Scrape	Single-page data extraction	Quick content capture
Crawl	Multi-page recursive crawling	Full site extraction
Map	URL discovery and site mapping	SEO analysis, URL lists
Crawl Job	Async job management	Long-running operations
Create Monitor	Scheduled re-checks of a target	Keeping ingested data fresh
Monitor Checks	Monitor details and check results	Acting only on changed pages

Best Practices

Agent Apps

Let agents decide when to scrape
Use natural language instructions
Enable tool calling in LLM settings
Monitor token usage with large scrapes

Workflow Apps

Use Map before Crawl for large sites
Set appropriate crawl limits
Add error handling nodes
Test with small datasets first

Dify vs Other Platforms

Feature	Dify	Make	Zapier	n8n
Type	LLM app platform	Workflow automation	Workflow automation	Workflow automation
Best For	AI agents & chatbots	Visual workflows	Quick automation	Developer control
Pricing	Open-source + Cloud	Operations-based	Per-task	Execution-based
AI-Native	Yes	Partial	Partial	Partial
Self-Hosted	Yes	No	No	Yes

Pro Tip: Dify excels at building AI-native applications where agents need dynamic web access. Perfect for chatbots, research assistants, and AI tools that need live data.

Sync websites into the Dify knowledge base

Firecrawl can also scrape a web page into Markdown and import it into the Dify knowledge base from Dify Cloud.

Configuring Firecrawl

Configure Firecrawl credentials in the Data Source section of the Settings page.

Scrape the target webpage

On the knowledge base creation page, select Sync from website and enter the URL to scrape.

Configuration options include: whether to crawl sub-pages, page crawling limit, page scraping max depth, excluded paths, include only paths, and content extraction scope. After configuring, click Run to preview the parsed pages.

Review import results

Imported page text is stored in knowledge base documents. View the results and click Add URL to import more pages.

Get Started

Core Endpoints

More

Quickstarts

Developer Guides

Webhooks

Use Cases

Other

Contributing