LangChain | Firecrawl

Official package: langchain-firecrawl · sourceScrape, crawl, map, extract, and search from LangChain, as a document loader or as agent tools

Install

pip install langchain-firecrawl

Get a key from firecrawl.dev and set it in your environment (or pass api_key=...):

export FIRECRAWL_API_KEY="fc-your-api-key"

Document loader

FirecrawlLoader returns LangChain Documents. Choose a mode:

Mode	What it loads
`scrape`	One page
`crawl`	A whole site
`map`	Discovered URLs
`extract`	Structured data
`search`	Web search results

from langchain_firecrawl import FirecrawlLoader

loader = FirecrawlLoader(url="https://www.firecrawl.dev", mode="scrape")
docs = loader.load()

print(docs[0].page_content[:200])
print(docs[0].metadata)

Agent tools

Each capability is also a BaseTool you can bind to an agent:

from langchain_firecrawl import (
    FirecrawlScrape,
    FirecrawlCrawl,
    FirecrawlMap,
    FirecrawlExtract,
    FirecrawlSearch,
)

scrape = FirecrawlScrape()
result = scrape.invoke({"url": "https://www.firecrawl.dev"})
print(result["markdown"])

search = FirecrawlSearch()
print(search.invoke({"query": "best web scraping libraries", "limit": 5}))

LangChain JS

The JavaScript loader ships in @langchain/community and takes the Firecrawl JS SDK as a peer dependency:

npm install @langchain/community @mendable/firecrawl-js

import { FireCrawlLoader } from "@langchain/community/document_loaders/web/firecrawl";

const loader = new FireCrawlLoader({
  url: "https://firecrawl.dev",
  apiKey: process.env.FIRECRAWL_API_KEY, // defaults to FIRECRAWL_API_KEY
  mode: "scrape", // "scrape" for a single page, "crawl" for subpages
});

const docs = await loader.load();

Resources