Repository navigation
Add Firecrawl toolkit (web search, URL reader) - #682
Conversation
|
Hi @JuampiHernandez , Recently we decided to limit vendor specific integrations, and reserve them only as an opportunity to partner with established companies. Once a component goes into the core of the framework it must be added to the official documentation, making us responsible for it. We can't do this with everyone, so we ask for a collaboration and a commitment to add Neuron AI and this integration in your official documentation as a supported PHP framework, and announce this support to your community via social media or newsletter. We'll obviously do the same. This simply allows a little commitment by both, Firecrawl and Neuron AI. If something changes in your integration pattern, you would remember to update the supported integration, or let us know at least. If this sounds good for you I'll review your PR soon, otherwise you can consider to ship a separate package under your management and publish it to NeuronHub. |
|
Hi Valerio, thanks for the quick reply. Sounds good on our side. Once the toolkit is merged and released, we'll add Neuron AI to the agent frameworks section of the Firecrawl docs, next to the other frameworks we support. If anything changes in our API that affects the toolkit, I'll open a PR here or let you know. We don't announce this kind of integration on our socials, so the docs page will be our side of it. The PR is ready for your review whenever you have time, and I'll send the docs section for neuron-php-doc as soon as it's released. |
|
Due to how 4.x manage toolkit guidelines and prompt caching, the tool description should go into each tool class, and put in the guidelines the general strategy to use the toolkit. Check the Taviliy toolkit as an example. Fixing the prompts makes this PR mergable. |
10a9eb9 to
acc48b4
Compare
|
Thanks, done. Each tool now carries its own guidance in its description, like the other web toolkits, and the guidelines only say when to read a page after a search. Also rebased on the latest 4.x. |
|
Released in the latest 4.1.3 - Send me the link to the Firecrawl documentation so I can reference it in our doc. |
|
Thanks for merging and releasing it! Our page is live at https://docs.firecrawl.dev/integrations/neuron-ai |
Hey, I'm Juampi from the Firecrawl team.
This adds a Firecrawl toolkit under
src/Tools/Toolkits/Firecrawl/with two tools:FirecrawlSearchTool(web_search) searches the web, or news when the model setstopic: news, with an optionaltime_range. If you passscrapeOptionsthroughwithOptions(), each result also carries the full page as Markdown, so the agent often doesn't need a second call.FirecrawlScrapeTool(url_reader) returns a page's main content as Markdown. An invalid URL, an unreachable host, or a page that answers 4xx/5xx goes back to the model as a tool error. API errors (bad key, rate limit) raiseHttpException.Both follow the shape of the existing web toolkits:
make()with the key and an optionalHttpClientInterface,withOptions()to replace the defaults,$topicsfor the search description. No new dependencies. I also added the toolkit toToolkitCredentialsTrustBoundarySecurityTest.I left crawl out because Firecrawl's crawl endpoint is asynchronous (start a job, then poll it), which doesn't fit a blocking tool call.
Tested with
vendor/bin/phpunit tests/Tools/Toolkits/Firecrawland the shared credentials test, andphpstan analyseon the new files, all clean, plus a live run: anAgentwithFirecrawlToolkit::make(key: getenv('FIRECRAWL_API_KEY'))searched, read php.net to confirm the current PHP release, and read docs.neuron-ai.dev/agent/tools.I have the docs section ready for neuron-php-doc and will open it once this is released.
Closes #681