plug.directory
Add my server

Install CrawlKit Scrapers in Goose

What you need first

  • Goose, installed.
  • Nothing else: the server runs on its publisher's machines and Goose connects to it.

Steps

  1. Open ~/.config/goose/config.yaml. On Windows it is %APPDATA%\Block\goose\config\config.yaml.
  2. Add this entry. If the file already has extensions, add only the entry inside it.
extensions:
  "crawlkit-scrapers":
    type: streamable_http
    name: "crawlkit-scrapers"
    enabled: true
    uri: "https://mcp.apify.com/?actors=sweet_stickball/booking-scraper,sweet_stickball/airbnb-scraper,sweet_stickball/amazon-reviews-scraper,sweet_stickball/aliexpress-scraper,sweet_stickball/google-shopping-scraper,sweet_stickball/zillow-scraper,sweet_stickball/reddit-scraper,sweet_stickball/google-jobs-scraper,sweet_stickball/indeed-job-scraper,sweet_stickball/yelp-scraper,sweet_stickball/google-news-scraper,sweet_stickball/google-flights-scraper,sweet_stickball/shopify-store-scraper,sweet_stickball/telegram-public-channel-scraper,sweet_stickball/bing-ddg-serp"
    headers: {}
    env_keys: []
    envs: {}
    timeout: 300
  1. Save the file and start a new Goose session.

Check it worked

Ask Goose: List the tools of crawlkit-scrapers.

If it does not start

401 Unauthorized
The endpoint wants a sign-in or a key. Sign in when Goose asks.
Nothing changes after a restart
The file is not valid YAML any more. Check that the entry is indented like the lines around it.

Back to CrawlKit Scrapers