Crawler results not being sent to webhook


#1

Hi there,

I was testing out one of the crawler templates (eg. Example_Hacker_News) to see how data is being sent to a webhook URL, but when viewing the request sent to the webhook, the data is null.

Steps I took:

  1. Added a webhook URL to the ‘Finish webhook URL’ field (used https://webhook.site to test this)
  2. Saved crawler (no other changes made to the crawler template)
  3. Ran crawler
  4. Webhook status in the Run Console > Info tab = HTTP status: 200
  5. Webhook request body:
    { “_id”: “QXGPd2zyQ58fxEFNc”, “actId”: “gRnNAv8buGxswZRHa”, “data”: null }

Is there anything in particular I need to configure to ensure the crawled data is sent to the webhook correctly?

Thanks!


#2

Hi @daveeeo,

Your webhook was sent correctly.
If you want to pass any data to your webhook data you need to use “Finish webhook data” for it.

If you want to get results from webhook you need to call one more API call to retrieve results.


#3

Thanks @drobnikj! Retrieving the results via the API call did the job :slight_smile:


#4

No problem.

Happy crawling! :slight_smile: