Puppeteer Crawler “BasicCrawler: handleRequestFunction …” error


#1

Hi,
I tested the examples on the SDK pages: Puppeteer Crawler

When I run the “Puppeteer Crawler”, in some cases, I get this error:

ERROR: BasicCrawler: handleRequestFunction failed, reclaiming failed request back to the list or queue {“url”:“http://example.com/logs/","retryCount”:1}
Error: Navigation failed because browser has disconnected!
at CDPSession.LifecycleWatcher._eventListeners.helper.addEventListener (D:\Developpement\NodeJS\CrawlUpDyn\node_modules\puppeteer\lib\FrameManager.js:1181:107)
at CDPSession.emit (events.js:182:13)
at CDPSession._onClosed (D:\Developpement\NodeJS\CrawlUpDyn\node_modules\puppeteer\lib\Connection.js:231:10)
at Connection._onClose (D:\Developpement\NodeJS\CrawlUpDyn\node_modules\puppeteer\lib\Connection.js:121:15)
at WebSocketTransport._ws.addEventListener.event (D:\Developpement\NodeJS\CrawlUpDyn\node_modules\puppeteer\lib\WebSocketTransport.js:45:22)
at WebSocket.onClose (D:\Developpement\NodeJS\CrawlUpDyn\node_modules\ws\lib\event-target.js:124:16)
at WebSocket.emit (events.js:182:13)
at WebSocket.emitClose (D:\Developpement\NodeJS\CrawlUpDyn\node_modules\ws\lib\websocket.js:180:10)
at Socket.socketOnClose (D:\Developpement\NodeJS\CrawlUpDyn\node_modules\ws\lib\websocket.js:803:15)
at Socket.emit (events.js:182:13)
at TCP._handle.close (net.js:610:12)
– ASYNC –
at Frame. (D:\Developpement\NodeJS\CrawlUpDyn\node_modules\puppeteer\lib\helper.js:144:27)
at Page.goto (D:\Developpement\NodeJS\CrawlUpDyn\node_modules\puppeteer\lib\Page.js:624:49)
at Page. (D:\Developpement\NodeJS\CrawlUpDyn\node_modules\puppeteer\lib\helper.js:145:23)
at PuppeteerCrawler.gotoFunction (D:\Developpement\NodeJS\CrawlUpDyn\main.js:60:25)
at PuppeteerCrawler._handleRequestFunction (D:\Developpement\NodeJS\CrawlUpDyn\node_modules\apify\build\puppeteer_crawler.js:316:35)
at process._tickCallback (internal/process/next_tick.js:68:7)

In the PuppeteerCrawler options, I didn’t find options.handleRequestFunction

What are the meaning of this error? Why BasicCrawler?

How can I fix/intercept them?

Regards,
Christian.


#2

Response in Issue #323 Puppeteer Crawler “BasicCrawler: handleRequestFunction …” error