Loop through URLs with ID


#1

Hello, newbe here!

I’m trying to crawl through a list of URLs that end in “id=XX”, where XX is a number from 40 to 50.

So I do as follows…

Start-URL: https://www.mytest/names.php?id=40
Pseudo-URLs: https://www.mytest/names.php?id=[(4[1-9]|50)]

but it doesn’t pass the first URL.

I’ve read the tutorial on “crawling multiple pages” but it didn’t help me much. The pages have no “Next” link or similar linking one to the next.

what am I missing?

Thanks a bunch!!!


#2

Hi @2Geckos_Tech,

for a Crawler, the best way is to use context.enqueuePage().
Check our these two approaches you can take:


#3

Awesome that made the trick! Thanks!