Pagination & Clickable element

Good evening everyone,

I’m trying to scrap this whole directory usine-digitale.fr/annuaire-start-up
My actual script works like a charm with the first page. But the script doesn’t grab properly next pages.
I suspect “Clickable elements” not being set properly. Any help with this ?

The next page button is :
<a class="nav" rel="next" href="/annuaire-start-up/3/">Suivant</a>

And my Clickable element is currently set to :
a:not([rel="next"])

Thanks by advance for any help!

John

Pseudo-URLs :
page : https://www.usine-digitale.fr/annuaire-start-up/[\d+]
detail : https://www.usine-digitale.fr/annuaire-start-up/[.+],[.+]

Any help would be grandly appreciated ! :smiley:

Hi @john,

I think you need to just small update in your Pseudo-URLs and clickable elements.
I checked the site and it looks that URLs for next pages looks like:
https://www.usine-digitale.fr/annuaire-start-up/2/
So you need to update Pseudo-URL for a page to
https://www.usine-digitale.fr/annuaire-start-up/[\d+]/

About clickable element. Your CSS selector a:not([rel="next"]) is wrong, because you pick just a element without rel="next" attribute.
I recommend using a more generic selector like a[href]. The crawler will pick up all elements with href attribute and match them against Pseudo-URLs.
After these two updates, it should work.