How to crawl the detail page

#1

I have been trying to crawl the detail page , can anybody help…
Url : https://www.bbb.org/en/ca/search?find_text=restaurant+chinese&find_loc=Calgary%2C+AB&find_latlng=51.039802%2C-114.076702&page=1

Two pseduo urls
page : https://www.bbb.org/en/ca/search?find_text=restaurant+chinese&find_loc=Calgary%2C+AB&find_latlng=51.039802%2C-114.076702&page=[.+]

details : https://www.bbb.org/calgary/business-reviews/caterer/[.+]

function pageFunction(context) {

var $ = context.jQuery;
var manufacturer = []; 

if (context.request.label === "details")
{
$('.dtm-search-listing').each(function(i){

     var company_name = $(this).find('.jss2').find('h3').text(); 
     var c_desc1 = $(this).find('.jss2').find('p').find('.dtm-search-listing-phone').text();
     var c_desc2 = $(this).find('.jss2').find('p').find('.dtm-search-listing-address').text();
     var email   = $(this).find('.business-buttons').find('a').eq(0).attr('href');
     var website = $(this).find('.business-buttons').find('a').eq(1).attr('href');
     
      var obj = {
         
         company_name:company_name,
         c_desc1:c_desc1,
         c_desc2:c_desc2,
         email:email,
         website:website
         
         
     }
    manufacturer.push(obj);      
});   
return manufacturer;
}else{
    
    context.skipOutput();
}

}

#2

Can anybody help me with the detail page ??

#3

Hi,

You need to set your pseudoUrl to this:
details : https://www.bbb.org/calgary/business-reviews/[.+]

You also need to add .dtm-search-listing-business-name
to your clickable elements.

#4

Thanks for the reply.

But how to combine both first page result and detail page result ???

#5

Here’s a topic about how to merge data from the 1st level and detailed page: How to get data when parsing urls of ‘Start URLs’?

#6

Thanks @jakubbalada was able to sort it .