GOSPELTRUTH31:
How can I make money from this

DataMina:

How it works often on Upwork or Fiverr clients make request to scrap specific websites or leads for cold email marketing.

GOSPELTRUTH31:


Do you offer classes online

DataMina:
Hello Nairalanders,

I want to share my experience in crawling leads from Jiji. As a web scraping enthusiast, I've tackled various sites, but this one proved to be difficult. This is because the phone numbers are in the product detail page and you have to be logged in to get it. Another issue I faced was that the website is loaded with java script and uses infinite scrolling.

I was able to circumvent these road blocks by using selenium to login to the site with my personal details after which I saved the cookies as a json file. As the cite uses infinite scrolling I scraped all the sellers url links to a csv file. I then read the url links so as to crawl the site using the links.

Remember my login details have been saved as cookies in a json file, In the process of crawling the site again using the URLs, I applied the cookies from the json file. The crawler visited each seller's detail page using the saved URL links and clicked the "Show Contact" button to extract the contact information.

Retailers specializing in second-hand items like laptops can leverage this approach coupled with applied analytics on the scraped the data to gain swift access to mouth-watering deals, faster than regular visitor

You can check out the code for executing the project in my github repo: https://github.com/StephDAnalyst/JijiLeadScraping

landiqa:


Finally. Your contact please

airsaylongcome:
OP,

Word of advice, never use your personal login details for webscrapping

blaskikv:
Is webscraping lucrative?

blaskikv:
Is webscraping lucrative?

DataMina:
Hello Nairalanders,

I want to share my experience in crawling leads from Jiji. As a web scraping enthusiast, I've tackled various sites, but this one proved to be difficult. This is because the phone numbers are in the product detail page and you have to be logged in to get it. Another issue I faced was that the website is loaded with java script and uses infinite scrolling.

I was able to circumvent these road blocks by using selenium to login to the site with my personal details after which I saved the cookies as a json file. As the cite uses infinite scrolling I scraped all the sellers url links to a csv file. I then read the url links so as to crawl the site using the links.

Remember my login details have been saved as cookies in a json file, In the process of crawling the site again using the URLs, I applied the cookies from the json file. The crawler visited each seller's detail page using the saved URL links and clicked the "Show Contact" button to extract the contact information.

Retailers specializing in second-hand items like laptops can leverage this approach coupled with applied analytics on the scraped the data to gain swift access to mouth-watering deals, faster than regular visitor

You can check out the code for executing the project in my github repo: https://github.com/StephDAnalyst/JijiLeadScraping

BlackhatMentor:


Sure

Felixitie:


Great job! Being able to handle the infinite scrolling on jiji page, different websites with different technicalities when it comes to data extraction. I did a project for a client recently, Airbnb and zillow, omoh the zillow gave me wahala small, reasons were, all the Web elements did not load except you scroll the page bit by bit down to the button, and to be able to scroll the page you have to minimise the driver window., very interesting.
Nice one op.
Thanks. You are right there is no one perfect fit all to scraping. If it was that straight forward the client wouldn't have needed your service 👌

blaskikv:
Is webscraping lucrative?