Web Scraping: An Introduction
Wolverine was here Web scraping has been used to compile extensive datasets on everything from UFO sightings to product reviews on the internet! If you have ever gone to a website, copied data from it, and pasted it into an Excel file, then congratulations, you have done web scraping manually. However, what if you had a bigger problem to solve and required more data from a website? For example, the wristband for your cheap Casio watch broke, and you decided it's time to upgrade to a more premium model. However, before spending your money, you wanted to compare the prices, reviews and materials of thousands of watches to ensure that you are making the best decision when it comes to durability and your budget. I hope we can both agree that this task is humanly impossible to get done within a reasonable amount of time, and that's where libraries like Scrapy and Beautiful Soup come into play. With great power comes great responsibility! Here are some pros and cons of web ...