{"id":8007,"date":"2020-12-29T00:26:16","date_gmt":"2020-12-29T00:26:16","guid":{"rendered":"https:\/\/healinglifespan.com\/data-science\/2020\/12\/29\/web-scraping-using-r\/"},"modified":"2020-12-29T00:26:16","modified_gmt":"2020-12-29T00:26:16","slug":"web-scraping-using-r","status":"publish","type":"post","link":"https:\/\/wealthrevelation.com\/data-science\/2020\/12\/29\/web-scraping-using-r\/","title":{"rendered":"Web Scraping Using R..!"},"content":{"rendered":"
\n

In this blog, I\u2019ll show you, How to Web Scrape using R..?<\/strong><\/span><\/p>\n

What is R..?<\/span><\/strong><\/p>\n

R is a programming language and its environment built for statistical analysis, graphical representation & reporting. R programming is mostly preferred by statisticians, data miners, and software programmers who want to develop statistical software.<\/span><\/p>\n

R<\/span><\/strong> is also available as Free Software under the terms of the Free Software Foundation\u2019s GNU General Public License in source code form.<\/span><\/p>\n

\"Reasons<\/a><\/p>\n

Reasons to choose R<\/strong><\/p>\n<\/div>\n

Let\u2019s begin our topic of Web Scraping using R.<\/span><\/strong><\/p>\n

Step 1- Select the website & the data you want to scrape.<\/span><\/strong><\/p>\n

I picked this website \u201chttps:\/\/www.alexa.com\/topsites\/countries\/IN<\/strong>\u201d and want to scrape data of Top 50 sites in India.<\/span><\/p>\n

\"Data<\/a><\/p>\n

Data we want to scrape<\/strong><\/p>\n<\/div>\n

Step 2- Get to know the HTML tags using SelectorGadget.<\/span><\/strong><\/p>\n

In my previous blog, I already discussed how to inspect & find the proper HTML tags. So, now I\u2019ll explain an easier way to get the HTML tags.<\/span><\/p>\n

You have to go to Google chrome extension (chrome:\/\/extensions) & search SelectorGadget<\/strong>. Add it to your browser, it\u2019s a quite good CSS selector.<\/span><\/p>\n

\"\"<\/a><\/p>\n

Step 3- R Code<\/span><\/strong><\/p>\n

Evoking Important Libraries or Packages<\/span><\/strong><\/p>\n

I\u2019m using RVEST package to scrape the data from the webpage; it is inspired by libraries like Beautiful Soup<\/strong>. If you didn\u2019t install the package yet, then follow the code in the snippet below.<\/span><\/p>\n

\"\"<\/a><\/p>\n

Step 4- Set the url of the website<\/span><\/strong><\/p>\n

\"\"<\/a><\/p>\n

Step 5- Find the HTML tags using SelectorGadget<\/span><\/strong><\/p>\n

It\u2019s quite easy to find the proper HTML tags in which your data is present.<\/span><\/p>\n

Firstly, I have to click on data using SelectorGadget which I want to scrape, it automatically selects the data which are similar to selected HTML tags. Before going forward, cross-check the selected values, are they correct or some junk data is also gets selected..? If you noticed our page has only 50 values, but you can see 156 values are selected.<\/span><\/p>\n

\"Selection<\/a><\/p>\n

Selection by SelectorGadget<\/strong><\/p>\n<\/div>\n

So I need to remove unwanted values who get selected, once you click on them to deselect it, it turns red and others will turn yellow except our primary selection which turn to green. Now you can see only 50 values are selected as per our primary requirement but it\u2019s not enough. I have to again cross-check that some required values are not exchanged with junk values<\/strong>.<\/span><\/p>\n

If we satisfy with our selection then copy the HTML tag & include it into the code, else repeat this exercise.<\/span><\/p>\n

\"\"<\/a><\/p>\n

Modified Selection by SelectorGadget<\/strong><\/p>\n<\/div>\n

Step 6- Include the tag in our Code<\/span><\/strong><\/p>\n

After including the tags, our code is like this.<\/span><\/p>\n

\"\"<\/a><\/p>\n

Code Snippet<\/strong><\/p>\n<\/div>\n

If I run the code, values in each list object will be 50.<\/span><\/p>\n

\"\"<\/a><\/p>\n

Data Stored in List Objects<\/p>\n<\/div>\n

Step 7- Creating DataFrame<\/span><\/strong><\/p>\n

Now, we create a dataframe with our list-objects. So for creating a dataframe, we always need to remember one thumb rule that is the number of rows (length of all the lists) should be equal, else we get an error.<\/span><\/p>\n

\"\"<\/a><\/p>\n

Error appears when number of rows differs<\/strong><\/p>\n<\/div>\n

Finally, Our DataFrame will look like this<\/span>:<\/p>\n

\"\"<\/a><\/p>\n

Our Final Data<\/strong><\/p>\n<\/div>\n

Step 8- Writing our DataFrame to CSV file<\/span><\/strong><\/p>\n

We need our scraped data to be available locally for further analysis & model building or other purposes.<\/span><\/p>\n

Our final piece of code to write it in CSV file is<\/span>:<\/p>\n

\"\"<\/a><\/p>\n

Writing to CSV file<\/strong><\/p>\n<\/div>\n

Step 9- Check the CSV file<\/span><\/strong><\/p>\n

\"\"<\/a><\/p>\n

Data written in CSV file<\/strong><\/p>\n<\/div>\n

Conclusion-<\/span><\/strong><\/p>\n

I tried to explain Web Scraping using R in a simple way, Hope this will help you in understanding it better.<\/span><\/p>\n

Find full code on<\/span><\/strong><\/p>\n

https:\/\/github.com\/vgyaan\/Alexa\/blob\/master\/webscrap.R<\/span><\/a><\/span><\/p>\n

If you have any questions about the code or web scraping in general, reach out to me on LinkedIn!
<\/span><\/strong><\/p>\n

Okay, we will meet again with the new exposer.<\/span><\/p>\n

Till then,<\/span><\/p>\n

Happy Coding..!<\/span><\/b><\/em><\/p>\n

\"\"<\/a><\/p>\n

\n

Gyan Vardhan<\/a><\/h3>\n
\"Gyan<\/div>\n

<\/a><\/p>\n

Gyan Vardhan is a free and independent data scientist based in India.<\/p>\n<\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"

https:\/\/data-science-blog.com\/en\/blog\/2020\/11\/18\/web-scraping-using-r\/<\/p>\n","protected":false},"author":0,"featured_media":8008,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[2],"tags":[],"_links":{"self":[{"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/posts\/8007"}],"collection":[{"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/types\/post"}],"replies":[{"embeddable":true,"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/comments?post=8007"}],"version-history":[{"count":0,"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/posts\/8007\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/media\/8008"}],"wp:attachment":[{"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/media?parent=8007"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/categories?post=8007"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/tags?post=8007"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}