Scrape Newspaper Articles with Machine Learning and Python

In this Tutorial I will show, how we can Scrape and Summarize Newspaper Articles using Python and Machine Learning techniques like NLP(Natural Language Processing).

You need to install these two packages before starting:
pip install nltk
pip install newspaper3k

Let’s start with importing libraries

import nltk
from newspaper import Article
# select a url of any news article
url = 'https://www.indiatoday.in/india/story/heavy-rain-strong-winds-storm-bengaluru-latest-updates-may-24-1681398-2020-05-24'
article = Article(url)

Now let’s do some NLP

article.download()
article.parse()
nltk.download('punkt')
article.nlp()

To get the authors of the article

article.authors

Unfortunately my Article doesn’t have any author.

To get the published date

article.publish_date
#Output
datetime.datetime(2020, 5, 24, 0, 0)

To get the top image of the article

article.top_image
#Output
'https://akm-img-a-in.tosshub.com/indiatoday/images/story/202005/rain_rep-647x363.jpeg?tBBX77JjgqP2dcddvic9tk53JXUxGCWc'

The output will give you a link, click the link to see the output

To  get the article text

print(article.text)
#Output
Heavy rain accompanied by strong winds lashed Bengaluru on Friday, leaving several trees uprooted near the BTM layout. Reports of waterlogging and roads caving-in also came to light.

Karnataka: Heavy rain accompanied by strong winds lashed Bengaluru today. Trees were uprooted near the BTM layout. pic.twitter.com/Tl2I2Mbj9j — ANI (@ANI) May 24, 2020

The India Meteorological Department (IMD) had predicted rainfall in Bengaluru till May 30.

The IMD had said that the sky would be partly cloudy with one or two spells of rain or thundershowers. Meanwhile, the maximum temperature in the city is likely to be 36 degrees Celcius. It is likely to further drop to 34 degrees as the week progresses.

The sudden change in weather brought respite for the Bangaloreans as they took to social media to post videos and pictures of the surprise spell of rain, calling it a "sign of relief".

"Rain and wind went frenzy in Bangalore," a Twitter user said.

Rain and wind went frenzy in Bangalore. pic.twitter.com/RvYolCdRkS — The Swacch Surgeon (@viswas_mr) May 24, 2020

"Sudden thunderstorm.. severe high speed lashing winds with rain.. hitting hard on South Bangalore, bangaloreans," another user said.

Meanwhile, some even said that they have not seen this kind of rain before in Bengaluru.

"Not sure if I've seen this kind of #rain before in #Bangalore!," a Twitter user said.

To get a summary of the article

print(article.summary)
#Output
Heavy rain accompanied by strong winds lashed Bengaluru on Friday, leaving several trees uprooted near the BTM layout.
Karnataka: Heavy rain accompanied by strong winds lashed Bengaluru today.
"Rain and wind went frenzy in Bangalore," a Twitter user said.
Rain and wind went frenzy in Bangalore.
"Not sure if I've seen this kind of #rain before in #Bangalore!," a Twitter user said.

Follow us on Instagram for all your Queries

Aman Kharwal
Aman Kharwal

I'm a writer and data scientist on a mission to educate others about the incredible power of data📈.

Articles: 1498

Leave a Reply