Task 1 is an automated online data acquisition challenge, i.e., to scrape some required
information from one or many static webpages with Python Requests and BeautifulSoup
packages in an automated manner, and store the information in a structured, machinereadable data format as the output.
The task
Starting from Q2, all the answers should be accompanied by the Python code
Q1. The problem: what is the webpage (URL) you want to scrape? [1 mark]
suggestion: priority will be given to “public-oriented” webpages, such as government official
pages, academic institutions, Wikipedia public pages, NGOs (please check their term of use).
Q2. Quick warming up: please figure out the following information from the webpage from
your Python code:
- What is the “title” of this webpage, as stipulated by the “head” section? [2 mark]
- please print all the text of the paragraphs of your webpage [2 mark].
Q3. Please identify the elements you want to scrape, by make a screen snapshot of your
browser, with the “development mode.” Hover your mouse above the area of the element you
want to scrape (a rough and approximate area will do). [1 mark]
Q4. Now the scraping. Please scrape the information from the webpage, with the following
Sample Solution