I need a script to automate the following web scraping process:

  1. Use search filter on the webpage by entering a pre-given ID in a TextInput field.

  2. Mark one of the listed results by clicking an icon next to it.

  3. Press a button and wait for a data fetching process to complete

  4. Extract HTML embedded text from the fetched data.

Answer

For Python I would recommend:

Selenium: Selenium is a web testing library. It is used to automate browser activities.
BeautifulSoup: Beautiful Soup is a Python package for parsing HTML and XML documents. It creates parse-trees that are helpful in extracting data easily.

Write a comment

Your email address will not be published. Required fields are marked *