Get inner div using Beautifulsoup : Scrape the div Content

Beautifulsoup allows you to scrap the data from the HTML or XML content. HTML content contains nested div. You can easily scrape the content from the inner div using the Beautifulsoup module. In this entire tutorial, you will know how to get the inner div content using the beautifulsoup.

What is and How beautifulsoup module Works?

Beautifulsoup is a python module that contains many functions like find, findall() e.t.c to find the tags of the HTLM or XML to scrape the content from the divs, paragraphs, links e.t.c. It extracts data in a hierarchical and more readable manner

Steps to get the inner div content using beautifulsoup

Let’s know all the steps that you will take to find the content from the inner div using the module.

Sample HTML document

html_doc = """
<div class="outer-div">
<div class="inner-div">
<p>Code The Best</p>
</div>
</div>
"""

Step 1: Import the beautifulsoup library

In the first step, you will import the beautifulsoup module. If you have not installed it then use the pip install beautifulsoup4 command() to install it.

from bs4 import BeautifulSoup

Step 2:  Read the HTML document

In the second step read the HTML content using the html.parser using the BeautifulSoup() constructor. Pass the two arguments to your HTML docs(html_doc) and the other the parser (html.paraser)

soup = BeautifulSoup(html_doc, 'html.parser')

Step 3: Find the inner div

After parsing the HTML document you will use the find() function to find the inner div. You can use the class name defined for the inner div to get it.

inner_div= soup.find("div", class_="inner-div")

Step 4: Display the content

Now you have to find the inner. You can easily get all the content of the div, You can use the dot operator to find the inner div text content. In this case, you use inner_div. text then it will display the text ” Code The Best”.

print(result.text)

Output

Code The Best

Conclusion

BeautifulSoup is a great module to scrap web content easily. The above steps will allow you to scrape or Get inner div content using beautifulsoup. There are also other methods to get the inner div content but in this tutorial, you will get the simplest one. You can contact us  for more help,