lesson1Title

lesson2Title

lesson3Title

lesson4Title

lesson5Title

lesson6Title

lesson7Title

lesson8Title

lesson9Title

lesson10Title

pythonIntroCrawlingChapter4Title

lesson11Title

pythonIntroCrawlingChapter1Title

pythonIntroCrawlingChapter2Title

pythonIntroCrawlingChapter3Title

import requests
from bs4 import BeautifulSoup

# URL of the Django repository
url = "https://github.com/django/django"

# Use requests to fetch the HTML of the webpage
response = requests.get(url)
html_content = response.text

# Parse the HTML using BeautifulSoup
soup = BeautifulSoup(html_content, "html.parser")

# Find the contents of the 'title' tag
title_content = soup.find("title")

# Output the title tag contents if present, otherwise display a message
if title_content:
    title_text = title_content.get_text()
else:
    title_text = "Could not find the title tag."

print(title_text)

# Crawling a Django Repository

Django is a popular web framework for developing web servers using Python.

In this project, we will crawl through the Django project repository on GitHub and gather the desired data.

<br />

## Important Notes

For crawling specific repository information on GitHub (for example, Django), it is most suitable to use GitHub's API.

However, in this guide, we will introduce how to use `BeautifulSoup` for the purpose of learning web crawling.

Keep in mind that the structure of GitHub's web pages can change at any time, which may require modifications to the example code.

For actual projects, it's advisable to use GitHub's official API whenever possible.

<br />

## Practice

Click the _`Run Code`_ button on the right side of the screen to check the crawling results or modify the code!

python_execution

Crawling a Django Repository