5.5 API & Web Service Interaction with Python

5.5 API & Web Service Interaction

Modern applications rarely work in isolation. Websites, mobile apps, dashboards, ERP systems, payment systems, cloud platforms, and data applications frequently communicate through APIs (Application Programming Interfaces).

Python provides excellent support for consuming and working with web APIs. This makes it possible to retrieve data, submit data, integrate different applications, and automate business workflows.

1. What Is an API?

An API is an interface that allows one software system to communicate with another according to defined rules.

For example, a weather application may request weather information from a weather service instead of maintaining its own weather database.

Python Application
        |
        | HTTP Request
        ↓
    REST API
        |
        | JSON Response
        ↓
Python Application

2. What Is a Web Service?

A web service allows applications to exchange information over a network, commonly using HTTP or HTTPS.

Common technologies include:

• REST APIs
• JSON
• XML
• HTTP/HTTPS
• Authentication mechanisms
• Webhooks

3. What Is REST?

REST stands for Representational State Transfer.

REST is an architectural style commonly used for designing web APIs.

A REST API generally exposes resources through URLs and uses standard HTTP methods to perform operations.

4. HTTP Methods Used by REST APIs

5. Anatomy of an API Request

An HTTP API request may contain:

• HTTP method
• URL
• Path parameters
• Query parameters
• Request headers
• Request body
• Authentication information

GET /users?id=101

Headers:
Authorization: Bearer TOKEN
Accept: application/json

6. Installing the requests Library

pip install requests

Import it using:

import requests

7. Making a GET Request

import requests

response = requests.get(
    "https://api.example.com/users"
)

print(
    response.status_code
)

The returned object is a requests.Response object containing information about the server response.

8. Understanding HTTP Status Codes

9. Reading the Response

response = requests.get(
    "https://api.example.com/users"
)

print(
    response.text
)

response.text returns the response body as text.

10. Working with JSON Responses

Many REST APIs return data in JSON (JavaScript Object Notation).

response = requests.get(
    "https://api.example.com/users"
)

data = response.json()

print(
    data
)

Python typically represents JSON objects as dictionaries and JSON arrays as lists.

11. JSON to Python Data Types

12. Understanding Nested JSON

{
    "student": {
        "name": "Alex",
        "grade": 11,
        "subjects": [
            "Computer Science",
            "Mathematics"
        ]
    }
}

Python representation:

data = {
    "student": {
        "name": "Alex",
        "grade": 11,
        "subjects": [
            "Computer Science",
            "Mathematics"
        ]
    }
}

Access nested values:

print(
    data["student"]["name"]
)

print(
    data["student"]["subjects"][0]
)

13. Query Parameters

Query parameters are values appended to a URL to control or filter a request.

params = {
    "page": 2,
    "limit": 10
}

response = requests.get(
    "https://api.example.com/users",
    params=params
)

print(
    response.url
)

Using the params argument is preferable to manually constructing a query string in most cases.

14. HTTP Headers

Headers provide additional information about a request or response.

headers = {
    "Accept": "application/json"
}

response = requests.get(
    "https://api.example.com/users",
    headers=headers
)

15. Custom Request Headers

headers = {
    "Accept": "application/json",
    "User-Agent": "Python-API-Client/1.0"
}

response = requests.get(
    "https://api.example.com/data",
    headers=headers
)

Only send headers required or permitted by the API.

16. Sending Data with POST

A POST request is commonly used to submit or create data.

payload = {
    "name": "Alex",
    "email": "alex@example.com"
}

response = requests.post(
    "https://api.example.com/users",
    json=payload
)

print(
    response.status_code
)

The json= argument automatically serializes the Python object as JSON and sets the appropriate content type.

17. json= vs data=

These arguments serve different purposes.

requests.post(
    url,
    json=payload
)

is appropriate when the API expects JSON.

requests.post(
    url,
    data=form_data
)

is commonly used for form-encoded data.

18. Updating Data with PUT

payload = {
    "name": "Alex",
    "grade": 12
}

response = requests.put(
    "https://api.example.com/users/101",
    json=payload
)

print(
    response.status_code
)

19. Partial Updates with PATCH

payload = {
    "grade": 12
}

response = requests.patch(
    "https://api.example.com/users/101",
    json=payload
)

PATCH is generally used when only selected fields need to be modified.

20. Deleting Data with DELETE

response = requests.delete(
    "https://api.example.com/users/101"
)

print(
    response.status_code
)

Never send destructive API requests against production data unless the operation is explicitly authorized and intended.

21. Handling HTTP Errors

The raise_for_status() method raises an exception for unsuccessful HTTP status codes.

response = requests.get(
    "https://api.example.com/users"
)

response.raise_for_status()

data = response.json()

22. API Exception Handling

import requests

try:

    response = requests.get(
        "https://api.example.com/users",
        timeout=10
    )

    response.raise_for_status()

    data = response.json()

except requests.exceptions.Timeout:

    print(
        "The API request timed out."
    )

except requests.exceptions.HTTPError as error:

    print(
        "HTTP error:",
        error
    )

except requests.exceptions.RequestException as error:

    print(
        "Request failed:",
        error
    )

23. Why API Timeouts Matter

Never assume that a remote server will always respond immediately. A timeout prevents your program from waiting indefinitely.

response = requests.get(
    url,
    timeout=10
)

24. API Authentication

APIs may require authentication before allowing access to protected resources.

Common mechanisms include:

• API keys
• Bearer tokens
• Basic authentication
• OAuth 2.0
• Session-based authentication

25. API Key Authentication

An API key may be provided through a request header or query parameter depending on the API specification.

headers = {
    "X-API-Key": "YOUR_API_KEY"
}

response = requests.get(
    "https://api.example.com/data",
    headers=headers
)

26. Bearer Token Authentication

token = "YOUR_ACCESS_TOKEN"

headers = {
    "Authorization": f"Bearer {token}",
    "Accept": "application/json"
}

response = requests.get(
    "https://api.example.com/profile",
    headers=headers
)

27. Storing API Credentials Securely

Environment variables are a basic approach for keeping secrets outside the source code.

import os

api_key = os.environ.get(
    "API_KEY"
)

headers = {
    "X-API-Key": api_key
}

Larger production systems may use dedicated secret-management platforms.

28. Basic Authentication

import requests
from requests.auth import HTTPBasicAuth

response = requests.get(
    "https://api.example.com/profile",
    auth=HTTPBasicAuth(
        "username",
        "password"
    )
)

29. Validating API Responses

response = requests.get(
    "https://api.example.com/users",
    timeout=10
)

response.raise_for_status()

data = response.json()

if "users" in data:

    users = data["users"]

    for user in users:

        print(
            user.get("name")
        )

Using dict.get() can be useful when an optional field may not exist.

30. JSON Serialization with the json Module

Python's standard library includes the json module.

import json

student = {
    "name": "Alex",
    "grade": 11,
    "active": True
}

json_text = json.dumps(
    student
)

print(
    json_text
)

31. Converting JSON Text to Python

json_text = '''
{
    "name": "Alex",
    "grade": 11
}
'''

student = json.loads(
    json_text
)

print(
    student["name"]
)

32. Reading JSON from a File

import json

with open(
    "students.json",
    "r",
    encoding="utf-8"
) as file:

    data = json.load(
        file
    )

print(data)

33. API Data with pandas

API data can be converted into a pandas DataFrame for analysis.

import requests
import pandas as pd

response = requests.get(
    "https://api.example.com/students",
    timeout=10
)

response.raise_for_status()

data = response.json()

df = pd.DataFrame(
    data["students"]
)

print(df.head())

34. API Pagination

APIs often divide large datasets into multiple pages instead of returning everything in a single response.

page = 1

while True:

    response = requests.get(
        "https://api.example.com/users",
        params={
            "page": page,
            "limit": 100
        },
        timeout=10
    )

    response.raise_for_status()

    data = response.json()

    records = data.get(
        "users",
        []
    )

    if not records:
        break

    for record in records:

        print(record)

    page += 1

The exact pagination mechanism depends on the API.

35. API Rate Limits

Many APIs restrict the number of requests a client can make during a given period.

A server may return:

429 Too Many Requests

Your program should respect the API's documented rate limits.

import time

time.sleep(
    1
)

For production systems, use the API's documented retry and backoff strategy rather than blindly sending repeated requests.

36. Basic Retry Logic

import time
import requests

url = (
    "https://api.example.com/data"
)

for attempt in range(3):

    try:

        response = requests.get(
            url,
            timeout=10
        )

        response.raise_for_status()

        data = response.json()

        break

    except requests.exceptions.RequestException:

        if attempt == 2:

            raise

        time.sleep(
            2 ** attempt
        )

Exponential backoff increases the waiting period between retries.

37. Using requests.Session()

A Session can persist settings such as headers and cookies across multiple requests.

import requests

session = requests.Session()

session.headers.update({
    "Accept": "application/json"
})

response = session.get(
    "https://api.example.com/users",
    timeout=10
)

print(
    response.status_code
)

session.close()

38. Mini Project — API Data Collector

Create a script that retrieves records from an authorized public or test API and stores them in a CSV file.

import requests
import pandas as pd

response = requests.get(
    "https://api.example.com/students",
    timeout=10
)

response.raise_for_status()

data = response.json()

df = pd.DataFrame(
    data["students"]
)

df.to_csv(
    "students.csv",
    index=False
)

print(
    "Data saved successfully."
)

39. Automating Email with Python

Python can send email notifications automatically using the standard-library smtplib module.

This is useful for:

• Automation reports
• Job completion notifications
• System alerts
• Data pipeline notifications
• Scheduled reports
• Failure notifications

40. What Is SMTP?

SMTP stands for Simple Mail Transfer Protocol.

It is a protocol used for sending email messages between mail systems.

Python Program
      |
      | SMTP
      ↓
Mail Server
      |
      ↓
Recipient Mailbox

41. Importing smtplib

import smtplib

No separate installation is normally required because smtplib is part of Python's standard library.

42. Creating an Email Message

Python's email package provides structured classes for creating email messages.

from email.message import EmailMessage

message = EmailMessage()

message["Subject"] = (
    "Automation Report"
)

message["From"] = (
    "sender@example.com"
)

message["To"] = (
    "recipient@example.com"
)

message.set_content(
    "The automation job completed successfully."
)

43. Connecting to an SMTP Server

SMTP server settings depend on your email provider or organization.

import smtplib

with smtplib.SMTP(
    "smtp.example.com",
    587
) as server:

    server.starttls()

    # Authenticate according
    # to the provider's requirements.

Port 587 is commonly associated with SMTP submission using STARTTLS, but the correct server, port, and security method must always come from the email provider's documentation.

44. Sending an Email

import os
import smtplib

from email.message import EmailMessage


sender = os.environ.get(
    "EMAIL_USERNAME"
)

password = os.environ.get(
    "EMAIL_PASSWORD"
)

recipient = (
    "recipient@example.com"
)


message = EmailMessage()

message["Subject"] = (
    "Automation Completed"
)

message["From"] = sender

message["To"] = recipient

message.set_content(
    "The scheduled automation task "
    "completed successfully."
)


with smtplib.SMTP(
    "smtp.example.com",
    587
) as server:

    server.starttls()

    server.login(
        sender,
        password
    )

    server.send_message(
        message
    )

45. Sending an HTML Email

from email.message import EmailMessage

message = EmailMessage()

message["Subject"] = (
    "Daily Data Report"
)

message["From"] = (
    "sender@example.com"
)

message["To"] = (
    "recipient@example.com"
)

message.set_content(
    "Your email client does not support HTML."
)

message.add_alternative(
    """
    <html>
        <body>
            <h2>Daily Data Report</h2>
            <p>The report has been generated successfully.</p>
        </body>
    </html>
    """,
    subtype="html"
)

46. Sending an Attachment

The EmailMessage class can attach files to messages.

from pathlib import Path
from email.message import EmailMessage

file_path = Path(
    "report.csv"
)

message = EmailMessage()

message["Subject"] = (
    "Daily Report"
)

message["From"] = (
    "sender@example.com"
)

message["To"] = (
    "recipient@example.com"
)

message.set_content(
    "Please find the report attached."
)

data = file_path.read_bytes()

message.add_attachment(
    data,
    maintype="text",
    subtype="csv",
    filename=file_path.name
)

47. Attaching a PDF

from pathlib import Path

pdf_path = Path(
    "report.pdf"
)

pdf_data = pdf_path.read_bytes()

message.add_attachment(
    pdf_data,
    maintype="application",
    subtype="pdf",
    filename=pdf_path.name
)

48. Sending to Multiple Recipients

message["To"] = (
    "one@example.com, "
    "two@example.com"
)

CC and BCC can also be represented using appropriate message headers and recipient handling.

49. Handling Email Errors

import smtplib

try:

    with smtplib.SMTP(
        "smtp.example.com",
        587,
        timeout=20
    ) as server:

        server.starttls()

        server.login(
            sender,
            password
        )

        server.send_message(
            message
        )

except smtplib.SMTPAuthenticationError:

    print(
        "SMTP authentication failed."
    )

except smtplib.SMTPException as error:

    print(
        "Email error:",
        error
    )

except OSError as error:

    print(
        "Network error:",
        error
    )

50. Combining API Automation with Email

One of the most useful automation patterns is:

API
 ↓
Retrieve Data
 ↓
Validate Data
 ↓
Process Data
 ↓
Generate Report
 ↓
Send Email
 ↓
Record Result

This pattern can be used for scheduled operational reports, monitoring systems, data pipelines, and many other legitimate automation workflows.

51. Mini Project — API Report + Email Notification

The following example demonstrates the overall architecture. Replace the example API and SMTP configuration with services for which you have authorization.

import os
import smtplib

import requests
import pandas as pd

from email.message import EmailMessage


API_URL = (
    "https://api.example.com/students"
)

SMTP_HOST = (
    "smtp.example.com"
)

SMTP_PORT = 587


sender = os.environ.get(
    "EMAIL_USERNAME"
)

password = os.environ.get(
    "EMAIL_PASSWORD"
)

recipient = (
    "recipient@example.com"
)


# --------------------------------------------------
# 1. Retrieve API data
# --------------------------------------------------

response = requests.get(
    API_URL,
    timeout=10
)

response.raise_for_status()

data = response.json()


# --------------------------------------------------
# 2. Convert data to DataFrame
# --------------------------------------------------

df = pd.DataFrame(
    data["students"]
)


# --------------------------------------------------
# 3. Generate report
# --------------------------------------------------

report_path = (
    "student_report.csv"
)

df.to_csv(
    report_path,
    index=False
)


# --------------------------------------------------
# 4. Create email
# --------------------------------------------------

message = EmailMessage()

message["Subject"] = (
    "Automated Student Data Report"
)

message["From"] = sender

message["To"] = recipient

message.set_content(
    "The student data report "
    "has been generated successfully."
)


# --------------------------------------------------
# 5. Attach report
# --------------------------------------------------

with open(
    report_path,
    "rb"
) as file:

    message.add_attachment(
        file.read(),
        maintype="text",
        subtype="csv",
        filename=report_path
    )


# --------------------------------------------------
# 6. Send email
# --------------------------------------------------

with smtplib.SMTP(
    SMTP_HOST,
    SMTP_PORT,
    timeout=20
) as server:

    server.starttls()

    server.login(
        sender,
        password
    )

    server.send_message(
        message
    )


print(
    "Report generated and email sent."
)

52. Sending Failure Notifications

Email automation can also notify an administrator when an automation job fails.

try:

    response = requests.get(
        API_URL,
        timeout=10
    )

    response.raise_for_status()

    data = response.json()

except Exception as error:

    print(
        "Automation failed:",
        error
    )

    # Send an appropriate
    # failure notification
    # through the configured
    # notification system.

In production systems, avoid catching every exception without logging or handling it appropriately. Capture enough diagnostic information to identify the actual cause of the failure.

53. Logging API Automation

import logging

logging.basicConfig(
    level=logging.INFO
)

logging.info(
    "Starting API data collection."
)

logging.info(
    "API request completed."
)

logging.error(
    "API request failed."
)

Logging is more useful than relying exclusively on print() in production automation.

54. API Security Best Practices

• Use HTTPS whenever supported.
• Never expose API keys or tokens.
• Store secrets outside source code.
• Use least-privilege credentials.
• Set request timeouts.
• Validate API responses.
• Handle authentication failures safely.
• Respect rate limits.
• Avoid logging sensitive information.
• Rotate credentials according to organizational policy.

55. API Automation vs Selenium

56. Reading API Documentation

Before writing an API integration, identify:

1. Base URL
2. Endpoint
3. HTTP method
4. Required parameters
5. Request body format
6. Required headers
7. Authentication method
8. Response structure
9. Status codes
10. Rate limits
11. Pagination rules
12. Error response format

57. Professional API Integration Workflow

Read API Documentation
        ↓
Identify Endpoint
        ↓
Choose HTTP Method
        ↓
Configure Authentication
        ↓
Prepare Parameters / JSON
        ↓
Send Request
        ↓
Check Status Code
        ↓
Parse JSON
        ↓
Validate Response
        ↓
Process Data
        ↓
Store / Report / Notify

58. API & Web Services Interview Questions

59. Examination Questions — MCQs

60. Practical Examination Questions

61. Real-World Project — Automated API Reporting System

Build an end-to-end Python automation system with the following architecture:

              REST API
                  ↓
             Python Requests
                  ↓
             JSON Response
                  ↓
           Validate Response
                  ↓
               pandas
                  ↓
          Data Transformation
                  ↓
             CSV / PDF Report
                  ↓
           Email Notification
                  ↓
          Recipient Mailbox

Project Requirements

1. Retrieve data from an authorized API.
2. Use a timeout.
3. Validate the HTTP response.
4. Parse the JSON response.
5. Convert the data into a DataFrame.
6. Perform basic analysis.
7. Export the results.
8. Create an email message.
9. Attach the generated report.
10. Send the email securely.
11. Log success or failure.

62. Expert Tips

1. Read the API documentation first. Do not guess endpoints or authentication formats.
2. Use timeouts. Network calls can fail or become slow.
3. Check status codes. A request returning a response does not automatically mean it succeeded.
4. Validate JSON structure. APIs can change or return unexpected data.
5. Protect credentials. Never expose API keys, tokens, passwords, or SMTP credentials.
6. Respect rate limits. API automation should not overload a service.
7. Prefer APIs over browser automation when a suitable documented API exists.
8. Use logging. Automated jobs should leave useful diagnostic information.
9. Design for failure. Network failures are normal possibilities, not exceptional impossibilities.
10. Separate configuration from code. Keep URLs, credentials, and environment-specific settings configurable.

63. API & Email Quick Reference Cheat Sheet

64. Self-Assessment Checklist

Before moving to Module 6, make sure you can:

• ☐ Explain what an API is.
• ☐ Explain REST APIs.
• ☐ Explain common HTTP methods.
• ☐ Send GET requests using Python.
• ☐ Send POST requests with JSON.
• ☐ Use PUT, PATCH, and DELETE.
• ☐ Read HTTP status codes.
• ☐ Parse JSON responses.
• ☐ Work with nested JSON.
• ☐ Send query parameters.
• ☐ Send request headers.
• ☐ Use API authentication appropriately.
• ☐ Protect API credentials.
• ☐ Configure network timeouts.
• ☐ Handle API exceptions.
• ☐ Understand rate limiting.
• ☐ Work with API pagination.
• ☐ Use requests.Session().
• ☐ Convert API JSON data into pandas DataFrames.
• ☐ Explain SMTP.
• ☐ Use smtplib.
• ☐ Create an EmailMessage.
• ☐ Send an automated email.
• ☐ Attach CSV or PDF reports.
• ☐ Handle email exceptions.
• ☐ Combine API automation, data processing, reporting, and email notification.