3.1 Data Structures — Pandas Series & DataFrames

3.1 Pandas Series & DataFrames

pandas is one of the most widely used Python libraries for data manipulation and analysis.

Before cleaning, transforming, visualizing, or analyzing a dataset, it is essential to understand the two fundamental pandas data structures:

• Series
• DataFrame

A good understanding of these structures makes later pandas operations such as filtering, grouping, merging, and data cleaning much easier.

1. What Is pandas?

pandas is an open-source Python library designed for working with structured and tabular data.

It provides convenient tools for:

• Reading datasets
• Cleaning data
• Filtering records
• Sorting data
• Grouping data
• Handling missing values
• Combining datasets
• Performing statistical analysis
• Preparing data for visualization and Machine Learning

pandas is built to work naturally with structured data such as spreadsheets and database tables.

2. Importing pandas

pandas is conventionally imported using the alias pd.

import pandas as pd

Once imported, pandas functionality can be accessed through pd.

import pandas as pd

data = pd.Series([10, 20, 30])

print(data)

3. The Two Core pandas Data Structures

4. What Is a Series?

A Series is a one-dimensional labeled array capable of holding data values along with an associated index.

A Series can be thought of as a single labeled column of data.

import pandas as pd

marks = pd.Series([78, 85, 92, 74])

print(marks)

Output:

0    78
1    85
2    92
3    74
dtype: int64

Notice that pandas automatically creates an index beginning at 0.

5. Understanding the Structure of a Series

0    78
1    85
2    92
3    74

The structure contains two important components:

• Index: Labels identifying each element.
• Values: Actual data stored in the Series.

Conceptually:

Index       Value

  0           78
  1           85
  2           92
  3           74

6. Accessing Series Values

The .values attribute provides the underlying values.

marks = pd.Series([78, 85, 92, 74])

print(marks.values)

Output:

[78 85 92 74]

The returned object is a NumPy-based array representation of the Series values.

7. Accessing the Series Index

marks = pd.Series([78, 85, 92, 74])

print(marks.index)

Output will describe the index range, for example:

RangeIndex(start=0, stop=4, step=1)

The index identifies the positions or labels associated with the values.

8. Series Data Type

The .dtype attribute identifies the data type of the values stored in the Series.

marks = pd.Series([78, 85, 92, 74])

print(marks.dtype)

Output:

int64

The exact dtype can depend on the values and platform.

9. Creating a Series with Custom Labels

A Series does not have to use the default numeric index. Custom labels can be supplied using the index parameter.

marks = pd.Series(
    [78, 85, 92],
    index=["Alice", "Brian", "Clara"]
)

print(marks)

Output:

Alice    78
Brian    85
Clara    92
dtype: int64

Now the names are used as labels.

10. Accessing Series Data Using Labels

marks = pd.Series(
    [78, 85, 92],
    index=["Alice", "Brian", "Clara"]
)

print(marks["Brian"])

Output:

85

The label "Brian" identifies the required value.

11. Creating a Series from a Dictionary

A dictionary is a natural way to create a labeled Series because dictionary keys can become the Series index.

marks = {
    "Alice": 78,
    "Brian": 85,
    "Clara": 92
}

series = pd.Series(marks)

print(series)

Output:

Alice    78
Brian    85
Clara    92
dtype: int64

12. Creating a Series from a Scalar

A scalar value can also be used to create a Series when an index is provided.

series = pd.Series(
    100,
    index=["A", "B", "C"]
)

print(series)

Output:

A    100
B    100
C    100
dtype: int64

The scalar value is assigned to every supplied index label.

13. Series Indexing

Series values can be accessed using labels or positional indexing.

marks = pd.Series(
    [78, 85, 92, 74],
    index=["A", "B", "C", "D"]
)

print(marks["C"])

Output:

92

Positional access can be performed using .iloc.

print(marks.iloc[2])

Output:

92

14. Label-Based Selection with .loc

.loc is used primarily for label-based selection.

marks = pd.Series(
    [78, 85, 92, 74],
    index=["A", "B", "C", "D"]
)

print(marks.loc["C"])

Output:

92

15. Position-Based Selection with .iloc

.iloc uses integer positions.

marks = pd.Series(
    [78, 85, 92, 74]
)

print(marks.iloc[1])

Output:

85

16. Slicing a Series

A Series can be sliced using .iloc for positional slicing.

marks = pd.Series(
    [78, 85, 92, 74, 88]
)

print(marks.iloc[1:4])

Output:

1    85
2    92
3    74
dtype: int64

As with standard Python slicing, the ending position is excluded.

17. Filtering a Series

Boolean conditions can be used to select values satisfying a condition.

marks = pd.Series([45, 67, 82, 91, 38])

passed = marks[marks >= 50]

print(passed)

Output:

1    67
2    82
3    91
dtype: int64

18. Modifying a Series

Existing values can be modified using their labels or positions.

marks = pd.Series(
    [78, 85, 92],
    index=["A", "B", "C"]
)

marks["B"] = 90

print(marks)

Output:

A    78
B    90
C    92
dtype: int64

19. Adding a New Series Element

A new label can be assigned a value.

marks = pd.Series(
    [78, 85, 92],
    index=["A", "B", "C"]
)

marks["D"] = 88

print(marks)

Output:

A    78
B    85
C    92
D    88
dtype: int64

20. Arithmetic Operations on Series

pandas Series support vectorized arithmetic operations.

marks = pd.Series([70, 80, 90])

print(marks + 5)
print(marks * 2)

Output:

0    75
1    85
2    95
dtype: int64

0    140
1    160
2    180
dtype: int64

21. What Is a DataFrame?

A DataFrame is a two-dimensional labeled data structure containing rows and columns.

It is similar to:

• A spreadsheet table
• A database table
• A collection of aligned Series

import pandas as pd

data = {
    "Name": ["Alice", "Brian", "Clara"],
    "Age": [20, 21, 19],
    "Score": [85, 91, 78]
}

df = pd.DataFrame(data)

print(df)

Output:

    Name  Age  Score
0  Alice   20     85
1  Brian   21     91
2  Clara   19     78

22. Understanding a DataFrame

    Name  Age  Score
0  Alice   20     85
1  Brian   21     91
2  Clara   19     78

The DataFrame contains:

• Rows: Individual records.
• Columns: Variables or attributes.
• Index: Labels identifying rows.
• Column labels: Names identifying fields.

23. Creating a DataFrame from a Dictionary

One of the most common ways to create a DataFrame is from a dictionary of lists.

data = {
    "Name": ["Alice", "Brian", "Clara"],
    "Age": [20, 21, 19],
    "Score": [85, 91, 78]
}

df = pd.DataFrame(data)

print(df)

Each dictionary key becomes a column name.

Each list provides the values for that column.

24. Creating a DataFrame from a List of Lists

A DataFrame can also be created from nested lists.

data = [
    ["Alice", 20, 85],
    ["Brian", 21, 91],
    ["Clara", 19, 78]
]

df = pd.DataFrame(
    data,
    columns=["Name", "Age", "Score"]
)

print(df)

Output:

    Name  Age  Score
0  Alice   20     85
1  Brian   21     91
2  Clara   19     78

25. Creating a DataFrame from Series

Multiple Series can be combined into a DataFrame.

names = pd.Series(
    ["Alice", "Brian", "Clara"]
)

scores = pd.Series(
    [85, 91, 78]
)

df = pd.DataFrame({
    "Name": names,
    "Score": scores
})

print(df)

26. DataFrame Index

By default, pandas creates a numeric index beginning at zero.

df = pd.DataFrame({
    "Name": ["Alice", "Brian", "Clara"],
    "Score": [85, 91, 78]
})

print(df.index)

The result describes a default RangeIndex.

27. Creating a DataFrame with a Custom Index

df = pd.DataFrame(
    {
        "Score": [85, 91, 78],
        "Grade": ["A", "A+", "B"]
    },
    index=["Alice", "Brian", "Clara"]
)

print(df)

Output:

       Score Grade
Alice     85     A
Brian     91    A+
Clara     78     B

28. Accessing DataFrame Columns

The .columns attribute returns the column labels.

df = pd.DataFrame({
    "Name": ["Alice", "Brian", "Clara"],
    "Age": [20, 21, 19],
    "Score": [85, 91, 78]
})

print(df.columns)

29. DataFrame Shape

The .shape attribute returns:

(number_of_rows, number_of_columns)

print(df.shape)

For the previous DataFrame:

(3, 3)

This means:

• 3 rows
• 3 columns

30. DataFrame Dimensions

The .ndim attribute returns the number of dimensions.

print(df.ndim)

Output:

2

A DataFrame is therefore a two-dimensional structure.

31. DataFrame Data Types

The .dtypes attribute displays the data type of each column.

print(df.dtypes)

Example output:

Name     object
Age       int64
Score     int64
dtype: object

The exact dtype can vary depending on the data and pandas/Python environment.

32. Selecting a Single Column

A column can be selected using square brackets.

print(df["Name"])

The result is a Series.

0    Alice
1    Brian
2    Clara
Name: Name, dtype: object

33. Selecting Multiple Columns

A list of column names can be supplied to select multiple columns.

result = df[
    ["Name", "Score"]
]

print(result)

Output:

    Name  Score
0  Alice     85
1  Brian     91
2  Clara     78

Selecting multiple columns returns a DataFrame.

34. Dot Notation for Columns

In some situations, a column can also be accessed using attribute notation.

print(df.Name)

This can be convenient, but bracket notation is generally more explicit and works for column names containing spaces or names that conflict with DataFrame attributes.

print(df["Name"])

35. Selecting Rows with .iloc

.iloc selects rows using integer positions.

print(df.iloc[0])

This returns the first row as a Series.

Name     Alice
Age          20
Score        85
Name: 0, dtype: object

36. Selecting Multiple Rows with .iloc

print(df.iloc[0:2])

Output:

    Name  Age  Score
0  Alice   20     85
1  Brian   21     91

The result is a DataFrame.

37. Selecting Rows with .loc

.loc is used for label-based selection.

df = pd.DataFrame(
    {
        "Name": ["Alice", "Brian", "Clara"],
        "Score": [85, 91, 78]
    },
    index=["S01", "S02", "S03"]
)

print(df.loc["S02"])

Output:

Name     Brian
Score      91
Name: S02, dtype: object

38. Selecting Rows and Columns with .loc

.loc can select both rows and columns simultaneously.

print(
    df.loc[
        ["S01", "S03"],
        ["Name", "Score"]
    ]
)

Output:

      Name  Score
S01  Alice     85
S03  Clara     78

39. Selecting Rows and Columns with .iloc

.iloc can also select both dimensions using integer positions.

print(
    df.iloc[
        [0, 2],
        [0, 1]
    ]
)

This selects the first and third rows and the first and second columns.

40. Adding a New Column

A new column can be created by assigning values to a new column name.

df["Passed"] = [
    True,
    True,
    True
]

print(df)

Output:

    Name  Score  Passed
0  Alice     85    True
1  Brian     91    True
2  Clara     78    True

41. Creating a Calculated Column

DataFrame columns support vectorized calculations.

df["Bonus"] = df["Score"] + 5

print(df)

This calculates a bonus-adjusted score for every record.

42. Modifying an Existing Column

df["Score"] = df["Score"] + 2

print(df["Score"])

Every score is increased by 2 using vectorized pandas operations.

43. Removing a Column

A column can be removed using drop().

df = df.drop(
    columns=["Bonus"]
)

print(df)

The returned DataFrame does not contain the specified column.

44. Number of Rows

The built-in len() function returns the number of rows in a DataFrame.

print(len(df))

Alternatively, the first value of shape represents the number of rows.

print(df.shape[0])

45. Number of Columns

print(df.shape[1])

Here:

df.shape[0] → number of rows
df.shape[1] → number of columns

46. Copying a DataFrame

When you need an independent copy of a DataFrame, use .copy().

df_copy = df.copy()

This is useful when transformations should be performed on a separate DataFrame without unintentionally modifying the original object.

47. Series vs DataFrame

48. A DataFrame Is Made of Series

Each column of a DataFrame can be viewed as a Series.

df = pd.DataFrame({
    "Name": ["Alice", "Brian", "Clara"],
    "Score": [85, 91, 78]
})

name_series = df["Name"]

print(type(name_series))

The result is a pandas Series.

This relationship is important:

49. Practical Example — Student Dataset

Consider a simple student dataset.

import pandas as pd

students = pd.DataFrame({
    "Name": [
        "Alice",
        "Brian",
        "Clara",
        "David"
    ],

    "Age": [
        20,
        21,
        19,
        22
    ],

    "Math": [
        82,
        91,
        76,
        88
    ],

    "Science": [
        85,
        89,
        81,
        92
    ]
})

print(students)

This DataFrame represents a typical tabular dataset.

50. Exploring the Student DataFrame

Select a Column

print(students["Math"])

Calculate Mean

print(students["Math"].mean())

Find Highest Score

print(students["Math"].max())

Filter Students

high_scorers = students[
    students["Math"] >= 85
]

print(high_scorers)

This demonstrates how a DataFrame becomes the foundation for practical data analysis.

51. Common Ways to Create a Series

52. Common Ways to Create a DataFrame

53. Important Series & DataFrame Attributes

54. Examination Questions

Multiple Choice Questions

Short Answer Questions

1. Define a pandas Series.
2. Define a pandas DataFrame.
3. Differentiate between Series and DataFrame.
4. What is the purpose of the index in pandas?
5. Differentiate between .loc and .iloc.
6. What is returned when a single DataFrame column is selected?
7. What does the .shape attribute represent?
8. How can a new column be added to a DataFrame?

55. Pandas Interview Questions

df[["Name", "Score"]]

56. Practical Lab Exercise

57. Mini Project — Student Performance Data Structure

Build a small student performance dataset using pandas.

import pandas as pd

students = pd.DataFrame({
    "Name": [
        "Alice",
        "Brian",
        "Clara",
        "David",
        "Emma"
    ],

    "Math": [
        82,
        91,
        76,
        88,
        95
    ],

    "Science": [
        85,
        89,
        81,
        92,
        90
    ],

    "English": [
        78,
        94,
        84,
        86,
        91
    ]
})

students["Total"] = (
    students["Math"]
    + students["Science"]
    + students["English"]
)

students["Average"] = (
    students["Total"] / 3
)

print(students)

This single example introduces an important Data Science workflow:

58. Common Mistakes to Avoid

60. Key Takeaways

• pandas is a major Python library for data manipulation and analysis.
• A Series is a one-dimensional labeled array.
• A DataFrame is a two-dimensional labeled table.
• A DataFrame consists of rows and columns and can be viewed as a collection of aligned Series.
• Every Series and DataFrame has an index.
• .shape returns (rows, columns).
• .loc is primarily used for label-based selection.
• .iloc is used for integer-position-based selection.
• df["Column"] normally returns a Series.
• df[["Column1", "Column2"]] returns a DataFrame.
• Columns can be created or transformed using vectorized operations.