Yuxu Ge
Article · 2023-04-23

Pandas Basics: Core Objects & Common Operations

Series: One-Dimensional Labeled Data

Creating a Series

A Series is a one-dimensional array with labeled indices, similar to a column in a spreadsheet. You can create a Series from a list, dictionary, or scalar value:

import pandas as pd
import numpy as np

# 创建一个最简单的 Series：不指定索引时，会自动使用0,1,2,...作为索引
s1 = pd.Series([10, 20, 30, 40])
print(s1)
# 输出:
# 0    10
# 1    20
# 2    30
# 3    40
# dtype: int64

# 创建 Series 时指定索引
s2 = pd.Series([100, 98, 67, 23], index=['zs', 'ls', 'ww', 'sl'])
print(s2)
# 输出:
# zs    100
# ls     98
# ww     67
# sl     23
# dtype: int64
# 这是一个 Series：左边是索引(index)，右边是对应的值，dtype 表示值的数据类型

# 通过字典创建 Series，键会成为索引，值成为数据
d = {'zs': 100, 'ls': 98, 'ww': 67, 'sl': 23}
s3 = pd.Series(d)
print(s3)
# 输出与上面 s2 相同（字典键的插入顺序即索引顺序）

A Series created from a list lets you specify indices explicitly; if omitted, Pandas uses default integer indices (0, 1, 2, …). Creating from a dictionary uses the dict's keys as indices, preserving insertion order. Both approaches yield a labeled sequence.

For repeated values, pass a scalar and an index:

# 创建一个包含10个0.2的 Series
s4 = pd.Series(0.2, index=range(10))
print(s4.head(3))  # 先看头3行
# 输出:
# 0    0.2
# 1    0.2
# 2    0.2
# dtype: float64

Here s4 contains 10 elements all equal to 0.2, indexed 0–9. The head(3) method displays the first three rows.

Indexing a Series

A Series combines features of Python lists and dictionaries: you can access elements by position or by label. Note that these two indexing schemes are independent.

Position indexing (0, 1, 2, …): Access elements by their order. s[0] retrieves the first element; s[1:3] returns a slice.

Label indexing: Access elements by their index label. If the index is 'zs', then s['zs'] retrieves that element.

Compare them:

s = pd.Series([100, 78, 98, 79], index=['zs', 'ls', 'ww', 'sl'])
# 这个 Series 的索引为 'zs','ls','ww','sl'，对应的值如列表所给

# 按位置索引访问
print(s[3])      # 第4个元素，位置从0开始计数
print(s[:2])     # 前2个元素的切片（不包含位置2）
print(s[[0, 2]]) # 位置0和2的元素，返回一个新的 Series

# 按标签索引访问
print(s['sl'])              # 索引标签为 'sl' 的元素
print(s['zs':'ww'])         # 按标签切片，包含 'zs' 到 'ww'（注意包含末端）
print(s[['zs', 'ww', 'sl']])# 指定标签列表，返回多个元素

Here s[3] fetches the element at position 3 (value 79), while s['sl'] fetches the element labeled 'sl' (also 79).

Slicing behavior differs:

This asymmetry requires attention.

Series does not support negative indexing like s[-1]. Use s.iloc[-1] instead. Position indexing works only forward.

Inspect a Series's structure:

print(s.index)   # Index(['zs', 'ls', 'ww', 'sl'], dtype='object')
print(s.values)  # [100  78  98  79]，值本身是一个 numpy.ndarray
print(s.dtype)   # int64，值的数据类型
print(s.shape)   # (4,)，Series 的形状，相当于长度为4的一维数组
print(s.size)    # 4，元素个数
print(s.ndim)    # 1，维度，对于Series永远是1

The index attribute returns an Index object with all labels. values returns a NumPy array of data. dtype shows the element type (int64 here; float64 if NaN or decimals are present). shape, size, and ndim follow NumPy conventions.

DataFrame: Two-Dimensional Labeled Tables

A DataFrame is a table with row and column labels—think of it as a spreadsheet or as multiple Series sharing one index. Each column is a Series; the column name is the key.

Example: a DataFrame with columns Name, Age, Gender represents multiple people's attributes across rows. Create a DataFrame from a dictionary, list-of-lists, or multiple Series:

import pandas as pd

# 通过字典创建 DataFrame，每个键代表一列
data = {
    'Name': ['Tom', 'Jerry', 'Jack', 'Rose'],
    'Age': [18, 18, 20, 20]
}
df = pd.DataFrame(data)
print(df)
# 输出:
#    Name  Age
# 0   Tom   18
# 1 Jerry   18
# 2  Jack   20
# 3  Rose   20

This DataFrame has columns Name and Age with default integer row indices (0, 1, 2, 3). Row and column labels appear when printed, along with cell values.

You can specify row indices and column order:

# 用字典包含多个 Series 来创建 DataFrame，各 Series 按索引自动对齐
data2 = {
    'Name': pd.Series(['Tom', 'Jerry', 'Jack', 'Rose'], index=['a', 'b', 'c', 'd']),
    'Age': pd.Series([18, 18, 20], index=['a', 'b', 'c']),      # 少了索引 'd'
    'Gender': pd.Series(['M', 'M', 'F'], index=['a', 'c', 'd']) # 少了索引 'b'
}
df2 = pd.DataFrame(data2)
print(df2)
# 输出:
#   Name   Age Gender
# a   Tom  18.0      M
# b Jerry  18.0    NaN
# c  Jack  20.0      F
# d  Rose   NaN      M

Here we set indices to 'a', 'b', 'c', 'd'. The Name column has all four values, but Age lacks a value for 'd' and Gender lacks one for 'b'. Pandas aligns rows by index, filling missing data with NaN. Index 'b' shows NaN for Gender; index 'd' shows NaN for Age. NaN (Not a Number) represents missing values—a special float type.

Note: When a column contains NaN, Pandas upcasts it to float, since NaN is a float value. For example, Age column becomes float64 despite initially being integers.

DataFrame Attributes and Operations

DataFrame offers attributes and methods to inspect structure:

Examining our DataFrame:

print(df2.index)    # Index(['a', 'b', 'c', 'd'], dtype='object')
print(df2.columns)  # Index(['Name', 'Age', 'Gender'], dtype='object')
print(df2.shape)    # (4, 3) -> 4 行 3 列
print(df2.dtypes)   
# Name      object
# Age      float64
# Gender    object
# dtype: object

print(df2.head(2))
#   Name   Age Gender
# a   Tom  18.0      M
# b Jerry  18.0    NaN

print(df2.describe())
#              Age
# count   3.000000
# mean   18.666667
# std     1.154701
# min    18.000000
# 25%    18.000000
# 50%    18.000000
# 75%    19.000000
# max    20.000000

index and columns list row and column labels. shape shows 4 rows, 3 columns. dtypes reveals Name and Gender as object (string), Age as float64. head(2) shows the first two rows; describe() summarizes the Age column (non-numeric columns are skipped).

Column Operations

Columns in a DataFrame resemble dictionary keys: each column is a Series. Use these patterns to read, add, modify, or delete columns.

Read Columns

Access a column by name using df['column_name'], which returns a Series. To get multiple columns, pass a list: df[['Name', 'Age']] returns a new DataFrame. Do not use slice notation like df['col1':'col3']—brackets default to column selection, slices default to row selection (this behavior is intentional for consistency).

Example:

df = df2  # 继续使用前面的 df2: 有 Name, Age, Gender 三列，索引 a,b,c,d

# 访问单列数据
name_col = df['Name']
print(name_col)           # 获取 Name 列（类型为 Series）

# 访问多列数据
subset = df[['Name', 'Age']]
print(subset)             # 获取 Name 和 Age 两列，返回 DataFrame

# 利用 df.columns 得到列名列表，比如去掉最后一列:
print(df[df.columns[:-1]])# 获取除最后一列外的所有列

name_col is a Series containing each person's name. subset is a DataFrame with Name and Age. The last line uses df.columns[:-1] to get all columns except the last, demonstrating an alternative retrieval method.

Pandas allows attribute access for valid column names: df.Name equals df['Name']. However, this fails if the name contains spaces or special characters, or conflicts with a DataFrame method. Prefer df[...] for clarity.

Note: If a column name has spaces ("Total Sales"), you must use df['Total Sales'], not df.Total Sales. The same applies to names with special characters or conflicting method names. To enable dot access, rename columns: df.columns = df.columns.str.replace(' ', '_').

Add Columns

Add a column by assignment: df['new_column'] = .... The right side can be:

Add multiple columns at once using pd.concat:

# 原始 DataFrame 回顾:
print(df)
#   Name   Age Gender
# a   Tom  18.0      M
# b Jerry  18.0    NaN
# c  Jack  20.0      F
# d  Rose   NaN      M

# 1. 直接赋值列表，添加一列
df['Math'] = [100, 100, 100, 100]   # 所有行的 Math 值都是100
print(df)
# 新增了 Math 列:
#   Name   Age Gender  Math
# a   Tom  18.0      M   100
# b Jerry  18.0    NaN   100
# c  Jack  20.0      F   100
# d  Rose   NaN      M   100

# 2. 赋值一个 Series，按索引对齐添加新列
df['English'] = pd.Series([95, 96, 97], index=['a', 'b', 'd'])
print(df)
# 新增 English 列（对齐索引，有缺失）:
#   Name   Age Gender  Math  English
# a   Tom  18.0      M   100     95.0
# b Jerry  18.0    NaN   100     96.0
# c  Jack  20.0      F   100      NaN  # c 没有 English 值，NaN
# d  Rose   NaN      M   100     97.0

# 3. 用 concat 横向拼接添加多列
new_cols = pd.DataFrame({
    'Chinese': [50, 50, 50, 50],
    'P.E.': [60, 60, 60, 60]
}, index=['a', 'b', 'c', 'd'])
df = pd.concat([df, new_cols], axis=1)
print(df)
# 新增了 Chinese 和 P.E. 两列:
#   Name   Age Gender  Math  English  Chinese  P.E.
# a   Tom  18.0      M   100     95.0      50    60
# b Jerry  18.0    NaN   100     96.0      50    60
# c  Jack  20.0      F   100      NaN      50    60
# d  Rose   NaN      M   100     97.0      50    60

This adds Math (all values 0.9), English (NaN for index 'c'), and two more columns via concat. Note that 'P.E.' contains a dot, so access it only via df['P.E.'], never df.P.E..

Modify Columns

Modifying a column is identical to adding one—assign to an existing column name to overwrite it:

df['Chinese'] = [100, 100, 100, 100]
print(df)
# Chinese 列的值已更新为100:
#   Name   Age Gender  Math  English  Chinese  P.E.
# a   Tom  18.0      M   100     95.0      100    60
# b Jerry  18.0    NaN   100     96.0      100    60
# c  Jack  20.0      F   100      NaN      100    60
# d  Rose   NaN      M   100     97.0      100    60

This replaces the Chinese column with all 100 values. Similarly, use df.loc to modify specific cells (covered under row operations).

Delete Columns

Remove columns with:

Example:

# 1. del 关键字删除列
del df['P.E.']
print(df.columns)  # Index(['Name', 'Age', 'Gender', 'Math', 'English', 'Chinese'], dtype='object')

# 2. pop 方法删除列
df.pop('Chinese')
print(df.columns)  # Index(['Name', 'Age', 'Gender', 'Math', 'English'], dtype='object')

# 3. drop 删除多列
df.drop(['Math', 'English'], axis=1, inplace=True)
print(df.columns)  # Index(['Name', 'Age', 'Gender'], dtype='object')
print(df)
#    Name   Age Gender
# a   Tom  18.0      M
# b Jerry  18.0    NaN
# c  Jack  20.0      F
# d  Rose   NaN      M

After deletion, the DataFrame retains only Name, Age, Gender, and the original row indices. When using drop, remember axis=1 for columns. Assign the result back if not using inplace: df = df.drop([...], axis=1).

Row Operations

Rows are accessed via their indices—either by label or by integer position. Both loc (label-based) and iloc (position-based) are essential.

Read Rows

df[...] does not directly select rows by label. df['a'] attempts to fetch a column named 'a', not a row. Instead, use:

Using loc:

# 使用 loc 按标签索引行
print(df.loc['a'])         # 索引为 'a' 的整行数据，返回一个 Series
print(df.loc['b':'c'])     # 从索引 'b' 到 'c' 的行（包括 'c' 行）
print(df.loc[['a', 'c'], ['Name', 'Gender']])
# 上面一行选取索引 a 和 c 的两行，且只取其中的 Name 和 Gender 两列

Using iloc:

# 使用 iloc 按位置索引行
print(df.iloc[0])          # 第 0 号位置的行（即索引 'a' 这一行）
print(df.iloc[1:3])        # 位置 1 到 2 的行（不包括位置3），对应索引 b, c 两行
print(df.iloc[[0, 2], [0, 2]])
# 上面一行选取位置 0 和 2 的两行，以及位置 0 和 2 的两列，实现行列交叉选取

Both loc and iloc accept single indices, slices, and lists. Key differences:

Add Rows

Pandas 1.4+ removed df.append(). Use pd.concat instead. Build a new DataFrame or Series matching the original's columns, then concatenate:

# 构造一行新数据，注意列名需与原 DataFrame 一致
new_row = pd.DataFrame([['ZhangSan', 21, 'M']], 
                       columns=['Name', 'Age', 'Gender'], 
                       index=['e'])
# 用 concat 连接新行
df = pd.concat([df, new_row], axis=0)
print(df)
# 输出:
#    Name   Age Gender
# a    Tom  18.0      M
# b  Jerry  18.0    NaN
# c   Jack  20.0      F
# d   Rose   NaN      M
# e ZhangSan 21.0      M

Here we add a row labeled 'e'. Without specifying index=['e'], the new row would default to index 0, potentially conflicting with existing indices. Always provide a unique index.

Modify Rows

Modify a row using loc or iloc:

df.loc['e'] = ['Peter', 22.0, 'M']  # 将索引 'e' 的整行重置为新的值
print(df.loc['e'])
# 输出:
# Name      Peter
# Age        22.0
# Gender        M
# Name: e, dtype: object

Row 'e' is updated to Name = Peter, Age = 22.0, Gender = 'M'. To modify a single cell, use df.at[row_label, col_name] = value or df.iat[row_pos, col_pos] = value.

Delete Rows

Use df.drop() with axis=0 (or omitted, since it's the default):

# 删除索引为 'b' 和 'd' 的两行
df = df.drop(['b', 'd'], axis=0)
print(df)
# 输出:
#    Name   Age Gender
# a   Tom  18.0      M
# c  Jack  20.0      F
# e Peter  22.0      M

Rows 'b' and 'd' are gone. The DataFrame now contains rows 'a', 'c', 'e'. With default integer indices, you can pass the index values directly (e.g., df.drop([0]) removes index 0). With custom indices, use labels. To delete rows by condition, construct a boolean mask: df.drop(df[condition].index, axis=0).

CSV File I/O

Most data lives in files. Pandas reads and writes many formats; CSV (Comma-Separated Values) is the most common.

Reading CSV Files

Use pd.read_csv(filepath):

import pandas as pd

# 基本读取（假设 CSV 第一行为表头）
df = pd.read_csv('data.csv')
print(df.head(5))  # 查看前5行

If the CSV has a header row, read_csv treats it as column names automatically. If not, set header=None and provide column names via names:

# 如果 CSV 没有表头行，指定 header=None 并提供列名
df = pd.read_csv('data_no_header.csv', header=None, names=['col1', 'col2', 'col3'])

Common parameters:

Combined example—reading stock data without headers, selecting the date (as index) and OHLC columns:

df = pd.read_csv('aapl.csv',
                 sep=',',         # 分隔符为逗号
                 header=None,     # 文件中无表头行
                 names=['name', 'date', '_', 'open', 'high', 'low', 'close', 'volume'],  # 列名列表
                 index_col='date',# 将"date"列作为行索引
                 usecols=['date', 'open', 'high', 'low', 'close']  # 只读取需要的五列
)
print(df.head(3))
# 输出示例:
#             open    high     low   close
# date
# 2011-01-28  344.17  344.40  333.53  336.10
# 2011-01-31  335.80  340.04  334.30  339.32
# 2011-02-01  341.30  345.65  340.98  345.03

Here names provides 8 column names (one marked _ to ignore), usecols selects 5 of them, and index_col='date' makes date the row index. head(3) confirms the read. usecols avoids reading unwanted data, speeding up the load.

Common question: "Will reading 50,000 rows be slow?"

No. Pandas handles 50k rows in milliseconds. Millions of rows demand attention to memory and performance. For very large files, use chunked reading: pd.read_csv(..., chunksize=10000) returns an iterable TextFileReader, yielding 10,000 rows per chunk. Process each chunk iteratively to avoid excess memory use. Alternatively, specify dtype to avoid Pandas guessing types, which saves overhead.

Writing CSV Files

Save a DataFrame to CSV with df.to_csv():

df.to_csv('output.csv', index=False)

This writes the DataFrame to output.csv with index=False, omitting row indices (often unnecessary). By default, to_csv saves both index and column headers; disable with index=False and header=False. The method supports sep and encoding parameters.

Pandas also handles other formats—Excel (read_excel, to_excel), JSON (read_json, to_json), SQL (read_sql, to_sql), Parquet, HTML tables, and more. This uniform interface makes data format conversion straightforward: move data between CSV, Excel, JSON, and databases seamlessly.

Closing

Series and DataFrame are Pandas's foundational objects for tabular data. Series indexes—both position-based and label-based—underpin all selection logic. The CRUD patterns for rows and columns generalize to larger datasets. File I/O operations integrate seamlessly, enabling rapid iteration from raw data to analysis. These fundamentals form the platform for data cleaning, merging, reshaping, and visualization tasks that follow.

© 2026 Yuxu Ge ·