Yuxu Ge
Article · 2024-01-30

NumPy Fundamentals and Practical Foundations

import numpy as np# 从列表创建一维数组
a = np.array([1, 2, 3, 4, 5])       # [1 2 3 4 5] -> 1行5列的一维数组
b = np.array((6, 7, 8, 9, 10))     # [ 6  7  8  9 10] -> 1行5列，由元组创建
print(a.dtype, b.shape)            # int64 (5,) 数据类型和形状

The code above creates one-dimensional arrays a and b from a list and tuple respectively. NumPy automatically converts all elements to a single type—here, int64—and stores them in contiguous memory. a.dtype indicates the array's element type, while b.shape gives the array's dimensions. Both outputs show length-5 one-dimensional arrays with elements as 64-bit integers.

Creating Arrays with NumPy Built-in Functions

NumPy provides several built-in functions to create arrays with specific content:

# 使用 np.arange 创建连续整数序列
c = np.arange(1, 6)                 # [1 2 3 4 5] -> 1行5列，类似 range()，生成1到5的整数
d = np.arange(1.0, 6.0)             # [1. 2. 3. 4. 5.] -> 1行5列，起止为浮点数，则元素为 float64
print(c, c.dtype)                   # [1 2 3 4 5] int64
print(d, d.dtype)                   # [1. 2. 3. 4. 5.] float64 # 创建全零或全一数组
e = np.zeros(5)                     # [0. 0. 0. 0. 0.] -> 长度5的全0数组，默认dtype=float64
f = np.ones(5, dtype=np.int32)      # [1 1 1 1 1] -> 长度5的全1数组，元素类型int32
g = np.ones_like(e)                 # [1. 1. 1. 1. 1.] -> 创建一个形状和e相同的全1数组，dtype跟e相同(float64)
print(e, f, g)                      # [0. 0. 0. 0. 0.] [1 1 1 1 1] [1. 1. 1. 1. 1.]# 创建等差数列数组的两种方式
h = np.linspace(1, 10, 4)           # [ 1.  4.  7. 10.] -> 等差序列，从1到10均匀取4个数
interval = 6.28 / 199
x = np.arange(-3.14, 3.14 + interval, interval)
y = np.linspace(-3.14, 3.14, 200)
print(len(x), x[0], x[-1])          # 200 -3.14 3.14 -> np.arange 实现等步长数列
print(len(y), y[0], y[-1])          # 200 -3.14 3.14 -> np.linspace 实现等步长数列# 创建随机数组
r = np.random.randn(2, 5)           # 创建一个2x5的标准正态分布随机数组
print(r.shape, r.mean())           # (2, 5) ~0 (二维数组形状，均值接近0)

These examples demonstrate common array creation patterns:

Array Operations

NumPy arrays support vectorized element-wise operations, allowing arithmetic or comparison on entire arrays without explicit loops. NumPy applies the operation to each element and leverages underlying optimizations.

Create a two-dimensional array and perform addition and comparison operations:

ary = np.array([range(4), range(4, 8)])   # 创建一个2x4的二维数组
print(ary)
# [[0 1 2 3]
#  [4 5 6 7]]
print(ary + 1)                            # 每个元素加1
# [[1 2 3 4]
#  [5 6 7 8]]
print(ary > 3)                            # 与标量比较，返回布尔矩阵
# [[False False False False]
#  [ True  True  True  True]]

Here, ary + 1 adds 1 to each element, producing a new array of identical shape. Simultaneously, ary > 3 compares each element and returns a Boolean array of the same shape, indicating whether each element exceeds 3. Element-wise operations make code both concise and efficient. In NumPy, operations between scalars and arrays automatically broadcast: the scalar is treated as compatible with the array's shape and expanded implicitly for element-wise computation. Similarly, if two arrays have compatible shapes (one dimension is 1 or dimensions match), NumPy applies broadcasting; this is an advanced topic beyond this scope.

Notably, NumPy's arithmetic and comparison operations produce new arrays without modifying the original. To modify an array in-place, use assignment with slicing or indexing (discussed later).

Basic Attributes of ndarray

NumPy ndarray objects contain important attributes describing array properties: shape, dimensionality, data type, and others. Understanding these attributes helps you master array structure and characteristics.

The code below creates a 2×4 array and demonstrates common ndarray attributes:

arr = np.arange(8, dtype=np.float32).reshape(2, 4)
print(arr.shape)    # (2, 4) -> 数组形状为2行4列
print(arr.ndim)     # 2 -> 数组维度数（秩），二维数组
print(arr.size)     # 8 -> 数组元素总个数
print(len(arr))     # 2 -> len 返回数组第一维长度，这里是2
print(arr.dtype)    # float32 -> 数组元素的数据类型
print(arr.itemsize) # 4 -> 每个元素占用字节数(float32每个4字节)
print(arr.nbytes)   # 32 -> 数组总字节大小(size * itemsize = 8*4)

We created arr containing elements 0 through 7 and reshaped it to 2×4. In the output:

These attributes help you understand array structure and storage. They are invaluable when verifying data import correctness or optimizing storage and performance.

Indexing and Slicing Arrays

NumPy array indexing and slicing follow syntax similar to Python lists but with notable differences and features.

Basic Indexing

One-dimensional arrays use list-like syntax with index arr[i] to access the element at position i (0-indexed). Multidimensional arrays use comma-separated indices: arr2d[i, j] retrieves the element at row i, column j. Layered indexing arr2d[i][j] is equivalent.

Slice to Get Subarrays

Applying slice syntax (e.g., arr[start:stop:step]) to an ndarray extracts a subarray. Unlike lists, multidimensional arrays allow slicing on each dimension independently: arr2d[0:2, 1:3] slices both rows and columns simultaneously.

Importantly, array slices return a view of the original array, not a copy. Modifying a slice affects the corresponding portion of the original array—unlike Python lists, which create independent copies. For an independent copy, use np.copy() or the array's .copy() method.

Indexing and slicing differ as shown below:

arr = np.arange(1, 9).reshape(2, 4)
print(arr[1], arr[1].shape)    # 访问第2行（索引1），返回1维数组，shape=(4,)
print(arr[1:2], arr[1:2].shape)  # 切片第2行，返回保留行维度的2维数组，shape=(1, 4)# 切片是视图，修改它会影响原数组
sub_arr = arr[0, 1:3]          # 取出第1行（索引0）的第2-3列子数组
sub_arr[0] = 99                # 修改子数组的第1个元素
print(arr[0])                  # 原数组第1行相应元素也变为99
# [ 1 99 99 4]

Here, arr[1] directly indexes the second row, returning a one-dimensional array (length 4), while arr[1:2] uses slicing to extract the second row, still returning a two-dimensional array with one row. Single indexing reduces dimensionality; slicing preserves it. After assigning arr[0, 1:3] (columns 2 and 3 of row 1) to sub_arr and modifying elements in sub_arr, the corresponding values in the original arr also change. This confirms that sub_arr shares memory with the original without independent data. To operate on extracted subarrays without affecting original data, add .copy() when slicing.

Data Types and Conversion

NumPy supports numerous data types (dtype): common numeric types (int, float), Booleans, strings, datetime, and composite types. Flexible dtype use sometimes improves efficiency or handles specialized data conveniently. We cover several categories below.

Basic Numeric Types and astype Conversion

In NumPy, integers default to int64 and floats to float64 (unless explicitly specified). The .astype() method converts between types:

x = np.array([1, 2, 3], dtype=np.int64)
print(x.dtype)           # int64
y = x.astype(np.float32) 
print(y, y.dtype)        # [1. 2. 3.] float32 -> 将整数转为32位浮点

We converted an integer array x to floating-point array y. astype returns a new array; the original x remains unchanged. NumPy performs safe conversions where possible—integer to float is safe. However, float to integer truncates the fractional part (does not round); note this carefully.

Boolean and String Types

NumPy's bool type represents Boolean values: True stores as 1, False as 0. Boolean arrays frequently serve as masks for filtering data. Boolean storage is compact, typically 1 byte per element.

NumPy's string type (typically displayed as <U<n>) represents fixed-length Unicode strings, where <U denotes Unicode and n is the maximum character length. NumPy determines n from the longest string in the array; all elements are truncated or padded to this length. Note that NumPy string dtype suits simple fixed-length text; for more flexible string operations, use Python's native strings or pandas string functionality.

Create Boolean and string arrays to observe their dtype and memory use:

bool_arr = np.zeros(5, dtype=bool)
print(bool_arr, bool_arr.dtype, bool_arr.itemsize)  
# [False False False False False] bool 1  -> 5个False，类型bool，每个占1字节str_arr = np.array(['NumPy', '数据分析', 'Python3.9'])
print(str_arr.dtype, str_arr.itemsize, str_arr.nbytes)  
# <U6 24 72 -> dtype为<U6，每个元素固定6个字符(24字节)，总数组占72字节(3*24)

The bool_arr is a length-5 Boolean array; printing shows element values as False, dtype as bool, and itemsize of 1 byte per element. str_arr contains three strings: 'NumPy' (length 5), '数据分析' (length 4—two Chinese characters count as 2, each character as 1), and 'Python3.9' (length 8). NumPy allocates a uniform length of 6 to the entire array—though whether this accommodates the longest 8 characters is uncertain; Chinese character representation varies by implementation, and the <U6 dtype may actually reflect the longest Unicode code points rather than simple character count. The resulting dtype is <U6, meaning each element is fixed at 6 characters. The itemsize is 24 bytes (6 Unicode characters at 4 bytes each per element), with total nbytes of 72. For short strings, this fixed-length storage wastes space.

Date and Time Types

NumPy introduces datetime64 for dates and timedelta64 for time intervals, enabling convenient time-related calculations. Convert date strings to datetime64:

dates = np.array(['2023', '2024-01-01', '2025-03-07 15:30:21'])
print(dates.dtype)            # <U19 (字符串类型)
dates = dates.astype('datetime64[s]')
print(dates, dates.dtype)
# ['2023-01-01T00:00:00' '2024-01-01T00:00:00' '2025-03-07T15:30:21'] datetime64[s]
print(dates.astype('int64'))
# [1672531200 1704067200 1741361421]  -> 转换为Unix时间戳(秒)

We defined three date strings: '2023' (year only), '2024-01-01' (year-month-day), and '2025-03-07 15:30:21' (full date-time). Converting to datetime64[s] (second precision), NumPy interprets them uniformly: missing date parts default to 01-01, missing time defaults to midnight. The output shows '2023' becomes '2023-01-01T00:00:00'. We then converted the datetime64 array to int64, yielding Unix timestamps (seconds since 1970-01-01). This is valuable for calculating date differences or sorting by time.

Structured Data Types (Composite Types)

Sometimes you need to store different data types in a single array—for example, a table with name (string), age (integer), and scores (integer list). NumPy structured arrays allow custom composite dtypes. Think of structured dtype as "arrays of tuples," where each element contains multiple typed values, like a database row.

Define a structured dtype with fields for name, three subject scores, and age:

data = [
    ('张三', [85, 92, 78], 21),
    ('李四', [75, 85, 89], 19),
    ('王五', [90, 88, 95], 20)
]
dtype = {
    'names':   ('name',   'scores',    'age'),
    'formats': ('U2',     '3int32',    'int32')
}
arr = np.array(data, dtype=dtype)
print(arr[0])          # ('张三', [85, 92, 78], 21)
print(arr['name'])     # ['张三' '李四' '王五']
print(arr['age'].mean())   # 20.0 -> 年龄字段的平均值

We constructed a list where each element is a tuple (name, [three scores], age). We then specified each field's name and type via the dtype dictionary: name as 2-character Unicode string ('U2'), scores as 3 32-bit integers ('3int32'), age as 32-bit integer. np.array creates the structured array arr from this dtype.

Printing arr[0] shows each element as a composite tuple. Access fields like a dictionary: arr['name'] extracts all name fields into a new array, arr['age'] extracts all ages. Thus, arr['age'].mean() directly computes average age. Structured arrays handle tabular data conveniently in NumPy, but note that elements are composite types and some NumPy universal functions do not directly support per-field operations—choose and apply carefully.

Reading External Data into Arrays

In practice, you often read external data (text files, CSV) into NumPy arrays. NumPy provides np.loadtxt, np.genfromtxt, and similar methods for direct file reading, but here we demonstrate manual parsing to illustrate data cleaning and array conversion.

Suppose you have a stock data file aapl.csv with rows containing date and daily open, high, low, close prices, and volume:

2021-01-04,133.52,133.6116,126.76,129.41,143301900
2021-01-05,128.89,131.74,128.43,131.01,97664900
...

Read the file line by line, parsing each row into a NumPy array:

f = open('aapl.csv')
data_list = []
for line in f:
    # 去除换行符并按逗号分割
    date, open_, high, low, close, volume = line.strip().split(',')
    # 将每行数据转换成对应类型的tuple后添加到列表
    data_list.append((date, float(open_), float(high), float(low), float(close), int(volume)))
f.close()
# 定义结构化dtype，与文件列一一对应
dtype = [('date', 'U10'), ('open', 'f4'), ('high', 'f4'), ('low', 'f4'), ('close', 'f4'), ('volume', 'u8')]
data = np.array(data_list, dtype=dtype)
print(data['high'].max())   # 145.09  -> 最高价（示例输出）

Read the file and parse each line using strip().split(',') to separate fields, then construct tuples with appropriate types: dates remain strings, prices convert to float, volume to int. Collect tuples into a list, then convert once via np.array(..., dtype=custom_dtype) to a structured NumPy array. Here, the dtype defines 6 fields: date (up to 10-character string), open, high, low, close prices (32-bit floating-point), and volume (unsigned 64-bit integer).

Finally, data['high'].max() exploits NumPy's convenience to directly find the maximum of the high-price column—the stock's historical highest price (example: 145.09). This approach easily performs statistical calculations on any structured array column.

Tip: In practice, use np.loadtxt or np.genfromtxt directly to read CSV and similar text data into arrays, specifying delimiter and dtype, minimizing manual parsing code. The example above helps you understand the data-reading and dtype-specification process.

Array Shape Transformation

Multidimensional array shape can be altered via several methods: reordering dimensions, adding or removing dimensions. NumPy's operations include reshape, flatten/ravel, and resize. Different operations have different memory behavior: some return views of the original array, others create new arrays.

reshape: View-based Reshaping

ndarray.reshape(new_shape) returns a reshaped array (view). Unless it violates memory continuity rules, reshape does not copy data; modifying the reshaped array also affects the original.

a = np.arange(12)               # 创建包含0~11的一维数组，共12个元素
b = a.reshape(3, 4)             # 将a视图reshape为3行4列的二维数组
b[0, 0] = 99                    # 修改b中的元素
print(a)                        # [99  1  2  3  4  5  6  7  8  9 10 11]
print(b)                        # [[99  1  2  3]
#                                [ 4  5  6  7]
#                                [ 8  9 10 11]]

We reshaped one-dimensional array a via reshape(3,4) to produce two-dimensional array b. After modifying b[0,0] to 99, the first element of original a is also 99. This shows a and b share underlying data memory, differing only in view.

Note: if a is not contiguous in memory (e.g., reshaping a non-contiguous slice), reshape may trigger copying or raise an error. For arrays derived from continuous sources as in this example, reshape is efficient and safe.

flatten: Copying and Reshaping

To obtain a one-dimensional independent copy, use flatten(). It returns a new array (deep copy) not sharing data with the original.

c = a.flatten()                # 将数组a展开为一维并复制
c[0] = 0
print(a)                       # [99  1  2  3  4  5  6  7  8  9 10 11] 原数组a不受影响
print(c)                       # [0 1 2 3 4 5 6 7 8 9 10 11] 新数组c的修改不影响a

After modifying the first element of a.flatten()'s result c to 0, original a remains unchanged (still starts with 99). This proves flatten returns a data copy. Similarly, np.ravel() returns a view where possible (like reshape(-1,)), copying only when unavoidable. You can also use arr.reshape(-1) for flattening, equivalent to ravel.

resize: In-Place Reshaping

ndarray.resize(new_shape) directly modifies the array's shape in-place. Unlike reshape, resize modifies the original array. If new size has fewer total elements, trailing elements are truncated; if larger, new positions fill with 0.

a.resize(3, 3)                 # 将数组a直接改为3x3形状
print(a)
# [[99  1  2]
#  [ 3  4  5]
#  [ 6  7  8]]

We called resize(3,3) on a. Originally a held 12 elements; resizing to 3×3 holds only 9, so the last three [9, 10, 11] are discarded. Resize modifies a's shape; it is now a 3×3 array containing the original first 9 elements (including the modified 99), with others dropped.

Use resize cautiously—it destroys original data. To obtain a copy of specific shape, prefer reshape returning a view, or np.copy() then reshape, avoiding unintended original data loss.

Boolean and Position Indexing

Section 3 introduced Boolean arrays; now we detail Boolean and position indexing—powerful NumPy mechanisms for filtering or reordering arrays.

Boolean Indexing

Boolean indexing uses a Boolean array to index another. Positions where the Boolean array is True are retained in the result.

Extract multiples of 3 from an array of 1 to 9:

ary = np.arange(1, 10)
print(ary % 3 == 0)           # [False False  True False False  True False False  True]
print(ary[ary % 3 == 0])      # [3 6 9] -> 筛选出3的倍数

ary % 3 == 0 produces a Boolean array of equal length; True appears where elements are multiples of 3. Using this Boolean array as an index retrieves all True positions: [3, 6, 9].

Construct your own Boolean list to manually select elements:

mask = [True, False, True, False, False, True, False, True, False]
print(ary[mask])              # [1 3 6 8] -> True 所在位置的元素被选出

The mask list must match ary's length; True at positions 0, 2, 5, 7 selects corresponding elements 1, 3, 6, 8.

Boolean indexing commonly filters arrays by condition, as shown. For complex conditions, combine bitwise operators. For example, filter elements that are multiples of both 3 and 7:

cond = (ary % 3 == 0) & (ary % 7 == 0)
print(cond)                   # [False False False False False False False False False]
print(ary[cond])              # [] -> 1~9中没有同时满足3和7倍数条件的数字

Here, & is bitwise AND, equivalent to logical "and." From 1 to 9, no numbers are multiples of both; the result is empty. In a larger range like 1 to 100, you'd find 21, 42, 63, 84.

Position Indexing

Position indexing uses integer arrays (or lists) to specify desired index positions, allowing reordering or arbitrary element extraction.

For example:

idx = [3, 2, 0, 1]            # 定义一个索引顺序
arr = np.arange(1, 6)         # [1 2 3 4 5]
print(arr[idx])               # [4 3 1 2] -> 按idx顺序重新排列得到的新数组

We use index list [3, 2, 0, 1] to index array [1, 2, 3, 4, 5], retrieving elements in order: the 3rd element (value 4), 2nd (value 3), 0th (value 1), 1st (value 2), combined as [4, 3, 1, 2]. This enables arbitrary array reordering.

Position indexing supports repeating indices:

idx2 = [0, 0, 1, 1]
print(arr[idx2])              # [1 1 2 2] -> 0和1索引各出现两次

Result [1, 1, 2, 2] shows original elements 0 and 1 retrieved twice each. The resulting array's size depends on the index list length, not the source array's.

Modifying Values with Boolean Indexing

Boolean indexing not only filters but also modifies elements meeting a condition. Suppose a matrix holds student scores; mark passing (≥60) as 1, failing as 0:

scores = np.array([[66.6, 54.2, 88.8],
                   [98.7, 45.6, 12.3],
                   [56.7, 89.0, 33.3]])
mask = scores >= 60
scores[mask] = 1
scores[~mask] = 0
print(scores)
# [[1. 0. 1.]
#  [1. 0. 0.]
#  [0. 1. 0.]]

First, scores >= 60 produces a same-shaped Boolean matrix mask indicating pass/fail. Then use Boolean indexing to assign directly: scores[mask] = 1 sets all passing positions to 1, scores[~mask] = 0 sets failing positions (mask inverted via ~) to 0. All values become 0 or 1, implementing the marking.

Note: operations like scores[mask] = 1 modify the original array; ensure this is safe and intended. To mark without modifying original data, copy first with scores.copy() before operating, or use (scores >= 60).astype(int) for a new 0/1 array unaffecting scores.

Combining and Splitting Arrays

With multiple arrays, you often merge them or split a large array into smaller parts. NumPy provides convenient functions for vertical/horizontal combining and splitting, plus higher-dimensional cases.

Array Combining (Concatenation)

Example vertical and horizontal combining:

a = np.array([[1, 2],
              [3, 4]])
b = np.array([[5, 6],
              [7, 8]])
v = np.vstack((a, b))
h = np.hstack((a, b))
d = np.dstack((a, b))
print(v)
# [[1 2]
#  [3 4]
#  [5 6]
#  [7 8]]
print(h)
# [[1 2 5 6]
#  [3 4 7 8]]
print(d.shape)                # (2, 2, 2) -> 深度组合后数组的形状

Both a and b are 2×2 matrices. vstack stacks rows, producing 4×2 matrix v; hstack concatenates columns, producing 2×4 matrix h. dstack combines along the third dimension into a (2, 2, 2) three-dimensional array. Printing d shows it essentially stacks a and b as two layers.

Array Splitting (Division)

Corresponding split functions reverse combination:

Split a 2×6 array horizontally into three 2×2 arrays, and vertically into two parts:

c = np.arange(1, 13).reshape(2, 6)
print(c)
# [[ 1  2  3  4  5  6]
#  [ 7  8  9 10 11 12]]
print(np.hsplit(c, 3))
# [array([[1, 2],
#        [7, 8]]), array([[3, 4],
#        [9, 10]]), array([[ 5,  6],
#        [11, 12]])]
print(np.vsplit(c, 2))
# [array([[1, 2, 3, 4, 5, 6]]), array([[ 7,  8,  9, 10, 11, 12]])]

We created c as a 2×6 matrix with contents 1 to 12. np.hsplit(c, 3) cuts by column into 3 parts of 2 columns each, returning a list of three 2×2 arrays. np.vsplit(c, 2) splits rows in half, producing two 1×6 row vectors. np.split achieves the same with axis parameters.

For unequal parts—for example, splitting c into first two columns and the rest—use an index list: np.hsplit(c, [2]) (cutting at column index 2). Combining and splitting flexibly adjust array structure, useful for chunked data processing or result grouping.

Other Common Attributes and Methods

Finally, several other useful ndarray attributes and methods:

Demonstrate these with a 3×3 complex matrix:

A = np.array([[1+1j, 2+4j, 3+7j],
              [4+2j, 5+5j, 6+8j],
              [7+3j, 8+6j, 9+9j]])
print(A.real)
# [[1. 2. 3.]
#  [4. 5. 6.]
#  [7. 8. 9.]]
print(A.imag)
# [[1. 4. 7.]
#  [2. 5. 8.]
#  [3. 6. 9.]]
print(A.T)
# [[1.+1.j 4.+2.j 7.+3.j]
#  [2.+4.j 5.+5.j 8.+6.j]
#  [3.+7.j 6.+8.j 9.+9.j]]
print([x for x in A.flat])   # [(1+1j), (2+4j), (3+7j), (4+2j), (5+5j), (6+8j), (7+3j), (8+6j), (9+9j)]
print(A.tolist())           # [[(1+1j), (2+4j), (3+7j)], [(4+2j), (5+5j), (6+8j)], [(7+3j), (8+6j), (9+9j)]]

In the 3×3 complex matrix A:

These attributes and methods streamline ndarray operations. For passing NumPy arrays to pure Python code, use .tolist(). For element-wise processing (though Python-level loops should be avoided), .flat provides convenient traversal. When working with complex arrays, .real and .imag separate real and imaginary parts for analysis.

© 2026 Yuxu Ge ·