进阶 tensorflow.org 2026-10-22 02:47:47 · 18 阅读

第3章 TensorFlow 第3章概览:基础入门指南

TensorFlow 学习 TensorFlow Core 指南

TensorFlow 基础

使用集合功能保持条理 根据你的偏好保存和分类内容。

在 TensorFlow.org 上查看 在 Google Colab 中运行 在 GitHub 上查看源码 下载 notebook

本指南简要介绍 TensorFlow 的基础知识。文档每一节都是某个更大主题的概览,各节末尾提供了完整指南的链接。

TensorFlow 是一个端到端的机器学习平台,支持以下功能:

基于多维数组的数值计算(类似 NumPy) GPU 和分布式处理 自动微分 模型的构建、训练和导出 等等

张量

TensorFlow 处理的对象是以 tf.Tensor 对象表示的多维数组(即张量)。下面是一个二维张量: import tensorflow as tf

x = tf.constant([[1., 2., 3.], [4., 5., 6.]])

print(x) print(x.shape) print(x.dtype)

2024-10-03 01:24:28.897515: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:485] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered 2024-10-03 01:24:28.918497: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:8454] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered 2024-10-03 01:24:28.924727: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1452] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered WARNING: All log messages before absl::InitializeLog() is called are written to STDERR I0000 00:00:1727918671.501067 10533 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1727918671.504962 10533 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1727918671.508951 10533 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1727918671.512662 10533 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1727918671.523975 10533 cuda_executor.cc:1015] successful NUMA node read from SysFS ha tf.Tensor( [[1. 2. 3.] [4. 5. 6.]], shape=(2, 3), dtype=float32) (2, 3) d negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1727918671.527575 10533 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1727918671.531095 10533 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1727918671.534452 10533 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1727918671.537818 10533 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1727918671.541314 10533 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1727918671.544748 10533 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1727918671.548158 10533 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1727918672.811574 10533 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1727918672.813683 10533 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1727918672.815800 10533 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1727918672.817784 10533 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1727918672.819747 10533 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1727918672.822369 10533 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1727918672.824361 10533 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1727918672.826310 10533 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1727918672.828145 10533 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1727918672.830199 10533 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1727918672.832189 10533 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1727918672.834056 10533 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1727918672.872509 10533 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1727918672.874567 10533 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1727918672.876725 10533 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1727918672.878642 10533 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1727918672.880502 10533 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1727918672.882472 10533 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1727918672.884483 10533 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1727918672.886365 10533 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1727918672.888219 10533 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1727918672.891799 10533 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1727918672.894304 10533 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1727918672.896575 10533 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355

tf.Tensor 最重要的两个属性是形状和数据类型:

Tensor.shape:张量在每个轴上的大小。 Tensor.dtype:张量中所有元素的数据类型。

TensorFlow 对张量实现了标准数学运算,以及许多机器学习专用的运算。

例如: x + x

5 * x

x @ tf.transpose(x)

tf.concat([x, x, x], axis=0)

tf.nn.softmax(x, axis=-1)

tf.reduce_sum(x)

tf.convert_to_tensor([1,2,3])

tf.reduce_sum([1,2,3])

在 CPU 上运行大规模计算可能很慢。只要配置得当,TensorFlow 可以利用 GPU 等加速硬件来极快地执行运算。 if tf.config.list_physical_devices('GPU'): print("TensorFlow IS using the GPU") else: print("TensorFlow IS NOT using the GPU")

TensorFlow IS using the GPU

详情请参阅 Tensor 指南。

变量

普通的 tf.Tensor 对象是不可变的。要在 TensorFlow 中存储模型权重(或其他可变状态),应使用 tf.Variable。 var = tf.Variable([0.0, 0.0, 0.0]) var.assign([1, 2, 3])

var.assign_add([1, 1, 1])

详情请参阅 Variables 指南。

自动微分

梯度下降及相关算法是现代机器学习的基石。

为此,TensorFlow 实现了自动微分(autodiff),利用微积分来计算梯度。通常,你会用它来计算模型误差或损失相对于权重的梯度。 x = tf.Variable(1.0)

def f(x): y = x2 + 2*x - 5 return y f(x)

在 x = 1.0 处,y = f(x) = (12 + 2*1 - 5) = -2。

y 的导数是 y' = f'(x) = (2*x + 2) = 4。TensorFlow 可以自动算出它: with tf.GradientTape() as tape: y = f(x)

g_x = tape.gradient(y, x) # g(x) = dy/dx

g_x

这个简化示例只对单个标量(x)求导,但 TensorFlow 可以同时计算任意数量非标量张量的梯度。

详情请参阅 Autodiff 指南。

计算图与 tf.function

虽然你可以像使用其他 Python 库一样交互式地使用 TensorFlow,但 TensorFlow 还提供了以下工具:

性能优化:加速训练和推理。 导出:训练完成后保存模型。

这些功能要求你使用 tf.function 把纯 TensorFlow 代码与 Python 代码分离开来。 @tf.function def my_func(x): print('Tracing.\n') return tf.reduce_sum(x)

第一次运行 tf.function 时,虽然它是在 Python 中执行的,但它会捕获一个完整且经过优化的计算图,表示函数内部完成的 TensorFlow 计算。 x = tf.constant([1, 2, 3]) my_func(x)

Tracing.

之后的调用中,TensorFlow 只执行优化后的计算图,跳过任何非 TensorFlow 的步骤。注意下面 my_func 没有再打印 tracing,因为 print 是 Python 函数而非 TensorFlow 函数。 x = tf.constant([10, 9, 8]) my_func(x)

如果输入的签名(形状和数据类型)不同,计算图无法复用,因此会生成一个新图: x = tf.constant([10.0, 9.1, 8.2], dtype=tf.float32) my_func(x)

Tracing.

这些捕获的计算图带来两个好处:

在很多情况下能显著加快执行速度(当然这个简单例子看不出来)。 可以使用 tf.saved_model 导出这些图,在服务器或移动设备等其他环境中运行,无需安装 Python。

详情请参阅 Intro to graphs。

模块、层和模型

tf.Module 是一个用于管理 tf.Variable 对象以及操作它们的 tf.function 对象的类。tf.Module 类支持两个重要功能:

可以使用 tf.train.Checkpoint 保存和恢复变量的值。这在训练过程中非常实用,可以快速保存和恢复模型状态。 可以使用 tf.saved_model 导入导出 tf.Variable 的值和 tf.function 的计算图。这使你可以在创建模型的 Python 程序之外独立运行模型。

下面是一个导出简单 tf.Module 对象的完整示例: class MyModule(tf.Module): def __init__(self, value): self.weight = tf.Variable(value)

@tf.function def multiply(self, x): return x * self.weight mod = MyModule(3) mod.multiply(tf.constant([1, 2, 3]))

保存这个 Module: save_path = './saved' tf.saved_model.save(mod, save_path)

INFO:tensorflow:Assets written to: ./saved/assets INFO:tensorflow:Assets written to: ./saved/assets

生成的 SavedModel 独立于创建它的代码。你可以在 Python、其他语言绑定或 TensorFlow Serving 中加载 SavedModel,也可以将其转换为 TensorFlow Lite 或 TensorFlow JS 格式来运行。 reloaded = tf.saved_model.load(save_path) reloaded.multiply(tf.constant([1, 2, 3]))

tf.keras.layers.Layer 和 tf.keras.Model 类都构建在 tf.Module 之上,为构建、训练和保存模型提供了额外的功能和便捷方法。下一节将演示其中一部分。

详情请参阅 Intro to modules。

训练循环

现在把上面的内容结合起来,从零开始构建一个基础模型并训练它。

首先,创建一些示例数据。下面的代码生成一团大致沿二次曲线分布的点: import matplotlib from matplotlib import pyplot as plt

matplotlib.rcParams['figure.figsize'] = [9, 6] x = tf.linspace(-2, 2, 201) x = tf.cast(x, tf.float32)

def f(x): y = x2 + 2*x - 5 return y

y = f(x) + tf.random.normal(shape=[201])

plt.plot(x.numpy(), y.numpy(), '.', label='Data') plt.plot(x, f(x), label='Ground truth') plt.legend();

创建一个权重和偏置随机初始化的二次模型: class Model(tf.Module):

def __init__(self): # 随机生成权重和偏置 rand_init = tf.random.uniform(shape=[3], minval=0., maxval=5., seed=22) # 初始化模型参数 self.w_q = tf.Variable(rand_init[0]) self.w_l = tf.Variable(rand_init[1]) self.b = tf.Variable(rand_init[2])

@tf.function def __call__(self, x): # 二次模型:quadratic_weight * x^2 + linear_weight * x + bias return self.w_q * (x2) + self.w_l * x + self.b

先看看训练前模型的表现: quad_model = Model() def plot_preds(x, y, f, model, title): plt.figure() plt.plot(x, y, '.', label='Data') plt.plot(x, f(x), label='Ground truth') plt.plot(x, model(x), label='Predictions') plt.title(title) plt.legend() plot_preds(x, y, f, quad_model, 'Before training')

接下来为模型定义一个损失函数:

由于该模型要预测的是连续值,均方误差(MSE)是损失函数的不错选择。给定预测值向量 \(\hat{y}\) 和真实目标值向量 \(y\),MSE 定义为预测值与真实值之差的平方的平均值。

\(MSE = \frac{1}{m}\sum_{i=1}^{m}(\hat{y}_i -y_i)^2\) def mse_loss(y_pred, y): return tf.reduce_mean(tf.square(y_pred - y))

为模型编写一个基本的训练循环。该循环利用 MSE 损失函数及其关于参数的梯度,迭代地更新模型参数。使用 mini-batch 训练既能节省内存,又能加快收敛。tf.data.Dataset API 提供了实用的批处理和打乱功能。 batch_size = 32 dataset = tf.data.Dataset.from_tensor_slices((x, y)) dataset = dataset.shuffle(buffer_size=x.shape[0]).batch(batch_size) # 设置训练参数 epochs = 100 learning_rate = 0.01 losses = []

# 格式化训练循环 for epoch in range(epochs): for x_batch, y_batch in dataset: with tf.GradientTape() as tape: batch_loss = mse_loss(quad_model(x_batch), y_batch) # 根据梯度计算更新参数 grads = tape.gradient(batch_loss, quad_model.variables) for g,v in zip(grads, quad_model.variables): v.assign_sub(learning_rate*g) # 记录每个 epoch 的模型损失 loss = mse_loss(quad_model(x), y) losses.append(loss) if epoch % 10 == 0: print(f'Mean squared error for step {epoch}: {loss.numpy():0.3f}')

# 绘制模型结果 print("\n") plt.plot(range(epochs), losses) plt.xlabel("Epoch") plt.ylabel("Mean Squared Error (MSE)") plt.title('MSE loss vs training iterations');

Mean squared error for step 0: 57.050 Mean squared error for step 10: 10.417 Mean squared error for step 20: 4.270 Mean squared error for step 30: 2.171 Mean squared error for step 40: 1.429 Mean squared error for step 50: 1.162 Mean squared error for step 60: 1.062 Mean squared error for step 70: 1.027 Mean squared error for step 80: 1.017 Mean squared error for step 90: 1.014

再看看训练后模型的表现: plot_preds(x, y, f, quad_model, 'After training')

运行效果不错。不过要注意,常用的训练工具在 tf.keras 模块中已有现成实现,动手自己写之前不妨先考虑用它们。首先,Model.compile 和 Model.fit 方法可以帮你实现训练循环:

先用 tf.keras.Sequential 在 Keras 中创建一个 Sequential 模型。最简单的 Keras 层之一是全连接层,可以用 tf.keras.layers.Dense 实例化。全连接层能学习 \(\mathrm{Y} = \mathrm{W}\mathrm{X} + \vec{b}\) 形式的多维线性关系。要学习 \(w_1x^2 + w_2x + b\) 这样的非线性方程,全连接层的输入应是包含 \(x^2\) 和 \(x\) 两个特征的数据矩阵。tf.keras.layers.Lambda 层可用于完成这种特征堆叠变换。 new_model = tf.keras.Sequential([ tf.keras.layers.Lambda(lambda x: tf.stack([x, x2], axis=1)), tf.keras.layers.Dense(units=1, kernel_initializer=tf.random.normal)]) new_model.compile( loss=tf.keras.losses.MSE, optimizer=tf.keras.optimizers.SGD(learning_rate=0.01))

history = new_model.fit(x, y, epochs=100, batch_size=32, verbose=0)

new_model.save('./my_new_model.keras')

WARNING: All log messages before absl::InitializeLog() is called are written to STDERR I0000 00:00:1727918680.462363 10695 service.cc:146] XLA service 0x7f51a0006eb0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: I0000 00:00:1727918680.462397 10695 service.cc:154] StreamExecutor device (0): Tesla T4, Compute Capability 7.5 I0000 00:00:1727918680.462401 10695 service.cc:154] StreamExecutor device (1): Tesla T4, Compute Capability 7.5 I0000 00:00:1727918680.462404 10695 service.cc:154] StreamExecutor device (2): Tesla T4, Compute Capability 7.5 I0000 00:00:1727918680.462407 10695 service.cc:154] StreamExecutor device (3): Tesla T4, Compute Capability 7.5 I0000 00:00:1727918680.790826 10695 device_compiler.h:188] Compiled cluster using XLA! This line is logged at most once for the lifetime of the process.

观察 Keras 模型训练后的表现: plt.plot(history.history['loss']) plt.xlabel('Epoch') plt.ylim([0, max(plt.ylim())]) plt.ylabel('Loss [Mean Squared Error]') plt.title('Keras training progress');

plot_preds(x, y, f, new_model, 'After Training: Keras')

详情请参阅 Basic training loops 和 Keras 指南。

评论 (0)