进阶 tensorflow.org 2026-10-22 02:47:47 · 16 阅读

第5章 TensorFlow 变量指南:创建、更新与管理 tf.Variable

TensorFlow

Learn TensorFlow Core Guide 变量简介 使用集合整理内容 根据个人喜好保存并分类内容。 在 TensorFlow.org 上查看 在 Google Colab 中运行 在 GitHub 上查看源代码 下载笔记本 TensorFlow 变量是表示程序所操作的可共享、持久化状态的推荐方式。本指南涵盖如何在 TensorFlow 中创建、更新和管理 tf.Variable 实例。 变量通过 tf.Variable 类创建和追踪。tf.Variable 表示一个张量,其值可以通过在其上运行 op 来改变。特定的 op 允许你读取和修改该张量的值。像 tf.keras 这样的高层库使用 tf.Variable 来存储模型参数。 设置 本笔记本讨论变量放置。 如果你想查看变量被放置在什么设备上,请取消注释下面这行代码。 import tensorflow as tf # 取消注释以查看变量被放置在哪里(见下文) # tf.debugging.set_log_device_placement(True) 2024-08-15 03:11:13.352524: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:485] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered 2024-08-15 03:11:13.373928: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:8454] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered 2024-08-15 03:11:13.380240: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1452] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered 创建一个变量 要创建变量,请提供一个初始值。 tf.Variable 将具有与初始化值相同的 dtype。 my_tensor = tf.constant([[1.0, 2.0], [3.0, 4.0]]) my_variable = tf.Variable(my_tensor) # 变量可以是各种类型,就像张量一样 bool_variable = tf.Variable([False, False, False, True]) complex_variable = tf.Variable([5 + 4j, 6 + 1j]) WARNING: All log messages before absl::InitializeLog() is called are written to STDERR I0000 00:00:1723691475.985751 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723691475.989650 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723691475.992820 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723691475.996591 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723691476.008057 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723691476.011646 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723691476.014444 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723691476.017845 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723691476.021212 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723691476.024742 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723691476.027614 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723691476.031160 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723691477.244355 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723691477.246449 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723691477.248442 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723691477.250525 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723691477.252548 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723691477.254492 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723691477.257197 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723691477.259182 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723691477.261129 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723691477.263068 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723691477.264968 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723691477.266962 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723691477.304803 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723691477.307421 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723691477.309352 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723691477.311366 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723691477.313328 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723691477.315262 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723691477.317178 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723691477.319165 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723691477.321104 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723691477.323540 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723691477.325885 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723691477.328388 199297 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 变量看起来和行为都和张量一样,实际上,它是由 tf.Tensor 支撑的数据结构。 和张量一样,它们具有 dtype 和 shape,并且可以导出到 NumPy。 print("Shape: ", my_variable.shape) print("DType: ", my_variable.dtype) print("As NumPy: ", my_variable.numpy()) Shape: (2, 2) DType: As NumPy: [[1. 2.] [3. 4.]] 大多数张量操作都可以按预期作用于变量,尽管变量不能被重塑(reshape)。 print("A variable:", my_variable) print("\nViewed as a tensor:", tf.convert_to_tensor(my_variable)) print("\nIndex of highest value:", tf.math.argmax(my_variable)) # 这创建了一个新的张量;它不会重塑该变量。 print("\nCopying and reshaping: ", tf.reshape(my_variable, [1,4])) A variable: Viewed as a tensor: tf.Tensor( [[1. 2.] [3. 4.]], shape=(2, 2), dtype=float32) Index of highest value: tf.Tensor([1 1], shape=(2,), dtype=int64) Copying and reshaping: tf.Tensor([[1. 2. 3. 4.]], shape=(1, 4), dtype=float32) 如上所述,变量由张量支撑。你可以使用 tf.Variable.assign 重新分配张量。 调用 assign 通常不会分配新的张量;相反,现有张量的内存会被重用。 a = tf.Variable([2.0, 3.0]) # 这将保持相同的 dtype,float32 a.assign([1, 2]) # 不允许,因为它会调整变量大小: try: a.assign([1.0, 2.0, 3.0]) except Exception as e: print(f"{type(e).__name__}: {e}") ValueError: Cannot assign value to variable ' Variable:0': Shape mismatch.The variable shape (2,), and the assigned value shape (3,) are incompatible. 如果你在操作中将变量用作张量,你通常操作的是底层张量。 从现有变量创建新变量会复制底层张量。两个变量不会共享相同的内存。 a = tf.Variable([2.0, 3.0]) # 基于 a 的值创建 b b = tf.Variable(a) a.assign([5, 6]) # a 和 b 是不同的 print(a.numpy()) print(b.numpy()) # assign 还有其他版本 print(a.assign_add([2,3]).numpy()) # [7. 9.] print(a.assign_sub([7,9]).numpy()) # [0. 0.] [5. 6.] [2. 3.] [7. 9.] [0. 0.] 生命周期、命名和监听 在基于 Python 的 TensorFlow 中,tf.Variable 实例与其他 Python 对象具有相同的生命周期。当没有引用指向变量时,它会自动被释放。 变量也可以命名,这有助于你追踪和调试它们。你可以给两个变量赋予相同的名称。 # 创建 a 和 b;它们将具有相同的名称,但由不同的张量支撑。 a = tf.Variable(my_tensor, name="Mark") # 具有相同名称但不同值的新变量 # 注意标量加法是广播的 b = tf.Variable(my_tensor + 1, name="Mark") # 尽管名称相同,但这些元素不相等 print(a == b) tf.Tensor( [[False False] [False False]], shape=(2, 2), dtype=bool) 保存和加载模型时会保留变量名称。默认情况下,模型中的变量会自动获取唯一的变量名称,因此除非你想手动分配,否则无需自行设置。 尽管变量对微分很重要,但某些变量不需要微分。你可以通过在创建时将 trainable 设置为 false 来关闭变量的梯度。一个不需要梯度的变量示例是训练步计数器。 step_counter = tf.Variable(1, trainable=False) 放置变量和张量 为了获得更好的性能,TensorFlow 会尝试将张量和变量放置在与其 dtype 兼容的最快设备上。 这意味着如果可用,大多数变量会被放置在 GPU 上。 但是,你可以覆盖此行为。 在此代码片段中,即使有 GPU 可用,也将 float 张量和变量放置在 CPU 上。通过开启设备放置日志记录(见设置部分),你可以查看变量被放置在何处。 如果你在带有和不带 GPU 的不同后端上运行此笔记本,你会看到不同的日志记录。 请注意,必须在会话开始时开启日志记录设备放置。 with tf.device('CPU:0'): # 创建一些张量 a = tf.Variable([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]]) b = tf.constant([[1.0, 2.0], [3.0, 4.0], [5.0, 6.0]]) c = tf.matmul(a, b) print(c) tf.Tensor( [[22. 28.] [49. 64.]], shape=(2, 2), dtype=float32) 可以设置变量或张量在一个设备上,而在另一个设备上执行计算。 这会引起延迟,因为数据需要在设备之间复制。 不过,如果你有多个 GPU 工作进程,但只想要变量的一个副本,你可能这样做。 with tf.device('CPU:0'): a = tf.Variable([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]]) b = tf.Variable([[1.0, 2.0, 3.0]]) with tf.device('GPU:0'): # 逐元素乘法 k = a * b print(k) tf.Tensor( [[ 1. 4. 9.] [ 4. 10. 18.]], shape=(2, 3), dtype=float32) 有关分布式训练的更多信息,请参考指南。 下一步 要了解变量通常如何使用,请参阅我们关于自动微分的指南。

评论 (0)