基于 OpenTelemetry 的时间可观测性与监控
Temporal 可观测性通过 OpenTelemetry 收集 traces、logs 和 metrics,让你实时掌握 workflow 执行和 AI agent 模式的情况。本指南将教你如何为基于 Temporal 的应用添加插桩,并把遥测数据发送到 SigNoz,从而监控 workflow 活动、调试 agent 执行,并实现端到端的性能优化。
借助 SigNoz 中的完整 Temporal 可观测性,你可以在同一仪表板中关联 traces、logs 和 metrics,针对 workflow 失败或高延迟设置告警,并持续分析 agent 执行模式,不断提升系统的可靠性和效率。
前提条件
- 拥有有效 ingestion key 的 SigNoz Cloud 账号,或 自托管的 SigNoz 实例
- 需要网络访问,以便向 SigNoz Cloud 发送遥测数据
- Python 3.10+,并已安装
temporalio - Python 环境:已安装用于管理依赖包的
pip
使用 OpenTelemetry 监控 Temporal Workflows
自动插桩(推荐)手动如果希望快速上手、尽量少改代码,推荐使用无代码自动插桩。它适合在不修改应用代码的情况下快速搭建可观测性,并利用标准的 instrumentor 库。 有关在 Python 环境中上手 Temporal 的更多信息,请参阅 Temporal Python Setup Guide
Step 1:在 Python 环境中安装必要的包
Copypip install \
opentelemetry-distro \
opentelemetry-exporter-otlp \
httpx \
opentelemetry-instrumentation-httpx \
opentelemetry-instrumentation-system-metrics \
temporalio \
openinference-instrumentation-openai-agents \
openai \
openai-agentsStep 2:添加自动插桩
Copyopentelemetry-bootstrap --action=installStep 3:设置环境变量
在项目根目录下创建 .env 文件,并根据你的 Temporal 部署方式添加以下环境变量:
OPENAI_API_KEY=<your-openai-api-key>
TEMPORAL_ADDRESS=<local-temporal-server>
<local-temporal-server>是你本地 Temporal server 的托管地址(默认:localhost:7233)
OPENAI_API_KEY=<your-openai-api-key>
TEMPORAL_ADDRESS=<your-temporal-cloud-address>
TEMPORAL_NAMESPACE=<your-temporal-namespace-name>
TEMPORAL_API_KEY=<your-temporal-api-key>
TEMPORAL_TLS=true第 4 步:创建示例 Temporal agent 工作流
main.pyfrom __future__ import annotations
from dotenv import load_dotenv
import asyncio
import os
from temporalio import workflow
from temporalio.client import Client
from temporalio.worker import Worker
from temporalio.contrib.openai_agents import OpenAIAgentsPlugin
from temporalio.worker import UnsandboxedWorkflowRunner
from agents import Agent, Runner
load_dotenv()
@workflow.defn
class HelloWorldAgent:
@workflow.run
async def run(self, prompt: str) -> str:
agent = Agent(
name="Assistant",
model="gpt-5",
instructions="You only respond in haikus.",
)
result = await Runner.run(agent, input=prompt)
return result.final_output
async def main():
tls = os.environ.get("TEMPORAL_TLS", "").lower() in ("1", "true", "yes")
api_key = os.environ.get("TEMPORAL_API_KEY")
plugin = OpenAIAgentsPlugin()
client = await Client.connect(
target_host=os.environ.get("TEMPORAL_ADDRESS", "localhost:7233"),
namespace=os.environ.get("TEMPORAL_NAMESPACE", "default"),
api_key=api_key or None,
tls=tls,
plugins=[plugin]
)
worker = Worker(
client,
task_queue=os.environ.get("TEMPORAL_TASK_QUEUE", "openai-agents-task-queue"),
workflows=[HelloWorldAgent],
workflow_runner=UnsandboxedWorkflowRunner()
)
async with worker:
handle = await client.start_workflow(
HelloWorldAgent.run,
id="hello-world-workflow-01",
task_queue=os.environ.get("TEMPORAL_TASK_QUEUE", "openai-agents-task-queue"),
args=["Tell me about SigNoz"],
)
result = await handle.result()
print("\nWorkflow result:\n", result)
asyncio.run(main())第 5 步:以自动埋点方式运行你的应用
使用以下环境变量运行你的应用。这会将 OpenTelemetry 配置为把 traces、logs 和 metrics 导出到 SigNoz Cloud,并启用自动日志关联:
CopyOTEL_RESOURCE_ATTRIBUTES="service.name=<service_name>" \
OTEL_EXPORTER_OTLP_ENDPOINT="https://ingest.<region>.signoz.cloud:443" \
OTEL_EXPORTER_OTLP_HEADERS="signoz-ingestion-key=<your-ingestion-key>" \
OTEL_EXPORTER_OTLP_PROTOCOL=grpc \
OTEL_TRACES_EXPORTER=otlp \
OTEL_METRICS_EXPORTER=otlp \
OTEL_LOGS_EXPORTER=otlp \
OTEL_PYTHON_LOG_CORRELATION=true \
OTEL_PYTHON_LOGGING_AUTO_INSTRUMENTATION_ENABLED=true \
opentelemetry-instrument <your_run_command><service_name>:你的服务名称<region>:你的 SigNoz Cloud 区域<your-ingestion-key>:你的 SigNoz ingestion key- 把
<your_run_command>替换为你实际运行应用的命令。本例中应使用:python main.py
使用自托管 SigNoz?大部分步骤相同。只需按照 Cloud → Self-Hosted 的说明更新 endpoint 并移除 ingestion key header 即可。
基于代码的手动埋点可以对遥测配置进行细粒度控制。当你需要自定义资源属性、采样策略,或与现有可观测性基础设施集成时,推荐使用这种方式。
Step 1: 安装额外的 OpenTelemetry 依赖
Copypip install \
opentelemetry-api \
opentelemetry-sdk \
opentelemetry-exporter-otlp \
opentelemetry-instrumentation-httpx \
opentelemetry-instrumentation-system-metrics \
temporalio \
openinference-instrumentation-openai-agentsStep 2: 在 Python 应用中导入所需模块
Traces:
Copyfrom opentelemetry import trace
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
日志:
Copyfrom opentelemetry.sdk._logs import LoggerProvider, LoggingHandler
from opentelemetry.sdk._logs.export import BatchLogRecordProcessor
from opentelemetry.exporter.otlp.proto.http._log_exporter import OTLPLogExporter
from opentelemetry._logs import set_logger_provider
import logging指标:
Copyfrom opentelemetry.sdk.metrics import MeterProvider
from opentelemetry.exporter.otlp.proto.http.metric_exporter import OTLPMetricExporter
from opentelemetry.sdk.metrics.export import PeriodicExportingMetricReader
from opentelemetry import metrics
from opentelemetry.instrumentation.system_metrics import SystemMetricsInstrumentor
from opentelemetry.instrumentation.httpx import HTTPXClientInstrumentor第 3 步:配置 OpenTelemetry Tracer Provider,将 traces 直接发送到 SigNoz Cloud
Copyfrom opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry import trace
import os
from openinference.instrumentation.openai_agents import OpenAIAgentsInstrumentor
resource = Resource.create({"service.name": "<service_name>"})
provider = TracerProvider(resource=resource)
span_exporter = OTLPSpanExporter(
endpoint= os.getenv("OTEL_EXPORTER_TRACES_ENDPOINT"),
headers={"signoz-ingestion-key": os.getenv("SIGNOZ_INGESTION_KEY")},
)
processor = BatchSpanProcessor(span_exporter)
provider.add_span_processor(processor)
trace.set_tracer_provider(provider)
# 开始对 Temporal Agents 进行插桩
OpenAIAgentsInstrumentor().instrument()<service_name>是你的服务名称OTEL_EXPORTER_TRACES_ENDPOINT→ SigNoz Cloud 的 trace 端点,需使用对应的 区域:https://ingest.<region>.signoz.cloud:443/v1/tracesSIGNOZ_INGESTION_KEY→ 你的 SigNoz ingestion key
使用自托管 SigNoz?大部分步骤完全相同。只需按Cloud → Self-Hosted文档说明,更新 endpoint 并移除 ingestion key 请求头即可。
第 4 步:配置日志
复制import logging
from opentelemetry.sdk.resources import Resource
from opentelemetry._logs import set_logger_provider
from opentelemetry.sdk._logs import LoggerProvider, LoggingHandler
from opentelemetry.sdk._logs.export import BatchLogRecordProcessor
from opentelemetry.exporter.otlp.proto.http._log_exporter import OTLPLogExporter
import os
resource = Resource.create({"service.name": "<service_name>"})
logger_provider = LoggerProvider(resource=resource)
set_logger_provider(logger_provider)
otlp_log_exporter = OTLPLogExporter(
endpoint= os.getenv("OTEL_EXPORTER_LOGS_ENDPOINT"),
headers={"signoz-ingestion-key": os.getenv("SIGNOZ_INGESTION_KEY")},
)
logger_provider.add_log_record_processor(
BatchLogRecordProcessor(otlp_log_exporter)
)
# Attach OTel logging handler to root logger
handler = LoggingHandler(level=logging.INFO, logger_provider=logger_provider)
logging.basicConfig(level=logging.INFO, handlers=[handler])
logger = logging.getLogger(__name__)<service_name>是你的服务名称OTEL_EXPORTER_LOGS_ENDPOINT→ SigNoz Cloud endpoint,需使用对应region:https://ingest.<region>.signoz.cloud:443/v1/logsSIGNOZ_INGESTION_KEY→ 你的 SigNoz ingestion key
使用自托管 SigNoz?大部分步骤完全相同。只需按Cloud → Self-Hosted文档说明,更新 endpoint 并移除 ingestion key 请求头即可。
第 5 步:配置指标
复制from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.metrics import MeterProvider
from opentelemetry.exporter.otlp.proto.http.metric_exporter import OTLPMetricExporter
from opentelemetry.sdk.metrics.export import PeriodicExportingMetricReader
from opente
