内容纲要
概要描述
本文主要介绍DBAService后台存储的csv文件每列的说明。
这些csv数据文件 在dbaservice节点 $dbaservice.persistence.datadir 目录下
详细介绍
1. sql
| 序号 | Sql字段名称 | Sql数据类型 | Sql字段中文解释 | 样例 |
|---|---|---|---|---|
| 1 | serverKey | String | 监控server的唯一标识，用于识别服务器实例 | Inceptor::kv1::d8da21f4-7912-463f-be5d-b17f5a93c5fa, |
| 2 | SqlId | long | Sql的唯一标识或ID | 961626, |
| 3 | sessionId | long | 会话的唯一标识，表示会话的ID | 52470, |
| 4 | extraJobId | long | localfast模式下的JobID | -2147483648, |
| 5 | sqlGroup | String | Sql所属的组别或分类 | inceptor, |
| 6 | sessionName | String | 会话的名称 | , |
| 7 | detailedDescription | String | 详细的Sql描述(Sql语句) | SELECT * FROM emp_holo t1 JOIN emp_holo t2 ON t1.empno =t2.empno, |
| 8 | message | String | Sql的报错信息 | , |
| 9 | state | String | Sql的状态 | SUCCESS, |
| 10 | tags | String | 与Sql相关的标签 | , |
| 11 | execution PlanGraph | Execution PlanGraph | Sql执行计划的图形表示 (查询执行计划图) | {"nodes":[{"opId":"FileSinkOperator-3084004","output":0,"stageId":211145,"jobId":240687,"table":null,"highlight":false,"skewTables":[],"nullSkewFlag":false},{"opId":"SelectOperator-3084005","output":0,"stageId":211145,"jobId":240687,"table":null,"highlight":false,"skewTables":[],"nullSkewFlag":false},{"opId":"HolodeskGLUMJoinOperator-3084014","output":0,"stageId":211145,"jobId":240687,"table":null,"highlight":false,"skewTables":[],"nullSkewFlag":false},{"opId":"FilterOperator-3084008","output":0,"stageId":211145,"jobId":240687,"table":null,"highlight":false,"skewTables":[],"nullSkewFlag":false},{"opId":"TableScanOperator-3084009","output":0,"stageId":211145,"jobId":240687,"table":{"database":"default","name":"emp_holo","alias":"t1","format":"HOLODESK","filterPartsNum":0,"rangePartition":false,"partKeys":null,"partsInfo":null,"columnsNeeded":[{"index":0,"name":"EMPNO","type":"Int","useGlobalIndex":false},{"index":1,"name":"ENAME","type":"String","useGlobalIndex":false},{"index":2,"name":"JOB","type":"String","useGlobalIndex":false},{"index":3,"name":"MGR","type":"Int","useGlobalIndex":false},{"index":4,"name":"HIREDATE","type":"Date","useGlobalIndex":false},{"index":5,"name":"SAL","type":"Int","useGlobalIndex":false},{"index":6,"name":"COMM","type":"Int","useGlobalIndex":false},{"index":7,"name":"DEPTNO","type":"Int","useGlobalIndex":false}]},"highlight":false,"skewTables":[],"nullSkewFlag":false},{"opId":"FilterOperator-3084011","output":0,"stageId":211144,"jobId":240686,"table":null,"highlight":false,"skewTables":[],"nullSkewFlag":false},{"opId":"TableScanOperator-3084012","output":0,"stageId":211144,"jobId":240686,"table":{"database":"default","name":"emp_holo","alias":"t2","format":"HOLODESK","filterPartsNum":0,"rangePartition":false,"partKeys":null,"partsInfo":null,"columnsNeeded":[{"index":0,"name":"EMPNO","type":"Int","useGlobalIndex":false},{"index":1,"name":"ENAME","type":"String","useGlobalIndex":false},{"index":2,"name":"JOB","type":"String","useGlobalIndex":false},{"index":3,"name":"MGR","type":"Int","useGlobalIndex":false},{"index":4,"name":"HIREDATE","type":"Date","useGlobalIndex":false},{"index":5,"name":"SAL","type":"Int","useGlobalIndex":false},{"index":6,"name":"COMM","type":"Int","useGlobalIndex":false},{"index":7,"name":"DEPTNO","type":"Int","useGlobalIndex":false}]},"highlight":false,"skewTables":[],"nullSkewFlag":false}],"edges":[{"h":"TableScanOperator-3084009","t":"FilterOperator-3084008"},{"h":"FilterOperator-3084008","t":"HolodeskGLUMJoinOperator-3084014"},{"h":"TableScanOperator-3084012","t":"FilterOperator-3084011"},{"h":"FilterOperator-3084011","t":"HolodeskGLUMJoinOperator-3084014"},{"h":"HolodeskGLUMJoinOperator-3084014","t":"SelectOperator-3084005"},{"h":"SelectOperator-3084005","t":"FileSinkOperator-3084004"}],"stageGroups":[{"stageId":211145,"jobId":240687,"submissionTime":1789889544110,"completionTime":1789889544485,"duration":375,"partitionSize":6,"status":"SUCCESS","opIds":["FileSinkOperator-3084004","SelectOperator-3084005","HolodeskGLUMJoinOperator-3084014","FilterOperator-3084008","TableScanOperator-3084009"],"stageCode":"Q09NTU9OK0luY2VwdG9yOjprdjE6OmQ4ZGEyMWY0LTc5MTItNDYzZi1iZTVkLWIxN2Y1YTkzYzVmYSsyMTExNDUrMTc4OTg4OTU0NDExMCsxNzg5ODg5NTQ0NDg1","highlight":false},{"stageId":211144,"jobId":240686,"submissionTime":1789889543785,"completionTime":1789889543936,"duration":151,"partitionSize":6,"status":"SUCCESS","opIds":["FilterOperator-3084011","TableScanOperator-3084012"],"stageCode":"Q09NTU9OK0luY2VwdG9yOjprdjE6OmQ4ZGEyMWY0LTc5MTItNDYzZi1iZTVkLWIxN2Y1YTkzYzVmYSsyMTExNDQrMTc4OTg4OTU0Mzc4NSsxNzg5ODg5NTQzOTM2","highlight":false}],"jobGroups":[{"jobId":240686,"opIds":["FilterOperator-3084011","TableScanOperator-3084012"],"jobCode":"Q09NTU9OK0luY2VwdG9yOjprdjE6OmQ4ZGEyMWY0LTc5MTItNDYzZi1iZTVkLWIxN2Y1YTkzYzVmYSsyNDA2ODYrMTc4OTg4OTU0Mzc4NCsxNzg5ODg5NTQzOTM2"},{"jobId":240687,"opIds":["FileSinkOperator-3084004","SelectOperator-3084005","HolodeskGLUMJoinOperator-3084014","FilterOperator-3084008","TableScanOperator-3084009"],"jobCode":"Q09NTU9OK0luY2VwdG9yOjprdjE6OmQ4ZGEyMWY0LTc5MTItNDYzZi1iZTVkLWIxN2Y1YTkzYzVmYSsyNDA2ODcrMTc4OTg4OTU0NDExMCsxNzg5ODg5NTQ0NDg1"}]}, |
| 12 | dagGraph | DagGraph | Sql的DAG（Job页面的有向无环图）图形表示 | null, |
| 13 | submissionTime | long | Sql提交的时间戳 | 1789889543351, |
| 14 | startTime | long | Sql开始执行的时间戳 | 1789889543665, |
| 15 | compute FinishTime | long | Sql计算完成的时间戳 | 1789889544503, |
| 16 | fileCommit FinishTime | long | 文件提交完成的时间戳 | 1789889544548, |
| 17 | fetchResult StartTime | long | 获取结果开始的时间戳 | 1789889544548, |
| 18 | realFetch ResultStartTime | long | 实际获取结果的开始时间戳 | 1789889544636, |
| 19 | localTime | long | Sql收到消息的最新本地时间 | 1789889544948, |
| 20 | fetchRowCount | long | 获取结果的行数 | 14, |
| 21 | fetchCount | long | 获取结果的次数 | 2, |
| 22 | fetchComputeTotal | long | 获取结果计算总时间 | 102, |
| 23 | durationBlockMap | Map<String,String> |
Sql执行时间映射 | {}, |
| 24 | durations | Map<String,Long> |
不同Stage的Sql执行时间 | {"Fetch Data":235,"File Commit":45,"compile":291,"Job 240687":375,"Job 240686":152}, |
| 25 | nonLinear Durations | Map<String,Long> |
非线性Sql执行时间 | {}, |
| 26 | accumulable Durations | Map<String,Long> |
累积Sql执行时间 | {}, |
| 27 | nonLinear Accumulable Durations | Map<String,Long> |
非线性累积Sql执行时间 | {}, |
| 28 | metrics | Map<String,String> |
与Sql相关的性能指标 | {"sqlId":"961626","phase":"FETCH_RESULT","sessionName":""}, |
| 29 | meta | Map<String,String> |
其他与Sql相关的元数据 | {}, |
| 30 | operationId | String | 操作的唯一标识(对应kill sql operationId) | d71525f0-9759-402b-afc5-a44194afe0a2, |
| 31 | SqlType | String | Sql的类型 | QUERY(1), |
| 32 | JobsCompleted | long | 完成的Job数 | 2, |
| 33 | JobsFailed | long | 失败的Job数 | 0, |
| 34 | stagesExpected | long | 预期的Stage数 | 2, |
| 35 | stagesCompleted | long | 完成的Stage数 | 2, |
| 36 | stagesFailed | long | 失败的Stage数 | 0, |
| 37 | stagesSkipped | long | 跳过的Stage数 | 0, |
| 38 | tasksExpected | long | 预期的Task数 | 12, |
| 39 | tasksCompleted | long | 完成的Task数 | 12, |
| 40 | tasksFailed | long | 失败的Task数 | 0, |
| 41 | tasksSkipped | long | 跳过的Task数 | 0, |
| 42 | totalTaskTime | long | Sql的所有Task总执行时间(task时间的累加和) | 2968, |
| 43 | mode | String | Sql执行模式 | linac-cluster, |
| 44 | user | String | 执行Sql的用户 | unknown, |
| 45 | poolName | String | 资源池的名称 | TTTT, |
| 46 | sessionTimeRange | String | SQL的会话时间范围(openTime::closeTime) | 1789889581058, |
| 47 | completionTime | Set<Long> |
Sql完成的时间戳 | 1789889544783, |
| 48 | inceptorTxnIds | Set<Long> |
Inceptor事务的ID | [], |
| 49 | argoHoloTxnIds | Set<Long> |
ArgoHolo事务的ID | [232563], |
| 50 | SqlLength | long | Sql的语句长度 | 71, |
| 51 | shuffleReadSize | long | Shuffle读取的数据大小(byte) | 0, |
| 52 | shuffleWriteSize | long | Shuffle写入的数据大小(byte) | 0, |
| 53 | taskCost | long | Task的总成本耗时(旧字段不用兼容保留) | 2968, |
| 54 | taskAbnormal | List<Integer> |
Task是否异常 | [], |
| 55 | waitTime | long | 等待时间 | 0, |
| 56 | wait ArgoHolo TxnIds | Set<Long> |
等待的ArgoHolo事务的ID | [], |
| 57 | block ArgoHolo TxnIds | Set<Long> |
阻塞的ArgoHolo事务的ID | [], |
| 58 | topOne TaskMax Duration MappingId | Object | 最长执行时间的Task的映射ID | {"id":122103,"maxDuration":365,"parentId":211145,"executorAddr":"kv4:46219"}, |
| 59 | topTwo TaskMax Duration MappingId | Object | 第二长执行时间的Task的映射ID | {"id":122105,"maxDuration":364,"parentId":211145,"executorAddr":"kv4:46219"}, |
| 60 | tagFinishTime | long | 根据标签主动标记SQL结束时间(对LvtMsgLost标记消息丢失SQL检测使用) | 0, |
| 61 | perfLogs | List<String> |
接收的quark perflog信息, 数组成员格式是method,startTime,duration (method是perflog里的方法名, startTime是这条perflog的时间, duration是对应perflog记录的耗时) |
["parse,1789889543352,1","getMeta,1789889543403,46","genResolvedTree,1789889543402,48","genPlan,1789889543451,4","RBO,1789889543456,182","doAuthorization,1789889543641,1","compile,1789889543352,290","TimeToSubmit,1789889543352,311","PreHook.GuardianPLFunctionHook,1789889543663,1","PreHook.DisallowTransformHook,1789889543664,0","PreHook.DisallowTransformHook,1789889543664,0","PreHook.DisallowTransformHook,1789889543664,0","task.MAPRED-SPARK.Stage-0,1789889543665,838","runTasks,1789889543665,838","PostHook.CatalogHook,1789889544503,1","Driver.execute,1789889543663,841","Driver.run,1789889543352,1195"], |
| 62 | nullSkewFlag | boolean | skewjoin倾斜诊断标记是否出现null的倾斜 | false, |
| 63 | sessionHandle | String | 会话的唯一标识符, 对应sessionUUID | , |
| 64 | microJobName | String | 增量计算的 microjob名称 | , |
| 65 | microJobIncrementRows | Map<String, Long> |
本次增量 sql 的触发源表及更新数据规模 (key是源表名, value是更新数据行数) | {}, |
| 66 | microJobTargetTable | String | 增量计算insert 的目标表表 | , |
| 67 | microJobOutput | long | 增量计算sql 影响行数 | 0 |
2. job
| 序号 | Job字段名称 | Job数据类型 | Job字段中文解释 | 样例 |
|---|---|---|---|---|
| 1 | serverKey | String | 监控server的唯一标识，用于识别服务器实例 | Inceptor::kv1::d8da21f4-7912-463f-be5d-b17f5a93c5fa, |
| 2 | jobId | long | Job的唯一标识或ID | 240686, |
| 3 | SqlId | long | Sql查询的唯一标识或ID，表示此Job属于哪个查询 | 961626, |
| 4 | description | String | Job的描述或概要信息 | SQL Job 240686 -> / === Linac === / SELECT * FROM emp_holo t1 JOIN emp_holo t2 ON t1.empno =t2.empno apply at HierachicalPerf.scala:230, |
| 5 | message | String | Job的报错信息 | , |
| 6 | poolName | String | Stage的执行状态，可能包括成功、失败等 | TTTT, |
| 7 | jobGroup | String | Job所属的组别 | hive_20260920153223_f5c00d51-bb02-4624-b67f-f1ba2ae982ab, |
| 8 | stageInfosExpected | List<StageInfo> |
期望的StageInfo (包括jobId, stageId,attemptId, numTasks, sources[对应stargate context]) | [{"jobId":240686,"stageId":211144,"attemptId":0,"numTasks":6,"sources":"============== Stargate Context ==============\nRDD: io.transwarp.windrunner.linac.ArgoPerformanceLinacPlanShiva2RDD[1563927]\n Dialect: ORACLE\n Schema: Table default.emp_holo (alias[t2]), Columns [EMPNO,ENAME,JOB,MGR,HIREDATE,SAL,COMM,DEPTNO,HOLODESKBLOCKID,HOLODESKRECORDID,HOLODESKROWID,SKEWROWID,CHANGELOG_OP_TYPE]\n Stargate Filter: (EMPNO[Int] not null[0.0][Equivalent Pushdown: true])\n Stargate Filter All Push Down: true\n Needed Column: EMPNO,ENAME,JOB,MGR,HIREDATE,SAL,COMM,DEPTNO\n Order By Pushed Down: false\n Stargate Dimensions Pushed Down: false\n \n Stargate Limit/Order By: \n Local Parallelism: -1\n==============================================\n"}], |
| 9 | stageSources | Map<Long, String> |
key: stageId, value: stage的sources字段，stargate context内容，主要包括谓词下推、表扫描等信息。 | {"211144":"============== Stargate Context ==============\nRDD: io.transwarp.windrunner.linac.ArgoPerformanceLinacPlanShiva2RDD[1563927]\n Dialect: ORACLE\n Schema: Table default.emp_holo (alias[t2]), Columns [EMPNO,ENAME,JOB,MGR,HIREDATE,SAL,COMM,DEPTNO,HOLODESKBLOCKID,HOLODESKRECORDID,HOLODESKROWID,SKEWROWID,CHANGELOG_OP_TYPE]\n Stargate Filter: (EMPNO[Int] not null[0.0][Equivalent Pushdown: true])\n Stargate Filter All Push Down: true\n Needed Column: EMPNO,ENAME,JOB,MGR,HIREDATE,SAL,COMM,DEPTNO\n Order By Pushed Down: false\n Stargate Dimensions Pushed Down: false\n \n Stargate Limit/Order By: \n Local Parallelism: -1\n==============================================\n"}, |
| 10 | nodes | List<Vertex> |
Job的DAG的点的列表 (包括id,desc,detail,group,status) | [{"id":"stage_211144","desc":"Stage 211144","detail":null,"group":null,"status":"SUCCESS"},{"id":"1335947","desc":"PerformancePlan","detail":null,"group":"stage_211144","status":null},{"id":"1335951","desc":"Local Sink","detail":null,"group":"stage_211144","status":null},{"id":"1335949","desc":"FilterPlan","detail":null,"group":"stage_211144","status":null}], |
| 11 | edges | List<Edge> |
Job的DAG的边的列表(f, t) | [{"f":"1335947","t":"1335949"},{"f":"1335949","t":"1335951"}], |
| 12 | submissionTime | long | job提交的时间戳 | 1789889543784, |
| 13 | updateTime | long | 接收到DAG相关消息的时间戳 | 1789889543785, |
| 14 | stagesExpected | long | 预期的Stage数 | 1, |
| 15 | stagesCompleted | long | 完成的Stage数 | 1, |
| 16 | stagesFailed | long | 失败的Stage数 | 0, |
| 17 | stagesSkipped | long | 跳过的Stage数 | 0, |
| 18 | tasksExpected | long | 预期的Task数 | 6, |
| 19 | tasksCompleted | long | 完成的Task数 | 6, |
| 20 | tasksFailed | long | 失败的Task数 | 0, |
| 21 | tasksSkipped | long | 跳过的Task数 | 0, |
| 22 | totalTaskTime | long | Job的所有Task总执行时间(task时间的累加和) | 846, |
| 23 | user | String | 执行所属用户 | unknown, |
| 24 | sqlGroup | String | 所属SQL的组 | inceptor, |
| 25 | sqlTimeRange | String | 所属SQL的执行时间范围(sqlSubmissionTime::sqlCompletionTime) | 1789889543351::1789889544783, |
| 26 | status | String | Job执行状态 | SUCCESS, |
| 27 | completionTime | long | Job完成的时间戳 | 1789889543936, |
| 28 | linacOperatorGraph | LinacOperatorGraph | Linac的算子的执行计划图(类似sql执行计划图,linac的在job维度) | {"nodes":[],"edges":[],"stageEdges":[],"stageGroups":[{"stageId":211144,"jobId":240686,"submissionTime":1789889543785,"completionTime":1789889543936,"duration":151,"partitionSize":6,"status":"SUCCESS","opIds":["FilterOperator-3084011","TableScanOperator-3084012"],"stageCode":"Q09NTU9OK0luY2VwdG9yOjprdjE6OmQ4ZGEyMWY0LTc5MTItNDYzZi1iZTVkLWIxN2Y1YTkzYzVmYSsyMTExNDQrMTc4OTg4OTU0Mzc4NSsxNzg5ODg5NTQzOTM2","highlight":false}]}, |
| 29 | sessionId | long | 所属session的ID | 52470 |
3. stage
| 序号 | stage字段名称 | stage数据类型 | stage字段中文解释 | 样例 |
|---|---|---|---|---|
| 1 | serverKey | String | 监控server的唯一标识，用于识别服务器实例 | Inceptor::kv1::d8da21f4-7912-463f-be5d-b17f5a93c5fa, |
| 2 | stageId | long | Stage的唯一标识或ID，用于区分不同的Stage | 211144, |
| 3 | jobId | long | Job的唯一标识或ID，表示此Stage属于哪个Job | 240686, |
| 4 | SqlId | long | Sql查询的唯一标识或ID，表示此Stage属于哪个查询 | 961626, |
| 5 | attemptId | int | Stage的尝试标识，通常用于区分重试的Stage | 0, |
| 6 | description | String | Stage的描述或概要信息 | Stage 211144, |
| 7 | message | String | Stage的报错信息和说明 | , |
| 8 | status | String | Stage的执行状态，包括运行中、成功、失败等 | SUCCESS, |
| 9 | tasksExpected | long | 预期的Task数 | 6, |
| 10 | tasksCompleted | long | 完成的Task数 | 6, |
| 11 | tasksFailed | long | 失败的Task数 | 0, |
| 12 | tasksSkipped | long | 跳过的Task数 | 0, |
| 13 | partitionSize | int | 所有尝试中预期的最大Task数 | 6, |
| 14 | inputRowsString | String | DAG中每个operator的输入数据量（JSON） | , |
| 15 | outputRowsString | String | DAG中每个operator的输出数据量（JSON） | {"FilterOperator-3084011" : 0, "TableScanOperator-3084012" : 0, "HolodeskGLUMJoinOperator-3084014" : 0}, |
| 16 | shuffleReadBytes | long | Shuffle读取的字节数 | 0, |
| 17 | shuffleWriteBytes | long | Shuffle写入的字节数 | 0, |
| 18 | actual TasksCompleted | int | 实际完成的Task数 (旧数据兼容保留) | 0, |
| 19 | actual TasksFailed | int | 实际失败的Task数 (旧数据兼容保留) | 0, |
| 20 | submissionTime | long | Stage提交的时间戳 | 1789889543785, |
| 21 | stageMode | String | Stage的执行模式 | linac-cluster, |
| 22 | poolName | String | Stage所属的队列资源池名称 | TTTT, |
| 23 | totalTaskTime | long | Stage的所有Task的总执行时间 | 846, |
| 24 | input | long | Stage的输入数据量(行数) | 0, |
| 25 | shuffleRead BytesAggregated | long | 聚合后的Shuffle读取字节数 | 0, |
| 26 | shuffleWrite BytesAggregated | long | 聚合后的Shuffle写入字节数 | 0, |
| 27 | attempts | Map<Integer,Stage Attempt> |
Stage的尝试次数及重试信息 | {"0":{"stageId":211144,"attemptId":0,"message":null,"submissionTime":1789889543785,"completionTime":1789889543936}}, |
| 28 | user | String | 执行Stage的用户 | unknown, |
| 29 | SqlGroup | String | Sql查询所属的组别或分类 | inceptor, |
| 30 | sources | String | Stage(调度阶段)源, stargate context内容 | "============== Stargate Context ============== RDD: io.transwarp.windrunner.linac.ArgoPerformanceLinacPlanShiva2RDD[1563927] Dialect: ORACLE Schema: Table default.emp_holo (alias[t2]), Columns [EMPNO,ENAME,JOB,MGR,HIREDATE,SAL,COMM,DEPTNO,HOLODESKBLOCKID,HOLODESKRECORDID,HOLODESKROWID,SKEWROWID,CHANGELOG_OP_TYPE] Stargate Filter: (EMPNO[Int] not null[0.0][Equivalent Pushdown: true]) Stargate Filter All Push Down: true Needed Column: EMPNO,ENAME,JOB,MGR,HIREDATE,SAL,COMM,DEPTNO Order By Pushed Down: false Stargate Dimensions Pushed Down: false Stargate Limit/Order By: |
| 31 | sqlTimeRange | String | 所属SQL的执行时间范围(sqlSubmissionTime::sqlCompletionTime) | 1789889543351::1789889544783, |
| 32 | JobTimeRange | String | 所属Job的执行时间范围(jobSubmissionTime::jobCompletionTime) | 1789889543784::1789889543936, |
| 33 | completionTime | long | Stage完成的时间戳 | 1789889543936, |
| 34 | scanOpNum | int | tablescan扫描算子的数量 | 1, |
| 35 | scanRows | long | tablescan扫描的行数 | 0, |
| 36 | numRemoteReadFilesTotalNum | long | 远程读文件数 | 0, |
| 37 | numLocalReadFilesTotalNum | long | 本地读文件数 | 6, |
| 38 | numRemoteReadSameDCFilesTotalNum | long | 远程读tddms时同dc的文件数量 | 0, |
| 39 | numRemoteReadDiffDCFilesTotalNum | long | 远程读tddms时不同dc的文件数 | 0, |
| 40 | stageNullSkewFlag | boolean | skewjoin倾斜诊断标记Stage是否出现null的倾斜 | false, |
| 41 | parentStageIds | String | 父亲stageId列表 | , |
| 42 | linacOperatorJson | String | linac算子的执行计划树json |
4. task
| 序号 | task字段名称 | task数据类型 | task字段中文解释 | 样例 |
|---|---|---|---|---|
| 1 | serverKey | String | 监控server的唯一标识，用于识别服务器实例。 | Inceptor::kv1::d8da21f4-7912-463f-be5d-b17f5a93c5fa, |
| 2 | taskId | long | Task的唯一标识或ID，用于区分不同的Task。 | 122097, |
| 3 | stageId | long | Task所属的阶段的唯一标识或ID。 | 211144, |
| 4 | taskIndex InStage | int | Task在阶段中的索引位置。 | 1, |
| 5 | attempt | int | Task的尝试次数，通常用于区分Task的重试。 | 0, |
| 6 | executorId | String | 执行Task的执行者（Executor）的ID唯一标识。 | 40, |
| 7 | taskLocality | String | Task执行的数据本地性信息，如本地、节点本地、远程等。 | CACHE_LOCAL, |
| 8 | message | String | Task的报错信息和说明 | , |
| 9 | shuffleReadBytes | long | 从Shuffle中读取的字节数。 | 0, |
| 10 | shuffleWriteBytes | long | 写入Shuffle的字节数。 | 0, |
| 11 | input | long | Task的输入数据量(行数)。 | 0, |
| 12 | garbage CollectionTime | long | 垃圾回收时间。 | 0, |
| 13 | result SerializationTime | long | 结果序列化时间。 | 0, |
| 14 | timeSpent Fetching TaskResults | long | 获取Task结果所花费的时间。 | 0, |
| 15 | shuffle SpillMemory | long | Shuffle写入到内存中的字节数。 | 0, |
| 16 | shuffleSpillDisk | long | Shuffle写入到磁盘中的字节数。 | 0, |
| 17 | writeTime | long | 写入操作所花费的时间。 | 0, |
| 18 | aggregation | String | 聚合操作相关信息。 | , |
| 19 | sequence | String | Task执行的顺序。 | , |
| 20 | schedulerDelay | long | Task执行的调度延迟。 | 27, |
| 21 | accumulators | Map <String,String> |
Task执行时的累加器信息。 | {}, |
| 22 | durations | Map <String,Long> |
Task执行时间的各个阶段的持续时间。 | {}, |
| 23 | metrics | Map <String,String> |
与Task执行相关的性能指标。 | {"GC_TIME":"0","RESULT_SERIALIZATION_TIME":"0","SHUFFLE_WRITE_BYTE":"0","FETCH_RESULT_TIME":"0","numRemoteReadSameDCFiles":"0","DISK_BYTES_SPILLED":"0","SCHEDULER_DELAY":"27","numRemoteReadDiffDCFiles":"0","numRemoteReadFiles":"0","SHUFFLE_WRITE_TIME":"0.0","numLocalReadFiles":"1","MEMORY_BYTES_SPILLED":"0"}, |
| 24 | submissionTime | long | Task提交的时间戳。 | 1789889543794, |
| 25 | host | String | 执行Task的executor的主机名。 | kv3, |
| 26 | port | int | 执行Task的executor的端口号。 | 36984, |
| 27 | status | String | Task的执行状态，包括运行中、成功、失败等。 | SUCCESS, |
| 28 | completionTime | long | Task完成的时间戳。 | 1789889543936 |
5. session
| 序号 | session字段名称 | session数据类型 | session字段中文解释 | 样例 |
|---|---|---|---|---|
| 1 | serverKey | long | 监控server的唯一标识，用于识别服务器实例。 | Inceptor::kv1::d8da21f4-7912-463f-be5d-b17f5a93c5fa, |
| 2 | sessionId | long | 会话的唯一标识或ID，表示会话的编号。 | 52477, |
| 3 | sessionUUID | String | 会话的唯一标识符，通常以UUID的形式表示。 | ad0fa05c-0766-4f5c-92a5-705920994b66, |
| 4 | sessionName | String | 会话的名称或标签。 | , |
| 5 | user | String | 执行查询的用户，即谁在会话中运行查询。 | hive, |
| 6 | ipAddress | String | 与会话关联的IP地址，即会话的来源客户端IP。 | 172.18.131.174, |
| 7 | openTime | long | 会话开启的时间戳，表示会话的开始时间。 | 1789889650298, |
| 8 | closeTime | long | 会话关闭的时间戳，表示会话的结束时间。 | 1789889829513, |
| 9 | message | String | 与会话相关的报错消息或说明。 | Closed by user. |
6. executor
| 序号 | executor字段名称 | executor数据类型 | executor字段中文解释 | 样例 |
|---|---|---|---|---|
| 1 | serverKey | String | 监控server的唯一标识，用于识别服务器实例。 | Inceptor::kv1::d8da21f4-7912-463f-be5d-b17f5a93c5fa, |
| 2 | executorId | String | Executor的唯一标识或ID，表示集群中的一个Executor。 | 42, |
| 3 | host | String | Executor所在的主机名或IP地址。 | kv2, |
| 4 | port | int | Executor的通信端口号，用于与集群通信。 | 44285, |
| 5 | totalCores | int | Executor可用的总核心数，表示Executor的计算资源。 | 4, |
| 6 | maxMemory | long | Executor可用的最大内存量，表示Executor的内存资源。 | 8418164736, |
| 7 | logLevel | String | Executor的日志级别，表示Executor的日志详细程度。 | INFO, |
| 8 | dirStatus | String | 目录状态，可能与Executor的目录信息有关。 | [{"used" : 0, "total" : 209131559239, "writable" : true, "path" : "\/vdir\/mnt\/disk1\/hadoop\/ngmr\/quark1\/spark-local-20260919220055-f910"}, {"used" : 0, "total" : 209131559239, "writable" : true, "path" : "\/vdir\/mnt\/disk2\/hadoop\/ngmr\/quark1\/spark-local-20260919220055-08fe"}, {"used" : 0, "total" : 209131559239, "writable" : true, "path" : "\/vdir\/mnt\/disk3\/hadoop\/ngmr\/quark1\/spark-local-20260919220055-a816"}], |
| 9 | firstSeen | long | executor启动时首次发送的时间 | 1789826453546, |
| 10 | lastSeen | long | executor最近一次汇报的时间 | 1789901694880, |
| 11 | firstSeenLocal | long | 收到executor启动时首次发送的本地时间 | 1789826453721, |
| 12 | lastSeenLocal | long | 收到executor最近一次汇报的本地时间 | 1789901695062 |