{"id":17531,"date":"2025-10-30T10:36:24","date_gmt":"2025-10-30T02:36:24","guid":{"rendered":"https:\/\/kbwp.transwarp.cn\/?p=17531"},"modified":"2026-01-08T15:47:05","modified_gmt":"2026-01-08T07:47:05","slug":"tdt%e5%8a%a0%e8%bd%bd%e4%bb%bb%e5%8a%a1%e5%a6%82%e4%bd%95%e6%9f%a5%e7%9c%8bexecutor%e6%97%a5%e5%bf%97%e6%8a%a5%e9%94%99","status":"publish","type":"post","link":"https:\/\/kbwp.transwarp.cn\/?p=17531","title":{"rendered":"TDT\u52a0\u8f7d\u4efb\u52a1\u5982\u4f55\u67e5\u770b\u670d\u52a1\u5668\u65e5\u5fd7\u62a5\u9519"},"content":{"rendered":"<h3>\u6982\u8981\u63cf\u8ff0<\/h3>\n<hr \/>\n<p>\u5f53\u524d\u7248\u672ctransporter\uff0c\u6682\u4e0d\u652f\u6301\u4ecetds\u9875\u9762\u76f4\u63a5\u5224\u65ad\u52a0\u8f7d\u4efb\u52a1\u5931\u8d25\u7684\u539f\u56e0\uff0c\u9700\u8981\u5230\u540e\u53f0\u670d\u52a1\u5668\u4e0a\u67e5\u770b\/var\/log\/quarkX\/quark-server.log<em> \u6216 \/var\/log\/quarkX\/quark-executor.log<\/em>\u65e5\u5fd7\uff0c\u672c\u6587\u4e3e\u4f8b\u8bf4\u660e\u65e5\u5fd7\u7684\u67e5\u770b\u65b9\u6cd5\u3002<\/p>\n<h3>\u8be6\u7ec6\u8bf4\u660e<\/h3>\n<hr \/>\n<h4>\u4e00. \u62a5\u9519\u5728quark executor\u4e0a<\/h4>\n<p>*<em>\u4ee5\u5982\u4e0b\u52a0\u8f7d\u4efb\u52a1\u6267\u884c\u62a5\u9519\u4e3a\u4f8b\uff0c\u5177\u4f53\u6307\u5411\u4e86\u67d0\u4e00\u4e2aquark executor\u8282\u70b9\uff0c\u5219\u8868\u660e\u62a5\u9519\u5728quark executor\u4e0a\uff0c\u8981\u5230\u5bf9\u5e94\u7684quark executor\u8282\u70b9\u540e\u53f0\u67e5\u770b \/var\/log\/quarkX\/quark-executor.log<\/em> \u65e5\u5fd7\uff0c**<\/p>\n<div style=\"box-shadow: 1px 1px 10px rgba(0,0,0,0.1); padding: 1px; display: inline-block; width: auto; margin-bottom: 10px;\">\n  <img decoding=\"async\" src=\"\/wp-content\/uploads\/2025\/10\/image-1763125975302.png\" style=\"display: block;\">\n<\/div>\n<p>\u62a5\u9519\u4fe1\u606f\u4f1a\u63d0\u793a <code>Caused by: java.sql.SQLExeception: EXECUTION FAILED: Task MAPRED-SPARK error SparkException: [Error 1] job aborted due to stage failure: Task 4 in stage 201946.0 failed 4 times, most recent failure: Lost task 4.3 in stage 201946.0(TID 6244229, 10.15.15.26, jobId 129073, sqlId 278135, sessionId 49153): org.apache.hadoop.hive.ql.metadata.HiveException: Got SQLException<\/code><\/p>\n<p><strong>\u6216\u8005\u8fd9\u79cd\uff1a<\/strong><\/p>\n<div style=\"box-shadow: 1px 1px 10px rgba(0,0,0,0.1); padding: 1px; display: inline-block; width: auto; margin-bottom: 10px;\">\n  <img decoding=\"async\" src=\"\/wp-content\/uploads\/2025\/10\/image-1761790758056.png\" style=\"display: block;\">\n<\/div>\n<p>\u62a5\u9519\u4fe1\u606f\u4f1a\u63d0\u793a <code>Caused by: java.lang.RuntimeException: java.sql.SQLException: EXECUTION FAILED: Task MAPRED-SPARK error SparkException: [Error 1] Job aborted due to stage failure: Task 4 in stage 24351.0 failed 4 times, most recent failure: Lost task 4.3 in stage 24351.0 (TID 959170, 10.15.15.29, jobId 11152, sqlId 18734, sessionId 3492): java.lang.RuntimeException: Exception found during closeOp. See more detail on executor log.<\/code><\/p>\n<p>\u4e0a\u8ff02\u79cd\u60c5\u51b5\u4e0b\uff0c\u65e5\u5fd7\u63d0\u793a\u90fd\u4e5f\u5f88\u660e\u786e\uff0c\u9700\u8981\u5230\u5177\u4f53\u7684executor\u8282\u70b9\u8282\u70b9\uff0c\u641c\u7d22TID\u6765\u67e5\u770b\u5177\u4f53\u62a5\u9519\u3002<\/p>\n<p>\u4ee5\u4e0a\u9762\u7684\u62a5\u9519\u4e3a\u4f8b\uff1a\u9996\u5148\u6211\u4eec\u5148\u786e\u8ba4 \u52a0\u8f7d\u4efb\u52a1\u662f\u7531\u54ea\u4e2aquark\u6267\u884c\u5f15\u64ce\u8fd0\u884c\u7684\uff0c\u627e\u5230\u6e90\u7aef\u914d\u7f6e\uff0c\u6570\u636e\u6e90\u5bf9\u5e94 quark6\u6570\u636e\u5e93\uff0c\uff08\u8fd9\u4e2a\u6570\u636e\u6e90\u7684\u8fde\u63a5\u914d\u7f6e\uff0c\u51b3\u5b9aexecutor\u8282\u70b9\u7684\u65e5\u5fd7\u76ee\u5f55\uff0c\u4e5f\u5c31\u662f\u540e\u9762\u7684 <code>\/var\/log\/quark6<\/code>\uff09\uff1b<\/p>\n<blockquote>\n<p>\u5982\u679c\u662f\u5c06\u6570\u636e\u52a0\u8f7d\u5230quark\uff0c\u5230\u76ee\u6807\u7aef\u67e5\u770b\u662f\u54ea\u4e2aquark<\/p>\n<\/blockquote>\n<div style=\"box-shadow: 1px 1px 10px rgba(0,0,0,0.1); padding: 1px; display: inline-block; width: auto; margin-bottom: 10px;\">\n  <img decoding=\"async\" src=\"\/wp-content\/uploads\/2025\/10\/image-1761790654818.png\" style=\"display: block;\">\n<\/div>\n<p>\u767b\u5f55\u5230 <code>10.15.15.29<\/code>\u670d\u52a1\u5668\uff0c\u8fdb\u5165\u5230 <code>\/var\/log\/quark6<\/code> \u76ee\u5f55\u4e0b\uff0c<code>ls -lrth quark-executor.log*<\/code>\uff0c\u627e\u5230\u5bf9\u5e94\u65f6\u95f4\u70b9\u7684\u65e5\u5fd7\u6587\u4ef6\uff0cvi\u6a21\u5f0f\u8fdb\u53bb\u641c\u7d22 <code>959170<\/code> \u8fd9\u4e2aTID\uff0c\u5f80\u524d\u7ffb\u4e00\u7ffb\u5373\u53ef\u770b\u5230\u5177\u4f53\u62a5\u9519\u4fe1\u606f\uff1a<\/p>\n<p><img decoding=\"async\" src=\"\/wp-content\/uploads\/2025\/10\/image-1761791014953.png\" alt=\"file\" \/><\/p>\n<h4>\u4e8c. \u62a5\u9519\u5728quark server\u4e0a<\/h4>\n<p>\u5982\u679c\u52a0\u8f7d\u65e5\u5fd7\u663e\u793a<code>Caused by: java.sql.SQLException: EXECUTION FAILED: Task MAPRED-SPARK error IOException: [Error 1] Got SQLException<\/code>\uff0c\u6ca1\u6709\u5177\u4f53\u6307\u5411\u67d0\u4e00\u4e2aquark executor\u8282\u70b9\uff0c\u5219\u8868\u660e\u62a5\u9519\u5728quark server\u4e0a\uff0c\u8981\u5230\u5bf9\u5e94\u7684quark server\u8282\u70b9\u540e\u53f0\u67e5\u770b \/var\/log\/quarkX\/quark-server.log* \u65e5\u5fd7\uff0c\u6bd4\u5982\u4e0b\u9762\u7684\u62a5\u9519<\/p>\n<div style=\"box-shadow: 1px 1px 10px rgba(0,0,0,0.1); padding: 1px; display: inline-block; width: auto; margin-bottom: 10px;\">\n  <img decoding=\"async\" src=\"\/wp-content\/uploads\/2025\/10\/image-1762844237648.png\" style=\"display: block;\">\n<\/div>\n<h3>FAQ<\/h3>\n<hr \/>\n<h4>1. \u5378\u6570\u5230mysql\u65f6\u9047\u5230Duplicate entry\u952e\u503c\u51b2\u7a81\u7684\u62a5\u9519 \u6709\u6240\u4e0d\u540c<\/h4>\n<blockquote>\n<p>\u7c7b\u4f3c\u7684\uff0c\u5378\u6570\u5230oracle\u65f6\uff0c\u62a5\u9519 ORA-00001: unique constraint \u4e5f\u8981\u7528\u8fd9\u79cd\u6392\u67e5\u903b\u8f91\u3002<\/p>\n<\/blockquote>\n<p>\u9700\u8981\u5230DBAService\u9875\u9762\uff0cstage\u9875\u9762\u627e\u5230\u6267\u884c\u5931\u8d25\u7684task\uff0c\u5230 <strong>\u5c1d\u8bd5<\/strong> \u4e3a0\u7684executor\u8282\u70b9\u53bb\u67e5\u770bquark-executor.log\u3002 \u56e0\u4e3a\u5728quark\u4e2dtask\u5931\u8d25\u4f1a\u9ed8\u8ba4\u91cd\u8bd53\u6b21\uff0c\u53ea\u6709\u7b2c\u4e00\u6b21\u7684\u65e5\u5fd7\u624d\u4f1a\u663e\u793a\u5177\u4f53\u91cd\u590d\u7684\u952e\u503c\uff0c\u540e\u7eed\u7b2c\u4e8c\/\u4e09\/\u56db\u6b21\u662f\u7531\u4e8etask\u91cd\u8bd5\u673a\u5236\u5bfc\u81f4\u7684\uff0c\u5e76\u975e\u662f\u91cd\u590dkey\u3002\u9047\u5230\u8fd9\u79cd\u62a5\u9519\u4e5f\u53ef\u4ee5\u76f4\u63a5\u5230\u6e90\u7aef\u6570\u636e\u5e93\uff0c\u5bf9\u6620\u5c04\u5230\u76ee\u6807\u7aef\u552f\u4e00\u952e\u7684\u5217\u505a\u805a\u5408\u67e5\u8be2\uff0c\u540c\u6837\u4e5f\u53ef\u4ee5\u627e\u5230\u91cd\u590d\u7684\u952e\u503c\u3002<\/p>\n<div style=\"box-shadow: 1px 1px 10px rgba(0,0,0,0.1); padding: 1px; display: inline-block; width: auto; margin-bottom: 10px;\">\n  <img decoding=\"async\" src=\"\/wp-content\/uploads\/2025\/10\/image-1761791204664.png\" style=\"display: block;\">\n<\/div>\n<h4>2. WARN\u7ea7\u522b\u65e5\u5fd7\u65e0\u6cd5\u663e\u793a\u5177\u4f53\u4fe1\u606f\uff0c\u9700\u8981\u8c03\u6574\u6210INFO\u7ea7\u522b\uff1f<\/h4>\n<p>\u53c2\u8003 <strong><a href=\"https:\/\/kb.transwarp.cn\/posts\/12306\" title=\"\u4fee\u6539quark\u65e5\u5fd7\u7ea7\u522b\u7684\u51e0\u79cd\u65b9\u5f0f - \u65b9\u6848\u4e8c\uff1aDBAService\u9875\u9762\u4fee\u6539\">\u4fee\u6539quark\u65e5\u5fd7\u7ea7\u522b\u7684\u51e0\u79cd\u65b9\u5f0f \u4e4b\u65b9\u6848\u4e8c\uff1aDBAService\u9875\u9762\u4fee\u6539<\/a><\/strong>\uff0c\u5c06quark\u5f15\u64ce\u7684\u65e5\u5fd7\u7ea7\u522b\u8c03\u6574\u4e3a <code>INFO<\/code>\u3002<\/p>\n<p>\u56e0\u4e3a\u65e5\u5fd7\u7ea7\u522b\u9ed8\u8ba4\u4e3a <code>WARN<\/code> \u7684\u60c5\u51b5\u4e0b\uff0c\u65e0\u6cd5\u6253\u5370executor\u65e5\u5fd7\u4e2d\u7684\u4e00\u4e9b\u5173\u952e\u4fe1\u606f\uff0c\u6bd4\u5982replace\u6a21\u5f0f\u5378\u6570\u4e0b\u7684sql\u8bed\u53e5\u7b49\u3002<\/p>\n<p>\u8c03\u8bd5\u5b8c\u6210\u4e4b\u540e\uff0c\u8bf7\u52a1\u5fc5\u4fee\u6539\u56deWARN\uff0c\u9632\u6b62\u5bf9\u65e5\u5e38\u4e1a\u52a1\u6027\u80fd\u9020\u6210\u5f71\u54cd\u3002<\/p>\n","protected":false},"excerpt":{"rendered":"<p>\u6982\u8981\u63cf\u8ff0 \u5f53\u524d\u7248\u672ctransporter\uff0c\u6682\u4e0d\u652f\u6301\u4ecetds\u9875\u9762\u76f4\u63a5\u5224\u65ad\u52a0\u8f7d\u4efb\u52a1\u5931\u8d25\u7684\u539f\u56e0\uff0c\u9700\u8981\u5230\u540e\u53f0\u670d\u52a1\u5668\u4e0a\u67e5 ..<\/p>\n<div class=\"clear-fix\"><\/div>\n<p><a href=\"https:\/\/kbwp.transwarp.cn\/?p=17531\" title=\"read more...\">Read more<\/a><\/p>\n","protected":false},"author":12,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[47],"tags":[],"class_list":["post-17531","post","type-post","status-publish","format-standard","hentry","category-third_part"],"acf":[],"_links":{"self":[{"href":"https:\/\/kbwp.transwarp.cn\/index.php?rest_route=\/wp\/v2\/posts\/17531","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/kbwp.transwarp.cn\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/kbwp.transwarp.cn\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/kbwp.transwarp.cn\/index.php?rest_route=\/wp\/v2\/users\/12"}],"replies":[{"embeddable":true,"href":"https:\/\/kbwp.transwarp.cn\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=17531"}],"version-history":[{"count":3,"href":"https:\/\/kbwp.transwarp.cn\/index.php?rest_route=\/wp\/v2\/posts\/17531\/revisions"}],"predecessor-version":[{"id":17891,"href":"https:\/\/kbwp.transwarp.cn\/index.php?rest_route=\/wp\/v2\/posts\/17531\/revisions\/17891"}],"wp:attachment":[{"href":"https:\/\/kbwp.transwarp.cn\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=17531"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/kbwp.transwarp.cn\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=17531"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/kbwp.transwarp.cn\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=17531"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}