展开查看详情
1 . JIT in GP7
--提升GP7查询性能的秘密武器
汤韬
GPDB内核开发工程师
© 2023 Greenplum Database, a VMware sponsored open source project
2 .JIT概况
• 功能背景
o GPDB和上游Postgres关系密切，GPDB7在merge Postgres12的过程中新
引⼊了JIT特性，并针对MPP架构进⾏了适配和增强
• 加速概况
© 2023 Greenplum Database, a VMware sponsored open source project
3 .JIT概念与GPDB
• 即时编译（Just-In-Time）
o 对接收的动态指令序列快速转成native code后运⾏的动态编译执⾏⽅法
o 兼顾对动态指令的适应性，以及编译执⾏的性能优势
o 适合短时⾼频使⽤的指令⽚段: 单次耗时的编译 + 多次更快的运⾏
• GPDB JIT
o 查询指令的不确定性，天然决定数据库的解释器性质
o ⽕⼭模型，逐元组的重复调⽤频繁
o 表达式计算（filter， projection，agg…)
o 元组分解 (tuple deforming) – 存储形式转换为计算使⽤的内存形式
© 2023 Greenplum Database, a VMware sponsored open source project
4 .JIT加速原理与GPDB的实现
• 相关的计算机执行性能的影响因素
o CPU流水线结构与分支预测
o 现代CPU通过指令流⽔线提升性能，遇到分⽀需预测加载后续指令，错误预测将
导致流⽔线失效重载。过多分⽀会产⽣⼤量预测失败，显著降低CPU执⾏效率
o 函数调用的上下文维护开销
o 函数调⽤会引⼊参数传递、返回地址处理、寄存器恢复等⼀系列上下⽂维护操作。
过多⼩函数的调⽤
© 2023 Greenplum Database, a VMware sponsored open source project
5 .JIT加速原理与GPDB的实现
• JIT提升性能的路线1 – 消除分⽀
o 利⽤具体查询提供的额外确定化信息，重构专⽤函数时消减分⽀
o 表达式计算 – 算⼦识别的逻辑，提前到⽬标表达式函数⽣成过程中固化消除
o 元组分解 – 列数、align，attlen等属性得到确认，相应的判别分⽀可消除
© 2023 Greenplum Database, a VMware sponsored open source project
6 .JIT加速原理与GPDB的实现
• JIT提升性能的路线2 – 编译优化
o 利⽤inline减少函数调⽤开销
o 启⽤O3优化提升产出native code性能
© 2023 Greenplum Database, a VMware sponsored open source project
7 .JIT加速原理与GPDB的实现
• JIT的分区表性能恶化问题与解决
o 分区表实际是按⼦表⽣成执⾏节点，表达式数量随分区数成⽐例增⻓
o 解决思路 – ⼦表间尽可能复⽤表达式的函数：
o JIT Cache for ORCA plan
© 2023 Greenplum Database, a VMware sponsored open source project
8 .JIT在GP7中的使⽤
• Workflow
o Plan阶段
o 基于cost确定JIT各功能项启⽤情况
o Executor Init阶段
o 重构专⽤⽬标函数（源码）
o Executor Run阶段
o ⾸次调⽤⽬标函数时完成编译再使⽤
© 2023 Greenplum Database, a VMware sponsored open source project
9 .JIT在GP7中的使⽤
• 常用JIT GUCs
o jit （默认关，使⽤JIT需开启）
o jit_above_cost *
o jit_expressions
o jit_tuple_deforming
o jit_optimize_above_cost *
o jit_inline_above_cost *
* ORCA时使⽤前缀optimizer_的同类参数
© 2023 Greenplum Database, a VMware sponsored open source project
10 .JIT在GP7中的使⽤
• Explain Analyze
JIT:
Options: Inlining false, Optimization false, Expressions true, Deforming true.
(slice0): Functions: 14.00. Timing: 8.045 ms total.
(slice1): Functions: 12.00 avg x 3 workers, 12.00 max (seg0). Timing: 7.219 ms avg x 3 workers, 7.399 ms max (seg2).
(slice2): Functions: 1.00 avg x 3 workers, 1.00 max (seg0). Timing: 1.219 ms avg x 3 workers, 1.305 ms max (seg0).
Execution Time: 977.086 ms
(26 rows)
© 2023 Greenplum Database, a VMware sponsored open source project
11 .JIT在GP7中的使⽤
• Explain Analyze Verbose
JIT:
Options: Inlining false, Optimization false, Expressions true, Deforming true.
(slice0): Functions: 14.00. Timing: 8.926 ms total.
seg-1: Functions 14, Generation 0.896 ms, Inlining 0.000 ms, Optimization 0.422 ms, Emission 7.607 ms, Total 8.926 ms.
(slice1): Functions: 12.00 avg x 3 workers, 12.00 max (seg0). Timing: 8.192 ms avg x 3 workers, 8.293 ms max (seg0).
seg0: Functions 12, Generation 1.022 ms, Inlining 0.000 ms, Optimization 0.359 ms, Emission 6.912 ms, Total 8.293 ms.
seg1: Functions 12, Generation 0.901 ms, Inlining 0.000 ms, Optimization 0.349 ms, Emission 6.840 ms, Total 8.091 ms
seg2: Functions 12, Generation 0.889 ms, Inlining 0.000 ms, Optimization 0.386 ms, Emission 6.917 ms, Total 8.191 ms.
(slice2): Functions: 1.00 avg x 3 workers, 1.00 max (seg0). Timing: 2.084 ms avg x 3 workers, 2.581 ms max (seg2).
seg0: Functions 1, Generation 0.069 ms, Inlining 0.000 ms, Optimization 0.108 ms, Emission 1.679 ms, Total 1.855 ms.
seg1: Functions 1, Generation 0.063 ms, Inlining 0.000 ms, Optimization 0.087 ms, Emission 1.665 ms, Total 1.815 ms.
seg2: Functions 1, Generation 0.063 ms, Inlining 0.000 ms, Optimization 0.115 ms, Emission 2.403 ms, Total 2.581 ms.
Execution Time: 1011.769 ms
(43 rows)
© 2023 Greenplum Database, a VMware sponsored open source project
12 . Thanks!
© 2023 Greenplum Database, a VMware sponsored open source project
热门城市
