InfluxDB vs GreptimeDB query performance #8525
Replies: 1 comment · 2 replies
|
how about |
All reactions
|
It definitely feels subjectively faster, but the variance is high enough that I'm not sure if it actually helped or not. A 7d query is still taking ~1.5s. Here's a new stage,node,plan
0,0," SortPreservingMergeExec: [channel@1 ASC NULLS LAST, time@0 ASC NULLS LAST] metrics=[output_rows: 8076, elapsed_compute: 1.2ms, output_bytes: 373.0KiB, output_batches: 1, ]
SortExec: expr=[channel@1 ASC NULLS LAST, time@0 ASC NULLS LAST], preserve_partitioning=[true] metrics=[output_rows: 8076, elapsed_compute: 48us, output_bytes: 0B, output_batches: 0, spill_count: 0, spilled_bytes: 0B, spilled_rows: 0, ]
ProjectionExec: expr=[time@0 as time, channel@1 as channel, channel_name@3 as channel_name, average_current_amps@2 as average_current_amps] metrics=[output_rows: 8076, elapsed_compute: 182us, output_bytes: 1.6MiB, output_batches: 12, expr_0_eval_time: 22us, expr_1_eval_time: 18us, expr_2_eval_time: 20us, expr_3_eval_time: 20us, ]
HashJoinExec: mode=Partitioned, join_type=Left, on=[(channel@1, channel@0)], projection=[time@0, channel@1, average_current_amps@2, channel_name@4] metrics=[output_rows: 8076, elapsed_compute: 9.3ms, output_bytes: 1.6MiB, output_batches: 11, array_map_created_count: 0, build_input_batches: 11, build_input_rows: 8076, input_batches: 28, input_rows: 13, build_mem_used: 404.2KiB, build_time: 6.8ms, join_time: 1.8ms, avg_fanout: 0, probe_hit_rate: 0, ]
ProjectionExec: expr=[time@0 as time, channel@1 as channel, (total_charge_coulombs@3 - lag(t1.total_charge_coulombs) PARTITION BY [t1.channel] ORDER BY [t1.time ASC NULLS LAST] RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW@4) / (total_time_seconds@2 - lag(t1.total_time_seconds) PARTITION BY [t1.channel] ORDER BY [t1.time ASC NULLS LAST] RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW@5) as average_current_amps] metrics=[output_rows: 8076, elapsed_compute: 248us, output_bytes: 198.2KiB, output_batches: 11, expr_0_eval_time: 19us, expr_1_eval_time: 19us, expr_2_eval_time: 129us, ]
BoundedWindowAggExec: wdw=[lag(t1.total_charge_coulombs) PARTITION BY [t1.channel] ORDER BY [t1.time ASC NULLS LAST] RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW: Field { ""lag(t1.total_charge_coulombs) PARTITION BY [t1.channel] ORDER BY [t1.time ASC NULLS LAST] RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW"": nullable Float64 }, frame: RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW, lag(t1.total_time_seconds) PARTITION BY [t1.channel] ORDER BY [t1.time ASC NULLS LAST] RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW: Field { ""lag(t1.total_time_seconds) PARTITION BY [t1.channel] ORDER BY [t1.time ASC NULLS LAST] RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW"": nullable Float64 }, frame: RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW], mode=[Sorted] metrics=[output_rows: 8076, elapsed_compute: 3.3ms, output_bytes: 444.2KiB, output_batches: 11, ]
SortExec: expr=[channel@1 ASC NULLS LAST, time@0 ASC NULLS LAST], preserve_partitioning=[true] metrics=[output_rows: 8076, elapsed_compute: 60us, output_bytes: 0B, output_batches: 0, spill_count: 0, spilled_bytes: 0B, spilled_rows: 0, ]
RepartitionExec: partitioning=Hash([channel@1], 16), input_partitions=1 metrics=[output_rows: 8076, elapsed_compute: 461us, output_bytes: 2.1MiB, output_batches: 11, spill_count: 0, spilled_bytes: 0B, spilled_rows: 0, fetch_time: 1.18s, repartition_time: 304us, send_time: 45us, ]
ProjectionExec: expr=[greptime_timestamp@2 as time, channel@4 as channel, last_value(profinet.caparoc.total_time_seconds) RANGE 15m FILL NULL@1 as total_time_seconds, last_value(profinet.caparoc.total_charge_coulombs) RANGE 15m FILL NULL@0 as total_charge_coulombs] metrics=[output_rows: 8076, elapsed_compute: 15us, output_bytes: 262.1KiB, output_batches: 1, expr_0_eval_time: 2.4us, expr_1_eval_time: 1.4us, expr_2_eval_time: 978ns, expr_3_eval_time: 1.5us, ]
RangeSelectExec: range_expr=[last_value(profinet.caparoc.total_charge_coulombs) RANGE 900s FILL NULL, last_value(profinet.caparoc.total_time_seconds) RANGE 900s FILL NULL], align=900000ms, align_to=0ms, align_by=[name_of_station@2, channel@0], time_index=greptime_timestamp metrics=[output_rows: 0, elapsed_compute: 1.10s, output_bytes: 0B, output_batches: 0, ]
CoalescePartitionsExec metrics=[output_rows: 7085136, elapsed_compute: 23us, output_bytes: 1.4GiB, output_batches: 909, ]
MergeScanExec: peers=[4518305595392(1052, 0), ], metrics={""partition_0"":{""regions"":1,""total_poll_duration"":""121.543142ms"",""total_do_get_cost"":""8.248476ms"",""ready_time"":""8.273829ms"",""first_consume_time"":""31.444627ms"",""finish_time"":""1.174661419s"",""region_metrics"":[{""region_id"":""4518305595392(1052, 0)"",""poll_duration"":""121.543142ms"",""do_get_cost"":""8.248476ms"",""total_cost"":""1.174580893s""}]}} metrics=[output_rows: 7085136, greptime_exec_read_cost: 0ns, finish_time: 1.17s, first_consume_time: 32ms, ready_time: 8.3ms, ]
RepartitionExec: partitioning=Hash([channel@0], 16), input_partitions=16 metrics=[output_rows: 13, elapsed_compute: 375us, output_bytes: 496B, output_batches: 12, spill_count: 0, spilled_bytes: 0B, spilled_rows: 0, fetch_time: 69ms, repartition_time: 70us, send_time: 38us, ]
MergeScanExec: peers=[4518305595392(1052, 0), ], metrics={""partition_0"":{""regions"":1,""total_poll_duration"":""64.687305ms"",""total_do_get_cost"":""2.846501ms"",""ready_time"":""2.905517ms"",""first_consume_time"":""66.184622ms"",""finish_time"":""68.837874ms"",""region_metrics"":[{""region_id"":""4518305595392(1052, 0)"",""poll_duration"":""64.687305ms"",""do_get_cost"":""2.846501ms"",""total_cost"":""68.759233ms""}]}} metrics=[output_rows: 13, greptime_exec_read_cost: 0ns, finish_time: 69ms, first_consume_time: 66ms, ready_time: 3.0ms, ]
"
1,0," CooperativeExec metrics=[]
SeqScan: region=4518305595392(1052, 0), {""partition_count"":{""count"":241, ""mem_ranges"":0, ""files"":2, ""file_ranges"":241}, ""projection"": [""channel"", ""channel_name"", ""name_of_station"", ""average_current_amps"", ""greptime_timestamp"", ""average_power_watts"", ""current_over_80percent"", ""defect"", ""delta_time_seconds"", ""max_current_amps"", ""max_power_watts"", ""min_current_amps"", ""min_power_watts"", ""nominal_current_amps"", ""overload"", ""short_circuit"", ""status"", ""total_charge_coulombs"", ""total_energy_joules"", ""total_time_seconds"", ""average_voltage_volts"", ""max_voltage_volts"", ""min_voltage_volts"", ""total_cycles"", ""total_voltage_time_volt_seconds""], ""filters"": [""name_of_station = Utf8(\""workshop-caparoc\"")"", ""channel != Utf8(\""total\"")"", ""greptime_timestamp >= TimestampNanosecond(1783299280593000000, None)"", ""greptime_timestamp <= TimestampNanosecond(1783904080593000000, None)""], ""files"": [{""file_id"":""4518305595392(1052, 0)/df7cd2ca-08bd-4e83-b4ea-c23b49fca916"",""time_range_start"":""1782950400218686464::Nanosecond"",""time_range_end"":""1783555199793495040::Nanosecond"",""rows"":12984589,""size"":229975659,""index_size"":0}, {""file_id"":""4518305595392(1052, 0)/9ee90964-0ab5-4401-9e62-3720abb6c20a"",""time_range_start"":""1783555200380481280::Nanosecond"",""time_range_end"":""1784097098919327744::Nanosecond"",""rows"":11624888,""size"":206728167,""index_size"":0}], ""flat_format"": true, ""metrics_per_partition"": [{""partition"":0, ""metrics"":{""prepare_scan_cost"":""2.43018ms"", ""build_reader_cost"":""11.130735ms"", ""scan_cost"":""215.692731ms"", ""yield_cost"":""805.332093ms"", ""total_cost"":""1.02395408s"", ""num_rows"":540871, ""num_batches"":69, ""num_mem_ranges"":0, ""num_file_ranges"":8, ""build_parts_cost"":""0ns"", ""sst_scan_cost"":""193.051663ms"", ""rg_total"":0, ""rows_before_filter"":0, ""num_sst_record_batches"":0, ""num_sst_batches"":69, ""num_sst_rows"":540871, ""first_poll"":""2.648363ms"", ""convert_cost"":""5.595787ms"", ""pruner_cache_hit"":11, ""fetch_metrics"":{""total_fetch_elapsed"":""42.268622ms"", ""page_cache_hit"":10, ""cache_miss"":8, ""pages_to_fetch_mem"":544, ""page_size_to_fetch_mem"":3816053, ""pages_to_fetch_store"":469, ""page_size_to_fetch_store"":8613886, ""page_size_needed"":11730132, ""store_fetch_elapsed"":""25.224747ms"", ""prefilter_cost"":""10.955087ms"", ""prefilter_filtered_rows"":278329}, ""dedup_metrics"":{""dedup_cost"":""1.846872ms""}, ""top_file_metrics"": {""4518305595392(1052, 0)/df7cd2ca-08bd-4e83-b4ea-c23b49fca916"": {""build_part_cost"":""0ns"", ""num_ranges"":4, ""num_rows"":277656, ""build_reader_cost"":""399.698µs"", ""scan_cost"":""116.328837ms""}, ""4518305595392(1052, 0)/9ee90964-0ab5-4401-9e62-3720abb6c20a"": {""build_part_cost"":""0ns"", ""num_ranges"":4, ""num_rows"":263215, ""build_reader_cost"":""10.731037ms"", ""scan_cost"":""76.722826ms""}}, ""range_cache_size"":91915157, ""range_cache_hit"":4, ""range_cache_miss"":11, ""build_ranges_peak_mem_size"":0, ""num_peak_range_builders"":0, ""stream_eof"":true}}, {""partition"":1, ""metrics"":{""prepare_scan_cost"":""2.674064ms"", ""build_reader_cost"":""2.235673ms"", ""scan_cost"":""114.567736ms"", ""yield_cost"":""377.22413ms"", ""total_cost"":""494.711769ms"", ""num_rows"":198753, ""num_batches"":26, ""num_mem_ranges"":0, ""num_file_ranges"":3, ""build_parts_cost"":""0ns"", ""sst_scan_cost"":""107.620114ms"", ""rg_total"":0, ""rows_before_filter"":0, ""num_sst_record_batches"":0, ""num_sst_batches"":26, ""num_sst_rows"":198753, ""first_poll"":""2.748375ms"", ""convert_cost"":""6.342382ms"", ""pruner_cache_hit"":4, ""fetch_metrics"":{""total_fetch_elapsed"":""31.890771ms"", ""page_cache_hit"":4, ""cache_miss"":3, ""pages_to_fetch_mem"":138, ""page_size_to_fetch_mem"":856912, ""pages_to_fetch_store"":238, ""page_size_to_fetch_store"":4914986, ""page_size_needed"":5451463, ""store_fetch_elapsed"":""24.233566ms"", ""prefilter_cost"":""2.184899ms"", ""prefilter_filtered_rows"":108447}, ""dedup_metrics"":{""dedup_cost"":""735.491µs""}, ""top_file_metrics"": {""4518305595392(1052, 0)/df7cd2ca-08bd-4e83-b4ea-c23b49fca916"": {""build_part_cost"":""0ns"", ""num_ranges"":1, ""num_rows"":102400, ""build_reader_cost"":""51.752µs"", ""scan_cost"":""62.723661ms""}, ""4518305595392(1052, 0)/9ee90964-0ab5-4401-9e62-3720abb6c20a"": {""build_part_cost"":""0ns"", ""num_ranges"":2, ""num_rows"":96353, ""build_reader_cost"":""2.183921ms"", ""scan_cost"":""44.896453ms""}}, ""range_cache_size"":33500255, ""range_cache_hit"":11, ""range_cache_miss"":4, ""build_ranges_peak_mem_size"":0, ""num_peak_range_builders"":0, ""stream_eof"":true}}, {""partition"":2, ""metrics"":{""prepare_scan_cost"":""1.430479ms"", ""build_reader_cost"":""13.400353ms"", ""scan_cost"":""243.736246ms"", ""yield_cost"":""895.452062ms"", ""total_cost"":""1.141321872s"", ""num_rows"":709662, ""num_batches"":90, ""num_mem_ranges"":0, ""num_file_ranges"":7, ""build_parts_cost"":""0ns"", ""sst_scan_cost"":""218.146237ms"", ""rg_total"":0, ""rows_before_filter"":0, ""num_sst_record_batches"":0, ""num_sst_batches"":90, ""num_sst_rows"":709662, ""first_poll"":""1.77654ms"", ""convert_cost"":""3.62187ms"", ""pruner_cache_hit"":12, ""fetch_metrics"":{""total_fetch_elapsed"":""53.359688ms"", ""page_cache_hit"":11, ""cache_miss"":9, ""pages_to_fetch_mem"":657, ""page_size_to_fetch_mem"":4046273, ""pages_to_fetch_store"":593, ""page_size_to_fetch_store"":15758434, ""page_size_needed"":19492509, ""store_fetch_elapsed"":""37.704108ms"", ""prefilter_cost"":""13.260809ms"", ""prefilter_filtered_rows"":7138}, ""dedup_metrics"":{""dedup_cost"":""2.383865ms""}, ""top_file_metrics"": {""4518305595392(1052, 0)/9ee90964-0ab5-4401-9e62-3720abb6c20a"": {""build_part_cost"":""0ns"", ""num_ranges"":4, ""num_rows"":409600, ""build_reader_cost"":""13.072031ms"", ""scan_cost"":""108.923903ms""}, ""4518305595392(1052, 0)/df7cd2ca-08bd-4e83-b4ea-c23b49fca916"": {""build_part_cost"":""0ns"", ""num_ranges"":3, ""num_rows"":300062, ""build_reader_cost"":""328.322µs"", ""scan_cost"":""109.222334ms""}}, ""range_cache_size"":119639636, ""range_cache_hit"":4, ""range_cache_miss"":12, ""build_ranges_peak_mem_size"":0, ""num_peak_range_builders"":0, ""stream_eof"":true}}, {""partition"":3, ""metrics"":{""prepare_scan_cost"":""1.79407ms"", ""build_reader_cost"":""8.96623ms"", ""scan_cost"":""141.37473ms"", ""yield_cost"":""501.383519ms"", ""total_cost"":""644.801087ms"", ""num_rows"":274684, ""num_batches"":36, ""num_mem_ranges"":0, ""num_file_ranges"":5, ""build_parts_cost"":""0ns"", ""sst_scan_cost"":""124.921628ms"", ""rg_total"":0, ""rows_before_filter"":0, ""num_sst_record_batches"":0, ""num_sst_batches"":36, ""num_sst_rows"":274684, ""first_poll"":""1.826965ms"", ""convert_cost"":""4.054118ms"", ""pruner_cache_hit"":8, ""fetch_metrics"":{""total_fetch_elapsed"":""35.235166ms"", ""page_cache_hit"":8, ""cache_miss"":6, ""pages_to_fetch_mem"":463, ""page_size_to_fetch_mem"":1622674, ""pages_to_fetch_store"":162, ""page_size_to_fetch_store"":6820442, ""page_size_needed"":6945857, ""store_fetch_elapsed"":""27.290426ms"", ""prefilter_cost"":""8.86084ms"", ""prefilter_filtered_rows"":237316}, ""dedup_metrics"":{""dedup_cost"":""1.0481ms""}, ""top_file_metrics"": {""4518305595392(1052, 0)/9ee90964-0ab5-4401-9e62-3720abb6c20a"": {""build_part_cost"":""0ns"", ""num_ranges"":3, ""num_rows"":224620, ""build_reader_cost"":""8.541249ms"", ""scan_cost"":""64.572343ms""}, ""4518305595392(1052, 0)/df7cd2ca-08bd-4e83-b4ea-c23b49fca916"": {""build_part_cost"":""0ns"", ""num_ranges"":2, ""num_rows"":50064, ""build_reader_cost"":""424.981µs"", ""scan_cost"":""60.349285ms""}}, ""range_cache_size"":46040667, ""range_cache_hit"":7, ""range_cache_miss"":8, ""build_ranges_peak_mem_size"":0, ""num_peak_range_builders"":0, ""stream_eof"":true}}, {""partition"":4, ""metrics"":{""prepare_scan_cost"":""1.338499ms"", ""build_reader_cost"":""1.57114ms"", ""scan_cost"":""147.632809ms"", ""yield_cost"":""537.477024ms"", ""total_cost"":""686.663027ms"", ""num_rows"":285006, ""num_batches"":36, ""num_mem_ranges"":0, ""num_file_ranges"":4, ""build_parts_cost"":""0ns"", ""sst_scan_cost"":""139.987823ms"", ""rg_total"":0, ""rows_before_filter"":0, ""num_sst_record_batches"":0, ""num_sst_batches"":36, ""num_sst_rows"":285006, ""first_poll"":""1.384803ms"", ""convert_cost"":""3.275045ms"", ""pruner_cache_hit"":6, ""fetch_metrics"":{""total_fetch_elapsed"":""72.887104ms"", ""page_cache_hit"":5, ""cache_miss"":4, ""pages_to_fetch_mem"":250, ""page_size_to_fetch_mem"":557512, ""pages_to_fetch_store"":260, ""page_size_to_fetch_store"":3986067, ""page_size_needed"":3791473, ""store_fetch_elapsed"":""12.594669ms"", ""prefilter_cost"":""1.474269ms"", ""prefilter_filtered_rows"":124594}, ""dedup_metrics"":{""dedup_cost"":""946.06µs""}, ""top_file_metrics"": {""4518305595392(1052, 0)/df7cd2ca-08bd-4e83-b4ea-c23b49fca916"": {""build_part_cost"":""0ns"", ""num_ranges"":2, ""num_rows"":143241, ""build_reader_cost"":""135.911µs"", ""scan_cost"":""93.412682ms""}, ""4518305595392(1052, 0)/9ee90964-0ab5-4401-9e62-3720abb6c20a"": {""build_part_cost"":""0ns"", ""num_ranges"":2, ""num_rows"":141765, ""build_reader_cost"":""1.435229ms"", ""scan_cost"":""46.575141ms""}}, ""range_cache_size"":48033401, ""range_cache_hit"":9, ""range_cache_miss"":6, ""build_ranges_peak_mem_size"":0, ""num_peak_range_builders"":0, ""stream_eof"":true}}, {""partition"":5, ""metrics"":{""prepare_scan_cost"":""1.178563ms"", ""build_reader_cost"":""16.050601ms"", ""scan_cost"":""213.26597ms"", ""yield_cost"":""789.82178ms"", ""total_cost"":""1.004602525s"", ""num_rows"":496715, ""num_batches"":63, ""num_mem_ranges"":0, ""num_file_ranges"":6, ""build_parts_cost"":""0ns"", ""sst_scan_cost"":""184.408942ms"", ""rg_total"":0, ""rows_before_filter"":0, ""num_sst_record_batches"":0, ""num_sst_batches"":63, ""num_sst_rows"":496715, ""first_poll"":""1.228081ms"", ""convert_cost"":""6.11575ms"", ""pruner_cache_hit"":8, ""fetch_metrics"":{""total_fetch_elapsed"":""44.458967ms"", ""page_cache_hit"":10, ""cache_miss"":7, ""pages_to_fetch_mem"":351, ""page_size_to_fetch_mem"":2601815, ""pages_to_fetch_store"":650, ""page_size_to_fetch_store"":9479712, ""page_size_needed"":11574672, ""store_fetch_elapsed"":""28.649179ms"", ""prefilter_cost"":""15.947726ms"", ""prefilter_filtered_rows"":117685}, ""dedup_metrics"":{""dedup_cost"":""3.010419ms""}, ""top_file_metrics"": {""4518305595392(1052, 0)/9ee90964-0ab5-4401-9e62-3720abb6c20a"": {""build_part_cost"":""0ns"", ""num_ranges"":4, ""num_rows"":371005, ""build_reader_cost"":""15.819498ms"", ""scan_cost"":""121.627382ms""}, ""4518305595392(1052, 0)/df7cd2ca-08bd-4e83-b4ea-c23b49fca916"": {""build_part_cost"":""0ns"", ""num_ranges"":2, ""num_rows"":125710, ""build_reader_cost"":""231.103µs"", ""scan_cost"":""62.78156ms""}}, ""range_cache_size"":83870919, ""range_cache_hit"":7, ""range_cache_miss"":8, ""build_ranges_peak_mem_size"":0, ""num_peak_range_builders"":0, ""stream_eof"":true}}, {""partition"":6, ""metrics"":{""prepare_scan_cost"":""647.005µs"", ""build_reader_cost"":""12.161793ms"", ""scan_cost"":""227.216105ms"", ""yield_cost"":""774.652496ms"", ""total_cost"":""1.002790785s"", ""num_rows"":420739, ""num_batches"":55, ""num_mem_ranges"":0, ""num_file_ranges"":7, ""build_parts_cost"":""0ns"", ""sst_scan_cost"":""203.511092ms"", ""rg_total"":0, ""rows_before_filter"":0, ""num_sst_record_batches"":0, ""num_sst_batches"":55, ""num_sst_rows"":420739, ""first_poll"":""683.043µs"", ""convert_cost"":""4.862528ms"", ""pruner_cache_hit"":9, ""fetch_metrics"":{""total_fetch_elapsed"":""78.656509ms"", ""page_cache_hit"":8, ""cache_miss"":8, ""pages_to_fetch_mem"":303, ""page_size_to_fetch_mem"":1874723, ""pages_to_fetch_store"":574, ""page_size_to_fetch_store"":11614357, ""page_size_needed"":11389079, ""store_fetch_elapsed"":""50.301292ms"", ""prefilter_cost"":""12.013522ms"", ""prefilter_filtered_rows"":296061}, ""dedup_metrics"":{""dedup_cost"":""1.476432ms""}, ""top_file_metrics"": {""4518305595392(1052, 0)/df7cd2ca-08bd-4e83-b4ea-c23b49fca916"": {""build_part_cost"":""0ns"", ""num_ranges"":3, ""num_rows"":164367, ""build_reader_cost"":""364.708µs"", ""scan_cost"":""107.848071ms""}, ""4518305595392(1052, 0)/9ee90964-0ab5-4401-9e62-3720abb6c20a"": {""build_part_cost"":""0ns"", ""num_ranges"":4, ""num_rows"":256372, ""build_reader_cost"":""11.797085ms"", ""scan_cost"":""95.663021ms""}}, ""range_cache_size"":71780289, ""range_cache_hit"":6, ""range_cache_miss"":9, ""build_ranges_peak_mem_size"":0, ""num_peak_range_builders"":0, ""stream_eof"":true}}, {""partition"":7, ""metrics"":{""prepare_scan_cost"":""777.258µs"", ""build_reader_cost"":""6.481436ms"", ""scan_cost"":""226.397369ms"", ""yield_cost"":""842.877034ms"", ""total_cost"":""1.070385562s"", ""num_rows"":566155, ""num_batches"":73, ""num_mem_ranges"":0, ""num_file_ranges"":7, ""build_parts_cost"":""0ns"", ""sst_scan_cost"":""201.137204ms"", ""rg_total"":0, ""rows_before_filter"":0, ""num_sst_record_batches"":0, ""num_sst_batches"":72, ""num_sst_rows"":561886, ""first_poll"":""805.334µs"", ""convert_cost"":""4.33111ms"", ""pruner_cache_hit"":11, ""pruner_cache_miss"":1, ""pruner_prune_cost"":""832.503µs"", ""fetch_metrics"":{""total_fetch_elapsed"":""48.332057ms"", ""page_cache_hit"":8, ""cache_miss"":9, ""pages_to_fetch_mem"":649, ""page_size_to_fetch_mem"":2815331, ""pages_to_fetch_store"":304, ""page_size_to_fetch_store"":9572137, ""page_size_needed"":11329844, ""store_fetch_elapsed"":""32.796326ms"", ""prefilter_cost"":""6.361938ms"", ""prefilter_filtered_rows"":154914}, ""dedup_metrics"":{""dedup_cost"":""1.804969ms""}, ""top_file_metrics"": {""4518305595392(1052, 0)/df7cd2ca-08bd-4e83-b4ea-c23b49fca916"": {""build_part_cost"":""0ns"", ""num_ranges"":5, ""num_rows"":357086, ""build_reader_cost"":""459.833µs"", ""scan_cost"":""152.047306ms""}, ""4518305595392(1052, 0)/9ee90964-0ab5-4401-9e62-3720abb6c20a"": {""build_part_cost"":""0ns"", ""num_ranges"":2, ""num_rows"":204800, ""build_reader_cost"":""6.021603ms"", ""scan_cost"":""49.089898ms""}}, ""range_cache_size"":95131106, ""range_cache_hit"":3, ""range_cache_miss"":12, ""build_ranges_peak_mem_size"":0, ""num_peak_range_builders"":0, ""stream_eof"":true}}, {""partition"":8, ""metrics"":{""prepare_scan_cost"":""963.943µs"", ""build_reader_cost"":""12.200278ms"", ""scan_cost"":""232.193591ms"", ""yield_cost"":""837.360166ms"", ""total_cost"":""1.070875355s"", ""num_rows"":551698, ""num_batches"":71, ""num_mem_ranges"":0, ""num_file_ranges"":7, ""build_parts_cost"":""0ns"", ""sst_scan_cost"":""200.090014ms"", ""rg_total"":0, ""rows_before_filter"":0, ""num_sst_record_batches"":0, ""num_sst_batches"":71, ""num_sst_rows"":551698, ""first_poll"":""1.006057ms"", ""convert_cost"":""6.0214ms"", ""pruner_cache_hit"":9, ""pruner_cache_miss"":1, ""pruner_prune_cost"":""489.375µs"", ""fetch_metrics"":{""total_fetch_elapsed"":""41.758856ms"", ""page_cache_hit"":9, ""cache_miss"":9, ""pages_to_fetch_mem"":513, ""page_size_to_fetch_mem"":2807055, ""pages_to_fetch_store"":531, ""page_size_to_fetch_store"":9743239, ""page_size_needed"":11482247, ""store_fetch_elapsed"":""27.390929ms"", ""prefilter_cost"":""12.042716ms"", ""prefilter_filtered_rows"":165102}, ""dedup_metrics"":{""dedup_cost"":""2.118061ms""}, ""top_file_metrics"": {""4518305595392(1052, 0)/df7cd2ca-08bd-4e83-b4ea-c23b49fca916"": {""build_part_cost"":""0ns"", ""num_ranges"":3, ""num_rows"":281032, ""build_reader_cost"":""386.57µs"", ""scan_cost"":""106.359489ms""}, ""4518305595392(1052, 0)/9ee90964-0ab5-4401-9e62-3720abb6c20a"": {""build_part_cost"":""0ns"", ""num_ranges"":4, ""num_rows"":270666, ""build_reader_cost"":""11.813708ms"", ""scan_cost"":""93.730525ms""}}, ""range_cache_size"":93139550, ""range_cache_hit"":5, ""range_cache_miss"":10, ""build_ranges_peak_mem_size"":0, ""num_peak_range_builders"":0, ""stream_eof"":true}}, {""partition"":9, ""metrics"":{""prepare_scan_cost"":""1.063186ms"", ""build_reader_cost"":""13.10674ms"", ""scan_cost"":""162.262771ms"", ""yield_cost"":""675.552474ms"", ""total_cost"":""839.135453ms"", ""num_rows"":378478, ""num_batches"":48, ""num_mem_ranges"":0, ""num_file_ranges"":4, ""build_parts_cost"":""248.424µs"", ""sst_scan_cost"":""141.93177ms"", ""rg_total"":114, ""rows_before_filter"":11624888, ""num_sst_record_batches"":0, ""num_sst_batches"":48, ""num_sst_rows"":378478, ""first_poll"":""1.091751ms"", ""convert_cost"":""2.389603ms"", ""rg_minmax_filtered"":55, ""minmax_cache_hit"":1, ""pruner_cache_hit"":6, ""pruner_cache_miss"":1, ""pruner_prune_cost"":""315.401µs"", ""fetch_metrics"":{""total_fetch_elapsed"":""18.63848ms"", ""page_cache_hit"":7, ""cache_miss"":5, ""pages_to_fetch_mem"":274, ""page_size_to_fetch_mem"":1634630, ""pages_to_fetch_store"":393, ""page_size_to_fetch_store"":3589316, ""page_size_needed"":5223438, ""store_fetch_elapsed"":""11.507667ms"", ""prefilter_cost"":""13.03236ms"", ""prefilter_filtered_rows"":31122}, ""metadata_cache_metrics"":{""metadata_load_cost"":""14.596µs"", ""mem_cache_hit"":2}, ""dedup_metrics"":{""dedup_cost"":""1.380127ms""}, ""top_file_metrics"": {""4518305595392(1052, 0)/9ee90964-0ab5-4401-9e62-3720abb6c20a"": {""build_part_cost"":""248.424µs"", ""num_ranges"":3, ""num_rows"":307200, ""build_reader_cost"":""12.701315ms"", ""scan_cost"":""85.211835ms""}, ""4518305595392(1052, 0)/df7cd2ca-08bd-4e83-b4ea-c23b49fca916"": {""build_part_cost"":""0ns"", ""num_ranges"":1, ""num_rows"":71278, ""build_reader_cost"":""405.425µs"", ""scan_cost"":""56.719935ms""}}, ""range_cache_size"":64016857, ""range_cache_hit"":8, ""range_cache_miss"":7, ""build_ranges_peak_mem_size"":3459683, ""num_peak_range_builders"":1, ""stream_eof"":true}}, {""partition"":10, ""metrics"":{""prepare_scan_cost"":""1.270334ms"", ""build_reader_cost"":""17.233355ms"", ""scan_cost"":""106.530805ms"", ""yield_cost"":""240.434102ms"", ""total_cost"":""348.410405ms"", ""num_rows"":118234, ""num_batches"":16, ""num_mem_ranges"":0, ""num_file_ranges"":3, ""build_parts_cost"":""0ns"", ""sst_scan_cost"":""84.773868ms"", ""rg_total"":0, ""rows_before_filter"":0, ""num_sst_record_batches"":0, ""num_sst_batches"":16, ""num_sst_rows"":118234, ""first_poll"":""1.327115ms"", ""convert_cost"":""832.147µs"", ""pruner_cache_hit"":2, ""pruner_cache_miss"":1, ""pruner_prune_cost"":""837.949µs"", ""fetch_metrics"":{""total_fetch_elapsed"":""41.621337ms"", ""page_cache_hit"":6, ""cache_miss"":6, ""pages_to_fetch_mem"":109, ""page_size_to_fetch_mem"":680097, ""pages_to_fetch_store"":196, ""page_size_to_fetch_store"":6007838, ""page_size_needed"":5668649, ""store_fetch_elapsed"":""34.320741ms"", ""prefilter_cost"":""17.161837ms"", ""prefilter_filtered_rows"":188966}, ""dedup_metrics"":{""dedup_cost"":""501.457µs""}, ""top_file_metrics"": {""4518305595392(1052, 0)/9ee90964-0ab5-4401-9e62-3720abb6c20a"": {""build_part_cost"":""0ns"", ""num_ranges"":3, ""num_rows"":118234, ""build_reader_cost"":""17.233355ms"", ""scan_cost"":""84.773868ms""}}, ""range_cache_size"":21147930, ""range_cache_hit"":12, ""range_cache_miss"":3, ""build_ranges_peak_mem_size"":0, ""num_peak_range_builders"":0, ""stream_eof"":true}}, {""partition"":11, ""metrics"":{""prepare_scan_cost"":""694.357µs"", ""build_reader_cost"":""16.939811ms"", ""scan_cost"":""187.044669ms"", ""yield_cost"":""793.655692ms"", ""total_cost"":""981.769267ms"", ""num_rows"":452281, ""num_batches"":58, ""num_mem_ranges"":0, ""num_file_ranges"":6, ""build_parts_cost"":""304.227µs"", ""sst_scan_cost"":""152.501271ms"", ""rg_total"":127, ""rows_before_filter"":12984589, ""num_sst_record_batches"":0, ""num_sst_batches"":58, ""num_sst_rows"":452281, ""first_poll"":""726.763µs"", ""convert_cost"":""13.363127ms"", ""rg_minmax_filtered"":80, ""minmax_cache_hit"":1, ""pruner_cache_hit"":8, ""pruner_cache_miss"":1, ""pruner_prune_cost"":""809.175µs"", ""fetch_metrics"":{""total_fetch_elapsed"":""30.225626ms"", ""page_cache_hit"":9, ""cache_miss"":8, ""pages_to_fetch_mem"":389, ""page_size_to_fetch_mem"":1997161, ""pages_to_fetch_store"":466, ""page_size_to_fetch_store"":6293467, ""page_size_needed"":7508378, ""store_fetch_elapsed"":""20.153537ms"", ""prefilter_cost"":""16.838195ms"", ""prefilter_filtered_rows"":162119}, ""metadata_cache_metrics"":{""metadata_load_cost"":""42.603µs"", ""mem_cache_hit"":2}, ""dedup_metrics"":{""dedup_cost"":""1.873056ms""}, ""top_file_metrics"": {""4518305595392(1052, 0)/9ee90964-0ab5-4401-9e62-3720abb6c20a"": {""build_part_cost"":""0ns"", ""num_ranges"":4, ""num_rows"":262749, ""build_reader_cost"":""16.777223ms"", ""scan_cost"":""79.937601ms""}, ""4518305595392(1052, 0)/df7cd2ca-08bd-4e83-b4ea-c23b49fca916"": {""build_part_cost"":""304.227µs"", ""num_ranges"":2, ""num_rows"":189532, ""build_reader_cost"":""162.588µs"", ""scan_cost"":""72.56367ms""}}, ""range_cache_size"":77256119, ""range_cache_hit"":6, ""range_cache_miss"":9, ""build_ranges_peak_mem_size"":3750535, ""num_peak_range_builders"":1, ""stream_eof"":true}}, {""partition"":12, ""metrics"":{""prepare_scan_cost"":""743.805µs"", ""build_reader_cost"":""1.550117ms"", ""scan_cost"":""159.340292ms"", ""yield_cost"":""678.592924ms"", ""total_cost"":""838.922578ms"", ""num_rows"":395263, ""num_batches"":50, ""num_mem_ranges"":0, ""num_file_ranges"":5, ""build_parts_cost"":""0ns"", ""sst_scan_cost"":""144.652912ms"", ""rg_total"":0, ""rows_before_filter"":0, ""num_sst_record_batches"":0, ""num_sst_batches"":50, ""num_sst_rows"":395263, ""first_poll"":""764.338µs"", ""convert_cost"":""7.211057ms"", ""pruner_cache_hit"":6, ""pruner_cache_miss"":1, ""pruner_prune_cost"":""818.185µs"", ""fetch_metrics"":{""total_fetch_elapsed"":""26.941503ms"", ""page_cache_hit"":6, ""cache_miss"":5, ""pages_to_fetch_mem"":430, ""page_size_to_fetch_mem"":871315, ""pages_to_fetch_store"":326, ""page_size_to_fetch_store"":7489629, ""page_size_needed"":7749185, ""store_fetch_elapsed"":""16.971451ms"", ""prefilter_cost"":""1.399119ms"", ""prefilter_filtered_rows"":116737}, ""dedup_metrics"":{""dedup_cost"":""1.575122ms""}, ""top_file_metrics"": {""4518305595392(1052, 0)/9ee90964-0ab5-4401-9e62-3720abb6c20a"": {""build_part_cost"":""0ns"", ""num_ranges"":3, ""num_rows"":245569, ""build_reader_cost"":""1.313776ms"", ""scan_cost"":""89.101059ms""}, ""4518305595392(1052, 0)/df7cd2ca-08bd-4e83-b4ea-c23b49fca916"": {""build_part_cost"":""0ns"", ""num_ranges"":2, ""num_rows"":149694, ""build_reader_cost"":""236.341µs"", ""scan_cost"":""55.551853ms""}}, ""range_cache_size"":65804564, ""range_cache_hit"":8, ""range_cache_miss"":7, ""build_ranges_peak_mem_size"":0, ""num_peak_range_builders"":0, ""stream_eof"":true}}, {""partition"":13, ""metrics"":{""prepare_scan_cost"":""1.011295ms"", ""build_reader_cost"":""19.640976ms"", ""scan_cost"":""220.504757ms"", ""yield_cost"":""850.805918ms"", ""total_cost"":""1.072616556s"", ""num_rows"":500888, ""num_batches"":64, ""num_mem_ranges"":0, ""num_file_ranges"":6, ""build_parts_cost"":""0ns"", ""sst_scan_cost"":""189.06565ms"", ""rg_total"":0, ""rows_before_filter"":0, ""num_sst_record_batches"":0, ""num_sst_batches"":63, ""num_sst_rows"":495438, ""first_poll"":""1.041326ms"", ""convert_cost"":""9.171841ms"", ""pruner_cache_hit"":8, ""pruner_cache_miss"":1, ""pruner_prune_cost"":""1.187643ms"", ""fetch_metrics"":{""total_fetch_elapsed"":""69.484459ms"", ""page_cache_hit"":9, ""cache_miss"":9, ""pages_to_fetch_mem"":386, ""page_size_to_fetch_mem"":2424104, ""pages_to_fetch_store"":532, ""page_size_to_fetch_store"":8067784, ""page_size_needed"":9499271, ""store_fetch_elapsed"":""37.999325ms"", ""prefilter_cost"":""19.526785ms"", ""prefilter_filtered_rows"":118962}, ""dedup_metrics"":{""dedup_cost"":""1.665498ms""}, ""top_file_metrics"": {""4518305595392(1052, 0)/9ee90964-0ab5-4401-9e62-3720abb6c20a"": {""build_part_cost"":""0ns"", ""num_ranges"":4, ""num_rows"":314622, ""build_reader_cost"":""19.423002ms"", ""scan_cost"":""85.21177ms""}, ""4518305595392(1052, 0)/df7cd2ca-08bd-4e83-b4ea-c23b49fca916"": {""build_part_cost"":""0ns"", ""num_ranges"":2, ""num_rows"":180816, ""build_reader_cost"":""217.974µs"", ""scan_cost"":""103.85388ms""}}, ""range_cache_size"":82703536, ""range_cache_hit"":6, ""range_cache_miss"":9, ""build_ranges_peak_mem_size"":0, ""num_peak_range_builders"":0, ""stream_eof"":true}}, {""partition"":14, ""metrics"":{""prepare_scan_cost"":""838.229µs"", ""build_reader_cost"":""13.137541ms"", ""scan_cost"":""267.361216ms"", ""yield_cost"":""848.879359ms"", ""total_cost"":""1.117726441s"", ""num_rows"":670871, ""num_batches"":87, ""num_mem_ranges"":0, ""num_file_ranges"":9, ""build_parts_cost"":""0ns"", ""sst_scan_cost"":""235.52227ms"", ""rg_total"":0, ""rows_before_filter"":0, ""num_sst_record_batches"":0, ""num_sst_batches"":87, ""num_sst_rows"":670871, ""first_poll"":""896.896µs"", ""convert_cost"":""7.578216ms"", ""pruner_cache_hit"":10, ""pruner_cache_miss"":1, ""pruner_prune_cost"":""573.463µs"", ""fetch_metrics"":{""total_fetch_elapsed"":""67.228459ms"", ""page_cache_hit"":9, ""cache_miss"":10, ""pages_to_fetch_mem"":568, ""page_size_to_fetch_mem"":2232572, ""pages_to_fetch_store"":637, ""page_size_to_fetch_store"":15833748, ""page_size_needed"":17030577, ""store_fetch_elapsed"":""44.936191ms"", ""prefilter_cost"":""12.964825ms"", ""prefilter_filtered_rows"":250729}, ""dedup_metrics"":{""dedup_cost"":""2.239785ms""}, ""top_file_metrics"": {""4518305595392(1052, 0)/df7cd2ca-08bd-4e83-b4ea-c23b49fca916"": {""build_part_cost"":""0ns"", ""num_ranges"":4, ""num_rows"":289063, ""build_reader_cost"":""2.639144ms"", ""scan_cost"":""140.023091ms""}, ""4518305595392(1052, 0)/9ee90964-0ab5-4401-9e62-3720abb6c20a"": {""build_part_cost"":""0ns"", ""num_ranges"":5, ""num_rows"":381808, ""build_reader_cost"":""10.498397ms"", ""scan_cost"":""95.499179ms""}}, ""range_cache_size"":112767605, ""range_cache_hit"":4, ""range_cache_miss"":11, ""build_ranges_peak_mem_size"":0, ""num_peak_range_builders"":0, ""stream_eof"":true}}, {""partition"":15, ""metrics"":{""prepare_scan_cost"":""1.125205ms"", ""build_reader_cost"":""7.424915ms"", ""scan_cost"":""200.923287ms"", ""yield_cost"":""867.884225ms"", ""total_cost"":""1.070254052s"", ""num_rows"":524838, ""num_batches"":67, ""num_mem_ranges"":0, ""num_file_ranges"":6, ""build_parts_cost"":""0ns"", ""sst_scan_cost"":""182.200088ms"", ""rg_total"":0, ""rows_before_filter"":0, ""num_sst_record_batches"":0, ""num_sst_batches"":67, ""num_sst_rows"":524838, ""first_poll"":""1.1625ms"", ""convert_cost"":""3.57299ms"", ""pruner_cache_hit"":10, ""fetch_metrics"":{""total_fetch_elapsed"":""41.424386ms"", ""page_cache_hit"":8, ""cache_miss"":7, ""pages_to_fetch_mem"":505, ""page_size_to_fetch_mem"":2003486, ""pages_to_fetch_store"":414, ""page_size_to_fetch_store"":7213902, ""page_size_needed"":8966632, ""store_fetch_elapsed"":""28.966817ms"", ""prefilter_cost"":""7.283835ms"", ""prefilter_filtered_rows"":89562}, ""dedup_metrics"":{""dedup_cost"":""1.950233ms""}, ""top_file_metrics"": {""4518305595392(1052, 0)/df7cd2ca-08bd-4e83-b4ea-c23b49fca916"": {""build_part_cost"":""0ns"", ""num_ranges"":3, ""num_rows"":307200, ""build_reader_cost"":""370.506µs"", ""scan_cost"":""114.715661ms""}, ""4518305595392(1052, 0)/9ee90964-0ab5-4401-9e62-3720abb6c20a"": {""build_part_cost"":""0ns"", ""num_ranges"":3, ""num_rows"":217638, ""build_reader_cost"":""7.054409ms"", ""scan_cost"":""67.484427ms""}}, ""range_cache_size"":88729628, ""range_cache_hit"":5, ""range_cache_miss"":10, ""build_ranges_peak_mem_size"":0, ""num_peak_range_builders"":0, ""stream_eof"":true}}]} metrics=[output_rows: 7085136, elapsed_compute: 14.5s, mem_used: 1.4GiB, build_parts_cost: 553us, build_reader_cost: 173ms, convert_cost: 88ms, elapsed_await: 3.18s, elapsed_poll: 2.77s, scan_cost: 3.07s, yield_cost: 11.3s, ]
"
1,0," ProjectionExec: expr=[channel@0 as channel, CASE WHEN last_value(profinet.caparoc.channel_name) ORDER BY [profinet.caparoc.greptime_timestamp ASC NULLS LAST]@1 IS NOT NULL THEN last_value(profinet.caparoc.channel_name) ORDER BY [profinet.caparoc.greptime_timestamp ASC NULLS LAST]@1 ELSE concat(Channel , channel@0) END as channel_name] metrics=[output_rows: 13, elapsed_compute: 139us, output_bytes: 107.5KiB, output_batches: 12, expr_0_eval_time: 23us, expr_1_eval_time: 50us, ]
AggregateExec: mode=FinalPartitioned, gby=[channel@0 as channel], aggr=[last_value(profinet.caparoc.channel_name) ORDER BY [profinet.caparoc.greptime_timestamp ASC NULLS LAST]] metrics=[output_rows: 13, elapsed_compute: 1.1ms, output_bytes: 108.4KiB, output_batches: 12, spill_count: 0, spilled_bytes: 0B, spilled_rows: 0, peak_mem_used: 249.3KiB, aggregate_arguments_time: 41us, aggregation_time: 243us, emitting_time: 96us, time_calculating_group_ids: 63us, ]
RepartitionExec: partitioning=Hash([channel@0], 16), input_partitions=16 metrics=[output_rows: 87, elapsed_compute: 1.6ms, output_bytes: 771.4KiB, output_batches: 12, spill_count: 0, spilled_bytes: 0B, spilled_rows: 0, fetch_time: 692ms, repartition_time: 431us, send_time: 366us, ]
AggregateExec: mode=Partial, gby=[channel@0 as channel], aggr=[last_value(profinet.caparoc.channel_name) ORDER BY [profinet.caparoc.greptime_timestamp ASC NULLS LAST]] metrics=[output_rows: 87, elapsed_compute: 2.3ms, output_bytes: 147.4KiB, output_batches: 16, spill_count: 0, spilled_bytes: 0B, spilled_rows: 0, skipped_aggregation_rows: 0, peak_mem_used: 181.9KiB, aggregate_arguments_time: 59us, aggregation_time: 920us, emitting_time: 642us, time_calculating_group_ids: 169us, reduction_factor: 0, ]
FilterExec: channel@0 IS NOT NULL metrics=[output_rows: 100, elapsed_compute: 2.0ms, output_bytes: 1.0MiB, output_batches: 16, selectivity: 0, ]
SeqScan: region=4518305595392(1052, 0), {""partition_count"":{""count"":241, ""mem_ranges"":0, ""files"":2, ""file_ranges"":241}, ""selector"":""LastRow"", ""projection"": [""channel"", ""channel_name"", ""greptime_timestamp""], ""filters"": [""channel IS NOT NULL"", ""name_of_station = Utf8(\""workshop-caparoc\"")"", ""greptime_timestamp >= TimestampNanosecond(1783299280593000000, None)"", ""greptime_timestamp <= TimestampNanosecond(1783904080593000000, None)""], ""dyn_filters"": [""DynamicFilter [ CASE hash_repartition % 16 WHEN 0 THEN channel@0 >= 1_1_2 AND channel@0 <= 1_1_2 AND true WHEN 4 THEN channel@0 >= 1_1_1 AND channel@0 <= 1_1_1 AND true WHEN 6 THEN channel@0 >= 3_1_3 AND channel@0 <= 3_1_3 AND true WHEN 7 THEN channel@0 >= 2_1_2 AND channel@0 <= 2_1_2 AND true WHEN 9 THEN channel@0 >= 3_1_1 AND channel@0 <= 3_1_1 AND true WHEN 10 THEN channel@0 >= 2_1_3 AND channel@0 <= 2_1_3 AND true WHEN 11 THEN channel@0 >= 2_1_1 AND channel@0 <= 2_1_1 AND true WHEN 12 THEN channel@0 >= 2_1_4 AND channel@0 <= 2_1_4 AND true WHEN 13 THEN channel@0 >= 3_1_2 AND channel@0 <= 3_1_2 AND true WHEN 14 THEN channel@0 >= 1_1_4 AND channel@0 <= 3_1_4 AND true WHEN 15 THEN channel@0 >= 1_1_3 AND channel@0 <= 1_1_3 AND true ELSE false END ]""], ""files"": [{""file_id"":""4518305595392(1052, 0)/df7cd2ca-08bd-4e83-b4ea-c23b49fca916"",""time_range_start"":""1782950400218686464::Nanosecond"",""time_range_end"":""1783555199793495040::Nanosecond"",""rows"":12984589,""size"":229975659,""index_size"":0}, {""file_id"":""4518305595392(1052, 0)/9ee90964-0ab5-4401-9e62-3720abb6c20a"",""time_range_start"":""1783555200380481280::Nanosecond"",""time_range_end"":""1784097098919327744::Nanosecond"",""rows"":11624888,""size"":206728167,""index_size"":0}], ""flat_format"": true, ""metrics_per_partition"": [{""partition"":0, ""metrics"":{""prepare_scan_cost"":""1.103136ms"", ""build_reader_cost"":""11.299052ms"", ""scan_cost"":""44.457925ms"", ""yield_cost"":""691.844µs"", ""total_cost"":""47.48964ms"", ""num_rows"":7, ""num_batches"":7, ""num_mem_ranges"":0, ""num_file_ranges"":8, ""build_parts_cost"":""0ns"", ""sst_scan_cost"":""28.07118ms"", ""rg_total"":0, ""rows_before_filter"":0, ""num_sst_record_batches"":0, ""num_sst_batches"":7, ""num_sst_rows"":7, ""first_poll"":""2.26794ms"", ""convert_cost"":""344.873µs"", ""pruner_cache_hit"":14, ""pruner_cache_miss"":1, ""pruner_prune_cost"":""1.005847ms"", ""fetch_metrics"":{""total_fetch_elapsed"":""7.423866ms"", ""page_cache_hit"":11, ""cache_miss"":3, ""pages_to_fetch_mem"":275, ""page_size_to_fetch_mem"":2325717, ""pages_to_fetch_store"":23, ""page_size_to_fetch_store"":371264, ""page_size_needed"":2696981, ""store_fetch_elapsed"":""5.010936ms"", ""prefilter_cost"":""10.734041ms"", ""prefilter_filtered_rows"":278329}, ""dedup_metrics"":{""dedup_cost"":""39.951µs""}, ""top_file_metrics"": {""4518305595392(1052, 0)/9ee90964-0ab5-4401-9e62-3720abb6c20a"": {""build_part_cost"":""0ns"", ""num_ranges"":4, ""num_rows"":3, ""build_reader_cost"":""10.90913ms"", ""scan_cost"":""15.824247ms""}, ""4518305595392(1052, 0)/df7cd2ca-08bd-4e83-b4ea-c23b49fca916"": {""build_part_cost"":""0ns"", ""num_ranges"":4, ""num_rows"":4, ""build_reader_cost"":""389.922µs"", ""scan_cost"":""12.246933ms""}}, ""build_ranges_peak_mem_size"":0, ""num_peak_range_builders"":0, ""stream_eof"":true}}, {""partition"":1, ""metrics"":{""prepare_scan_cost"":""1.208176ms"", ""build_reader_cost"":""13.555606ms"", ""scan_cost"":""39.031501ms"", ""yield_cost"":""432.594µs"", ""total_cost"":""41.822968ms"", ""num_rows"":4, ""num_batches"":4, ""num_mem_ranges"":0, ""num_file_ranges"":5, ""build_parts_cost"":""0ns"", ""sst_scan_cost"":""14.210435ms"", ""rg_total"":0, ""rows_before_filter"":0, ""num_sst_record_batches"":0, ""num_sst_batches"":4, ""num_sst_rows"":4, ""first_poll"":""2.29895ms"", ""convert_cost"":""187.453µs"", ""pruner_cache_hit"":13, ""pruner_cache_miss"":2, ""pruner_prune_cost"":""8.004523ms"", ""fetch_metrics"":{""total_fetch_elapsed"":""8.386063ms"", ""page_cache_hit"":7, ""cache_miss"":2, ""pages_to_fetch_mem"":138, ""page_size_to_fetch_mem"":728085, ""pages_to_fetch_store"":12, ""page_size_to_fetch_store"":971982, ""page_size_needed"":1211140, ""store_fetch_elapsed"":""7.087653ms"", ""prefilter_cost"":""13.126297ms"", ""prefilter_filtered_rows"":247926}, ""dedup_metrics"":{""dedup_cost"":""31.917µs""}, ""top_file_metrics"": {""4518305595392(1052, 0)/9ee90964-0ab5-4401-9e62-3720abb6c20a"": {""build_part_cost"":""0ns"", ""num_ranges"":3, ""num_rows"":2, ""build_reader_cost"":""13.387989ms"", ""scan_cost"":""4.919586ms""}, ""4518305595392(1052, 0)/df7cd2ca-08bd-4e83-b4ea-c23b49fca916"": {""build_part_cost"":""0ns"", ""num_ranges"":2, ""num_rows"":2, ""build_reader_cost"":""167.617µs"", ""scan_cost"":""9.290849ms""}}, ""build_ranges_peak_mem_size"":0, ""num_peak_range_builders"":0, ""stream_eof"":true}}, {""partition"":2, ""metrics"":{""prepare_scan_cost"":""1.264468ms"", ""build_reader_cost"":""10.336855ms"", ""scan_cost"":""42.817155ms"", ""yield_cost"":""639.811µs"", ""total_cost"":""46.031784ms"", ""num_rows"":7, ""num_batches"":7, ""num_mem_ranges"":0, ""num_file_ranges"":9, ""build_parts_cost"":""0ns"", ""sst_scan_cost"":""27.687262ms"", ""rg_total"":0, ""rows_before_filter"":0, ""num_sst_record_batches"":0, ""num_sst_batches"":7, ""num_sst_rows"":7, ""first_poll"":""2.494364ms"", ""convert_cost"":""270.424µs"", ""pruner_cache_hit"":15, ""pruner_cache_miss"":1, ""pruner_prune_cost"":""583.101µs"", ""fetch_metrics"":{""total_fetch_elapsed"":""4.593429ms"", ""page_cache_hit"":11, ""pages_to_fetch_mem"":361, ""page_size_to_fetch_mem"":3365088, ""page_size_needed"":3365088, ""prefilter_cost"":""9.844757ms"", ""prefilter_filtered_rows"":211938}, ""dedup_metrics"":{""dedup_cost"":""49.796µs""}, ""top_file_metrics"": {""4518305595392(1052, 0)/9ee90964-0ab5-4401-9e62-3720abb6c20a"": {""build_part_cost"":""0ns"", ""num_ranges"":4, ""num_rows"":4, ""build_reader_cost"":""9.82178ms"", ""scan_cost"":""15.003266ms""}, ""4518305595392(1052, 0)/df7cd2ca-08bd-4e83-b4ea-c23b49fca916"": {""build_part_cost"":""0ns"", ""num_ranges"":5, ""num_rows"":3, ""build_reader_cost"":""515.075µs"", ""scan_cost"":""12.683996ms""}}, ""build_ranges_peak_mem_size"":0, ""num_peak_range_builders"":0, ""stream_eof"":true}}, {""partition"":3, ""metrics"":{""prepare_scan_cost"":""1.31608ms"", ""build_reader_cost"":""2.906425ms"", ""scan_cost"":""29.956529ms"", ""yield_cost"":""548.528µs"", ""total_cost"":""33.08833ms"", ""num_rows"":5, ""num_batches"":5, ""num_mem_ranges"":0, ""num_file_ranges"":6, ""build_parts_cost"":""0ns"", ""sst_scan_cost"":""13.216321ms"", ""rg_total"":0, ""rows_before_filter"":0, ""num_sst_record_batches"":0, ""num_sst_batches"":5, ""num_sst_rows"":5, ""first_poll"":""2.50931ms"", ""convert_cost"":""245.211µs"", ""pruner_cache_hit"":13, ""pruner_cache_miss"":2, ""pruner_prune_cost"":""10.303402ms"", ""fetch_metrics"":{""total_fetch_elapsed"":""1.430549ms"", ""page_cache_hit"":8, ""pages_to_fetch_mem"":164, ""page_size_to_fetch_mem"":987255, ""page_size_needed"":987255, ""prefilter_cost"":""2.397985ms"", ""prefilter_filtered_rows"":339716}, ""dedup_metrics"":{""dedup_cost"":""38.481µs""}, ""top_file_metrics"": {""4518305595392(1052, 0)/9ee90964-0ab5-4401-9e62-3720abb6c20a"": {""build_part_cost"":""0ns"", ""num_ranges"":4, ""num_rows"":3, ""build_reader_cost"":""2.623919ms"", ""scan_cost"":""9.319835ms""}, ""4518305595392(1052, 0)/df7cd2ca-08bd-4e83-b4ea-c23b49fca916"": {""build_part_cost"":""0ns"", ""num_ranges"":2, ""num_rows"":2, ""build_reader_cost"":""282.506µs"", ""scan_cost"":""3.896486ms""}}, ""build_ranges_peak_mem_size"":0, ""num_peak_range_builders"":0, ""stream_eof"":true}}, {""partition"":4, ""metrics"":{""prepare_scan_cost"":""1.362175ms"", ""build_reader_cost"":""1.53887ms"", ""scan_cost"":""24.585076ms"", ""yield_cost"":""504.878µs"", ""total_cost"":""27.576496ms"", ""num_rows"":5, ""num_batches"":5, ""num_mem_ranges"":0, ""num_file_ranges"":5, ""build_parts_cost"":""0ns"", ""sst_scan_cost"":""16.032651ms"", ""rg_total"":0, ""rows_before_filter"":0, ""num_sst_record_batches"":0, ""num_sst_batches"":5, ""num_sst_rows"":5, ""first_poll"":""2.408739ms"", ""convert_cost"":""240.182µs"", ""pruner_cache_hit"":13, ""pruner_cache_miss"":2, ""pruner_prune_cost"":""2.667849ms"", ""fetch_metrics"":{""total_fetch_elapsed"":""1.322924ms"", ""page_cache_hit"":6, ""pages_to_fetch_mem"":172, ""page_size_to_fetch_mem"":1092508, ""page_size_needed"":1092508, ""prefilter_cost"":""1.134495ms"", ""prefilter_filtered_rows"":124594}, ""dedup_metrics"":{""dedup_cost"":""44.137µs""}, ""top_file_metrics"": {""4518305595392(1052, 0)/df7cd2ca-08bd-4e83-b4ea-c23b49fca916"": {""build_part_cost"":""0ns"", ""num_ranges"":3, ""num_rows"":3, ""build_reader_cost"":""628.566µs"", ""scan_cost"":""10.388327ms""}, ""4518305595392(1052, 0)/9ee90964-0ab5-4401-9e62-3720abb6c20a"": {""build_part_cost"":""0ns"", ""num_ranges"":2, ""num_rows"":2, ""build_reader_cost"":""910.304µs"", ""scan_cost"":""5.644324ms""}}, ""build_ranges_peak_mem_size"":0, ""num_peak_range_builders"":0, ""stream_eof"":true}}, {""partition"":5, ""metrics"":{""prepare_scan_cost"":""1.436416ms"", ""build_reader_cost"":""8.329629ms"", ""scan_cost"":""42.446019ms"", ""yield_cost"":""489.582µs"", ""total_cost"":""45.516638ms"", ""num_rows"":7, ""num_batches"":7, ""num_mem_ranges"":0, ""num_file_ranges"":7, ""build_parts_cost"":""0ns"", ""sst_scan_cost"":""26.668845ms"", ""rg_total"":0, ""rows_before_filter"":0, ""num_sst_record_batches"":0, ""num_sst_batches"":7, ""num_sst_rows"":7, ""first_poll"":""2.498205ms"", ""convert_cost"":""213.504µs"", ""pruner_cache_hit"":13, ""pruner_cache_miss"":2, ""pruner_prune_cost"":""2.801313ms"", ""fetch_metrics"":{""total_fetch_elapsed"":""2.34302ms"", ""page_cache_hit"":11, ""pages_to_fetch_mem"":324, ""page_size_to_fetch_mem"":2521907, ""page_size_needed"":2521907, ""prefilter_cost"":""7.169157ms"", ""prefilter_filtered_rows"":117685}, ""dedup_metrics"":{""dedup_cost"":""79.134µs""}, ""top_file_metrics"": {""4518305595392(1052, 0)/9ee90964-0ab5-4401-9e62-3720abb6c20a"": {""build_part_cost"":""0ns"", ""num_ranges"":4, ""num_rows"":4, ""build_reader_cost"":""8.05774ms"", ""scan_cost"":""15.290036ms""}, ""4518305595392(1052, 0)/df7cd2ca-08bd-4e83-b4ea-c23b49fca916"": {""build_part_cost"":""0ns"", ""num_ranges"":3, ""num_rows"":3, ""build_reader_cost"":""271.889µs"", ""scan_cost"":""11.378809ms""}}, ""build_ranges_peak_mem_size"":0, ""num_peak_range_builders"":0, ""stream_eof"":true}}, {""partition"":6, ""metrics"":{""prepare_scan_cost"":""1.495571ms"", ""build_reader_cost"":""9.818358ms"", ""scan_cost"":""46.63668ms"", ""yield_cost"":""610.828µs"", ""total_cost"":""49.814361ms"", ""num_rows"":7, ""num_batches"":7, ""num_mem_ranges"":0, ""num_file_ranges"":9, ""build_parts_cost"":""0ns"", ""sst_scan_cost"":""28.842008ms"", ""rg_total"":0, ""rows_before_filter"":0, ""num_sst_record_batches"":0, ""num_sst_batches"":7, ""num_sst_rows"":7, ""first_poll"":""2.492898ms"", ""convert_cost"":""246.051µs"", ""pruner_cache_hit"":13, ""pruner_cache_miss"":2, ""pruner_prune_cost"":""3.230485ms"", ""fetch_metrics"":{""total_fetch_elapsed"":""11.430423ms"", ""page_cache_hit"":10, ""cache_miss"":4, ""pages_to_fetch_mem"":219, ""page_size_to_fetch_mem"":1527621, ""pages_to_fetch_store"":30, ""page_size_to_fetch_store"":181212, ""page_size_needed"":1708833, ""store_fetch_elapsed"":""9.225478ms"", ""prefilter_cost"":""9.20767ms"", ""prefilter_filtered_rows"":500861}, ""dedup_metrics"":{""dedup_cost"":""61.32µs""}, ""top_file_metrics"": {""4518305595392(1052, 0)/9ee90964-0ab5-4401-9e62-3720abb6c20a"": {""build_part_cost"":""0ns"", ""num_ranges"":5, ""num_rows"":4, ""build_reader_cost"":""9.158991ms"", ""scan_cost"":""19.354417ms""}, ""4518305595392(1052, 0)/df7cd2ca-08bd-4e83-b4ea-c23b49fca916"": {""build_part_cost"":""0ns"", ""num_ranges"":4, ""num_rows"":3, ""build_reader_cost"":""659.367µs"", ""scan_cost"":""9.487591ms""}}, ""build_ranges_peak_mem_size"":0, ""num_peak_range_builders"":0, ""stream_eof"":true}}, {""partition"":7, ""metrics"":{""prepare_scan_cost"":""1.555215ms"", ""build_reader_cost"":""16.42453ms"", ""scan_cost"":""48.578244ms"", ""yield_cost"":""940.825µs"", ""total_cost"":""52.140967ms"", ""num_rows"":9, ""num_batches"":9, ""num_mem_ranges"":0, ""num_file_ranges"":9, ""build_parts_cost"":""0ns"", ""sst_scan_cost"":""27.797887ms"", ""rg_total"":0, ""rows_before_filter"":0, ""num_sst_record_batches"":0, ""num_sst_batches"":9, ""num_sst_rows"":9, ""first_poll"":""2.542135ms"", ""convert_cost"":""494.473µs"", ""pruner_cache_hit"":14, ""pruner_cache_miss"":1, ""pruner_prune_cost"":""512.771µs"", ""fetch_metrics"":{""total_fetch_elapsed"":""15.27935ms"", ""page_cache_hit"":12, ""cache_miss"":6, ""pages_to_fetch_mem"":304, ""page_size_to_fetch_mem"":1883380, ""pages_to_fetch_store"":17, ""page_size_to_fetch_store"":1045222, ""page_size_needed"":2928602, ""store_fetch_elapsed"":""11.794014ms"", ""prefilter_cost"":""15.880958ms"", ""prefilter_filtered_rows"":289744}, ""dedup_metrics"":{""dedup_cost"":""60.06µs""}, ""top_file_metrics"": {""4518305595392(1052, 0)/9ee90964-0ab5-4401-9e62-3720abb6c20a"": {""build_part_cost"":""0ns"", ""num_ranges"":3, ""num_rows"":3, ""build_reader_cost"":""15.640984ms"", ""scan_cost"":""13.744735ms""}, ""4518305595392(1052, 0)/df7cd2ca-08bd-4e83-b4ea-c23b49fca916"": {""build_part_cost"":""0ns"", ""num_ranges"":6, ""num_rows"":6, ""build_reader_cost"":""783.546µs"", ""scan_cost"":""14.053152ms""}}, ""build_ranges_peak_mem_size"":0, ""num_peak_range_builders"":0, ""stream_eof"":true}}, {""partition"":8, ""metrics"":{""prepare_scan_cost"":""1.619608ms"", ""build_reader_cost"":""23.342885ms"", ""scan_cost"":""58.491028ms"", ""yield_cost"":""748.484µs"", ""total_cost"":""61.690507ms"", ""num_rows"":8, ""num_batches"":8, ""num_mem_ranges"":0, ""num_file_ranges"":9, ""build_parts_cost"":""358.632µs"", ""sst_scan_cost"":""26.060951ms"", ""rg_total"":127, ""rows_before_filter"":12984589, ""num_sst_record_batches"":0, ""num_sst_batches"":8, ""num_sst_rows"":8, ""first_poll"":""2.310962ms"", ""convert_cost"":""320.571µs"", ""rg_minmax_filtered"":76, ""pruner_cache_hit"":14, ""pruner_cache_miss"":1, ""pruner_prune_cost"":""5.288833ms"", ""fetch_metrics"":{""total_fetch_elapsed"":""7.258971ms"", ""page_cache_hit"":12, ""cache_miss"":3, ""pages_to_fetch_mem"":339, ""page_size_to_fetch_mem"":2447988, ""pages_to_fetch_store"":13, ""page_size_to_fetch_store"":798817, ""page_size_needed"":3246805, ""store_fetch_elapsed"":""3.939928ms"", ""prefilter_cost"":""22.672276ms"", ""prefilter_filtered_rows"":267502}, ""metadata_cache_metrics"":{""metadata_load_cost"":""50.006µs"", ""mem_cache_hit"":2}, ""dedup_metrics"":{""dedup_cost"":""48.26µs""}, ""top_file_metrics"": {""4518305595392(1052, 0)/9ee90964-0ab5-4401-9e62-3720abb6c20a"": {""build_part_cost"":""0ns"", ""num_ranges"":5, ""num_rows"":5, ""build_reader_cost"":""22.848272ms"", ""scan_cost"":""16.654864ms""}, ""4518305595392(1052, 0)/df7cd2ca-08bd-4e83-b4ea-c23b49fca916"": {""build_part_cost"":""358.632µs"", ""num_ranges"":4, ""num_rows"":3, ""build_reader_cost"":""494.613µs"", ""scan_cost"":""9.406087ms""}}, ""build_ranges_peak_mem_size"":3750759, ""num_peak_range_builders"":1, ""stream_eof"":true}}, {""partition"":9, ""metrics"":{""prepare_scan_cost"":""1.679113ms"", ""build_reader_cost"":""9.367675ms"", ""scan_cost"":""49.296143ms"", ""yield_cost"":""362.403µs"", ""total_cost"":""52.156472ms"", ""num_rows"":5, ""num_batches"":5, ""num_mem_ranges"":0, ""num_file_ranges"":5, ""build_parts_cost"":""0ns"", ""sst_scan_cost"":""25.409894ms"", ""rg_total"":0, ""rows_before_filter"":0, ""num_sst_record_batches"":0, ""num_sst_batches"":5, ""num_sst_rows"":5, ""first_poll"":""2.431158ms"", ""convert_cost"":""165.872µs"", ""pruner_cache_hit"":13, ""pruner_cache_miss"":2, ""pruner_prune_cost"":""11.294862ms"", ""fetch_metrics"":{""total_fetch_elapsed"":""10.196338ms"", ""page_cache_hit"":9, ""cache_miss"":3, ""pages_to_fetch_mem"":200, ""page_size_to_fetch_mem"":1405556, ""pages_to_fetch_store"":32, ""page_size_to_fetch_store"":963856, ""page_size_needed"":2020405, ""store_fetch_elapsed"":""7.880625ms"", ""prefilter_cost"":""8.976497ms"", ""prefilter_filtered_rows"":63505}, ""dedup_metrics"":{""dedup_cost"":""34.292µs""}, ""top_file_metrics"": {""4518305595392(1052, 0)/9ee90964-0ab5-4401-9e62-3720abb6c20a"": {""build_part_cost"":""0ns"", ""num_ranges"":4, ""num_rows"":4, ""build_reader_cost"":""9.309009ms"", ""scan_cost"":""21.913175ms""}, ""4518305595392(1052, 0)/df7cd2ca-08bd-4e83-b4ea-c23b49fca916"": {""build_part_cost"":""0ns"", ""num_ranges"":1, ""num_rows"":1, ""build_reader_cost"":""58.666µs"", ""scan_cost"":""3.496719ms""}}, ""build_ranges_peak_mem_size"":0, ""num_peak_range_builders"":0, ""stream_eof"":true}}, {""partition"":10, ""metrics"":{""prepare_scan_cost"":""1.751258ms"", ""build_reader_cost"":""8.948211ms"", ""scan_cost"":""42.467463ms"", ""yield_cost"":""441.323µs"", ""total_cost"":""45.518454ms"", ""num_rows"":4, ""num_batches"":4, ""num_mem_ranges"":0, ""num_file_ranges"":5, ""build_parts_cost"":""0ns"", ""sst_scan_cost"":""12.172131ms"", ""rg_total"":0, ""rows_before_filter"":0, ""num_sst_record_batches"":0, ""num_sst_batches"":4, ""num_sst_rows"":4, ""first_poll"":""2.546396ms"", ""convert_cost"":""217.204µs"", ""pruner_cache_hit"":13, ""pruner_cache_miss"":2, ""pruner_prune_cost"":""17.835381ms"", ""fetch_metrics"":{""total_fetch_elapsed"":""4.051812ms"", ""page_cache_hit"":8, ""cache_miss"":1, ""pages_to_fetch_mem"":141, ""page_size_to_fetch_mem"":1052793, ""pages_to_fetch_store"":5, ""page_size_to_fetch_store"":301632, ""page_size_needed"":1354425, ""store_fetch_elapsed"":""2.117154ms"", ""prefilter_cost"":""8.261397ms"", ""prefilter_filtered_rows"":291366}, ""dedup_metrics"":{""dedup_cost"":""35.131µs""}, ""top_file_metrics"": {""4518305595392(1052, 0)/9ee90964-0ab5-4401-9e62-3720abb6c20a"": {""build_part_cost"":""0ns"", ""num_ranges"":5, ""num_rows"":4, ""build_reader_cost"":""8.948211ms"", ""scan_cost"":""12.172131ms""}}, ""build_ranges_peak_mem_size"":0, ""num_peak_range_builders"":0, ""stream_eof"":true}}, {""partition"":11, ""metrics"":{""prepare_scan_cost"":""1.815163ms"", ""build_reader_cost"":""11.645742ms"", ""scan_cost"":""43.214824ms"", ""yield_cost"":""516.89µs"", ""total_cost"":""46.40173ms"", ""num_rows"":6, ""num_batches"":6, ""num_mem_ranges"":0, ""num_file_ranges"":8, ""build_parts_cost"":""0ns"", ""sst_scan_cost"":""20.548207ms"", ""rg_total"":0, ""rows_before_filter"":0, ""num_sst_record_batches"":0, ""num_sst_batches"":6, ""num_sst_rows"":6, ""first_poll"":""2.597938ms"", ""convert_cost"":""260.576µs"", ""pruner_cache_hit"":13, ""pruner_cache_miss"":2, ""pruner_prune_cost"":""7.367085ms"", ""fetch_metrics"":{""total_fetch_elapsed"":""6.192851ms"", ""page_cache_hit"":9, ""cache_miss"":2, ""pages_to_fetch_mem"":231, ""page_size_to_fetch_mem"":2079421, ""pages_to_fetch_store"":24, ""page_size_to_fetch_store"":240753, ""page_size_needed"":2320174, ""store_fetch_elapsed"":""3.83384ms"", ""prefilter_cost"":""11.181089ms"", ""prefilter_filtered_rows"":366919}, ""dedup_metrics"":{""dedup_cost"":""42.673µs""}, ""top_file_metrics"": {""4518305595392(1052, 0)/9ee90964-0ab5-4401-9e62-3720abb6c20a"": {""build_part_cost"":""0ns"", ""num_ranges"":4, ""num_rows"":4, ""build_reader_cost"":""11.402974ms"", ""scan_cost"":""13.025376ms""}, ""4518305595392(1052, 0)/df7cd2ca-08bd-4e83-b4ea-c23b49fca916"": {""build_part_cost"":""0ns"", ""num_ranges"":4, ""num_rows"":2, ""build_reader_cost"":""242.768µs"", ""scan_cost"":""7.522831ms""}}, ""build_ranges_peak_mem_size"":0, ""num_peak_range_builders"":0, ""stream_eof"":true}}, {""partition"":12, ""metrics"":{""prepare_scan_cost"":""1.872851ms"", ""build_reader_cost"":""3.796474ms"", ""scan_cost"":""34.324161ms"", ""yield_cost"":""492.238µs"", ""total_cost"":""37.375018ms"", ""num_rows"":5, ""num_batches"":5, ""num_mem_ranges"":0, ""num_file_ranges"":6, ""build_parts_cost"":""0ns"", ""sst_scan_cost"":""19.504716ms"", ""rg_total"":0, ""rows_before_filter"":0, ""num_sst_record_batches"":0, ""num_sst_batches"":5, ""num_sst_rows"":5, ""first_poll"":""2.491012ms"", ""convert_cost"":""207.567µs"", ""pruner_cache_hit"":13, ""pruner_cache_miss"":2, ""pruner_prune_cost"":""7.583243ms"", ""fetch_metrics"":{""total_fetch_elapsed"":""4.199947ms"", ""page_cache_hit"":8, ""cache_miss"":1, ""pages_to_fetch_mem"":207, ""page_size_to_fetch_mem"":1391395, ""pages_to_fetch_store"":3, ""page_size_to_fetch_store"":180305, ""page_size_needed"":1571700, ""store_fetch_elapsed"":""2.904609ms"", ""prefilter_cost"":""3.322744ms"", ""prefilter_filtered_rows"":219137}, ""dedup_metrics"":{""dedup_cost"":""32.406µs""}, ""top_file_metrics"": {""4518305595392(1052, 0)/9ee90964-0ab5-4401-9e62-3720abb6c20a"": {""build_part_cost"":""0ns"", ""num_ranges"":4, ""num_rows"":3, ""build_reader_cost"":""3.641498ms"", ""scan_cost"":""12.868025ms""}, ""4518305595392(1052, 0)/df7cd2ca-08bd-4e83-b4ea-c23b49fca916"": {""build_part_cost"":""0ns"", ""num_ranges"":2, ""num_rows"":2, ""build_reader_cost"":""154.976µs"", ""scan_cost"":""6.636691ms""}}, ""build_ranges_peak_mem_size"":0, ""num_peak_range_builders"":0, ""stream_eof"":true}}, {""partition"":13, ""metrics"":{""prepare_scan_cost"":""1.925511ms"", ""build_reader_cost"":""10.555316ms"", ""scan_cost"":""43.143583ms"", ""yield_cost"":""653.92µs"", ""total_cost"":""46.40173ms"", ""num_rows"":7, ""num_batches"":7, ""num_mem_ranges"":0, ""num_file_ranges"":7, ""build_parts_cost"":""0ns"", ""sst_scan_cost"":""25.775021ms"", ""rg_total"":0, ""rows_before_filter"":0, ""num_sst_record_batches"":0, ""num_sst_batches"":7, ""num_sst_rows"":7, ""first_poll"":""2.522929ms"", ""convert_cost"":""321.895µs"", ""pruner_cache_hit"":13, ""pruner_cache_miss"":2, ""pruner_prune_cost"":""3.426528ms"", ""fetch_metrics"":{""total_fetch_elapsed"":""8.992489ms"", ""page_cache_hit"":10, ""cache_miss"":3, ""pages_to_fetch_mem"":252, ""page_size_to_fetch_mem"":1517544, ""pages_to_fetch_store"":11, ""page_size_to_fetch_store"":662122, ""page_size_needed"":2179666, ""store_fetch_elapsed"":""6.881483ms"", ""prefilter_cost"":""9.89197ms"", ""prefilter_filtered_rows"":215912}, ""dedup_metrics"":{""dedup_cost"":""49.166µs""}, ""top_file_metrics"": {""4518305595392(1052, 0)/9ee90964-0ab5-4401-9e62-3720abb6c20a"": {""build_part_cost"":""0ns"", ""num_ranges"":4, ""num_rows"":4, ""build_reader_cost"":""10.239776ms"", ""scan_cost"":""15.367976ms""}, ""4518305595392(1052, 0)/df7cd2ca-08bd-4e83-b4ea-c23b49fca916"": {""build_part_cost"":""0ns"", ""num_ranges"":3, ""num_rows"":3, ""build_reader_cost"":""315.54µs"", ""scan_cost"":""10.407045ms""}}, ""build_ranges_peak_mem_size"":0, ""num_peak_range_builders"":0, ""stream_eof"":true}}, {""partition"":14, ""metrics"":{""prepare_scan_cost"":""1.981733ms"", ""build_reader_cost"":""13.4535ms"", ""scan_cost"":""47.167743ms"", ""yield_cost"":""530.581µs"", ""total_cost"":""50.350878ms"", ""num_rows"":8, ""num_batches"":8, ""num_mem_ranges"":0, ""num_file_ranges"":9, ""build_parts_cost"":""0ns"", ""sst_scan_cost"":""24.441134ms"", ""rg_total"":0, ""rows_before_filter"":0, ""num_sst_record_batches"":0, ""num_sst_batches"":8, ""num_sst_rows"":8, ""first_poll"":""2.572167ms"", ""convert_cost"":""223.771µs"", ""pruner_cache_hit"":14, ""pruner_cache_miss"":1, ""pruner_prune_cost"":""3.962836ms"", ""fetch_metrics"":{""total_fetch_elapsed"":""6.560635ms"", ""page_cache_hit"":13, ""cache_miss"":4, ""pages_to_fetch_mem"":324, ""page_size_to_fetch_mem"":1703733, ""pages_to_fetch_store"":16, ""page_size_to_fetch_store"":983470, ""page_size_needed"":2687203, ""store_fetch_elapsed"":""3.844735ms"", ""prefilter_cost"":""11.973996ms"", ""prefilter_filtered_rows"":250729}, ""dedup_metrics"":{""dedup_cost"":""46.444µs""}, ""top_file_metrics"": {""4518305595392(1052, 0)/9ee90964-0ab5-4401-9e62-3720abb6c20a"": {""build_part_cost"":""0ns"", ""num_ranges"":5, ""num_rows"":4, ""build_reader_cost"":""12.716121ms"", ""scan_cost"":""12.412105ms""}, ""4518305595392(1052, 0)/df7cd2ca-08bd-4e83-b4ea-c23b49fca916"": {""build_part_cost"":""0ns"", ""num_ranges"":4, ""num_rows"":4, ""build_reader_cost"":""737.379µs"", ""scan_cost"":""12.029029ms""}}, ""build_ranges_peak_mem_size"":0, ""num_peak_range_builders"":0, ""stream_eof"":true}}, {""partition"":15, ""metrics"":{""prepare_scan_cost"":""2.04417ms"", ""build_reader_cost"":""13.562453ms"", ""scan_cost"":""41.067079ms"", ""yield_cost"":""615.018µs"", ""total_cost"":""44.404074ms"", ""num_rows"":6, ""num_batches"":6, ""num_mem_ranges"":0, ""num_file_ranges"":7, ""build_parts_cost"":""542.872µs"", ""sst_scan_cost"":""21.360107ms"", ""rg_total"":114, ""rows_before_filter"":11624888, ""num_sst_record_batches"":0, ""num_sst_batches"":6, ""num_sst_rows"":6, ""first_poll"":""2.608833ms"", ""convert_cost"":""237.878µs"", ""rg_minmax_filtered"":51, ""pruner_cache_hit"":13, ""pruner_cache_miss"":2, ""pruner_prune_cost"":""1.60969ms"", ""fetch_metrics"":{""total_fetch_elapsed"":""9.518181ms"", ""page_cache_hit"":9, ""cache_miss"":2, ""pages_to_fetch_mem"":256, ""page_size_to_fetch_mem"":1816322, ""pages_to_fetch_store"":6, ""page_size_to_fetch_store"":366650, ""page_size_needed"":2182972, ""store_fetch_elapsed"":""7.076198ms"", ""prefilter_cost"":""13.059808ms"", ""prefilter_filtered_rows"":191962}, ""metadata_cache_metrics"":{""metadata_load_cost"":""30.451µs"", ""mem_cache_hit"":2}, ""dedup_metrics"":{""dedup_cost"":""46.442µs""}, ""top_file_metrics"": {""4518305595392(1052, 0)/9ee90964-0ab5-4401-9e62-3720abb6c20a"": {""build_part_cost"":""542.872µs"", ""num_ranges"":3, ""num_rows"":3, ""build_reader_cost"":""13.149553ms"", ""scan_cost"":""9.683704ms""}, ""4518305595392(1052, 0)/df7cd2ca-08bd-4e83-b4ea-c23b49fca916"": {""build_part_cost"":""0ns"", ""num_ranges"":4, ""num_rows"":3, ""build_reader_cost"":""412.9µs"", ""scan_cost"":""11.676403ms""}}, ""build_ranges_peak_mem_size"":3459907, ""num_peak_range_builders"":1, ""stream_eof"":true}}]} metrics=[output_rows: 100, elapsed_compute: 691ms, mem_used: 31.8MiB, build_parts_cost: 902us, build_reader_cost: 169ms, convert_cost: 4.2ms, elapsed_await: 680ms, elapsed_poll: 529ms, scan_cost: 678ms, yield_cost: 9.2ms, ]
"
,,Total rows: 8076With this updated query, a 30d query shows the gap more clearly. InfluxDB: 4.52s |
All reactions
Yeah LastRow pushdown worked and reduce a lot of unnecessary scan. There is still some optimize to do for this query like range select pushdown&more projection prune, I will see if I can get them to merge recently |
Uh oh!
There was an error while loading. Please reload this page.
I migrated a 125GB InfluxDB 2 table into a 25GB GreptimeDB table. The disk and RAM savings are amazing, but query performance seems ~25%-5x slower than InfluxDB. @evenyag requested that I open a discussion to look at queries.
Here's the slowest Flux query:
I converted this into the following GreptimeDB SQL query:
When querying over the same 7d period, the InfluxDB query takes ~1s. The GreptimeDB query takes anywhere from 1.5s to 10s depending on how warm the cache is. Here's an
explain analyze verbosefor a cold-cache run:All reactions