task2 - Details for Query 2

Details for Query 2

Submitted Time: 2024/11/28 21:54:57
Duration: 5 s
Succeeded Jobs: 4

Show the Stage ID and Task ID that corresponds to the max metric

digraph G { 0 [labelType="html" label=" TakeOrderedAndProject "]; subgraph cluster1 { isCluster="true"; label="WholeStageCodegen (2)\n \nduration: total (min, med, max (stageId: taskId))\n571 ms (1 ms, 2 ms, 18 ms (stage 5.0: task 14))"; 2 [labelType="html" label="HashAggregate time in aggregation build total (min, med, max (stageId: taskId)) 157 ms (0 ms, 1 ms, 8 ms (stage 5.0: task 14)) peak memory total (min, med, max (stageId: taskId)) 12.5 GiB (64.3 MiB, 64.3 MiB, 64.3 MiB (stage 5.0: task 14)) number of output rows: 2,083 avg hash probe bucket list iters (min, med, max (stageId: taskId)): (1, 1, 1 (stage 5.0: task 14))"]; } 3 [labelType="html" label="Exchange shuffle records written: 3,237 shuffle write time total (min, med, max (stageId: taskId)) 135 ms (9 ms, 25 ms, 43 ms (stage 4.0: task 10)) records read: 3,237 local bytes read total (min, med, max (stageId: taskId)) 113.2 KiB (112.0 B, 563.0 B, 1192.0 B (stage 5.0: task 142)) fetch wait time total (min, med, max (stageId: taskId)) 22 ms (0 ms, 0 ms, 3 ms (stage 5.0: task 112)) remote bytes read total (min, med, max (stageId: taskId)) 108.8 KiB (0.0 B, 559.0 B, 1103.0 B (stage 5.0: task 180)) local blocks read: 430 remote blocks read: 410 data size total (min, med, max (stageId: taskId)) 227.6 KiB (8.9 KiB, 31.9 KiB, 113.8 KiB (stage 4.0: task 9)) shuffle bytes written total (min, med, max (stageId: taskId)) 222.0 KiB (12.2 KiB, 34.7 KiB, 97.9 KiB (stage 4.0: task 9))"]; subgraph cluster4 { isCluster="true"; label="WholeStageCodegen (1)\n \nduration: total (min, med, max (stageId: taskId))\n6.7 s (165 ms, 1.6 s, 1.7 s (stage 4.0: task 9))"; 5 [labelType="html" label="HashAggregate time in aggregation build total (min, med, max (stageId: taskId)) 6.3 s (145 ms, 1.5 s, 1.6 s (stage 4.0: task 12)) peak memory total (min, med, max (stageId: taskId)) 1280.0 KiB (256.0 KiB, 256.0 KiB, 256.0 KiB (stage 4.0: task 10)) number of output rows: 3,237"]; } 6 [labelType="html" label="Scan csv number of files read: 1 metadata time: 0 ms size of files read: 519.0 MiB number of output rows: 500,001"]; 2->0; 3->2; 5->3; 6->5; }

TakeOrderedAndProject(limit=5, orderBy=[sum(size)#120L DESC NULLS LAST], output=[miner#25,sum(size)#126])

HashAggregate(keys=[miner#25], functions=[sum(cast(size#28 as bigint))])

WholeStageCodegen (2)

Exchange hashpartitioning(miner#25, 200), true, [id=#57]

HashAggregate(keys=[miner#25], functions=[partial_sum(cast(size#28 as bigint))])

WholeStageCodegen (1)

FileScan csv [miner#25,size#28] Batched: false, DataFilters: [], Format: CSV, Location: InMemoryFileIndex[s3a://data-repository-bkt/ECS765/ethereum/blocks.csv], PartitionFilters: [], PushedFilters: [], ReadSchema: struct<miner:string,size:int>

Details

== Parsed Logical Plan ==
GlobalLimit 5
+- LocalLimit 5
   +- Project [cast(miner#25 as string) AS miner#125, cast(sum(size)#120L as string) AS sum(size)#126]
      +- Sort [sum(size)#120L DESC NULLS LAST], true
         +- Aggregate [miner#25], [miner#25, sum(cast(size#28 as bigint)) AS sum(size)#120L]
            +- Relation[number#16,hash#17,parent_hash#18,nonce#19,sha3_uncles#20,logs_bloom#21,transactions_root#22,state_root#23,receipts_root#24,miner#25,difficulty#26L,total_difficulty#27,size#28,extra_data#29,gas_limit#30,gas_used#31,timestamp#32,transaction_count#33,base_fee_per_gas#34] csv

== Analyzed Logical Plan ==
miner: string, sum(size): string
GlobalLimit 5
+- LocalLimit 5
   +- Project [cast(miner#25 as string) AS miner#125, cast(sum(size)#120L as string) AS sum(size)#126]
      +- Sort [sum(size)#120L DESC NULLS LAST], true
         +- Aggregate [miner#25], [miner#25, sum(cast(size#28 as bigint)) AS sum(size)#120L]
            +- Relation[number#16,hash#17,parent_hash#18,nonce#19,sha3_uncles#20,logs_bloom#21,transactions_root#22,state_root#23,receipts_root#24,miner#25,difficulty#26L,total_difficulty#27,size#28,extra_data#29,gas_limit#30,gas_used#31,timestamp#32,transaction_count#33,base_fee_per_gas#34] csv

== Optimized Logical Plan ==
GlobalLimit 5
+- LocalLimit 5
   +- Project [miner#25, cast(sum(size)#120L as string) AS sum(size)#126]
      +- Sort [sum(size)#120L DESC NULLS LAST], true
         +- Aggregate [miner#25], [miner#25, sum(cast(size#28 as bigint)) AS sum(size)#120L]
            +- Project [miner#25, size#28]
               +- Relation[number#16,hash#17,parent_hash#18,nonce#19,sha3_uncles#20,logs_bloom#21,transactions_root#22,state_root#23,receipts_root#24,miner#25,difficulty#26L,total_difficulty#27,size#28,extra_data#29,gas_limit#30,gas_used#31,timestamp#32,transaction_count#33,base_fee_per_gas#34] csv

== Physical Plan ==
TakeOrderedAndProject(limit=5, orderBy=[sum(size)#120L DESC NULLS LAST], output=[miner#25,sum(size)#126])
+- *(2) HashAggregate(keys=[miner#25], functions=[sum(cast(size#28 as bigint))], output=[miner#25, sum(size)#120L])
   +- Exchange hashpartitioning(miner#25, 200), true, [id=#57]
      +- *(1) HashAggregate(keys=[miner#25], functions=[partial_sum(cast(size#28 as bigint))], output=[miner#25, sum#130L])
         +- FileScan csv [miner#25,size#28] Batched: false, DataFilters: [], Format: CSV, Location: InMemoryFileIndex[s3a://data-repository-bkt/ECS765/ethereum/blocks.csv], PartitionFilters: [], PushedFilters: [], ReadSchema: struct<miner:string,size:int>