task2 - Details for Query 2

Details for Query 2

Submitted Time: 2024/12/04 20:22:11
Duration: 9 s
Succeeded Jobs: 4

Show the Stage ID and Task ID that corresponds to the max metric

digraph G { 0 [labelType="html" label=" CollectLimit "]; subgraph cluster1 { isCluster="true"; label="WholeStageCodegen (3)\n \nduration: 0 ms"; 2 [labelType="html" label=" Project "]; } 3 [labelType="html" label="TakeOrderedAndProject shuffle records written: 1,804 shuffle write time total (min, med, max (stageId: taskId)) 59 ms (0 ms, 0 ms, 0 ms (stage 5.0: task 86)) records read: 1,804 local bytes read: 37.8 KiB fetch wait time: 2 ms remote bytes read: 70.4 KiB local blocks read: 71 remote blocks read: 129 shuffle bytes written total (min, med, max (stageId: taskId)) 108.1 KiB (225.0 B, 600.0 B, 624.0 B (stage 5.0: task 149))"]; subgraph cluster4 { isCluster="true"; label="WholeStageCodegen (2)\n \nduration: total (min, med, max (stageId: taskId))\n1.1 s (1 ms, 4 ms, 36 ms (stage 5.0: task 15))"; 5 [labelType="html" label="HashAggregate time in aggregation build total (min, med, max (stageId: taskId)) 196 ms (0 ms, 1 ms, 16 ms (stage 5.0: task 14)) peak memory total (min, med, max (stageId: taskId)) 12.5 GiB (64.3 MiB, 64.3 MiB, 64.3 MiB (stage 5.0: task 15)) number of output rows: 2,083 avg hash probe bucket list iters (min, med, max (stageId: taskId)): (1, 1, 1 (stage 5.0: task 15))"]; } 6 [labelType="html" label="Exchange shuffle records written: 3,237 shuffle write time total (min, med, max (stageId: taskId)) 153 ms (9 ms, 25 ms, 66 ms (stage 4.0: task 9)) records read: 3,237 local bytes read total (min, med, max (stageId: taskId)) 107.4 KiB (113.0 B, 511.0 B, 1192.0 B (stage 5.0: task 142)) fetch wait time total (min, med, max (stageId: taskId)) 26 ms (0 ms, 0 ms, 2 ms (stage 5.0: task 152)) remote bytes read total (min, med, max (stageId: taskId)) 114.6 KiB (0.0 B, 606.0 B, 1177.0 B (stage 5.0: task 170)) local blocks read: 438 remote blocks read: 402 data size total (min, med, max (stageId: taskId)) 227.6 KiB (8.9 KiB, 31.9 KiB, 113.8 KiB (stage 4.0: task 9)) shuffle bytes written total (min, med, max (stageId: taskId)) 222.0 KiB (12.2 KiB, 34.7 KiB, 97.9 KiB (stage 4.0: task 9))"]; subgraph cluster7 { isCluster="true"; label="WholeStageCodegen (1)\n \nduration: total (min, med, max (stageId: taskId))\n10.3 s (151 ms, 1.9 s, 3.4 s (stage 4.0: task 9))"; 8 [labelType="html" label="HashAggregate time in aggregation build total (min, med, max (stageId: taskId)) 9.8 s (128 ms, 1.8 s, 3.1 s (stage 4.0: task 9)) peak memory total (min, med, max (stageId: taskId)) 1280.0 KiB (256.0 KiB, 256.0 KiB, 256.0 KiB (stage 4.0: task 10)) number of output rows: 3,237"]; } 9 [labelType="html" label="Scan csv number of files read: 1 metadata time: 0 ms size of files read: 519.0 MiB number of output rows: 500,001"]; 2->0; 3->2; 5->3; 6->5; 8->6; 9->8; }

CollectLimit 21

Project [miner#25, cast(total_size#120L as string) AS total_size#126]

WholeStageCodegen (3)

TakeOrderedAndProject(limit=10, orderBy=[total_size#120L DESC NULLS LAST], output=[miner#25,total_size#120L])

HashAggregate(keys=[miner#25], functions=[sum(cast(size#28 as bigint))])

WholeStageCodegen (2)

Exchange hashpartitioning(miner#25, 200), true, [id=#67]

HashAggregate(keys=[miner#25], functions=[partial_sum(cast(size#28 as bigint))])

WholeStageCodegen (1)

FileScan csv [miner#25,size#28] Batched: false, DataFilters: [], Format: CSV, Location: InMemoryFileIndex[s3a://data-repository-bkt/ECS765/ethereum/blocks.csv], PartitionFilters: [], PushedFilters: [], ReadSchema: struct<miner:string,size:int>

Details

== Parsed Logical Plan ==
GlobalLimit 21
+- LocalLimit 21
   +- Project [cast(miner#25 as string) AS miner#125, cast(total_size#120L as string) AS total_size#126]
      +- GlobalLimit 10
         +- LocalLimit 10
            +- Sort [total_size#120L DESC NULLS LAST], true
               +- Aggregate [miner#25], [miner#25, sum(cast(size#28 as bigint)) AS total_size#120L]
                  +- Relation[number#16,hash#17,parent_hash#18,nonce#19,sha3_uncles#20,logs_bloom#21,transactions_root#22,state_root#23,receipts_root#24,miner#25,difficulty#26L,total_difficulty#27,size#28,extra_data#29,gas_limit#30,gas_used#31,timestamp#32,transaction_count#33,base_fee_per_gas#34] csv

== Analyzed Logical Plan ==
miner: string, total_size: string
GlobalLimit 21
+- LocalLimit 21
   +- Project [cast(miner#25 as string) AS miner#125, cast(total_size#120L as string) AS total_size#126]
      +- GlobalLimit 10
         +- LocalLimit 10
            +- Sort [total_size#120L DESC NULLS LAST], true
               +- Aggregate [miner#25], [miner#25, sum(cast(size#28 as bigint)) AS total_size#120L]
                  +- Relation[number#16,hash#17,parent_hash#18,nonce#19,sha3_uncles#20,logs_bloom#21,transactions_root#22,state_root#23,receipts_root#24,miner#25,difficulty#26L,total_difficulty#27,size#28,extra_data#29,gas_limit#30,gas_used#31,timestamp#32,transaction_count#33,base_fee_per_gas#34] csv

== Optimized Logical Plan ==
GlobalLimit 21
+- LocalLimit 21
   +- Project [miner#25, cast(total_size#120L as string) AS total_size#126]
      +- GlobalLimit 10
         +- LocalLimit 10
            +- Sort [total_size#120L DESC NULLS LAST], true
               +- Aggregate [miner#25], [miner#25, sum(cast(size#28 as bigint)) AS total_size#120L]
                  +- Project [miner#25, size#28]
                     +- Relation[number#16,hash#17,parent_hash#18,nonce#19,sha3_uncles#20,logs_bloom#21,transactions_root#22,state_root#23,receipts_root#24,miner#25,difficulty#26L,total_difficulty#27,size#28,extra_data#29,gas_limit#30,gas_used#31,timestamp#32,transaction_count#33,base_fee_per_gas#34] csv

== Physical Plan ==
CollectLimit 21
+- *(3) Project [miner#25, cast(total_size#120L as string) AS total_size#126]
   +- TakeOrderedAndProject(limit=10, orderBy=[total_size#120L DESC NULLS LAST], output=[miner#25,total_size#120L])
      +- *(2) HashAggregate(keys=[miner#25], functions=[sum(cast(size#28 as bigint))], output=[miner#25, total_size#120L])
         +- Exchange hashpartitioning(miner#25, 200), true, [id=#67]
            +- *(1) HashAggregate(keys=[miner#25], functions=[partial_sum(cast(size#28 as bigint))], output=[miner#25, sum#130L])
               +- FileScan csv [miner#25,size#28] Batched: false, DataFilters: [], Format: CSV, Location: InMemoryFileIndex[s3a://data-repository-bkt/ECS765/ethereum/blocks.csv], PartitionFilters: [], PushedFilters: [], ReadSchema: struct<miner:string,size:int>