task2 - Details for Query 2

Details for Query 2

Submitted Time: 2024/10/29 20:22:52
Duration: 7 s
Succeeded Jobs: 2

Show the Stage ID and Task ID that corresponds to the max metric

digraph G { 0 [labelType="html" label=" TakeOrderedAndProject "]; subgraph cluster1 { isCluster="true"; label="WholeStageCodegen (2)\n \nduration: total (min, med, max (stageId: taskId))\n719 ms (1 ms, 3 ms, 34 ms (stage 3.0: task 8))"; 2 [labelType="html" label="HashAggregate time in aggregation build total (min, med, max (stageId: taskId)) 48 ms (0 ms, 0 ms, 8 ms (stage 3.0: task 7)) peak memory total (min, med, max (stageId: taskId)) 12.5 GiB (64.3 MiB, 64.3 MiB, 64.3 MiB (stage 3.0: task 9)) number of output rows: 2,083 avg hash probe bucket list iters (min, med, max (stageId: taskId)): (1, 1, 1 (stage 3.0: task 9))"]; } 3 [labelType="html" label="Exchange shuffle records written: 3,237 shuffle write time total (min, med, max (stageId: taskId)) 106 ms (10 ms, 17 ms, 34 ms (stage 2.0: task 2)) records read: 3,237 local bytes read total (min, med, max (stageId: taskId)) 111.0 KiB (165.0 B, 562.0 B, 1103.0 B (stage 3.0: task 157)) fetch wait time total (min, med, max (stageId: taskId)) 7 ms (0 ms, 0 ms, 5 ms (stage 3.0: task 77)) remote bytes read total (min, med, max (stageId: taskId)) 104.4 KiB (0.0 B, 525.0 B, 1152.0 B (stage 3.0: task 163)) local blocks read: 448 remote blocks read: 392 data size total (min, med, max (stageId: taskId)) 227.6 KiB (8.9 KiB, 31.9 KiB, 113.8 KiB (stage 2.0: task 2)) shuffle bytes written total (min, med, max (stageId: taskId)) 215.4 KiB (12.0 KiB, 33.8 KiB, 94.6 KiB (stage 2.0: task 2))"]; subgraph cluster4 { isCluster="true"; label="WholeStageCodegen (1)\n \nduration: total (min, med, max (stageId: taskId))\n9.1 s (181 ms, 1.6 s, 3.6 s (stage 2.0: task 2))"; 5 [labelType="html" label="HashAggregate time in aggregation build total (min, med, max (stageId: taskId)) 8.7 s (157 ms, 1.6 s, 3.4 s (stage 2.0: task 2)) peak memory total (min, med, max (stageId: taskId)) 1280.0 KiB (256.0 KiB, 256.0 KiB, 256.0 KiB (stage 2.0: task 3)) number of output rows: 3,237"]; } 6 [labelType="html" label="Scan csv number of files read: 1 metadata time: 0 ms size of files read: 519.0 MiB number of output rows: 500,001"]; 2->0; 3->2; 5->3; 6->5; }

TakeOrderedAndProject(limit=11, orderBy=[total_size#120 DESC NULLS LAST], output=[miner#25,total_size#126])

HashAggregate(keys=[miner#25], functions=[sum(cast(size#28 as double))])

WholeStageCodegen (2)

Exchange hashpartitioning(miner#25, 200), true, [id=#57]

HashAggregate(keys=[miner#25], functions=[partial_sum(cast(size#28 as double))])

WholeStageCodegen (1)

FileScan csv [miner#25,size#28] Batched: false, DataFilters: [], Format: CSV, Location: InMemoryFileIndex[s3a://data-repository-bkt/ECS765/ethereum/blocks.csv], PartitionFilters: [], PushedFilters: [], ReadSchema: struct<miner:string,size:string>

Details

== Parsed Logical Plan ==
GlobalLimit 11
+- LocalLimit 11
   +- Project [cast(miner#25 as string) AS miner#125, cast(total_size#120 as string) AS total_size#126]
      +- Sort [total_size#120 DESC NULLS LAST], true
         +- Aggregate [miner#25], [miner#25, sum(cast(size#28 as double)) AS total_size#120]
            +- Relation[number#16,hash#17,parent_hash#18,nonce#19,sha3_uncles#20,logs_bloom#21,transactions_root#22,state_root#23,receipts_root#24,miner#25,difficulty#26,total_difficulty#27,size#28,extra_data#29,gas_limit#30,gas_used#31,timestamp#32,transaction_count#33,base_fee_per_gas#34] csv

== Analyzed Logical Plan ==
miner: string, total_size: string
GlobalLimit 11
+- LocalLimit 11
   +- Project [cast(miner#25 as string) AS miner#125, cast(total_size#120 as string) AS total_size#126]
      +- Sort [total_size#120 DESC NULLS LAST], true
         +- Aggregate [miner#25], [miner#25, sum(cast(size#28 as double)) AS total_size#120]
            +- Relation[number#16,hash#17,parent_hash#18,nonce#19,sha3_uncles#20,logs_bloom#21,transactions_root#22,state_root#23,receipts_root#24,miner#25,difficulty#26,total_difficulty#27,size#28,extra_data#29,gas_limit#30,gas_used#31,timestamp#32,transaction_count#33,base_fee_per_gas#34] csv

== Optimized Logical Plan ==
GlobalLimit 11
+- LocalLimit 11
   +- Project [miner#25, cast(total_size#120 as string) AS total_size#126]
      +- Sort [total_size#120 DESC NULLS LAST], true
         +- Aggregate [miner#25], [miner#25, sum(cast(size#28 as double)) AS total_size#120]
            +- Project [miner#25, size#28]
               +- Relation[number#16,hash#17,parent_hash#18,nonce#19,sha3_uncles#20,logs_bloom#21,transactions_root#22,state_root#23,receipts_root#24,miner#25,difficulty#26,total_difficulty#27,size#28,extra_data#29,gas_limit#30,gas_used#31,timestamp#32,transaction_count#33,base_fee_per_gas#34] csv

== Physical Plan ==
TakeOrderedAndProject(limit=11, orderBy=[total_size#120 DESC NULLS LAST], output=[miner#25,total_size#126])
+- *(2) HashAggregate(keys=[miner#25], functions=[sum(cast(size#28 as double))], output=[miner#25, total_size#120])
   +- Exchange hashpartitioning(miner#25, 200), true, [id=#57]
      +- *(1) HashAggregate(keys=[miner#25], functions=[partial_sum(cast(size#28 as double))], output=[miner#25, sum#130])
         +- FileScan csv [miner#25,size#28] Batched: false, DataFilters: [], Format: CSV, Location: InMemoryFileIndex[s3a://data-repository-bkt/ECS765/ethereum/blocks.csv], PartitionFilters: [], PushedFilters: [], ReadSchema: struct<miner:string,size:string>