Task3 - Details for Query 5

Details for Query 5

Submitted Time: 2025/04/10 23:50:57
Duration: 22 s
Succeeded Jobs: 31
Show the Stage ID and Task ID that corresponds to the max metric
Details
== Parsed Logical Plan ==
GlobalLimit 6
+- LocalLimit 6
   +- Project [cast(src#397 as string) AS src#409, cast(dst#398 as string) AS dst#410, cast(weight#399 as string) AS weight#411]
      +- Sort [src#397 DESC NULLS LAST], true
         +- Filter AtLeastNNulls(n, src#397,dst#398,weight#399)
            +- Project [Pickup Community Area#24 AS src#397, Dropoff Community Area#25 AS dst#398, cast(Trip Miles#21 as double) AS weight#399]
               +- Relation[Trip ID#16,Taxi ID#17,Trip Start Timestamp#18,Trip End Timestamp#19,Trip Seconds#20,Trip Miles#21,Pickup Census Tract#22L,Dropoff Census Tract#23L,Pickup Community Area#24,Dropoff Community Area#25,Fare#26,Tips#27,Tolls#28,Extras#29,Trip Total#30,Payment Type#31,Company#32,Pickup Centroid Latitude#33,Pickup Centroid Longitude#34,Pickup Centroid Location#35,Dropoff Centroid Latitude#36,Dropoff Centroid Longitude#37,Dropoff Centroid  Location#38] csv

== Analyzed Logical Plan ==
src: string, dst: string, weight: string
GlobalLimit 6
+- LocalLimit 6
   +- Project [cast(src#397 as string) AS src#409, cast(dst#398 as string) AS dst#410, cast(weight#399 as string) AS weight#411]
      +- Sort [src#397 DESC NULLS LAST], true
         +- Filter AtLeastNNulls(n, src#397,dst#398,weight#399)
            +- Project [Pickup Community Area#24 AS src#397, Dropoff Community Area#25 AS dst#398, cast(Trip Miles#21 as double) AS weight#399]
               +- Relation[Trip ID#16,Taxi ID#17,Trip Start Timestamp#18,Trip End Timestamp#19,Trip Seconds#20,Trip Miles#21,Pickup Census Tract#22L,Dropoff Census Tract#23L,Pickup Community Area#24,Dropoff Community Area#25,Fare#26,Tips#27,Tolls#28,Extras#29,Trip Total#30,Payment Type#31,Company#32,Pickup Centroid Latitude#33,Pickup Centroid Longitude#34,Pickup Centroid Location#35,Dropoff Centroid Latitude#36,Dropoff Centroid Longitude#37,Dropoff Centroid  Location#38] csv

== Optimized Logical Plan ==
GlobalLimit 6
+- LocalLimit 6
   +- Project [cast(src#397 as string) AS src#409, cast(dst#398 as string) AS dst#410, cast(weight#399 as string) AS weight#411]
      +- Sort [src#397 DESC NULLS LAST], true
         +- Project [Pickup Community Area#24 AS src#397, Dropoff Community Area#25 AS dst#398, Trip Miles#21 AS weight#399]
            +- Filter AtLeastNNulls(n, Pickup Community Area#24,Dropoff Community Area#25,Trip Miles#21)
               +- Relation[Trip ID#16,Taxi ID#17,Trip Start Timestamp#18,Trip End Timestamp#19,Trip Seconds#20,Trip Miles#21,Pickup Census Tract#22L,Dropoff Census Tract#23L,Pickup Community Area#24,Dropoff Community Area#25,Fare#26,Tips#27,Tolls#28,Extras#29,Trip Total#30,Payment Type#31,Company#32,Pickup Centroid Latitude#33,Pickup Centroid Longitude#34,Pickup Centroid Location#35,Dropoff Centroid Latitude#36,Dropoff Centroid Longitude#37,Dropoff Centroid  Location#38] csv

== Physical Plan ==
TakeOrderedAndProject(limit=6, orderBy=[src#397 DESC NULLS LAST], output=[src#409,dst#410,weight#411])
+- *(1) Project [Pickup Community Area#24 AS src#397, Dropoff Community Area#25 AS dst#398, Trip Miles#21 AS weight#399]
   +- *(1) Filter AtLeastNNulls(n, Pickup Community Area#24,Dropoff Community Area#25,Trip Miles#21)
      +- FileScan csv [Trip Miles#21,Pickup Community Area#24,Dropoff Community Area#25] Batched: false, DataFilters: [AtLeastNNulls(n, Pickup Community Area#24,Dropoff Community Area#25,Trip Miles#21)], Format: CSV, Location: InMemoryFileIndex[s3a://data-repository-bkt/ECS765/Chicago_Taxitrips/chicago_taxi_trips.csv], PartitionFilters: [], PushedFilters: [], ReadSchema: struct<Trip Miles:double,Pickup Community Area:int,Dropoff Community Area:int>