GlobalAirportsAnalysis - Details for Query 2

Details for Query 2

Submitted Time: 2025/02/17 15:09:41
Duration: 0.5 s
Succeeded Jobs: 4

Show the Stage ID and Task ID that corresponds to the max metric

digraph G { 0 [labelType="html" label=" CollectLimit "]; subgraph cluster1 { isCluster="true"; label="WholeStageCodegen (1)"; 2 [labelType="html" label=" Project "]; } 3 [labelType="html" label="Scan csv number of files read: 1 metadata time: 0 ms size of files read: 2.7 GiB number of output rows: 21"]; 2->0; 3->2; }

CollectLimit 21

Project [Trip ID#78, Taxi ID#79, cast(CASE WHEN isnull(gettimestamp(Trip Start Timestamp#80, MM/dd/yyyy hh:mm:ss a, Some(GMT))) THEN gettimestamp(gettimestamp(Trip Start Timestamp#80, MM/dd/yyyy hh:mm:ss a, Some(GMT)), MM/dd/yyyy HH:mm, Some(GMT)) ELSE gettimestamp(Trip Start Timestamp#80, MM/dd/yyyy hh:mm:ss a, Some(GMT)) END as string) AS Trip Start Timestamp#244, Trip End Timestamp#81, cast(Trip Seconds#82 as string) AS Trip Seconds#224, cast(Trip Miles#83 as string) AS Trip Miles#225, cast(Pickup Census Tract#84L as string) AS Pickup Census Tract#226, cast(Dropoff Census Tract#85L as string) AS Dropoff Census Tract#227, cast(Pickup Community Area#86 as string) AS Pickup Community Area#228, cast(Dropoff Community Area#87 as string) AS Dropoff Community Area#229, cast(Fare#88 as string) AS Fare#230, cast(Tips#89 as string) AS Tips#231, cast(Tolls#90 as string) AS Tolls#232, cast(Extras#91 as string) AS Extras#233, cast(Trip Total#92 as string) AS Trip Total#234, Payment Type#93, Company#94, cast(Pickup Centroid Latitude#95 as string) AS Pickup Centroid Latitude#237, cast(Pickup Centroid Longitude#96 as string) AS Pickup Centroid Longitude#238, Pickup Centroid Location#97, cast(Dropoff Centroid Latitude#98 as string) AS Dropoff Centroid Latitude#240, cast(Dropoff Centroid Longitude#99 as string) AS Dropoff Centroid Longitude#241, Dropoff Centroid Location#100, cast(hour(CASE WHEN isnull(gettimestamp(Trip Start Timestamp#80, MM/dd/yyyy hh:mm:ss a, Some(GMT))) THEN gettimestamp(gettimestamp(Trip Start Timestamp#80, MM/dd/yyyy hh:mm:ss a, Some(GMT)), MM/dd/yyyy HH:mm, Some(GMT)) ELSE gettimestamp(Trip Start Timestamp#80, MM/dd/yyyy hh:mm:ss a, Some(GMT)) END, Some(GMT)) as string) AS Start Hour#243]

WholeStageCodegen (1)

FileScan csv [Trip ID#78,Taxi ID#79,Trip Start Timestamp#80,Trip End Timestamp#81,Trip Seconds#82,Trip Miles#83,Pickup Census Tract#84L,Dropoff Census Tract#85L,Pickup Community Area#86,Dropoff Community Area#87,Fare#88,Tips#89,Tolls#90,Extras#91,Trip Total#92,Payment Type#93,Company#94,Pickup Centroid Latitude#95,Pickup Centroid Longitude#96,Pickup Centroid Location#97,Dropoff Centroid Latitude#98,Dropoff Centroid Longitude#99,Dropoff Centroid Location#100] Batched: false, DataFilters: [], Format: CSV, Location: InMemoryFileIndex[s3a://data-repository-bkt/ECS765/Chicago_Taxitrips/chicago_taxi_trips.csv], PartitionFilters: [], PushedFilters: [], ReadSchema: struct<Trip ID:string,Taxi ID:string,Trip Start Timestamp:string,Trip End Timestamp:string,Trip S...

Details

== Parsed Logical Plan ==
GlobalLimit 21
+- LocalLimit 21
   +- Project [cast(Trip ID#78 as string) AS Trip ID#221, cast(Taxi ID#79 as string) AS Taxi ID#222, cast(Trip Start Timestamp#148 as string) AS Trip Start Timestamp#244, cast(Trip End Timestamp#81 as string) AS Trip End Timestamp#223, cast(Trip Seconds#82 as string) AS Trip Seconds#224, cast(Trip Miles#83 as string) AS Trip Miles#225, cast(Pickup Census Tract#84L as string) AS Pickup Census Tract#226, cast(Dropoff Census Tract#85L as string) AS Dropoff Census Tract#227, cast(Pickup Community Area#86 as string) AS Pickup Community Area#228, cast(Dropoff Community Area#87 as string) AS Dropoff Community Area#229, cast(Fare#88 as string) AS Fare#230, cast(Tips#89 as string) AS Tips#231, cast(Tolls#90 as string) AS Tolls#232, cast(Extras#91 as string) AS Extras#233, cast(Trip Total#92 as string) AS Trip Total#234, cast(Payment Type#93 as string) AS Payment Type#235, cast(Company#94 as string) AS Company#236, cast(Pickup Centroid Latitude#95 as string) AS Pickup Centroid Latitude#237, cast(Pickup Centroid Longitude#96 as string) AS Pickup Centroid Longitude#238, cast(Pickup Centroid Location#97 as string) AS Pickup Centroid Location#239, cast(Dropoff Centroid Latitude#98 as string) AS Dropoff Centroid Latitude#240, cast(Dropoff Centroid Longitude#99 as string) AS Dropoff Centroid Longitude#241, cast(Dropoff Centroid  Location#100 as string) AS Dropoff Centroid  Location#242, cast(Start Hour#172 as string) AS Start Hour#243]
      +- Project [Trip ID#78, Taxi ID#79, Trip Start Timestamp#148, Trip End Timestamp#81, Trip Seconds#82, Trip Miles#83, Pickup Census Tract#84L, Dropoff Census Tract#85L, Pickup Community Area#86, Dropoff Community Area#87, Fare#88, Tips#89, Tolls#90, Extras#91, Trip Total#92, Payment Type#93, Company#94, Pickup Centroid Latitude#95, Pickup Centroid Longitude#96, Pickup Centroid Location#97, Dropoff Centroid Latitude#98, Dropoff Centroid Longitude#99, Dropoff Centroid  Location#100, hour(Trip Start Timestamp#148, Some(GMT)) AS Start Hour#172]
         +- Project [Trip ID#78, Taxi ID#79, CASE WHEN isnull(Trip Start Timestamp#124) THEN to_timestamp('Trip Start Timestamp, Some(MM/dd/yyyy HH:mm)) ELSE Trip Start Timestamp#124 END AS Trip Start Timestamp#148, Trip End Timestamp#81, Trip Seconds#82, Trip Miles#83, Pickup Census Tract#84L, Dropoff Census Tract#85L, Pickup Community Area#86, Dropoff Community Area#87, Fare#88, Tips#89, Tolls#90, Extras#91, Trip Total#92, Payment Type#93, Company#94, Pickup Centroid Latitude#95, Pickup Centroid Longitude#96, Pickup Centroid Location#97, Dropoff Centroid Latitude#98, Dropoff Centroid Longitude#99, Dropoff Centroid  Location#100]
            +- Project [Trip ID#78, Taxi ID#79, to_timestamp('Trip Start Timestamp, Some(MM/dd/yyyy hh:mm:ss a)) AS Trip Start Timestamp#124, Trip End Timestamp#81, Trip Seconds#82, Trip Miles#83, Pickup Census Tract#84L, Dropoff Census Tract#85L, Pickup Community Area#86, Dropoff Community Area#87, Fare#88, Tips#89, Tolls#90, Extras#91, Trip Total#92, Payment Type#93, Company#94, Pickup Centroid Latitude#95, Pickup Centroid Longitude#96, Pickup Centroid Location#97, Dropoff Centroid Latitude#98, Dropoff Centroid Longitude#99, Dropoff Centroid  Location#100]
               +- Relation[Trip ID#78,Taxi ID#79,Trip Start Timestamp#80,Trip End Timestamp#81,Trip Seconds#82,Trip Miles#83,Pickup Census Tract#84L,Dropoff Census Tract#85L,Pickup Community Area#86,Dropoff Community Area#87,Fare#88,Tips#89,Tolls#90,Extras#91,Trip Total#92,Payment Type#93,Company#94,Pickup Centroid Latitude#95,Pickup Centroid Longitude#96,Pickup Centroid Location#97,Dropoff Centroid Latitude#98,Dropoff Centroid Longitude#99,Dropoff Centroid  Location#100] csv

== Analyzed Logical Plan ==
Trip ID: string, Taxi ID: string, Trip Start Timestamp: string, Trip End Timestamp: string, Trip Seconds: string, Trip Miles: string, Pickup Census Tract: string, Dropoff Census Tract: string, Pickup Community Area: string, Dropoff Community Area: string, Fare: string, Tips: string, Tolls: string, Extras: string, Trip Total: string, Payment Type: string, Company: string, Pickup Centroid Latitude: string, Pickup Centroid Longitude: string, Pickup Centroid Location: string, Dropoff Centroid Latitude: string, Dropoff Centroid Longitude: string, Dropoff Centroid  Location: string, Start Hour: string
GlobalLimit 21
+- LocalLimit 21
   +- Project [cast(Trip ID#78 as string) AS Trip ID#221, cast(Taxi ID#79 as string) AS Taxi ID#222, cast(Trip Start Timestamp#148 as string) AS Trip Start Timestamp#244, cast(Trip End Timestamp#81 as string) AS Trip End Timestamp#223, cast(Trip Seconds#82 as string) AS Trip Seconds#224, cast(Trip Miles#83 as string) AS Trip Miles#225, cast(Pickup Census Tract#84L as string) AS Pickup Census Tract#226, cast(Dropoff Census Tract#85L as string) AS Dropoff Census Tract#227, cast(Pickup Community Area#86 as string) AS Pickup Community Area#228, cast(Dropoff Community Area#87 as string) AS Dropoff Community Area#229, cast(Fare#88 as string) AS Fare#230, cast(Tips#89 as string) AS Tips#231, cast(Tolls#90 as string) AS Tolls#232, cast(Extras#91 as string) AS Extras#233, cast(Trip Total#92 as string) AS Trip Total#234, cast(Payment Type#93 as string) AS Payment Type#235, cast(Company#94 as string) AS Company#236, cast(Pickup Centroid Latitude#95 as string) AS Pickup Centroid Latitude#237, cast(Pickup Centroid Longitude#96 as string) AS Pickup Centroid Longitude#238, cast(Pickup Centroid Location#97 as string) AS Pickup Centroid Location#239, cast(Dropoff Centroid Latitude#98 as string) AS Dropoff Centroid Latitude#240, cast(Dropoff Centroid Longitude#99 as string) AS Dropoff Centroid Longitude#241, cast(Dropoff Centroid  Location#100 as string) AS Dropoff Centroid  Location#242, cast(Start Hour#172 as string) AS Start Hour#243]
      +- Project [Trip ID#78, Taxi ID#79, Trip Start Timestamp#148, Trip End Timestamp#81, Trip Seconds#82, Trip Miles#83, Pickup Census Tract#84L, Dropoff Census Tract#85L, Pickup Community Area#86, Dropoff Community Area#87, Fare#88, Tips#89, Tolls#90, Extras#91, Trip Total#92, Payment Type#93, Company#94, Pickup Centroid Latitude#95, Pickup Centroid Longitude#96, Pickup Centroid Location#97, Dropoff Centroid Latitude#98, Dropoff Centroid Longitude#99, Dropoff Centroid  Location#100, hour(Trip Start Timestamp#148, Some(GMT)) AS Start Hour#172]
         +- Project [Trip ID#78, Taxi ID#79, CASE WHEN isnull(Trip Start Timestamp#124) THEN to_timestamp('Trip Start Timestamp, Some(MM/dd/yyyy HH:mm)) ELSE Trip Start Timestamp#124 END AS Trip Start Timestamp#148, Trip End Timestamp#81, Trip Seconds#82, Trip Miles#83, Pickup Census Tract#84L, Dropoff Census Tract#85L, Pickup Community Area#86, Dropoff Community Area#87, Fare#88, Tips#89, Tolls#90, Extras#91, Trip Total#92, Payment Type#93, Company#94, Pickup Centroid Latitude#95, Pickup Centroid Longitude#96, Pickup Centroid Location#97, Dropoff Centroid Latitude#98, Dropoff Centroid Longitude#99, Dropoff Centroid  Location#100]
            +- Project [Trip ID#78, Taxi ID#79, to_timestamp('Trip Start Timestamp, Some(MM/dd/yyyy hh:mm:ss a)) AS Trip Start Timestamp#124, Trip End Timestamp#81, Trip Seconds#82, Trip Miles#83, Pickup Census Tract#84L, Dropoff Census Tract#85L, Pickup Community Area#86, Dropoff Community Area#87, Fare#88, Tips#89, Tolls#90, Extras#91, Trip Total#92, Payment Type#93, Company#94, Pickup Centroid Latitude#95, Pickup Centroid Longitude#96, Pickup Centroid Location#97, Dropoff Centroid Latitude#98, Dropoff Centroid Longitude#99, Dropoff Centroid  Location#100]
               +- Relation[Trip ID#78,Taxi ID#79,Trip Start Timestamp#80,Trip End Timestamp#81,Trip Seconds#82,Trip Miles#83,Pickup Census Tract#84L,Dropoff Census Tract#85L,Pickup Community Area#86,Dropoff Community Area#87,Fare#88,Tips#89,Tolls#90,Extras#91,Trip Total#92,Payment Type#93,Company#94,Pickup Centroid Latitude#95,Pickup Centroid Longitude#96,Pickup Centroid Location#97,Dropoff Centroid Latitude#98,Dropoff Centroid Longitude#99,Dropoff Centroid  Location#100] csv

== Optimized Logical Plan ==
GlobalLimit 21
+- LocalLimit 21
   +- Project [Trip ID#78, Taxi ID#79, cast(CASE WHEN isnull(gettimestamp(Trip Start Timestamp#80, MM/dd/yyyy hh:mm:ss a, Some(GMT))) THEN gettimestamp(gettimestamp(Trip Start Timestamp#80, MM/dd/yyyy hh:mm:ss a, Some(GMT)), MM/dd/yyyy HH:mm, Some(GMT)) ELSE gettimestamp(Trip Start Timestamp#80, MM/dd/yyyy hh:mm:ss a, Some(GMT)) END as string) AS Trip Start Timestamp#244, Trip End Timestamp#81, cast(Trip Seconds#82 as string) AS Trip Seconds#224, cast(Trip Miles#83 as string) AS Trip Miles#225, cast(Pickup Census Tract#84L as string) AS Pickup Census Tract#226, cast(Dropoff Census Tract#85L as string) AS Dropoff Census Tract#227, cast(Pickup Community Area#86 as string) AS Pickup Community Area#228, cast(Dropoff Community Area#87 as string) AS Dropoff Community Area#229, cast(Fare#88 as string) AS Fare#230, cast(Tips#89 as string) AS Tips#231, cast(Tolls#90 as string) AS Tolls#232, cast(Extras#91 as string) AS Extras#233, cast(Trip Total#92 as string) AS Trip Total#234, Payment Type#93, Company#94, cast(Pickup Centroid Latitude#95 as string) AS Pickup Centroid Latitude#237, cast(Pickup Centroid Longitude#96 as string) AS Pickup Centroid Longitude#238, Pickup Centroid Location#97, cast(Dropoff Centroid Latitude#98 as string) AS Dropoff Centroid Latitude#240, cast(Dropoff Centroid Longitude#99 as string) AS Dropoff Centroid Longitude#241, Dropoff Centroid  Location#100, cast(hour(CASE WHEN isnull(gettimestamp(Trip Start Timestamp#80, MM/dd/yyyy hh:mm:ss a, Some(GMT))) THEN gettimestamp(gettimestamp(Trip Start Timestamp#80, MM/dd/yyyy hh:mm:ss a, Some(GMT)), MM/dd/yyyy HH:mm, Some(GMT)) ELSE gettimestamp(Trip Start Timestamp#80, MM/dd/yyyy hh:mm:ss a, Some(GMT)) END, Some(GMT)) as string) AS Start Hour#243]
      +- Relation[Trip ID#78,Taxi ID#79,Trip Start Timestamp#80,Trip End Timestamp#81,Trip Seconds#82,Trip Miles#83,Pickup Census Tract#84L,Dropoff Census Tract#85L,Pickup Community Area#86,Dropoff Community Area#87,Fare#88,Tips#89,Tolls#90,Extras#91,Trip Total#92,Payment Type#93,Company#94,Pickup Centroid Latitude#95,Pickup Centroid Longitude#96,Pickup Centroid Location#97,Dropoff Centroid Latitude#98,Dropoff Centroid Longitude#99,Dropoff Centroid  Location#100] csv

== Physical Plan ==
CollectLimit 21
+- *(1) Project [Trip ID#78, Taxi ID#79, cast(CASE WHEN isnull(gettimestamp(Trip Start Timestamp#80, MM/dd/yyyy hh:mm:ss a, Some(GMT))) THEN gettimestamp(gettimestamp(Trip Start Timestamp#80, MM/dd/yyyy hh:mm:ss a, Some(GMT)), MM/dd/yyyy HH:mm, Some(GMT)) ELSE gettimestamp(Trip Start Timestamp#80, MM/dd/yyyy hh:mm:ss a, Some(GMT)) END as string) AS Trip Start Timestamp#244, Trip End Timestamp#81, cast(Trip Seconds#82 as string) AS Trip Seconds#224, cast(Trip Miles#83 as string) AS Trip Miles#225, cast(Pickup Census Tract#84L as string) AS Pickup Census Tract#226, cast(Dropoff Census Tract#85L as string) AS Dropoff Census Tract#227, cast(Pickup Community Area#86 as string) AS Pickup Community Area#228, cast(Dropoff Community Area#87 as string) AS Dropoff Community Area#229, cast(Fare#88 as string) AS Fare#230, cast(Tips#89 as string) AS Tips#231, cast(Tolls#90 as string) AS Tolls#232, cast(Extras#91 as string) AS Extras#233, cast(Trip Total#92 as string) AS Trip Total#234, Payment Type#93, Company#94, cast(Pickup Centroid Latitude#95 as string) AS Pickup Centroid Latitude#237, cast(Pickup Centroid Longitude#96 as string) AS Pickup Centroid Longitude#238, Pickup Centroid Location#97, cast(Dropoff Centroid Latitude#98 as string) AS Dropoff Centroid Latitude#240, cast(Dropoff Centroid Longitude#99 as string) AS Dropoff Centroid Longitude#241, Dropoff Centroid  Location#100, cast(hour(CASE WHEN isnull(gettimestamp(Trip Start Timestamp#80, MM/dd/yyyy hh:mm:ss a, Some(GMT))) THEN gettimestamp(gettimestamp(Trip Start Timestamp#80, MM/dd/yyyy hh:mm:ss a, Some(GMT)), MM/dd/yyyy HH:mm, Some(GMT)) ELSE gettimestamp(Trip Start Timestamp#80, MM/dd/yyyy hh:mm:ss a, Some(GMT)) END, Some(GMT)) as string) AS Start Hour#243]
   +- FileScan csv [Trip ID#78,Taxi ID#79,Trip Start Timestamp#80,Trip End Timestamp#81,Trip Seconds#82,Trip Miles#83,Pickup Census Tract#84L,Dropoff Census Tract#85L,Pickup Community Area#86,Dropoff Community Area#87,Fare#88,Tips#89,Tolls#90,Extras#91,Trip Total#92,Payment Type#93,Company#94,Pickup Centroid Latitude#95,Pickup Centroid Longitude#96,Pickup Centroid Location#97,Dropoff Centroid Latitude#98,Dropoff Centroid Longitude#99,Dropoff Centroid  Location#100] Batched: false, DataFilters: [], Format: CSV, Location: InMemoryFileIndex[s3a://data-repository-bkt/ECS765/Chicago_Taxitrips/chicago_taxi_trips.csv], PartitionFilters: [], PushedFilters: [], ReadSchema: struct<Trip ID:string,Taxi ID:string,Trip Start Timestamp:string,Trip End Timestamp:string,Trip S...