Real-Time Taxi Fare Calculation

The subject

In a ride-hailing app, the fare shows up before the customer confirms the ride. The computation cannot wait for a batch job: it happens in the stream, event by event, between the request and the answer on screen. The project rebuilds that chain end to end, from the event producer to revenue analysis per geographic zone.

The stream is simulated: a Python producer emits 10,000 ride requests spread across eight customer / driver pairs on four continents, with three comfort tiers priced at 0.5, 0.3 and 0.2 per kilometer. Positions are drawn at random, which makes the distances deliberately unrealistic but provides enough message volume and variety to validate the architecture.

The architecture

Four stages, a single path for the data: Kafka carries the events, NiFi orchestrates them and computes the fare, ElasticSearch keeps them observable, BigQuery analyses them once consolidated.

In-stream fare computation

The core of the project is a NiFi processor written in Java and packaged as a NAR. TravelProcessor extends AbstractProcessor, takes every FlowFile coming off the Kafka topic, computes the great-circle distance between driver and customer with the haversine formula (Earth radius 6,371 km), multiplies it by the per-kilometer price of the requested comfort tier, then rewrites the JSON enriched with distance and fare.

Three decisions shape that component:

• Compute inside the flow, not beside it. An external service called from NiFi would have added a network round trip and a failure point outside the pipeline. A native processor inherits the framework’s backpressure, data provenance and error routing.
• Isolate failure instead of absorbing it. Two relationships, SUCCESS and FAILURE: malformed JSON leaves on the error branch with its trace in the component log, the rest of the stream keeps flowing. No lost message, no stalled pipeline.
• Count money in decimal. Distance and fare go through BigDecimal values rounded HALF_UP, to three decimals for kilometers and two for the amount. Floating point has no place in a billing computation.

The processor is covered by a unit test that replays a full message and checks the output, which makes the pricing rule safe to change without redeploying the flow blind. Downstream, a JoltTransformJSON flattens the nested structure into a ten-column record and stamps it with the time it passed through the flow.

Indexing and monitoring

Records are indexed in ElasticSearch with an explicitly declared mapping rather than an inferred one: geographic type for positions, date for the timestamp, keyword for identifiers. That is what keeps aggregations reliable and the map usable, where a dynamic mapping would have typed the coordinates as text.

Kibana acts as the control room: stream throughput, ride distribution per comfort tier, trip map, and message-by-message inspection whenever a value looks off.

Warehouse and analytics

• Consolidation: a MergeRecord groups events into 10,000-record batches and writes them as compressed Parquet, roughly 24 KB for 10,000 rows and 10 columns, one timestamped file per batch dropped on Cloud Storage.
• Warehouse: BigQuery reads the bucket through an external table declared over gs://.../*.parquet. No load step, no duplication, the schema is read on the fly and every new batch becomes queryable as soon as it lands.
• Model: a K-Means trained directly in SQL with BigQuery ML (8 clusters, standardized features) segments rides on their coordinates, extracted from the location field with a regular expression.
• Reporting: ML.PREDICT crosses cluster and comfort tier to aggregate revenue, surfaced in a Looker Studio dashboard.

The model recovers the eight zones of the generator, which counts here as an end-to-end consistency check rather than a discovery. The analysis does surface a clear imbalance though: the top comfort tier accounts for 60% of revenue while representing only a third of the rides. On a real stream, that is exactly the kind of signal that drives a pricing grid or a fleet allocation.

What the project demonstrates

An event-driven chain owned end to end: ingestion, business computation in the stream, indexing, monitoring, warehouse and analytical model. The ingestion framework is extended with a deliberate, tested JVM component rather than worked around with a script on the side, and the warehouse avoids needless copies (external table, schema on read, model trained in SQL). It is the same event backbone that today carries feature pipelines and the telemetry of AI systems in production.