Author
Here are three distinct ways to frame this story for your readers, depending on the specific angle your audience prefers:
You are standing on a busy street corner in downtown Chicago, surrounded by steel-and-glass skyscrapers that stretch fifty stories into the sky. It is rush hour. The sky is overcast, and a light drizzle is starting to fall.
You open your phone, request a ride, and watch the map. A small vehicle icon appears. It smoothly turns a corner, avoids a dead-end alley, and glides down the avenue toward your exact position.
On the surface, this feels like a simple utility. Your phone talks to a satellite, the satellite talks to the driver's phone, and the app draws the coordinates on your screen.
But if you have ever tried to build location-based software, you know that this smooth movement is an illusion.
If the app relied on raw GPS data, that little car would jump erratically from street to street, teleport through solid brick buildings, and spin in circles at every red light. The physical world is noisy, chaotic, and hostile to clean data.
To make that car glide smoothly across your screen, engineers had to build a system that rejects the idea of "continuous real-time tracking" entirely. Instead, they built a massive state-synchronization engine powered by probabilistic math, structural mapping, and fast, low-latency network protocols.
Let's explore how we turn a chaotic mess of satellite noise into a clean, predictable digital reality.
Before writing any code or configuring message brokers, we need to understand the fundamental limitation of mobile tracking: continuous telemetry is a physical impossibility.
If a driver's phone continuously transmitted raw, high-frequency GPS data over cellular networks, the device would overheat, the battery would drain in less than an hour, and the network gateway would collapse under the weight of billions of concurrent connections.
To understand how we solve this, let's use an analogy.
Imagine a theater director trying to track the movements of actors on a pitch-black stage. The director cannot see the actors directly. They only have access to a few weak, flickering spotlights that occasionally cast distorted shadows against the set pieces.
+─────────────────────────────────────────────────────────+
| THE PIT-BLACK STAGE |
| |
| [Skyscraper Set] [Skyscraper Set] |
| | | |
| v v |
| (Dark Shadow) (Dark Shadow) |
| |
| o <-- Actual Actor (No Direct Light)|
| /|\ |
| / \ |
| |
| * <-- Distorted Shadow Cast on Wall |
| (Raw, Messy GPS Coordinate) |
+─────────────────────────────────────────────────────────+
If the director trusted those distorted shadows (the raw GPS signals) directly, they would assume the actors are teleporting across the stage or walking through solid walls.
Instead, the director uses three critical pieces of context:
By combining these imperfect clues, the director calculates the most probable path of the actor in real-time, even when they are moving through complete darkness.
In a ride-sharing system, the driver's mobile device is the actor, and our backend services are the theater director. The smooth car moving on your screen is a client-side interpolation—a calculated animation designed to bridge the gaps between asynchronous, algorithmically corrected server updates.
To understand how this works in production, let’s follow a single location update as it travels from the driver's mobile hardware, through our processing pipeline, and onto the rider's screen.
+─────────────────────────────────────────────────────────────────────────+
| 1. MOBILTY LAYER (The Edge) |
| - Polling: 4-6s (Active) / 10-30s (Idle) |
| - Payload: Delta Encoded (Heading Shifts, Velocity Vectors) |
+────────────────────────────────────────┬────────────────────────────────+
│
▼
+─────────────────────────────────────────────────────────────────────────+
| 2. INGESTION LAYER |
| - Edge Gateway -> Apache Kafka REST Proxy |
| - Decoupled, Partitioned Distributed Logs |
+────────────────────────────────────────┬────────────────────────────────+
│
▼
+─────────────────────────────────────────────────────────────────────────+
| 3. SENSOR FUSION & PROCESSING |
| - Accelerometer & Gyroscope Calibration |
| - 3D Ray-Traced Shadow Matching (SNR Filtering) |
| - Particle Filter State Hypotheses |
+────────────────────────────────────────┬────────────────────────────────+
│
▼
+─────────────────────────────────────────────────────────────────────────+
| 4. TOPOLOGICAL ALIGNMENT (Map Matching) |
| - Hidden Markov Model (HMM) Grid Snapping |
| - S2 Cell Sharding (Parallel Processing) |
+────────────────────────────────────────┬────────────────────────────────+
│
▼
+─────────────────────────────────────────────────────────────────────────+
| 5. SPATIAL QUANTIZATION |
| - GeoToH3 Conversion (64-bit Hexagonal Indexing) |
| - Ephemeral State Caching (Redis/Twemproxy) |
+────────────────────────────────────────┬────────────────────────────────+
│
▼
+─────────────────────────────────────────────────────────────────────────+
| 6. DISTRIBUTION LAYER (RAMEN) |
| - Fireball Microservice (State Change Detection) |
| - Bi-directional Streaming: gRPC over QUIC (HTTP/3) |
+─────────────────────────────────────────────────────────────────────────+
Everything starts at the mobile hardware layer. Because we cannot stream data continuously, we must be highly efficient with when and what we transmit.
When a driver is on an active trip, the mobile application polls the operating system's location providers at optimized intervals of four to six seconds. When the driver is idling or waiting for a trip request, the application throttles this transmission back to every ten to thirty seconds to save battery.
Instead of sending heavy, absolute coordinate objects, we use delta encoding. The client transmits only what has changed: velocity vectors, heading shifts, and coordinate differences.
These compressed payloads travel over cellular networks and hit our load balancers, which immediately route them to Apache Kafka. Kafka serves as our distributed log, persisting incoming telemetry streams across partitioned, fault-tolerant clusters. This decouples our write-heavy ingestion pipeline from the complex downstream processing services.
When a driver enters an "urban canyon"—a street bordered by massive skyscrapers—direct satellite lines-of-sight are broken. Signals reflect off concrete and glass, creating multipath errors that throw off location calculations.
To solve this, we implement Sensor Fusion using the device's Inertial Measurement Unit (IMU).
[Raw GNSS Ping] (Falsely indicates 50m leap due to wall reflection)
o
|
v
[Kinematic Validator] <─── [IMU Accelerometer Vector] (Indicates constant speed)
│
├─> Leap exceeds physical capabilities? YES.
│
▼
[Anomaly Filtered Out] ───> Fallback to IMU Dead Reckoning Vector
The system continuously reads two primary hardware components to maintain an accurate estimation of the vehicle's position:
The Accelerometer: This measures linear acceleration. If a raw GPS ping suggests a sudden jump that would require the vehicle to accelerate faster than physically possible, our kinematic filters identify this as a multipath anomaly, discard the coordinate, and rely on accelerometer data to estimate the actual path.
The Gyroscope: This measures rotational velocity. By calculating the change in angle over time, we estimate a clean, stable vehicle heading:
$$\theta_{Gyro}(t) = \theta_{Gyro}(t-1) + \Delta \theta_{Gyro}(t)$$
This integration eliminates the erratic, spinning vehicle icons often seen on simpler mapping apps, replacing them with a stabilized direction vector based on true physics.
Rather than discarding weak or obstructed satellite signals, our system uses them as a feature. This is called Probabilistic Shadow Matching.
We ingest the raw Signal-to-Noise Ratio (SNR) and satellite positions from the phone's hardware and compare them to a 3D building map of the city.
If a specific satellite's SNR is low, we know the receiver must be located in the physical "shadow" of a nearby building. By simulating ray-traced paths from the satellites to a grid of candidate points on the ground, we can narrow down where the vehicle is.
Because these calculations generate irregular, multi-modal probability distributions (e.g., the car could be on the main street or a parallel alley), traditional linear filters fail.
Instead, we run a Particle Filter on our backend. The filter generates hundreds of simulated position hypotheses (particles) on the map, continuously weighting and updating them based on real-time signal strength and the physical constraints of the road.
Now we have a clean coordinate, but we must align it with the actual road network. If we simply snapped the GPS coordinates to the nearest road, a car on an elevated highway might appear to hop down onto the surface streets below.
To prevent this, our map-matching engine models the road network as a Hidden Markov Model (HMM). In this framework, the physical road segments are the hidden states, and our processed GPS coordinates are the observations.
We calculate two probability matrices:
[Segment A] ───────────(Routable Path)───────────> [Segment B]
▲ ▲
│ (High EP) │ (High EP)
▼ ▼
[GPS Ping G1] ───────────(Impossible Detour)─────────> [GPS Ping G2]
If moving between two segments requires an impossible turn or a massive, five-mile detour just to reach the opposite side of a divided highway, the transition probability drops to near zero.
We then apply the Viterbi algorithm to find the most probable sequence of road segments, snapping the vehicle to the logical path.
To process this at scale, our CatchME map-matching system shards incoming data using S2 spatial cells. By dividing the globe into small, manageable geographic regions, independent workers can calculate these paths in parallel without blocking one another.
Once the vehicle is snapped to the road, we must make it discoverable for marketplace matching, surge calculations, and dispatch queries. Running complex spatial database queries on raw coordinates under heavy load would introduce significant latency.
To solve this, we translate physical coordinates into the H3 Hexagonal Hierarchical Spatial Index. H3 replaces continuous geographic space with a discrete grid of hexagons, representing locations as compact, 64-bit integer IDs.
/\
__/ \__
/ \ / \
/ \/ \
\ /\ /
\__/ \__/
\ /
\/
We choose hexagons because they have an elegant mathematical property: the distance between a hexagon's center and all of its immediate neighbors is uniform. This is not true for square or triangular grids, where diagonal neighbors are further away than edge-sharing neighbors.
Using H3, geographic queries do not require heavy spatial calculations. Instead, we can execute operations like finding nearby drivers directly in memory using fast bitwise shifts on the 64-bit cell IDs.
The final step is sending this synchronized, indexed state back to the rider's phone. Our RAMEN (Realtime Asynchronous Messaging Network) platform handles this through a push-based model.
Our Fireball microservice monitors the Kafka event bus for updates. When a driver's state changes, Fireball packages the update and routes it to the RAMEN gateway.
Instead of traditional HTTP polling, RAMEN maintains long-running gRPC connections over the QUIC transport protocol (HTTP/3).
gRPC uses highly compressed, binary Protocol Buffers to keep payloads small. Running over QUIC allows us to use stream multiplexing, which prevents head-of-line blocking on lossy cellular networks.
Because this connection is bi-directional, our servers receive instant acknowledgments (ACKs) from the mobile clients, allowing us to monitor round-trip times and automatically adjust update rates based on network quality.
Building a real-time, global-scale geospatial engine requires a highly specialized set of technologies. Here is how these components work together:
+─────────────────────────────────────+
| MOBILE CLIENT |
| - React Native & Native SDKs |
+──────────────────┬──────────────────+
│
│ gRPC over QUIC (HTTP/3)
▼
+─────────────────────────────────────+
| API GATEWAY |
| - Envoy / Custom Push Proxy |
+──────────────────┬──────────────────+
│
│ High-throughput Ingress
▼
+─────────────────────────────────────+
| APACHE KAFKA |
| - Distributed Event Streaming |
+──────────────────┬──────────────────+
│
┌──────────────────┴──────────────────┐
▼ ▼
+───────────────────────────────────+ +───────────────────────────────────+
| PROCESSING PIPELINE (Go) | | STATE MANAGEMENT (Node) |
| - Sensor Fusion & Kinematics | | - Active Trip Coordination |
| - CatchME Viterbi Map Snapping | | - API Gateway State Sync |
+─────────────────┬─────────────────+ +─────────────────┬─────────────────+
│ │
▼ ▼
+───────────────────────────────────+ +───────────────────────────────────+
| GEOSPATIAL INDEXING (H3) | | FAST STORAGE & CACHING |
| - In-Memory Bitwise Operations | | - Redis via Twemproxy |
| - Hexagonal Spatial Sharding | | - Cassandra / Schemaless DB |
+───────────────────────────────────+ +───────────────────────────────────+
When designing a planetary-scale location system, you must make deliberate, realistic trade-offs. Here are the core engineering decisions we have to navigate:
[1Hz Polling Engine] ──> High Battery Drain / Network Load ──> Quick Device Thermal Limits
[Adaptive Polling] ──> Low Power Consumption / Steady Network ──> Requires Complex Client Interpolation
┌────────────────────────────────────────┐
│ CHOOSE H3 RESOLUTION │
└───────────────────┬────────────────────┘
│
┌─────────────────────────┴─────────────────────────┐
▼ ▼
[Coarse Resolution (Large)] [Fine Resolution (Small)]
- Lower DB lookup memory footprint - Higher query count (kRing neighbor checks)
- Large vehicle candidate sets per cell - Smaller candidate sets per cell
- Heavy in-memory CPU filtering required - Rapid spatial querying performance
Behind the simple experience of watching a car move across an app lies a complex engineering architecture designed to handle a chaotic physical world. Here is what we can learn from how these systems are built:
Have you built real-time tracking systems or faced challenges with noisy sensor data? How did you approach the balance between battery usage and data accuracy in your own applications? Let’s talk about it in the comments below!