Article

← Retour au blog

🇫🇷 Lire en français

2006: Mobile Earth GIS Technology

2006-06-18 #333 · rév. 1 · EN

You certainly already know “*Google Earth“:http://earth.google.com, a system that enables you to see satellite photographs of any location on Earth. It offers both 2D and 3D representations.*
This service is accessible on desktop computers, but unfortunately, nothing like it is available for your mobile phone.

Mobile Earth is a technology dedicated to the mobile wireless industry: its purpose is to provide the mobile industry with an innovative technology that enables users to play in a real-time 3D environment; the graphics are true-to-life representations of cities and landscapes that show off an “infinite”, unmatched level of detail.

Stéphane de Luca chez In-Fusio le 23 juin 2006

Mobile Earth 3D: First part

I started the project in late April this year, with the objective of providing a streaming technology capable of displaying interactive real-time 3D with unmatched quality for high-performance 3.5G wireless networks: HSDPA.

A handset user will play a game — and later use Geographic Information System-based services — that displays very high-quality backgrounds based on real-world satellite photographs of the Earth.

The idea behind it is to have a server that renders the world in real-time 3D at the location of the player’s avatar. The world is rendered with a very large z-far value — typically more than one hundred kilometers, giving the sensation of an extremely large persistent world.

Every single frame generated by the engine is compressed on the fly to feed an upstream video.
At the other end, a handset is connected to the server via a TCP connection over an HSDPA radio network, itself connected to the Internet. HSDPA is the 3.5G network capable of 1 Mbps downstream bandwidth. HSDPA deployments are currently starting up in France.

The video is then streamed down to the handset, and the client decodes the stream and displays the background at the phone’s resolution — typically QVGA, 320×240 pixels.

As a result, the player sees incredibly high-quality graphics.

The player can move their avatar up, down, left, and right while flying forward. This information is sent up to the server. Upon receipt, the game engine interprets the player’s move and positions the camera accordingly, altering the video stream.

As a result, the player feels that they can freely move through the environment and interact with it.

Welcome to GIS and its gigantic memory space

Playing such a game requires very high precision in order to see buildings, houses, and even cars on highways, for example.
Partnering with IGN, I was provided with a set of very high-resolution satellite photographs: four pixels per square meter. At this resolution, we can clearly see cars, and even parasols on sunny beaches!
Unfortunately, the other side of the coin is the very large size of the related files. For example, the storage required to hold all photographs for a single French department is 60 gigabytes.
I would then need 6 terabytes to store satellite photographs for all of France — equivalent to 1,257 DVDs!

Stage I: High-performance real-time rendering

I started off with an LOD-based system for managing the world.

I set up a server that processes data by implementing a data pipeline from the IGN photographs and altitude images into the engine database. 60 GB is required to hold the original “Alpes-Maritimes” data.
— Altitude: Altitude information is contained in a 38 MB 8-bit indexed TIFF image where the index indicates the altitude level at a resolution of 50 meters; I slice the image into many 2048×2048-pixel squares using a down-sampling algorithm.
— Photographs: The satellite photographs are converted from 2000×2000-pixel RGB TIFF images into 2048×2048-pixel RGB JPEGs, from which I generate a set of 10 mipmap images. At the moment, this takes 20+ minutes to generate on a P4 2 GHz for the small region I use for the game.

The rendering engine currently uses only a small subset of the department’s data. The LOD system drives tessellation on demand in order to minimize the required bandwidth.
Each frame is captured from the framebuffer and compressed as JPEG files on disk for a second process in charge of streaming.

Stage II: Video Streaming

The generated images are compressed on the fly and sent down to the handset using a PacketVideo server.
This project aims to showcase the real-world performance of HSDPA technology, which offers a theoretical bandwidth of 1 Mbps! Meanwhile, the application is scalable and also supports UMTS networks.

Stage III: J2ME client

The client needs to play back video in a background layer while displaying interactive assets on top of it. It therefore requires proper handling of alpha blending, or punch-through at the very least.
Oddly enough, there is no standard J2ME solution for doing this at the time of writing; this is why we turned to a different approach and chose to use a Flash player where video playback is supported — specifically, the Bluestreak client.

This solution raised two important issues that impact us: first, rendering vector-based graphics is less efficient than simple bitmaps, and second, the foreground result looks less realistic, which somewhat contrasts with the project’s goal of rendering real-life imagery.
Regardless, we went ahead with it.

The game displays a spacecraft that must pass through big 3D “donut” bonuses laid out along a chained cubic-spline path.
The player can move freely within the virtual tunnel, though in the first version they cannot roam freely everywhere in the world.

Stage IV: Liberté, liberté chérie

In this stage, the player will be totally free to go wherever they want and explore the entire landscape.
Player inputs will be uploaded in real time to the server through a TCP connection. Upon receipt, the engine will perform the requested action — including moving the camera — and generate new video frames.
The big challenge here is that we have 6 seconds of buffered video on the client, which introduces 6 seconds of latency. For this reason, the client anticipates the background movement by applying the 3D transformation directly to the spacecraft while the background is still in its previous position.
When the updated frames begin to stream down, the client compensates for the movement of the background by canceling out the anticipated transformations already applied to the aircraft. For this purpose, I embedded 3D transformation metadata directly within the video frames themselves.

But that is for the next stages… be patient and enjoy the first images and video!