UTFGrid

UTFGrid

UTFGrid is a specification for rasterized interaction data. As of version 1.2, it was removed from incubation in the MBTiles Specification and split into its own repository.

See CHANGELOG.md for per-version changes.

License

This specification is licensed under a Creative Commons Attribution 3.0 United States License.

Applications which make use of the specification are not subject to any restrictions.

Implementations

Write

• Mapnik's grid_renderer (Uses Antigrain Geometry library for rendering).
• Various python sample implementations
• GeoScript groovy example
• TileStache
• create-utfgrids

Read

• Wax for adding support to the Leaflet, Modest Maps, OpenLayers, and Google Maps clients: examples, code
• Leaflet.utfgrid
• OpenLayers Native: blog post, general demo,

----------------------------------------------------------------------------------------------------------------------------------------------------------------------

UTFGrid

UTFGrid is a specification for rasterized interaction data. As of version 1.2, it was removed from incubation in the MBTiles Specification and split into its own repository.

See CHANGELOG.md for per-version changes.

License

This specification is licensed under a Creative Commons Attribution 3.0 United States License.

Applications which make use of the specification are not subject to any restrictions.

Implementations

Write

• Mapnik's grid_renderer (Uses Antigrain Geometry library for rendering).
• Various python sample implementations
• GeoScript groovy example
• TileStache
• create-utfgrids

Read

• Wax for adding support to the Leaflet, Modest Maps, OpenLayers, and Google Maps clients: examples, code
• Leaflet.utfgrid
• OpenLayers Native: blog post, general demo, example multi-grid, example geography-class
• GDAL MBTiles support
• Landez - Python module

Servers

• TileMill - via tilelive.js which uses tilelive-mapnik to request grid creation and node-mbtiles to cache and fetch grids stored in MBTiles format.
• TileStache's TileStache.Goodies.Providers.MapnikGrid:Provider
• django-mbtiles which serves grids stored in MBTiles along with tiles using a simple template tag.

Authors

• Tom MacWright (tmcw)
• Will White (willwhite)
• Konstantin Kaefer (kkaefer)
• Justin Miller (incanus)
• Dane Springmeyer (springmeyer)

----------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------------------------------------------------------------------------------------------

Interaction

Tile servers can enhance tilesets with interactivity by implementing two additional HTTP endpoints.

• [base path]/layer.json: A layer manifest JSON containing the interaction formatter function and other optional attributes.
• [base path]/{n}/{n}/{n}.grid.json: A UTFGrid JSON file corresponding to its adjacent tile image.

[base path] refers to the full layer URL prior to any x, y or z coordinates.

Examples:

OSM-style URL schema

http://example.com/0/0/0.png       // tile image for 0/0/0
http://example.com/0/0/0.grid.json // utfgrid for 0/0/0
http://example.com/layer.json      // layer manifest

TileJSON

UTFGrid requires additions to the TileJSON payload for a layer:

• template: String. In the format of a mustache template.

• legend: String. Self-contained HTML that may be displayed as a legend for this layer. Optional.

Example response from layer.json:

{
    "template": "{{NAME}}",
    "legend": "<strong>Countries of the World</strong>"
}

Each layer.json item should be represented by a single row in the metadata table where key,value match its key and value in the layer.json object.

Template

As of UTFGrid 1.1, the formatter key is deprecated and replaced by template. Template is to be a mustache format string that produces HTML, which will be cleaned with an HTML whitelist after generation.

Mustache

Template data is specified according to the mustache specification. The full specification is supported, but no partials are provided, or should be provided by implementations.

Given the switch to templates from formatters, the options object is no longer available. Its functionality is emulated by setting 'format flags' on each data object.

For an example data object like

{
    "id": "helloworld"
}

This will be transformed into

{
    "id": "helloworld"
    "__location__": true
}

By the tooltip/interaction implementation, in order to trigger the location template. Note that true, 1, and all non-false values are equal to template, so implementations may set "__location__": 1 to save bytes.

The template implementation could be:

{{#__location__}}
    http://your.com/{{id}}
{{/__location__}}
{{#__full__}}
    This content has the id {{id}}
{{/__full__}}
{{#__teaser__}}
    {{id}}
{{/__teaser__}}

Which, for this implementation, will produce the output

http://your.com/helloworld

legend

A tileset may provide an HTML string that can be rendered by the client as a legend. The string should be self-contained and not reference external stylesheets, scripts or images. The Data URI scheme may be used to embed images or other data if necessary.

<div><span style='padding:0px 10px; background:#333;'></span> +10% population</div>
<div><span style='padding:0px 10px; background:#666;'></span> +5% population</div>
<div><span style='padding:0px 10px; background:#999;'></span> +0% population</div>
<div><span style='padding:0px 10px; background:#ccc;'></span> -5% population</div>

grid.json

See utfgrid.md for the format and storage of UTFGrid JSON.

----------------------------------------------------------------------------------------------------------------------------------------------------------------------

----------------------------------------------------------------------------------------------------------------------------------------------------------------------

UTFGrid

The UTFGrid encoding scheme encodes interactivity data for a tile in a space efficient manner. It is designed to be used in browsers, e.g. for displaying tooltips when hovering over certain features of a map tile.

Since slower browsers and machines can't cope with rendering the actual polygons used to draw vectors on the map tile, we use a grid-based approach where we store the associated information for each pixel.

UTFGrid uses JSON as a container format. It is exclusively geared towards square tiles.

Grid

To achieve reasonable speed in browsers, we store information for a pixel in long strings, where each character's Unicode code point is the key for retrieving the information associated with that pixel. When we have less than 96 unique IDs, this means that the space taken up by storing each pixel separately is 256 * 256 = 64 KB. Gzipping the grid data typically reduces it to a size below 2K.

By default, UTFGrid operates on a 4x4 grid, meaning that a tile at zoom level 0 that contains the entire world in its extent will have an grid resolution of 64x64. We take advantage of UTF-8's variable length codepoint encoding: all ASCII characters are encoded as is, that means that the first 94 codepoints are encoded with their code number as a single byte (codes 0x20,0x21, 0x23-0x5B and 0x5D-0x7F). IDs with a number larger than that will get encoded as multiple bytes.

Encoding IDs

JSON doesn't allow control characters, " and \ to be encoded as their literal UTF-8 representation. Encoding an ID works as follows:

• Add 32₁₀.

• If the result is >= 34₁₀, add 1.
• If the result is >= 92₁₀, add 1.

This ensures that all characters that cannot be represented natively are skipped.

Decoding works as follows:

• If the codepoint is >= 93₁₀, subtract 1.

• If the codepoint is >= 35₁₀, subtract 1.
• Subtract 32₁₀.

Mapping an ID to a key

The UTFgrid file contains an array in a property named grid at the root level. Each entry represents a row in the grid. Each array entry is a string that contains the UTF-8 encoded codepoint for each column. The string length corresponds to the number of entries in the grid array. Only powers of two are allowed.

The keys are stored in an array named keys at the root level. The index of each key represents the ID that it is associated to.

Retrieving a key from a coordinate works as follows (json is the root level object, x and y are the coordinates, starting from top left at 0, and size is the number of entries in the grid key):

• var factor = 256 / size, row = y / factor, col = x / factor

• var id = json.grid[row].charCodeAt(col); is the character that contains the encoded ID.
• Decode the id as described in "Encoding IDs".
• var key = json.keys[id]; retrieves the ID associated with the coordinate.

All divisions are integer divisions.

Mapping a key to data

The JSON file may contain an optional data property at the root level. If it isn't present, the client looks up the obtained key in its internal data store. If the lookup key is not present, it queries the server with the missing keys. If the data property is present, but the key cannot be found, the client must behave as if there were no data property.

An empty key signifies the unavailability of information for that pixel. No action may be taken to retrieve data for an empty ("") key.

Example UTFGrid JSON file

{ "grid":
[ "                                                    !!!#########",
  "                                                    !!!#########",
  "                                                   !!!!#########",
  "                                                   !!!##########",
  "                        !!                         !!!##########",
  "                                                    !!!#########",
  "                                                    !!######### ",
  "                            !                      !!! #######  ",
  "                                                       ###      ",
  "                                                        $       ",
  "                                                        $$    %%",
  "                                                       $$$$$$$%%",
  "                                                       $$$$$$$%%",
  "                                                     $$$$$$$$$%%",
  "                                                    $$$$$$$$$$%%",
  "                                                   $$$$$$$$$$$$%",
  "                                                   $$$$$$$$$%%%%",
  "                                                  $$$$$$$$$%%%%%",
  "                                                  $$$$$$$$%%%%%%",
  "                                                  $$$$$$$%%%%%%%",
  "                                                  $$$$%%%%%%%%%%",
  "                                                 $$$$%%%%%%%%%%%",
  "                                        # # #  $$$$$%%%%%%%%%%%%",
  "                                             $$$$$$$%%%%%%%%%%%%",
  "                                             $$$&&&&'%%%%%%%%%%%",
  "                                            $$$$&&&&'''%%%%%%%%%",
  "                                           $$$$'''''''''%%%%%%%%",
  "                                           $$$$'''''''''''%%%%%%",
  "                                          $$$$&''''''''((((%%%%%",
  "                                          $$$&&''''''''(((((%%%%",
  "                                         $$$&&'''''''''(((((((%%",
  "                                         $$$&&''''''''''(((((((%",
  "                                        $$$&&&''''''''''((((((((",
  "                                        ''''''''''''''''((((((((",
  "                                         '''''''''''''''((((((((",
  "                                         '''''''''''''''((((((((",
  "                                         '''''''''''''''((((((((",
  "                                         '''''''''''''''((((((((",
  "                                         '''''''''''''''((((((((",
  "                            )            '''''''''''''''((((((((",
  "                                         ***'''''''''''''(((((((",
  "                                         *****'''''''''''(((((((",
  "                              ))        ******'''(((((((((((((((",
  "                                        *******(((((((((((((((++",
  "                                        *******(((((((((((((++++",
  "                                        ********((((((((((((++++",
  "                                        ***,,-**((((((((((++++++",
  "                                         ,,,,-------(((((+++++++",
  "                                         ,,---------(((((+++++.+",
  "                                            --------(((((+++....",
  "                                             -///----0000000....",
  "                                             ////----0000000....",
  "                                             /////1---0000000...",
  "                                              ///11--0000000....",
  "                                                111110000000....",
  "                                                 11110000000....",
  "                                                  1111000000....",
  "                                                    1100     .  ",
  "                                                                ",
  "                                                                ",
  "                                                                ",
  "                                                                ",
  "                                                                ",
  "                                                                " ],
"keys":
 [ "",
   "1",
   "2",
   "3",
   "4",
   "5",
   "6",
   "7",
   "8",
   "9",
   "10",
   "11",
   "12",
   "13",
   "14",
   "15",
   "16" ],
"data":
 { "1": { "admin": "Portugal" },
   "2": { "admin": "Spain" },
   "3": { "admin": "Morocco" },
   "4": { "admin": "Algeria" },
   "5": { "admin": "Western Sahara" },
   "6": { "admin": "Mauritania" },
   "7": { "admin": "Mali" },
   "8": { "admin": "Cape Verde" },
   "9": { "admin": "Senegal" },
   "10": { "admin": "Burkina Faso" },
   "11": { "admin": "Guinea Bissau" },
   "12": { "admin": "Guinea" },
   "13": { "admin": "Ghana" },
   "14": { "admin": "Sierra Leone" },
   "15": { "admin": "Ivory Coast" },
   "16": { "admin": "Liberia" } } }

To test implementations, demo.json contains a grid that consists of 65501 different keys. This is the maximum possible in this version of UTFGrid. Implementors should check that obtaining a coordinate should return the key y * 256 + x for all x/y, with the exception of y = 255 and x >= 222 and x <= 255 returning 65501 due to the maximum codepoint allowed in JSON.

A dummy code validation routine is given here:

var json = JSON.parse(/* demo.json */);

// the resolution of the grid. adjust this for your grid.
var resolution = 4;

var key = 0,
    dimension = 256 / resolution;

for (var y = 0; y < dimension; y++) {
    for (var x = 0; x < dimension; x++) {
        var code = json.grid[y].charCodeAt(x);
        if (code >= 93) code--;
        if (code >= 35) code--;
        code -= 32;

        assert(code == key);

        if (key < 65501) key++;
    }
}