25 Commits

Author SHA1 Message Date
Frédéric Rodrigo
cb2cc09d20
Remove support of relation type=street, few valuable data, most false positive (#754)
Thank you @frodrigo for this PR. Tests are looking good.
2020-04-28 10:36:44 +02:00
Frédéric Rodrigo
5da406f4a2
Filter osm_building_polygon on polygon (#758) 2020-04-23 08:23:52 +02:00
Frédéric Rodrigo
b850819218
Narrow osm_building_relation_building_idx with condition building = '' (#771) 2020-04-21 11:12:29 +02:00
Jorge Sanz
ace759590e
Parallel capability to layer functions (#728)
This PR allows queries to be parallelized on recent versions of Postgres. The `PARALLEL SAFE` modifier has been added to the layer functions and a PLPGSQL function to convert strings into number has been replaced.

`PARALLEL SAFE` is a modifier for `CREATE FUNCTION` available since Postgres 9.6, so this change does not break current OpenMapTiles supported database version. More details about this topic [here](https://www.postgresql.org/docs/current/parallel-safety.html) and at the reference documentation for [`CREATE FUNCTION`](https://www.postgresql.org/docs/current/sql-createfunction.html).

### Testing procedure

The procedure to test this was:

* Imported `spain.pbf` in a clean environment
* Dumped the OpenMapTiles database from the Postgres Docker image
* Created a clean Postgres 12 database using the default Docker image
* Installed `postgis` 3 from the default Debian package and `osml10n` 2.5.8 from the repository (`make`, etc.)
* Restored the dump
* Lowered the postgres planner parameters for triggering parallel plans:
```sql
set parallel_setup_cost = 5;
set parallel_tuple_cost = 0.005;
```
* Manually added the `PARALLEL SAFE` modifier to each function involved in layer queries (not on updates or inserting functions).
* For each layer, run a testing query to confirm parallel workers were created, something like this:
```sql
explain analyze 
select * from layer_aerodrome_label(tilebbox(8,128,95),10,null)
union all
select * from layer_aerodrome_label(tilebbox(8,128,97),10,null);
```
* After all the layers were processed and confirmed to start parallel executions, a more complete example was run. This example just retrieves the geometries for all the layers from the same tile but without using any MVT related function.

<details><summary>Testing query</summary>

```sql
-- Using the function layer_landuse
explain analyze 
select geometry from layer_water(tilebbox(14,8020,6178),14)
union all
select geometry from layer_waterway(tilebbox(14,8020,6178),14)
union all
select geometry from layer_landcover(tilebbox(14,8020,6178),14)
union all
select geometry from layer_landuse(tilebbox(14,8020,6178),14)
union all
select geometry from layer_mountain_peak(tilebbox(14,8020,6178),14)
union all
select geometry from layer_park(tilebbox(14,8020,6178),14)
union all
select geometry from layer_boundary(tilebbox(14,8020,6178),14)
union all
select geometry from layer_aeroway(tilebbox(14,8020,6178),14)
union all
select geometry from layer_transportation(tilebbox(14,8020,6178),14)
union all
select geometry from layer_building(tilebbox(14,8020,6178),14)
union all
select geometry from layer_water_name(tilebbox(14,8020,6178),14)
union all
select geometry from layer_transportation_name(tilebbox(14,8020,6178),14)
union all
select geometry from layer_place(tilebbox(14,8020,6178),14)
union all
select geometry from layer_housenumber(tilebbox(14,8020,6178),14)
union all
select geometry from layer_poi(tilebbox(14,8020,6178),14)
union all
select geometry from layer_aerodrome_label(tilebbox(14,8020,6178),14);
```
</details>

You can inspect the execution plan and results on [this page](https://explain.dalibo.com/plan/3z). Also [attaching](https://github.com/openmaptiles/openmaptiles/files/3951822/explain-tile-simple.tar.gz) the query and JSON output for future reference. The website gives a ton of details, but you may want to search for nodes mentioning `workers` or `parallel` like in this area referring to `osm_border` or `osm_aeroway_linestring` entities

![image](https://user-images.githubusercontent.com/188264/70647153-9cac9300-1c48-11ea-96ea-ac7a1e2f4a79.png)

### Next steps

Since the execution plan is not showing a parallel append at the top level, meaning it's not running each layer individually, I want to continue experimenting with parameters and queries to see if it's possible to even parallelize more the request.

I will post my finding here, even no change in the code should happen.


cc. @nyurik

Co-authored-by: Yuri Astrakhan <yuriastrakhan@gmail.com>
2020-01-31 19:36:02 -05:00
Frédéric Rodrigo
b1eeff573e
Remove table building_multipolygon as multipolygon are already in table osm_building_polygon (#756)
Buildings from ways and multipolygons are loaded in table `osm_building_polygon`. But a table for `osm_building_multipolygon` is also loaded, the content is not used except to ensure an `osm_id` is from a multipolygon. To check if the object is from a multipolygon we have only to check if `osm_id` is negative. It is the counter part of e0c8ece375/layers/building/building.sql (L89)

I checked the objects are the same after this change.
2020-01-31 19:34:08 -05:00
Yuri Astrakhan
c9e7ad90c6
Remove unneeded "else null" in conditions (#732)
Minor code cleanup:
SQL already returns NULL in the "WHEN" condition
if it is not matched by any of the cases.

Co-authored-by: Eva Jelinkova <evka.jelinkova@gmail.com>
2020-01-22 17:24:28 -05:00
Frédéric Rodrigo
e0c8ece375 Filter building index on polygons 2019-12-09 11:03:21 +01:00
Frédéric Rodrigo
4d43ee9136 Add colour to buildings (#555) 2019-05-24 11:09:09 +02:00
Jiri Kozel
f3f8b9804e
Improve 3D buildings (#533)
Add `hide_3d` building attribute to hide some building parts in 3D.

Fix #128
2018-11-14 15:26:24 +01:00
jirik
ff99a02aae Treat buildings with fault heights 2018-06-13 14:51:25 +02:00
jirik
3cee3c7f80 Update diagrams 2017-12-04 12:06:28 +01:00
Jiri Kozel
a5cf15bdbd Consider also buildings with levels=null 2017-10-31 09:52:45 +01:00
jirik
1195126c90 Fix #315: Prevent OSM valndalism on buildings 2017-10-31 09:52:45 +01:00
stirringhalo
237278eb84 Import street and associatedStreet to get the houses (#284)
* Import street and associatedStreet to get the houses

* Some artifcating on the associatedstreet and street member buildings
2017-06-18 10:16:57 -04:00
stirringhalo
90690d2a39 Switch to ZRes (#214) 2017-04-25 18:03:29 -04:00
stirringhalo
199095a2ba Major breakthrough, just some artifacting to resolve 2017-01-28 10:55:08 -05:00
stirringhalo
a271e8c70c Fix building, switch to geometry 2016-12-21 21:26:48 -05:00
ImreSamu
f5ac3def5e etldoc fixes 2016-12-04 02:52:52 +01:00
Lukas Martinelli
071201fe52 No render_height attributes on z13 2016-11-30 12:17:01 +00:00
stirringhalo
373a28e7ad Add building heights (#56)
* Add better support for 3d buildings

Change key from "levels" to "building:levels" as the "levels" key is non-standard and sees much less use than "building:levels"

* Fix typo in sql definition and missing render_min_height

* Add render_height and min_height to building.yaml

* Simplify the render, switch numeric to int.

Assume min_height cannot be negative. There are a few examples in overpass, but can't show below ground anyways in mapbox (https://github.com/mapbox/mapbox-gl-js/issues/3456). Also don't attempt to fix min_height>height

* Remove colour as it is not currently used
2016-11-26 20:26:27 +01:00
ImreSamu
7a0ec0e40d add layer_building etldoc comments 2016-11-09 23:39:44 +01:00
lukasmartinelli
0379bde353 Simplify building layer 2016-10-30 16:30:55 +01:00
lukasmartinelli
f3b316a300 Round render height as int 2016-10-24 15:44:09 +02:00
lukasmartinelli
908e6dc68c Use buildings from imposm3 mapping. 2016-10-10 19:35:07 +02:00
lukasmartinelli
480d720585 Move to new modular layer structure 2016-10-09 21:27:09 +02:00