885 Commits

Author SHA1 Message Date
Frédéric Rodrigo
b1eeff573e
Remove table building_multipolygon as multipolygon are already in table osm_building_polygon (#756)
Buildings from ways and multipolygons are loaded in table `osm_building_polygon`. But a table for `osm_building_multipolygon` is also loaded, the content is not used except to ensure an `osm_id` is from a multipolygon. To check if the object is from a multipolygon we have only to check if `osm_id` is negative. It is the counter part of e0c8ece375/layers/building/building.sql (L89)

I checked the objects are the same after this change.
2020-01-31 19:34:08 -05:00
Yuri Astrakhan
6801353e15
Declared field mapping 2 (#734)
Make a few more mappings declarative, and removes values declared in both SQL and the yaml file.

Here's the diff comparing `build/tileset.sql` in master vs the new PR. The changes are mostly stylistic, except when a nested `if` statement is expanded into individual `if ... and ...` conditions (logically identical)

```diff
55c55
diff --git a/build/tileset.sql b/build/tileset.sql
index 4e59357..7c6c444 100644
--- a/build/tileset.sql
+++ b/build/tileset.sql
@@ -52,7 +52,7 @@ CREATE INDEX IF NOT EXISTS osm_ocean_polygon_gen4_idx ON osm_ocean_polygon_gen4
 CREATE OR REPLACE FUNCTION water_class(waterway TEXT) RETURNS TEXT AS $$
     SELECT CASE
            WHEN "waterway" IN ('', 'lake') THEN 'lake'
-           WHEN "waterway"='dock' THEN 'dock'
+           WHEN "waterway" = 'dock' THEN 'dock'
            ELSE 'river'
    END;
 $$ LANGUAGE SQL IMMUTABLE;
@@ -1813,24 +1813,41 @@ CREATE OR REPLACE FUNCTION highway_class(highway TEXT, public_transport TEXT, co
         WHEN "highway" IN ('tertiary', 'tertiary_link') THEN 'tertiary'
         WHEN "highway" IN ('unclassified', 'residential', 'living_street', 'road') THEN 'minor'
         WHEN "highway" IN ('pedestrian', 'path', 'footway', 'cycleway', 'steps', 'bridleway', 'corridor')
-            OR "public_transport"='platform'
+            OR "public_transport" = 'platform'
             THEN 'path'
-        WHEN "highway"='service' THEN 'service'
-        WHEN "highway"='track' THEN 'track'
-        WHEN "highway"='raceway' THEN 'raceway'
-        WHEN highway = 'construction' THEN CASE
-          WHEN construction IN ('motorway', 'motorway_link') THEN 'motorway_construction'
-          WHEN construction IN ('trunk', 'trunk_link') THEN 'trunk_construction'
-          WHEN construction IN ('primary', 'primary_link') THEN 'primary_construction'
-          WHEN construction IN ('secondary', 'secondary_link') THEN 'secondary_construction'
-          WHEN construction IN ('tertiary', 'tertiary_link') THEN 'tertiary_construction'
-          WHEN construction IN ('', 'unclassified', 'residential', 'living_street', 'road') THEN 'minor_construction'
-          WHEN construction IN ('pedestrian', 'path', 'footway', 'cycleway', 'steps', 'bridleway', 'corridor')
-              OR public_transport = 'platform' THEN 'path_construction'
-          WHEN construction = 'service' THEN 'service_construction'
-          WHEN construction = 'track' THEN 'track_construction'
-          WHEN construction = 'raceway' THEN 'raceway_construction'
-        END
+        WHEN "highway" = 'service' THEN 'service'
+        WHEN "highway" = 'track' THEN 'track'
+        WHEN "highway" = 'raceway' THEN 'raceway'
+        WHEN "highway" = 'construction'
+            AND "construction" IN ('motorway', 'motorway_link')
+            THEN 'motorway_construction'
+        WHEN "highway" = 'construction'
+            AND "construction" IN ('trunk', 'trunk_link')
+            THEN 'trunk_construction'
+        WHEN "highway" = 'construction'
+            AND "construction" IN ('primary', 'primary_link')
+            THEN 'primary_construction'
+        WHEN "highway" = 'construction'
+            AND "construction" IN ('secondary', 'secondary_link')
+            THEN 'secondary_construction'
+        WHEN "highway" = 'construction'
+            AND "construction" IN ('tertiary', 'tertiary_link')
+            THEN 'tertiary_construction'
+        WHEN "highway" = 'construction'
+            AND "construction" IN ('', 'unclassified', 'residential', 'living_street', 'road')
+            THEN 'minor_construction'
+        WHEN "highway" = 'construction'
+            AND ("construction" IN ('pedestrian', 'path', 'footway', 'cycleway', 'steps', 'bridleway', 'corridor') OR "public_transport" = 'platform')
+            THEN 'path_construction'
+        WHEN "highway" = 'construction'
+            AND "construction" = 'service'
+            THEN 'service_construction'
+        WHEN "highway" = 'construction'
+            AND "construction" = 'track'
+            THEN 'track_construction'
+        WHEN "highway" = 'construction'
+            AND "construction" = 'raceway'
+            THEN 'raceway_construction'
     END;
 $$ LANGUAGE SQL IMMUTABLE;
 
@@ -4073,6 +4090,12 @@ RETURNS TEXT AS $$
         WHEN "subclass" IN ('fast_food', 'food_court') THEN 'fast_food'
         WHEN "subclass" IN ('park', 'bbq') THEN 'park'
         WHEN "subclass" IN ('bus_stop', 'bus_station') THEN 'bus'
+        WHEN ("subclass" = 'station' AND "mapping_key" = 'railway')
+            OR "subclass" IN ('halt', 'tram_stop', 'subway')
+            THEN 'railway'
+        WHEN "subclass" = 'station'
+            AND "mapping_key" = 'aerialway'
+            THEN 'aerialway'
         WHEN "subclass" IN ('subway_entrance', 'train_station_entrance') THEN 'entrance'
         WHEN "subclass" IN ('camp_site', 'caravan_site') THEN 'campsite'
         WHEN "subclass" IN ('laundry', 'dry_cleaning') THEN 'laundry'
@@ -4082,7 +4105,7 @@ RETURNS TEXT AS $$
         WHEN "subclass" IN ('hotel', 'motel', 'bed_and_breakfast', 'guest_house', 'hostel', 'chalet', 'alpine_hut', 'dormitory') THEN 'lodging'
         WHEN "subclass" IN ('chocolate', 'confectionery') THEN 'ice_cream'
         WHEN "subclass" IN ('post_box', 'post_office') THEN 'post'
-        WHEN "subclass"='cafe' THEN 'cafe'
+        WHEN "subclass" = 'cafe' THEN 'cafe'
         WHEN "subclass" IN ('school', 'kindergarten') THEN 'school'
         WHEN "subclass" IN ('alcohol', 'beverages', 'wine') THEN 'alcohol_shop'
         WHEN "subclass" IN ('bar', 'nightclub') THEN 'bar'
@@ -4098,9 +4121,6 @@ RETURNS TEXT AS $$
         WHEN "subclass" IN ('bag', 'clothes') THEN 'clothing_store'
         WHEN "subclass" IN ('swimming_area', 'swimming') THEN 'swimming'
         WHEN "subclass" IN ('castle', 'ruins') THEN 'castle'
-        WHEN (subclass = 'station' AND mapping_key = 'railway')
-          OR (subclass IN ('halt', 'tram_stop', 'subway')) THEN 'railway'
-        WHEN (subclass = 'station' AND mapping_key = 'aerialway') THEN 'aerialway'
         ELSE subclass
     END;
 $$ LANGUAGE SQL IMMUTABLE;
@@ -4301,22 +4321,22 @@ $$
     COALESCE(NULLIF(name_de, ''), name, name_en) AS name_de,
     tags,
     CASE
-      WHEN "aerodrome"='international'
-          OR "aerodrome_type"='international'
+      WHEN "aerodrome" = 'international'
+          OR "aerodrome_type" = 'international'
           THEN 'international'
-      WHEN "aerodrome"='public'
-          OR "aerodrome_type"='civil'
+      WHEN "aerodrome" = 'public'
+          OR "aerodrome_type" = 'civil'
           OR "aerodrome_type" LIKE '%public%'
           THEN 'public'
-      WHEN "aerodrome"='regional'
-          OR "aerodrome_type"='regional'
+      WHEN "aerodrome" = 'regional'
+          OR "aerodrome_type" = 'regional'
           THEN 'regional'
-      WHEN "aerodrome"='military'
+      WHEN "aerodrome" = 'military'
           OR "aerodrome_type" LIKE '%military%'
-          OR "military"='airfield'
+          OR "military" = 'airfield'
           THEN 'military'
-      WHEN "aerodrome"='private'
-          OR "aerodrome_type"='private'
+      WHEN "aerodrome" = 'private'
+          OR "aerodrome_type" = 'private'
           THEN 'private'
       ELSE 'other'
     END AS class,
```
2020-01-31 00:22:21 -05:00
Yuri Astrakhan
9f4d48cf25
Remove unused import-sql image (#738)
import-sql image is not used anywhere in OMT, deleting.
2020-01-24 14:40:03 -05:00
Yuri Astrakhan
9d6dbfc64f
Use one pass docs image generation (#751)
quicker and cleaner diagram image generation.
Remove etl-graph and mapping-graph targets - redundant

Also, the obsolete "fields" is still in Imposm's code and both names are accepted,
but "fields" is not documented anywhere (PR submitted), and could be removed at any moment.

Our docs were not supporting it until this PR, so renaming it at the same time.

Several images have been updated due to a more inclusive mapping scan
Requires https://github.com/openmaptiles/openmaptiles-tools/pull/147 (merged)
2020-01-22 21:55:22 -05:00
Yuri Astrakhan
c9e7ad90c6
Remove unneeded "else null" in conditions (#732)
Minor code cleanup:
SQL already returns NULL in the "WHEN" condition
if it is not matched by any of the cases.

Co-authored-by: Eva Jelinkova <evka.jelinkova@gmail.com>
2020-01-22 17:24:28 -05:00
Yuri Astrakhan
3449cecb22
Migrate to new Wikidata importer (#735)
* Use _resolve_wikidata in layer mapping.yaml

Mark all tables that should not be populated with the Wikidata
international labels with a special OMT-specific flag.

This should be ok to merge even before the new tools version
is used because imposm seems to ignore anything it doesn't understand.

The next tools version will remove it when generating imposm mapping file.

* Migrate to new Wikidata importer

Uses latest tools to populate the wd_names table
during the quickstart.  This can be merged already,
or we can wait for the next tools version.
2020-01-22 16:16:38 -05:00
Yuri Astrakhan
1bc3f138b4
Create boundary materialized views (#740)
* Move simplified border tables to OMT as MAT VIEWs

Consolidate derived table creation in OMT repository.
Move all non-original simplified geometry tables from import-osmborder
image to this repo, allowing further optimization.

Later we can remove derived table creation fro mthe import-osmborder image.
2020-01-21 09:44:29 -05:00
Yuri Astrakhan
e6e92036a0
Use _resolve_wikidata in layer mapping.yaml (#731)
Mark all tables that should not be populated with the Wikidata
international labels with a special OMT-specific flag.

This should be ok to merge even before the new tools version
is used because imposm seems to ignore anything it doesn't understand.

The next tools version will remove it when generating imposm mapping file.
2020-01-21 09:43:21 -05:00
Yuri Astrakhan
11f4856d48
Add simplified water materialized views (#739)
Move materialized view creation from the tools repo.

This PR should be merged before the https://github.com/openmaptiles/openmaptiles-tools/pull/115

Merge the other PR shortly after this one to avoid doing the same work twice - first creating simplified table, then dropping it and recreating them as materialized views.
2020-01-20 12:12:19 -05:00
Yuri Astrakhan
1d91b9ef6e
Noop: tag sql MAT VIEWS with a special comment (#733)
Tag all SQL materialized views with a machine-readable comment
to indicate that this materialized view can be created without
data:

   /* DELAY_MATERIALIZED_VIEW_CREATION */

In the next version of tools this comment can be optionally
replaced with the "WITH NO DATA" parameter, thus allowing
a much faster execution of the SQL script. All materialized
viewes will be populated with data in parallel afterwards
using the `refresh-views` tools script.
2020-01-20 12:02:49 -05:00
Dalibor Janák
4e13a17e1a
Merge pull request #693 from Sophox/field-injection
Manage field mapping in SQL declaratively
2020-01-17 13:48:14 +01:00
Yuri Astrakhan
e4a9d90dec
noop: minor makefile targets cleanup (#737)
Avoid duplicating target name with `$@`.
This prevents accidental typo bugs.
2019-12-20 12:09:08 -05:00
Yuri Astrakhan
1614a4656c Manage field mapping in SQL declaratively
Simplify some of the OSM->OMT field value mappings using declarative syntax.

This approach is not for all cases, but in many it removes
the need of storing the same field in both the .yaml and .sql files.

TODO: support more complex AND/OR cases
2019-12-19 11:25:29 -05:00
Eva Jelinkova
f3667170f6
Merge pull request #745 from nyurik/fix-travis
TravisCI should raise error on fail
2019-12-19 12:02:02 +01:00
Eva Jelinkova
831bb6fe4d
Merge pull request #736 from MartinMikita/fix-docker-compose-tool-version
Fixed docker-compose commands.
2019-12-19 11:59:07 +01:00
Eva Jelinkova
bbaf743166
Merge pull request #743 from nyurik/detect-warnings
Detect sql warnings and abort quickstart
2019-12-19 11:58:45 +01:00
Eva Jelinkova
9f00db0245
Merge pull request #747 from eva-j/water_z4
Change NE source of lakes in zoom 4
2019-12-19 11:31:12 +01:00
Eva Jelinkova
36ddca1f98
Merge pull request #748 from MartinMikita/minor-readme-updates
Clean README with make commands to easy reading.
2019-12-19 11:30:40 +01:00
Eva Jelinkova
99ff133cef
Merge pull request #749 from eva-j/742_datatype
Add data types for boundary_z0
2019-12-19 11:29:05 +01:00
Eva J
4c43c6ea3b add data types for boundary_z0 2019-12-18 19:31:28 +01:00
MartinMikita
75a94f615f Fixed merge conflicts. 2019-12-18 09:40:10 +01:00
Eva J
c40e092381 lakes z4 2019-12-18 09:14:47 +01:00
Yuri Astrakhan
d26c621b2b TravisCI should raise error on fail
Added missing `set -e` to travis script
so that if the first line fails, it stops
2019-12-16 14:07:53 -05:00
MartinMikita
6aaabb820c Clean README with make commands to easy reading. 2019-12-16 15:39:50 +01:00
MartinMikita
434c0500f3 Removed TOOLS_VERSION, rewritten commands to use only docker-compose with TOOLS_VERSION inside .env. 2019-12-16 15:37:19 +01:00
Yuri Astrakhan
358939e912
etl graph update, add graph test to travis (#741)
* etl graph update

re-ran `make generate-devdoc` which updated landuse graph

* Add travis test to check docs are up to date
2019-12-16 08:37:34 -05:00
Yuri Astrakhan
d46af923dd Detect sql warnings and abort quickstart
If quickstart generates an SQL `WARNING` while running
the import-sql script, abort.
2019-12-14 03:11:38 -05:00
MartinMikita
32433a53a3 Fixed docker-compose commands, missing default TOOLS_VERSION. 2019-12-13 14:41:39 +01:00
Eva Jelinkova
ef3b0b04d0
OMT version update
Release of OMT 3.11.0 is coming soon!
2019-12-13 09:39:40 +01:00
Eva Jelinkova
afb9ccfa84
Merge pull request #730 from eva-j/723_water_sources
Changing ocean data source in z6-7
2019-12-12 19:05:27 +01:00
Eva Jelinkova
dae8e055ae
Merge pull request #720 from lun/add-garages
Add garages
2019-12-12 19:04:26 +01:00
Eva Jelinkova
d82ede81ce
Merge pull request #727 from eva-j/disp_borders_rel
Disputed borders - adding relations with claimed_by
2019-12-12 19:03:06 +01:00
Yuri Astrakhan
132747d9b5
Use unified tools version 3.1.0 for all images (#707)
* Use unified tools version for all images
* do not start postserve as part of quickstart, but added a help message how to start it
* wait for SQL start with pgwait
2019-12-12 12:40:31 -05:00
Eva J
8ff4a5acb4 changing ocean data source in z6-7 2019-12-12 12:44:32 +01:00
Eva J
6381451964 just newlines 2019-12-12 09:48:35 +01:00
Eva Jelinkova
c02ec2176f
Merge pull request #729 from openmaptiles/revert-645-include-pedestrian-multipolygon
Revert "Remove the area condition on highway polygon."
2019-12-11 22:53:53 +01:00
Eva Jelinkova
02c202e2f7
Revert "Remove the area condition on highway polygon." 2019-12-11 22:22:59 +01:00
Eva Jelinkova
a049d88db8
Merge pull request #721 from frodrigo/building_poly
Filter building on polygons
2019-12-11 17:55:02 +01:00
Yuri Astrakhan
8876e3a4d8
Clean up quickstart and makefile (#726)
Other than the change from `exit 404` to `exit 1`,
all other changes are noops - just cleaning things up
so that various linting tools don't complain.
2019-12-11 09:26:25 -05:00
Eva J
fee22d4eb9 adding disp. relations into z1-4 2019-12-11 12:05:42 +01:00
Eva J
0181513b1a 2 attributes, generalization in mapping, edit unions 2019-12-10 19:48:24 +01:00
Eva J
fd940684eb relation member values, diagrams 2019-12-10 08:49:54 +01:00
Eva Jelinkova
b97d6de626
Merge pull request #718 from frodrigo/golf_grass
Add leisure=golf_course, map it to grass
2019-12-09 15:21:07 +01:00
Eva J
89815c04cf mapping_diagram update 2019-12-09 14:48:52 +01:00
Frédéric Rodrigo
e0c8ece375 Filter building index on polygons 2019-12-09 11:03:21 +01:00
Eva J
a05228381a add relations with claimed_by 2019-12-07 12:18:06 +01:00
golubev
7f591ffebc update the 'layers/landuse/mapping_diagram.png' 2019-12-06 19:48:56 +02:00
golubev
1247972951 merge master into add-garages 2019-12-06 19:40:12 +02:00
Eva Jelinkova
0545f1e27d
Merge pull request #717 from eva-j/584_urban_areas
Landuse - removing ne_10m data source
2019-12-03 20:30:10 +01:00
Frédéric Rodrigo
f916e29be0 Add leisure=golf_course, map it to grass 2019-11-29 13:43:10 +01:00