[core] Offset viewport center when edge insets are specified

The change is implemented in TransformState::getProjMatrix, the rest of the code is making sure that existing API contracts stay and there are tests verifyingrendering and render query processing only items within screen and given tolerance around screen edges.

MapView: don't bake edge insets into relalculated camera center. Keep edge insets as property of camera in TransformState (similar to pitch, zoom, bearing) independent from specified camera center. Interpolate edge insets in animation.

iOS Demo app: "Turn On/Off Content Insets" pitch the camera and navigate to convenient location in Denver, where streets are parallel to cardinal directions, to illustrate viewport center offset when edge insets are set.

Tests:
ViewFrustumCulling: although Annotations are deprecated, queryRenderedFeatures related tests in Annotations would need to get ported and decided to add the edge insets related query tests next to them. Verify frustum culling (render+queryRenderedFeatures) With different camera and edge insets setups. TODO: port Annotations tests.

Transform.Padding: Verify that coordinates take proper place on screen after applying edge insets.

LocalGlyphRasterizer: verify text rendering when applying padding.

Related to #11882: both use projection matrix elements [8] and [9].

Alternative approach to this was to increase and offset map origin so that the screen would be a sub-rectangle in larger map viewport. This approach has a drawback of unecessary processing the items that are outside screen area.

Fixes #12107, #12728, navigation-sdks/issues/120

7 files changed, 79 insertions, 71 deletions

diff --git a/src/mbgl/layout/symbol_projection.cpp b/src/mbgl/layout/symbol_projection.cpp
index a58d90d4c..aeea10ffa 100644
--- a/src/mbgl/layout/symbol_projection.cpp
+++ b/src/mbgl/layout/symbol_projection.cpp
@@ -47,7 +47,7 @@ namespace mbgl {
 	 * For horizontal labels we want to do step 1 in the shader for performance reasons (no cpu work).
 	 *      This is what `u_label_plane_matrix` is used for.
 	 * For labels aligned with lines we have to steps 1 and 2 on the cpu since we need access to the line geometry.
-	 *      This is what `updateLineLabels(...)` does.
+	 *      This is what `updateLineLabels(...)` in JS, `reprojectLineLabels()` in gl-native, does.
 	 *      Since the conversion is handled on the cpu we just set `u_label_plane_matrix` to an identity matrix.
 	 *
 	 * Steps 3 and 4 are done in the shaders for all labels.
diff --git a/src/mbgl/map/transform.cpp b/src/mbgl/map/transform.cpp
index e97f0da3f..88b559592 100644
--- a/src/mbgl/map/transform.cpp
+++ b/src/mbgl/map/transform.cpp
@@ -77,13 +77,13 @@ void Transform::jumpTo(const CameraOptions& camera) {
 }
 
 /**
- * Change any combination of center, zoom, bearing, and pitch, with a smooth animation
- * between old and new values. The map will retain the current values for any options
- * not included in `options`.
+ * Change any combination of center, zoom, bearing, pitch and edgeInsets, with a
+ * smooth animation between old and new values. The map will retain the current
+ * values for any options not included in `options`.
  */
 void Transform::easeTo(const CameraOptions& camera, const AnimationOptions& animation) {
-    const EdgeInsets& padding = camera.padding;
-    LatLng startLatLng = getLatLng(padding, LatLng::Unwrapped);
+    const EdgeInsets& padding = camera.padding.value_or(state.edgeInsets);
+    LatLng startLatLng = getLatLng(LatLng::Unwrapped);
     const LatLng& unwrappedLatLng = camera.center.value_or(startLatLng);
     const LatLng& latLng = state.bounds != LatLngBounds::unbounded() ? unwrappedLatLng : unwrappedLatLng.wrapped();
     double zoom = camera.zoom.value_or(getZoom());
@@ -110,9 +110,6 @@ void Transform::easeTo(const CameraOptions& camera, const AnimationOptions& anim
     const Point<double> startPoint = Projection::project(startLatLng, state.scale);
     const Point<double> endPoint = Projection::project(latLng, state.scale);
 
-    ScreenCoordinate center = getScreenCoordinate(padding);
-    center.y = state.size.height - center.y;
-
     // Constrain camera options.
     zoom = util::clamp(zoom, state.getMinZoom(), state.getMaxZoom());
     const double scale = state.zoomScale(zoom);
@@ -130,6 +127,7 @@ void Transform::easeTo(const CameraOptions& camera, const AnimationOptions& anim
     state.panning = unwrappedLatLng != startLatLng;
     state.scaling = scale != startScale;
     state.rotating = bearing != startBearing;
+    const EdgeInsets startEdgeInsets = state.edgeInsets;
 
     startTransition(camera, animation, [=](double t) {
         Point<double> framePoint = util::interpolate(startPoint, endPoint, t);
@@ -143,9 +141,14 @@ void Transform::easeTo(const CameraOptions& camera, const AnimationOptions& anim
         if (pitch != startPitch) {
             state.pitch = util::interpolate(startPitch, pitch, t);
         }
-
-        if (!padding.isFlush()) {
-            state.moveLatLng(frameLatLng, center);
+        if (padding != startEdgeInsets) {
+            // Interpolate edge insets
+            state.edgeInsets = {
+                util::interpolate(startEdgeInsets.top(), padding.top(), t),
+                util::interpolate(startEdgeInsets.left(), padding.left(), t),
+                util::interpolate(startEdgeInsets.bottom(), padding.bottom(), t),
+                util::interpolate(startEdgeInsets.right(), padding.right(), t)
+            };
         }
     }, duration);
 }
@@ -159,8 +162,8 @@ void Transform::easeTo(const CameraOptions& camera, const AnimationOptions& anim
     Where applicable, local variable documentation begins with the associated
     variable or function in van Wijk (2003). */
 void Transform::flyTo(const CameraOptions &camera, const AnimationOptions &animation) {
-    const EdgeInsets& padding = camera.padding;
-    const LatLng& latLng = camera.center.value_or(getLatLng(padding, LatLng::Unwrapped)).wrapped();
+    const EdgeInsets& padding = camera.padding.value_or(state.edgeInsets);
+    const LatLng& latLng = camera.center.value_or(getLatLng(LatLng::Unwrapped)).wrapped();
     double zoom = camera.zoom.value_or(getZoom());
     double bearing = camera.bearing ? -*camera.bearing * util::DEG2RAD : getBearing();
     double pitch = camera.pitch ? *camera.pitch * util::DEG2RAD : getPitch();
@@ -170,15 +173,12 @@ void Transform::flyTo(const CameraOptions &camera, const AnimationOptions &anima
     }
 
     // Determine endpoints.
-    LatLng startLatLng = getLatLng(padding, LatLng::Unwrapped).wrapped();
+    LatLng startLatLng = getLatLng(LatLng::Unwrapped).wrapped();
     startLatLng.unwrapForShortestPath(latLng);
 
     const Point<double> startPoint = Projection::project(startLatLng, state.scale);
     const Point<double> endPoint = Projection::project(latLng, state.scale);
 
-    ScreenCoordinate center = getScreenCoordinate(padding);
-    center.y = state.size.height - center.y;
-
     // Constrain camera options.
     zoom = util::clamp(zoom, state.getMinZoom(), state.getMaxZoom());
     pitch = util::clamp(pitch, util::PITCH_MIN, util::PITCH_MAX);
@@ -278,6 +278,7 @@ void Transform::flyTo(const CameraOptions &camera, const AnimationOptions &anima
     state.panning = true;
     state.scaling = true;
     state.rotating = bearing != startBearing;
+    const EdgeInsets startEdgeInsets = state.edgeInsets;
 
     startTransition(camera, animation, [=](double k) {
         /// s: The distance traveled along the flight path, measured in
@@ -304,9 +305,14 @@ void Transform::flyTo(const CameraOptions &camera, const AnimationOptions &anima
         if (pitch != startPitch) {
             state.pitch = util::interpolate(startPitch, pitch, k);
         }
-
-        if (!padding.isFlush()) {
-            state.moveLatLng(frameLatLng, center);
+        if (padding != startEdgeInsets) {
+            // Interpolate edge insets
+            state.edgeInsets = {
+                util::interpolate(startEdgeInsets.top(), padding.top(), us),
+                util::interpolate(startEdgeInsets.left(), padding.left(), us),
+                util::interpolate(startEdgeInsets.bottom(), padding.bottom(), us),
+                util::interpolate(startEdgeInsets.right(), padding.right(), us)
+            };
         }
     }, duration);
 }
@@ -314,28 +320,16 @@ void Transform::flyTo(const CameraOptions &camera, const AnimationOptions &anima
 #pragma mark - Position
 
 void Transform::moveBy(const ScreenCoordinate& offset, const AnimationOptions& animation) {
-    ScreenCoordinate centerOffset = { offset.x, -offset.y, };
-    ScreenCoordinate centerPoint = getScreenCoordinate() - centerOffset;
-    easeTo(CameraOptions().withCenter(state.screenCoordinateToLatLng(centerPoint)), animation);
-}
-
-LatLng Transform::getLatLng(const EdgeInsets& padding, LatLng::WrapMode wrap) const {
-    if (padding.isFlush()) {
-        return state.getLatLng(wrap);
-    } else {
-        return screenCoordinateToLatLng(padding.getCenter(state.size.width, state.size.height));
-    }
+    ScreenCoordinate centerOffset = { offset.x, offset.y };
+    ScreenCoordinate pointOnScreen = state.edgeInsets.getCenter(state.size.width, state.size.height) - centerOffset;
+    // Use unwrapped LatLng to carry information about moveBy direction.
+    easeTo(CameraOptions().withCenter(screenCoordinateToLatLng(pointOnScreen, LatLng::Unwrapped)), animation);
 }
 
-ScreenCoordinate Transform::getScreenCoordinate(const EdgeInsets& padding) const {
-    if (padding.isFlush()) {
-        return { state.size.width / 2., state.size.height / 2. };
-    } else {
-        return padding.getCenter(state.size.width, state.size.height);
-    }
+LatLng Transform::getLatLng(LatLng::WrapMode wrap) const {
+    return state.getLatLng(wrap);
 }
 
-
 #pragma mark - Zoom
 
 double Transform::getZoom() const {
@@ -364,7 +358,7 @@ void Transform::setMaxZoom(const double maxZoom) {
 #pragma mark - Bearing
 
 void Transform::rotateBy(const ScreenCoordinate& first, const ScreenCoordinate& second,  const AnimationOptions& animation) {
-    ScreenCoordinate center = getScreenCoordinate();
+    ScreenCoordinate center = state.edgeInsets.getCenter(state.size.width, state.size.height);
     const ScreenCoordinate offset = first - center;
     const double distance = std::sqrt(std::pow(2, offset.x) + std::pow(2, offset.y));
 
@@ -576,10 +570,10 @@ ScreenCoordinate Transform::latLngToScreenCoordinate(const LatLng& latLng) const
     return point;
 }
 
-LatLng Transform::screenCoordinateToLatLng(const ScreenCoordinate& point) const {
+LatLng Transform::screenCoordinateToLatLng(const ScreenCoordinate& point, LatLng::WrapMode wrapMode) const {
     ScreenCoordinate flippedPoint = point;
     flippedPoint.y = state.size.height - flippedPoint.y;
-    return state.screenCoordinateToLatLng(flippedPoint).wrapped();
+    return state.screenCoordinateToLatLng(flippedPoint, wrapMode);
 }
 
 } // namespace mbgl
diff --git a/src/mbgl/map/transform.hpp b/src/mbgl/map/transform.hpp
index 29ca9bd14..0c018a6e4 100644
--- a/src/mbgl/map/transform.hpp
+++ b/src/mbgl/map/transform.hpp
@@ -47,8 +47,7 @@ public:
         @param offset The distance to pan the map by, measured in pixels from
             top to bottom and from left to right. */
     void moveBy(const ScreenCoordinate& offset, const AnimationOptions& = {});
-    LatLng getLatLng(const EdgeInsets& = {}, LatLng::WrapMode = LatLng::Wrapped) const;
-    ScreenCoordinate getScreenCoordinate(const EdgeInsets& = {}) const;
+    LatLng getLatLng(LatLng::WrapMode = LatLng::Wrapped) const;
 
     // Bounds
 
@@ -105,7 +104,7 @@ public:
 
     // Conversion and projection
     ScreenCoordinate latLngToScreenCoordinate(const LatLng&) const;
-    LatLng screenCoordinateToLatLng(const ScreenCoordinate&) const;
+    LatLng screenCoordinateToLatLng(const ScreenCoordinate&, LatLng::WrapMode = LatLng::Wrapped) const;
 
 private:
     MapObserver& observer;
diff --git a/src/mbgl/map/transform_state.cpp b/src/mbgl/map/transform_state.cpp
index 596ed0519..f1d45c043 100644
--- a/src/mbgl/map/transform_state.cpp
+++ b/src/mbgl/map/transform_state.cpp
@@ -33,24 +33,36 @@ void TransformState::getProjMatrix(mat4& projMatrix, uint16_t nearZ, bool aligne
         return;
     }
 
-     // Find the distance from the center point [width/2, height/2] to the
-    // center top point [width/2, 0] in Z units, using the law of sines.
+    const double cameraToCenterDistance = getCameraToCenterDistance();
+    auto offset = getCenterOffset();
+
+    // Find the distance from the viewport center point
+    // [width/2 + offset.x, height/2 + offset.y] to the top edge, to point
+    // [width/2 + offset.x, 0] in Z units, using the law of sines.
     // 1 Z unit is equivalent to 1 horizontal px at the center of the map
     // (the distance between[width/2, height/2] and [width/2 + 1, height/2])
-    const double halfFov = getFieldOfView() / 2.0;
+    const double fovAboveCenter = getFieldOfView() * (0.5 + offset.y / size.height);
     const double groundAngle = M_PI / 2.0 + getPitch();
-    const double topHalfSurfaceDistance = std::sin(halfFov) * getCameraToCenterDistance() / std::sin(M_PI - groundAngle - halfFov);
+    const double aboveCenterSurfaceDistance = std::sin(fovAboveCenter) * cameraToCenterDistance / std::sin(M_PI - groundAngle - fovAboveCenter);
 
 
     // Calculate z distance of the farthest fragment that should be rendered.
-    const double furthestDistance = std::cos(M_PI / 2 - getPitch()) * topHalfSurfaceDistance + getCameraToCenterDistance();
+    const double furthestDistance = std::cos(M_PI / 2 - getPitch()) * aboveCenterSurfaceDistance + cameraToCenterDistance;
     // Add a bit extra to avoid precision problems when a fragment's distance is exactly `furthestDistance`
     const double farZ = furthestDistance * 1.01;
 
     matrix::perspective(projMatrix, getFieldOfView(), double(size.width) / size.height, nearZ, farZ);
 
+    // Move the center of perspective to center of specified edgeInsets.
+    // Values are in range [-1, 1] where the upper and lower range values
+    // position viewport center to the screen edges. This is overriden
+    // if using axonometric perspective (not in public API yet, Issue #11882).
+    // TODO(astojilj): Issue #11882 should take edge insets into account, too.
+    projMatrix[8] = -offset.x * 2.0 / size.width;
+    projMatrix[9] = offset.y * 2.0 / size.height;
+
     const bool flippedY = viewportMode == ViewportMode::FlippedY;
-    matrix::scale(projMatrix, projMatrix, 1, flippedY ? 1 : -1, 1);
+    matrix::scale(projMatrix, projMatrix, 1.0, flippedY ? 1 : -1, 1);
 
     matrix::translate(projMatrix, projMatrix, 0, 0, -getCameraToCenterDistance());
 
@@ -134,16 +146,8 @@ ViewportMode TransformState::getViewportMode() const {
 #pragma mark - Camera options
 
 CameraOptions TransformState::getCameraOptions(const EdgeInsets& padding) const {
-    LatLng center;
-    if (padding.isFlush()) {
-        center = getLatLng();
-    } else {
-        ScreenCoordinate point = padding.getCenter(size.width, size.height);
-        point.y = size.height - point.y;
-        center = screenCoordinateToLatLng(point).wrapped();
-    }
     return CameraOptions()
-        .withCenter(center)
+        .withCenter(getLatLng())
         .withPadding(padding)
         .withZoom(getZoom())
         .withBearing(-bearing * util::RAD2DEG)
@@ -222,6 +226,10 @@ double TransformState::getMaxZoom() const {
     return scaleZoom(max_scale);
 }
 
+ScreenCoordinate TransformState::getCenterOffset() const {
+    return { 0.5 * (edgeInsets.left() - edgeInsets.right()), 0.5 * (edgeInsets.top() - edgeInsets.bottom()) };
+}
+
 #pragma mark - Rotation
 
 float TransformState::getBearing() const {
@@ -240,7 +248,6 @@ float TransformState::getPitch() const {
     return pitch;
 }
 
-
 #pragma mark - State
 
 bool TransformState::isChanging() const {
@@ -328,8 +335,7 @@ LatLng TransformState::screenCoordinateToLatLng(const ScreenCoordinate& point, L
 mat4 TransformState::coordinatePointMatrix() const {
     mat4 proj;
     getProjMatrix(proj);
-    float s = util::tileSize;
-    matrix::scale(proj, proj, s, s, 1);
+    matrix::scale(proj, proj, util::tileSize, util::tileSize, 1);
     matrix::multiply(proj, getPixelMatrix(), proj);
     return proj;
 }
diff --git a/src/mbgl/map/transform_state.hpp b/src/mbgl/map/transform_state.hpp
index f51660912..9e501d600 100644
--- a/src/mbgl/map/transform_state.hpp
+++ b/src/mbgl/map/transform_state.hpp
@@ -62,6 +62,10 @@ public:
     void setMaxZoom(double);
     double getMaxZoom() const;
 
+    // Viewport center offset, from [size.width / 2, size.height / 2], defined
+    // by |edgeInsets| in screen coordinates, with top left origin.
+    ScreenCoordinate getCenterOffset() const;
+
     // Rotation
     float getBearing() const;
     float getFieldOfView() const;
@@ -137,6 +141,8 @@ private:
     double ySkew = 1.0;
     bool axonometric = false;
 
+    EdgeInsets edgeInsets;
+
     // cache values for spherical mercator math
     double Bc = Projection::worldSize(scale) / util::DEGREES_MAX;
     double Cc = Projection::worldSize(scale) / util::M2PI;
diff --git a/src/mbgl/programs/symbol_program.cpp b/src/mbgl/programs/symbol_program.cpp
index b538a2333..8a7dccd55 100644
--- a/src/mbgl/programs/symbol_program.cpp
+++ b/src/mbgl/programs/symbol_program.cpp
@@ -66,7 +66,7 @@ Values makeValues(const bool isText,
     const bool pitchWithMap = values.pitchAlignment == style::AlignmentType::Map;
     const bool rotateWithMap = values.rotationAlignment == style::AlignmentType::Map;
 
-    // Line label rotation happens in `updateLineLabels`
+    // Line label rotation happens in `updateLineLabels`/`reprojectLineLabels``
     // Pitched point labels are automatically rotated by the labelPlaneMatrix projection
     // Unpitched point labels need to have their rotation applied after projection
     const bool rotateInShader = rotateWithMap && !pitchWithMap && !alongLine;
diff --git a/src/mbgl/text/collision_index.cpp b/src/mbgl/text/collision_index.cpp
index 88e59bf51..e3d832a85 100644
--- a/src/mbgl/text/collision_index.cpp
+++ b/src/mbgl/text/collision_index.cpp
@@ -348,25 +348,28 @@ std::pair<float,float> CollisionIndex::projectAnchor(const mat4& posMatrix, cons
 std::pair<Point<float>,float> CollisionIndex::projectAndGetPerspectiveRatio(const mat4& posMatrix, const Point<float>& point) const {
     vec4 p = {{ point.x, point.y, 0, 1 }};
     matrix::transformMat4(p, p, posMatrix);
+    auto offset = transformState.getCenterOffset();
+    auto size = transformState.getSize();
     return std::make_pair(
         Point<float>(
-            (((p[0]  / p[3] + 1) / 2) * transformState.getSize().width) + viewportPadding,
-            (((-p[1] / p[3] + 1) / 2) * transformState.getSize().height) + viewportPadding
+            (((p[0] / p[3] + 1) / 2) * size.width) + viewportPadding + offset.x,
+            (((-p[1] / p[3] + 1) / 2) * size.height) + viewportPadding + offset.y
         ),
         // See perspective ratio comment in symbol_sdf.vertex
         // We're doing collision detection in viewport space so we need
         // to scale down boxes in the distance
-        0.5 + 0.5 * (transformState.getCameraToCenterDistance() / p[3])
+        0.5 + 0.5 * transformState.getCameraToCenterDistance() / p[3]
     );
 }
 
 Point<float> CollisionIndex::projectPoint(const mat4& posMatrix, const Point<float>& point) const {
     vec4 p = {{ point.x, point.y, 0, 1 }};
     matrix::transformMat4(p, p, posMatrix);
-    return Point<float>(
-        (((p[0]  / p[3] + 1) / 2) * transformState.getSize().width) + viewportPadding,
-        (((-p[1] / p[3] + 1) / 2) * transformState.getSize().height) + viewportPadding
-    );
+    auto offset = transformState.getCenterOffset();
+    auto size = transformState.getSize();
+    return Point<float> {
+        static_cast<float>((((p[0] / p[3] + 1) / 2) * size.width) + viewportPadding + offset.x),
+        static_cast<float>((((-p[1] / p[3] + 1) / 2) * size.height) + viewportPadding + offset.y) };
 }
 
 } // namespace mbgl