[WIP] Simplify Graph

[rush42]

This PR implements an algorithm for simplifying the topology of an OSMGraph object.
It is adapted from osmnx.

This PR is work in progress, and a few issues have to be discussed.

TODO:

1. the output format of the simplification procedure (rn it returns a DiGraph and two DataFrames: one for nodes and the other for edges (which also contains the edge geometry) )
2. reference to the original OSM objects (an edge can consist of serveral ways)
3. turn restrictions
4. type stability and code generalization for types

Below is an example for Tiergarten district in Mitte, Berlin, Germany:

Before the simplification:
nodes: 2976,
egdes: 4727

After the simplification:
nodes: 683,
edges: 1384

I will upload an example script the following days.

[mmiller-max]

This is a great idea, thanks for working on it. Concerning your TODOs:

1. the output format of the simplification procedure (rn it returns a DiGraph and two DataFrames: one for nodes and the other for edges (which also contains the edge geometry) )
2. reference to the original OSM objects (an edge can consist of serveral ways)

I think it should return something like SimplifiedOSMGraph where the graph type and other parametric types match that of the input OSMGraph. This simplified graph could then contain then mapping between the original and new edge IDs. We could add something that gets the way IDs from the new edge IDs too.

We could add a parent type to SimplifiedOSMGraph and OSMGraph for the functions with which either graph works.

Ideally the NodeIDs should be the same as then we can use all the additional OSM data for them - are they?

3. turn restrictions

Yep this is pretty important, but hopefully just a small extension to what you've got already.

4. type stability and code generalization for types

Does my answer to 1 above remove all the instabilities, or do other functions need improving as well?

[mmiller-max]

Am I correct in thinking that the use of IGeometry, createpoint and createlinestring are not necessary for the graph reduction, they just give more meta data? If that's the case then I don't think they should be here. They could perhaps be added in new functions for all OSMGraphs.

[rush42]

I think it should return something like SimplifiedOSMGraph where the graph type and other parametric types match that of the input OSMGraph.

Yes, adding a dedicated type is probably necessary .

If that's the case then I don't think they should be here. They could perhaps be added in new functions for all OSMGraphs.

I would also prefer to keep the geometric types out, so one can decide on their own which library to use.

When the SimplifiedOSMGraph keeps a reference to the parent, it would even be sufficient to just store a list of all the interstitial NodeIDs from the original graph for every new edge.
Generating the geometries could then be achieved by indexing OSMG.node_coordinates with that list

IMHO that would also solve most issues related to TODO 2 and 3.

They could perhaps be added in new functions for all OSMGraphs

Agree, I already implemented two function generating something similar for plotting with Plots.jl . I'll upload them together with the example script.

I'll keep you updated about further ideas!

[rush42]

I added the SimplifiedOSMGraph type and methods for generating "GeoDataFrames" either from OSMGraph or SimplifiedOSMGraph objects. Additional I added a small PlotRecipe for visualization.

The example below makes use of the above mentioned methods and was mainly for checking if every thing looks correct.

using LightOSM, Plots

g = graph_from_download(
    :place_name, 
    place_name="tiergarten, berlin germany",
    network_type=:bike
)
sg = simplify_graph(g)

# check for missing edges
plot(g, color=:red, linewidth=0.8)
plot!(edge_gdf(sg).geom, linewidth=1.1, color=:black)
savefig("edge_validation")

# show original nodes
plot(sg)
plot!(node_gdf(g).geom, color=:red, markersize=2.2)
savefig("original_nodes")

# show relevant nodes
plot(sg)
plot!(node_gdf(sg).geom, color=:green, markersize=2.2)
savefig("relevant_nodes")

Output:

this should only show black edges:

original nodes:

relevant noes:

I will write some tests when I'll find the time to check if the topology is preserved correctly and if the weights are correct.

[mmiller-max]

Hey sorry for the delay, will try to review soon. It'd also be good for @captchanjack to take a look at this as it's a relatively big addition and I'm keen to know how it fits in with his thoughts for the package.

[mmiller-max]

It would be helpful to fix the merge conflicts via a rebase or merge - in particular the renaming of the field highway to way should be observed in the new graph. Once the conflicts are sorted I can kick off CI, it seems that GitHub won't let me do it before then.

[rush42]

I've merged the changes, and added the edge_to_way dict to simplifiedOSMGraph.

[mmiller-max]

@rush42 Just wanted to say sorry we haven't got around to reviewing this yet. Hopefully someone will be able to soon!

[ctbaum]

Awesome PR! will this be merged soon? I've tried using as is but cannot get shortest_path calculations from SimplifiedOSMGraph

[rush42]

@ctrebbau yes I haven't worked on the shortest path algorithms yet. And right now I can't find the time to do so. IMO it's the only thing missing, the rest should be implemented...

[rush42]

As I have some spare time atm I picked up this PR. I rebased my fork and will now try to get the routing functions to work.

[kkdd]

Hello,
When simplifying graph, interstitial vertices would be identified as (inward degree, outward degree, undirectional degree) = (1, 1, 0) or (0, 0, 2), if described by using a mixed multigraph model.

If a LightOSM graph is generated in the way that each two-way highway is converted to two opposite directional edges, I recommend to store its directionality in an additional variable when generating and use it when simplifying.

It could distinguish against the case of such one-way highway network as shown in the following exaggerated example, where I consider that the vertices 3 and 4 should not be deleted:

[rush42]

Thanks for the example! I think in that case 3 and 4 should be removed. I also thought about updating the condition to something that works for directed and undirected graphs. Maybe removing vertices with degree 2 and outneighbors(g, v) == inneighbors(g, v) would work for both cases.

[kkdd]

Can I ask whether you observe that the vertices 3 and 4 should be deleted or not?

I consider that they should not be deleted because they don't satisfy the condition of (inward degree, outward degree, undirectional degree) = (1, 1, 0) nor (0, 0, 2), if described by using a mixed multigraph model.

For your information, you can see https://github.com/kkdd/GraphFromOSM.

[rush42]

Because both vertices are neither sink, source nor do they have a self loop. Both have 2 neighbors and same in- and outdegree that's why they can be removed. They would then be replaced by two edges in either direction between 2 and 5.

[kkdd]

I consider that both vertices, or intersections, should remain in simplification. Because you can select the directions of turn at these intersections when traveling in this road network. Consider that both paths of 2->3->4 and 2->3->2 are allowed in the same way as airline routes with transfer.

[rush42]

I think as long as there are no turn restrictions they should be removed. But it's true that if there are turn restrictions you might need a subpath like 2->3->2. I haven't thought of it before and will have a look at it tomorrow to see how to integrate the turn restrictions into the simplification. Thanks for mentioning it!

[kkdd]

Thank you for your consideration.
I think universally that graph simplification should reserve its mathematical homeomorphicity. It also reserves the quantity of the number of vertices minus that of edges even on a mixed multigraph model. This concept is independent of turn restrictions.

[rush42]

The shortest_path algorithm are working now. I added a basic test to example.jl which computes all path between 200 randomly sampled nodes in both graphs and compares their length. The only thing missing at the moment are the turn restrictions which also need to be transformed when simplifying the graph.

[kkdd]

Note that length(neighbors) != 2 || indegree(g, v) != outdegree(g, v) cannot always reserve graph homeomorphicity, but indegree(g, v) != 1 || outdegree(g, v) != 1 can. For your information, see above.

[rush42]

I think that is not a problem as long as we ignore the turn restrictions(as mentioned above). But it significantly reduces the number of nodes which is why I'd like to keep it that way.

[rush42]

For nodes with turn restrictions though we need to add the closest node which allows a u-turn to the end points iterator.

[kkdd]

Note that I use a mixed multigraph model in https://github.com/kkdd/GraphFromOSM and its homeomorphic simplification also significantly reduces the number of nodes even for undirected edges.

[rush42]

So your point is keeping every node that connects two OSM ways ?

[kkdd]

No, every node which connects two OSM ways is deleted and they are concatenated by homeomorphic smoothing there, but under the following normal conditions:

  function shallBeConcatenated(vertex) {
    if (!vertex.inGraph) return false;
    const edgeIDs = vertex.edges.filter(id => graph.edges[id].inGraph);
    if (edgeIDs.length != 2) return false;  // It isn't a degree-2 vertex.
    if (edgeIDs.bothAreEqual()) return false;  // It's an isolated loop as the first and second edges are identical.
    const edges = edgeIDs.map(id => graph.edges[id]);  // depicted as [ --- v --- ] where v denotes the vertex and ---'s denote edges.
    const directed = edges.map(e => e.directed);  // represent wheather each edge is directed or not
    if (!directed.bothAreEqual()) return false;  // A mixture of directed and undirected edges
    if (directed[0]) {  // collect the vertex of inward-degree 1 and outward-degree 1 for directed edges
      const dirsOutward = edges.map(e => e.vertices[0] == vertex.id);  // represents wheather each edge is outward
      if (dirsOutward.bothAreEqual()) return false;  // can't be concatenated, as depicted as [ <-- v --> ] or [ --> v <-- ].
      if (dirsOutward[0]) vertex.edges.reverse(); // make them in the processing order depicted as [ p --> v --> s ] where p and s denote the predecessor and successor vertices.
    }
    return true;
  }