Add pmtiles support

[acalcutt]

This PR adds pmtiles support into tileserver-gl

I've made a new file pmtiles_adapter.js which contains functions tileserver-gl needs read the metadata from the file and to serve specific zxy tiles.

Whats working

• TileJSON generation from pmtiles file
• Serving zxy tiles from the normal tileserver-gl endpoint
• Vector tiles + inspect
• Maplibre-Native raster tiles

Whats needed

• I could use someone who knows more about proper javascript programming to look at this, since I really was throwing bits together here and could use a more skilled eye.
• There is probably a lot of duplication here. I tried not to touch a lot of mbtiles code, so the pmtiles code sometimes duplicated it.
• There are a lot of places in configuration that use 'mbtiles://' or mbtiles in the variable names. Right now to use a pmtiles file you still need to use this designator. Eventually I want to rework that but maybe changing it to something like 'LocalFile://' or maybe supporting both 'pmtiles://' and 'mbtiles://'

[acalcutt]

I made more progress on this and actually got maplibre-native rendering working with the pmtiles files. I think pretty much everything is at least working now :-)

For example I used a planetiler pmtiles file and ran tileserver-gl with
node . --file planetiler_2023-09-02.pmtiles

and it loads the data and style on the index with images made in maplibre-native

inspect works

vector maps works

raster maps works

[Caerbannog]

I tried reviewing since you asked for JS developers. I did not try to understand how pmtiles work. So only small suggestions.

[acalcutt]

I tried reviewing since you asked for JS developers. I did not try to understand how pmtiles work. So only small suggestions.

Thanks for the reviews.

If you or anyone does have any interest in how the pmtiles work, their spec was really useful
https://github.com/protomaps/PMTiles/blob/main/spec/v3/spec.md

but I also was reverencing and using bits of the pmtiles javascript code a lot from
https://github.com/protomaps/PMTiles/blob/main/js/index.ts

PMTiles is interesting because you can use it without a tilesserver and just host on http. however having a tileserver does have some advantages I think, like our built in styling and inspect. I also personally don't like the idea that people could download any of the pmtiles maps served over http, so bringing it into tileserver kind of hides your file locally in a place people can't get to it.

[acalcutt]

I figured out how to rework this so I am able to use more of the functions included in the pmtiles npm package.

In my original attempt I hade tried to use code based on pmtiles FileAPISource , however I had issues with this because it tried to load the whole file into memory, which would fail with files over 2GB and was very slow.

Because of that my second attempt went to reading just the bits of the files I needed, however because I went that route, I was not able to use many of the inbuilt pmtiles functions, and had to include a bunch of extra code to read the file that I didn't need in the first attempt.

my third attempt tries to combine the first and second attempt. I took the functions I made to read bytes from the second attempt and moved them into the custom source based on FileAPISource

[acalcutt]

I think I have gotten this to also support web based pmtiles..

For example, I can now specify a http(s) url as my file like
node . --file https://wifidb.net/demo/pmtiles/sources/planetiler_2023-09-02.pmtiles

If you add the --verbose flag you can see this generates a config like this which specifies a 'pmtiles' data source with a url

{
  "options": {
    "paths": {
      "root": "C:\\Users\\Andrew\\Documents\\GitHub\\tileserver-gl-wdb\\node_modules\\tileserver-gl-styles",
      "fonts": "fonts",
      "styles": "styles",
      "mbtiles": "C:\\Users\\Andrew\\Documents\\GitHub\\tileserver-gl-wdb",
      "pmtiles": "C:\\Users\\Andrew\\Documents\\GitHub\\tileserver-gl-wdb"
    }
  },
  "styles": {
    "basic-preview": {
      "style": "basic-preview/style.json",
      "tilejson": {
        "bounds": [
          -180,
          -85.0511287,
          180,
          85.0511287
        ]
      }
    }
  },
  "data": {
    "v3": {
      "pmtiles": "https://wifidb.net/demo/pmtiles/sources/planetiler_2023-09-02.pmtiles"
    }
  }
}

One thing to note is when running this way, tileserver-gl is bascailly acting as the pmtiles client, so it is basically proxying requests to a remote server. this means the server is going to be both downloading and uploading tiles.