Add a guide on working with large datasets

[jokroese]

Launch Checklist

Resolves #2823

• Confirm your changes do not include backports from Mapbox projects (unless with compliant license) - if you are not sure about this, please ask!
• Briefly describe the changes in this PR.
• Link to related issues.
• Include before/after visuals or gifs if this PR includes visual changes.
• Write tests for all new functionality.
• Document any changes to public APIs.
• Post benchmark scores.
• Add an entry to CHANGELOG.md under the ## main section.

The pull request:

• adds a 'Guides' section to the documentation
• adds a guide on working with large GeoJSON datasets, following the discussion in Add documentation on best practices for working with large datasets #2823

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (381b1de) 75.45% compared to head (29b76cd) 75.45%.
Report is 46 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #3307   +/-   ##
=======================================
  Coverage   75.45%   75.45%           
=======================================
  Files         241      241           
  Lines       19260    19260           
  Branches     4339     4339           
=======================================
  Hits        14532    14532           
  Misses       4728     4728           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[HarelM]

THANKS!!
I added a few minor comments.

[HarelM]

I would think about creating more guides before creating this folder.
Maybe move some of the instruction from the main page into a separate guide here, what do you think?
IDK...

[HarelM]

I've added this PR to the topic to discuss in out next monthly meeting this Wednesday. Feel free to join and present your work :-)

[jokroese]

Thanks for looking over this @HarelM! The changes all look good, only I'm not sure about the note on the clustering. Once you clue me into the restriction, I'll update that too.

I would think about creating more guides before creating this folder.
Maybe move some of the instruction from the main page into a separate guide here, what do you think?
IDK...

It is certainly a bit strange having a guides section with only one guide! That said, I'm not sure where else to put it and I would prefer to put it out somewhere rather than waiting for more guides. Have you had any thoughts on this in the meantime?

I've added this PR to the topic to discuss in out next monthly meeting this Wednesday. Feel free to join and present your work :-)

Unfortunately, I wasn't available but glad that you could share it and I hope to come another time. Were there any thoughts on the 'guides' section at the meeting?

[acalcutt]

I didn't see any mention of tippecanoe in this guide, which I think is really useful for cutting down large geojson datasources. I use it in my wifidb.net projected to make milions of points more workable. Maybe this tool could be mentioned under "Vector Tiling"?

There is also a felt version which supports pmtiles
https://github.com/felt/tippecanoe
but I'm not sure if it is better to link to the original.

[jokroese]

I didn't see any mention of tippecanoe in this guide, which I think is really useful for cutting down large geojson datasources. I use it in my wifidb.net projected to make milions of points more workable. Maybe this tool could be mentioned under "Vector Tiling"?

There is also a felt version which supports pmtiles https://github.com/felt/tippecanoe but I'm not sure if it is better to link to the original.

@acalcutt Thanks for this! I remember you suggesting this in the original discussion I started. I haven't figured out how to use it yet but I'd love it if you have the time to modify the guide to include it!

[acalcutt]

For an example, I use it like this with my geojson file of wifi points
https://github.com/acalcutt/wifidb-tileserver-gl/blob/master/convert/convert.sh#L32-L34
but it also has lots of other options

but this basically makes a few vector tiles files with layers of wifi points. per zoom level there are tile size limits it tries to reach, so at higher zooms it will prune or combine based one the options you set. you can also tell it to keep making zoom levels until all data can fit into a tile.

In the end it gives a nice vector tiles file like
https://tiles.wifidb.net/data/WifiDB_newest/#18/42.26664/-71.84578

[HarelM]

I decided that this should go in, even if this is the only guide available ATM.
I didn't get a meaningful input in the monthly meeting, but no one objected to having a guides section with a single guide.
A possible second guide can be a single page saying something like:
"Your guide goes here, just write something that the community would like to read about" to invite maybe other people to write their guides... Just a thought...

[jokroese]

I'm keen to write a second guide on "Accessibility in MapLibre" soon as it is something I'm researching and implementing for a project anyway. I'm happy to add this single guide on large data for now and add a draft of the accessibility guide within a month.

Until then, I'm happy to publish this one by itself. If you want to add an empty one to encourage other contributors that sounds fine too. Alternatively, that encouragement could go on the main Guides page (i.e. https://maplibre.org/maplibre-gl-js/docs/guides/ main page). I don't think either is strictly necessary though.

[HarelM]

@jokroese can you check the open issue related to accessibility (or maybe they got closed)? A person said that this library is not meeting the standards, but I have no clue how to tell what should or shouldn't be...

[jokroese]

@jokroese can you check the open issue related to accessibility (or maybe they got closed)? A person said that this library is not meeting the standards, but I have no clue how to tell what should or shouldn't be...

I can see that @Malvoz shared a bunch of suggestions for accessibility in 2021 that seem really well-thought-through. Unfortunately, those issues were marked stale and closed. I'll have a look through them :) It would be great if MapLibre could act as an example of best practices for digital map accessibility!

[HarelM]

I couldn't figure out how to turn those into actual code, and I guess no one else either.
Let me know if you need me to reopen those, or if you want to open a new one to group them.
I would definitely like maplibre to be more accessible!