constantinmedia Blog

Just another guy blogging…



Google Maps JavaScript API v3: Handling large amounts of features using clustering in data layers

So, I started this Google Maps based project last week, and the last time I had to do with the Google Maps API was several years ago (just right before the release of either v3 or maybe even v2) and I didn’t do much beyond playing around by adding some markers and lines back then. This time, I needed to display real-life data coming from a database in the form of markers, paths and areas, and when the project is live and as time proceeds, there will hopefully soon be hundreds or thousands of items to display.

Zooming out of a map with that amount of overlayed elements, will sooner or later end up looking very crowded, like this:

Not much of an eye candy, is it?

Beginning to dive into the Google Maps API documentation once again, one of the novelties I encountered were Data Layers and their ability to comfortably load GeoJSON data that is automatically converted to overlay objects – that can easily be exported to GeoJSON again after editing the map. If you don’t know about the GeoJSON format already, look at the specs, it’s worth a read. It’s a standardized JSON structure for geometrical features (Points, Lines, Polygons or collections of them). But I won’t go into any depth on this in this post, as it is intended to help those who are already working with GeoJSON data or who delayed working with data layers because of the problem related to handling large amounts of data as described above.

In fact, I was on the brink of discarding data layers altogether myself, because I just couldn’t find a solution. I knew about Marker Clustering using the JavaScript library js-marker-clusterer, made available by the Google Maps team themselves. However, this was only meant to work with traditional marker overlays, so it wouldn’t work with the Data Layer feature. It took me quite some time, working through pages of search results and related posts on StackOverflow, where several people described that problem and were looking for a solution. Actually, this appears to be such a fundamental and obvious problem in any usecase where you have to display more than just a handful of elements on a map, that I couldn’t – and still can’t – believe that this is not handled out-of-the-box, or at least that there would be plenty information and approaches to this available.

People were suggesting to either manually parse the GeoJSON data and create every point as a marker object, or listen to the ‘featureadded’ event on map.data when loading the GeoJSON and then remove it from the data layer and add it as a marker. This wouldn’t just be quite combersome, it would also make retrieving the markers as GeoJSON again more complex, and, quite bluntly, challenge the usefulness of the GeoJSON-to-data-layer feature in general, if you needed to something more than the rather simple demos provided in the API docs. So, just before I would have given up and done just that, I finally stumbled upon the data-layer-clusterer project by GitHub user “nantunes”.

He aimed at rebuilding the js-marker-clusterer functionality for use with data layers. And indeed, it worked great for points/markers, except for the fact that the marker cluster images had meanwhile moved from their old location at Google Code to GitHub. That would have been an easy fix (even without changing the code itself, as the URL can be changed in a variable), however, the project hadn’t been worked on for almost a year, so it didn’t seem that it was still actively maintained and any fixes or improvements could be expected sooner or later. There was only one fork by GitHub user “jesusr”, who implemented some optimizations and did some rewriting. Albeit, using the code in its current status just lead to immediate JavaScript errors in the console, which I had to fix first in order to give it a try.

Alas, needless to say that LineStrings and Polygons were not implemented either – but I needed them so badly (, I grew tired of my confusion… Sorry – I don’t even really know that song, I just felt the urgent need to add a reference. Cat pics next time, it is.)

But, my ambition had been roused and I really wanted to make this work. And it actually did in less than a day, astonishingly. So, here it is, ladies and gentlemen and damsels in dire dispair, trying to add more than a dozen markers onto a Google Map (whoever could have imagined that people might feel the need to do this) – drumroll, please…

*ba-dum-tsss* 1

https://github.com/Connum/data-layer-clusterer

Lines and Polygons are currently clustered based on their center point (the center point of their bounding rectangle, to be exact). I also fixed the cluster marker images and created SVG versions of them, which are used by default if the browser supports SVG. I am also planning to add an option to force Lines and Polygons to be clustered when they fall below a pixel threshold value in either width or height.

Ahhh, a feast for the eyes.

Documentation is also to come, as there hadn’t been any (except for comments in the code) for the original repo of data-layer-clusterer. But its usage is pretty straight-forward, you can look at the JSFiddle code and into the code of the library itself. Basically, you can call most of the methods that you would call on the map.data object directly on a DataLayerClusterer, which are then passed on to the internally used data layer object. Or, as a last resort, you can get the internal data layer itself via the _dataLayer property of any DataLayerClusterer instance.

I hope this will help many others in the future and lead to plenty of maps with plenty of GeoJSON content without making the users poke their eyes out with a fork due to sensory overload. Contributions are welcome, so bring on your shiny sparkly pull requests.

\\update 1:
I added new boolean option, ‘setProperty’: If set to true, instead of changing the StyleOption attribute ‘visible’ of the features directly, a boolean property ‘in_cluster’ (or a configurable property name defined in the constant DataLayerClusterer.CLUSTER_PROPERTY_NAME)is set on the features, which can then be used to toggle visibility (for example in order to take into account other properties for additonal filtering, like I needed to do).

\\update 2:
new option ‘recolorSvg’: (string) only takes action if SVG is supported and being used: a selector string for an SVG element in the set imagePath that can be used for re-coloring the cluster marker image. This saves requests and prevents the different marker images popping up after loading.

Updated the second demo to use the new version and a few LineStrings.


Was this post of any help for you? Say thank you by buying me a coffee via a PayPal donation! :-)
Donate via PayPal: $ USD | £ GBP | € EUR

Notes:

  1. This is actually the sound of a sheep, a snare drum and a snake falling off a cliff. Ok, never mind.

9 thoughts on “Google Maps JavaScript API v3: Handling large amounts of features using clustering in data layers
  • Alexander says:

    Hi! Cool library! But how to get markers in the same lat and long position, which makes the marker cluster be visible even at the highest zoom level (level 21)?

    • Constantin says:

      I’m afraid there is currently no solution for markers in the exact same spot. I think this is an extreme edge case, but feel free to contribute to the project! I will look into this when I got the time.

  • Fritz Zehrer says:

    Vielen Dank! Eigentlich genau was ich suche, aber ich finde nirgends eine Möglichkeit, meine vorhandene und “hart erarbeitete” Karte in das System inklusive Beschreibung, verschiedenfarbige Marker und Link zu einer Detailbeschreibung zu migrieren: https://www.ushikes.com/usa_hiking_database_hiking_map.htm

    • Constantin says:

      Hallo Fritz!
      Woran genau hapert es denn? Die Marker selbst sollten eigentlich beibehalten werden, inklusive eventueller custom icons. Es werden lediglich die cluster icons hinzugefügt und zusammen mit den Markeren entsprechend aus- und eingeblendet!

      • Fritz Zehrer says:

        Hallo Constantin, vermutlich hapert’s an mir :-).

        Also momentan ist die Google-Karte über iframe in eine html5 Seite eingebunden (kein API). Wenn ich nun mit Deiner js clustern will, ist mir einfach nicht klar, wie die bestehende Karte mit Deinem Clusterer verbinden kann, respektive muss.

  • Fritz Zehrer says:

    Ok, offensichtlich habe ich es nun einigermaßen auf die Reihe bekommen. Ich habe aus der Google Map eine kml generiert und daraus Geodaten erzeugt, die ich in Dein JavaScript als Points eingebunden habe. Die Marker erscheinen, die Clusterung funktioniert einwandfrei.

    Jedoch habe ich noch das Problem, dass weder das Bild, noch der Text, noch der Link zum Hike mit Klick auf den Marker erscheinen. Vermutlich ist hier die Einbindung repektive meine Grunderfassung falsch, denn sie erscheint nach der Geodaten-Konvertierung in Description:

    {
    “geometry”: {
    “coordinates”: [
    -155.09813,
    19.295217000000001,
    13.720000000000001
    ],
    “type”: “Point”
    },
    “properties”: {
    “description”: “Beschreibung – GPS – KartenDescription – GPS – Mapshttps://www.ushikes.com/htm_hikes_kamoamoa_lava_flow.htm”,
    “name”: “Kamoamoa Lava Flow”
    },
    “type”: “Feature”
    },

    Ich bin mir jetzt nicht sicher, ob Du Dich da noch als zuständig erklärst :-) Dankbar wäre ich natürlich für jeden Hinweis!

    • Fritz Zehrer says:

      Karte “on air” zur Ansicht:
      https://www.ushikes.com/usa_hiking_database_hiking_map_cluster.htm

      Dein System ist wirklich top, jetzt muss ich nur noch json-Daten-Darstellungen für
      – unterschiedliche Marker, d.h. in blau, rot und schwarz,
      – Link href zur Wanderung,
      – Titel und
      – Bild bzw. Thumb

      finden und zudem hoffen, dass Dein JS das irgendwie entsprechend verarbeitet und auf Datenebene darstellt.

      PS: Falls Dir die hier stattfindende Konversation zu “spammig” wird, – weg damit! Trotzdem erneut herzlichen Dank, denn soweit war ich mit dem Clustern noch nie :-)

      • Constantin says:

        Das stört mich ganz und gar nicht, im Gegenteil – ich freue mich über regen Austausch! Allerdings bin ich zur Zeit etwas beschäftigt, deshalb hatte ich noch keine Zeit, mir das genauer anzuschauen. Hier aber etwas zur Umsetzung verschiedener Icon-Styles, vielleicht hilft dir das schonmal weiter:

        cluster.setStyle(featureStyling);

        Und featureStyling ist dann eine Funktion, die ein Style-Objekt zurückgibt, anhand von Abfragen der Marker-Properties: https://jsfiddle.net/gjdjLwgd/

        Die Infos zu Link, Titel, Bild-URL etc. kannst du auch in den Properties speichern.

        Bin gespannt, ob du bald Erfolg vermelden kannst! :) Ansonsten schaue ich gerne mal in den Code, sobald ich etwas mehr Luft habe.

        Viele Grüße
        Constantin

  • Fritz Zehrer says:

    Ok, ich glaube das macht so keinen Sinn. Ich muss mich länger mit JS und Deinem JS und möglichweise überhaupt mit der Funktionsweise des google clusterns auseinandersetzen. Beispiele, um zu kopieren finde ich leider auch keine.

    Ich habe nun mal bei einem Wegpunkt eine URL unter Properties erfasst, was aber auch nicht viel hilft, da der Marker auf diese Eingabe nicht reagiert, weil sie vermutlich völlig falsch ist.

    {
    “geometry”: {
    “coordinates”: [
    -112.716111,
    32.038333000000002
    ],
    “type”: “Point”
    },
    “properties”: {
    “description”: “bla bla”,
    “name”: “Ajo Window”,
    “url”: “https://www.ushikes.com/htm_hikes_ajo_window.htm”
    },
    “type”: “Feature”
    },

    Schade, aber trotzdem danke Constantin für Deine Hilfe und Unterstützung. Vielleicht gibt es irgendwann einen Generator, der Nicht-Programmierern dabei hilft einen validen Code zu generieren. Oder ich klemm’ mich mehr dahinter … :-)

Leave a Reply

Your email address will not be published. Required fields are marked *