40k+ points from Shapefile, clustering them and populating on Google Maps/Webapp

[المشرف العام]

I just started a new Python/Django webapp job related to GIS (and i'm totally new to GIS). Here is my task and what i have found so far, and i am stuck as to finding out what's the best way or the global standard way of doing things.

• We are building a webapp where a user can upload a shapefile, and take the records and points from the file then populate them on a map.
  
  
  1. Upload a shapefile
  2. Parsed the shapefile using pyshp library - this seems OK
  3. I'm able to read the shapes (coordinates) and the records (value for each field on each shape)
  4. Now say for instance i've got about 40,000+ coordinates, each of them with field records in it
  5. I'll need to populate those 40k points onto Google Maps - those points should be represented with a marker, and the field records can be represented with a heatmap (depending on which field the user choose to filter it as)
  6. Certainly loading 40k points onto Google Maps wouldn't make sense as it will slow down the client side of things.

The question is, what can i do to make this fast or make this work? Thinking of clustering nearby coordinates in a grid, and not only that, each coordinate has a field record and we have to somehow cluster them(?) so that the clustered coordinate has an accurate representation of the values in it. A few options that i have looked at are :-

1. Use http://google-maps-utility-library-v3.googlecode.com/svn/trunk/markerclusterer/docs/reference.html, this would speed things up.. but the server will still be sending 40k points, which may not work at a point where we need to load more than 40k points for each different section on the map
2. Was looking at how we can achieve http://www.gdal.org/grid_tutorial.html by using http://www.gdal.org/gdal_grid.html - I've not been able to get the .tiff generation to work yet
3. Also to achieve grid data, i found http://docs.scipy.org/doc/scipy/reference/generated/scipy.interpolate.griddata.html - though i don't think this actually does it
4. I just happen to learn about GeoDjango and https://github.com/biodiv/anycluster, and am looking into it to see if this should be something i should use
5. and i'm still continuing to research

I don't know if there is any global standard, it looks like a common problem that plenty of people may have come across. I am probably unaware of the lingo terms to search for.

A pointer to the right direction, would be appreciated.



أكثر...