I have 7000 points, and for each point I want to create a OD cost matrix of that point to all other points and write the result in separate tables - that is 7000 tables with 6999 records each.
Solving the OD cost matrix (arcpy.na.solve) is initially fast (takes about 3 seconds) for each origin (to 6999 destinations), but as the loop goes further, it gradually slows down. For example after 100 iterations solving OD matrix takes about a minute, and finally it gets so slow that it is not practical to continue -- for all 7000 points (it probably takes several days). The solving time doesn't depend on where the origin point is located (in the center or periphery of the network).
Just to figure out what is wrong here, I modified the code such that for every 30 iterations, it redefined the OD Cost Matrix layer, and added the destinations again. This made the solving part as fast as it is in the beginning (about 3 second), but it is not a useful workaround as adding 7000 destination locations itself is a very slow process.
The memory usage doesn't change much, but CPU usage increases as the it loops further.
Should I del something after solving the OD matrix (after arcpy.na.solve) in the loop?here is my code (I took out the part that redefined the OD Matrix for every 30 iteration, that was just for test):
import arcpy,datetime, sys arcpy.env.overwriteOutput=1 network=r"C:\Users\Reza\Documents\NCSU\2015SUMMER\LUC_Simulation\Datasets\street_centerlines_ND.nd" inOrgins=r"C:\Users\Reza\Documents\NCSU\2015SUMMER\LUC_Simulation\Parcel_points.shp" inDestinations=r"C:\Users\Reza\Documents\NCSU\2015SUMMER\LUC_Simulation\Parcel_points.shp" outLayerFile=r"C:\Users\Reza\Documents\NCSU\2015SUMMER\LUC_Simulation\Parcel_points.lyr" searchTolerance = "1000 Meters" arcpy.CheckOutExtension("Network") g = arcpy.Geometry() geometryList = arcpy.CopyFeatures_management(inOrgins, g) print "Creating OD matrix",datetime.datetime.now().ctime() outNALayer=arcpy.na.MakeODCostMatrixLayer(network,'OD_layer','Minutes','','','','','','','','No_Lines') print "OD matrix created",datetime.datetime.now().ctime() outNALayer = outNALayer.getOutput(0) subLayerNames = arcpy.na.GetNAClassNames(outNALayer) originsLayerName = subLayerNames["Origins"] destinationsLayerName = subLayerNames["Destinations"] print "adding destintions",datetime.datetime.now().ctime() arcpy.na.AddLocations(outNALayer, destinationsLayerName, geometryList,"", searchTolerance) print "destintions added",datetime.datetime.now().ctime() i=0 for inOrgin in geometryList: print "myTable_{0}.dbf".format(i) arcpy.na.AddLocations(outNALayer, originsLayerName, inOrgin, "",searchTolerance,"","","",'CLEAR') print "origin {0} added".format(i),datetime.datetime.now().ctime() arcpy.na.Solve(outNALayer) print "solved for origin {0}".format(i),datetime.datetime.now().ctime() lines_lyr=arcpy.mapping.ListLayers(outNALayer, 'Lines')[0] arcpy.TableToTable_conversion(lines_lyr,r"C:\Users\Reza\Documents\NCSU\2015SUMMER\LUC_Simulation\tables","myTable_{0}.dbf".format(i)) print "dBase for origin {0} creatd".format(i),datetime.datetime.now().ctime() i=i+1
أكثر...
Solving the OD cost matrix (arcpy.na.solve) is initially fast (takes about 3 seconds) for each origin (to 6999 destinations), but as the loop goes further, it gradually slows down. For example after 100 iterations solving OD matrix takes about a minute, and finally it gets so slow that it is not practical to continue -- for all 7000 points (it probably takes several days). The solving time doesn't depend on where the origin point is located (in the center or periphery of the network).
Just to figure out what is wrong here, I modified the code such that for every 30 iterations, it redefined the OD Cost Matrix layer, and added the destinations again. This made the solving part as fast as it is in the beginning (about 3 second), but it is not a useful workaround as adding 7000 destination locations itself is a very slow process.
The memory usage doesn't change much, but CPU usage increases as the it loops further.
Should I del something after solving the OD matrix (after arcpy.na.solve) in the loop?here is my code (I took out the part that redefined the OD Matrix for every 30 iteration, that was just for test):
import arcpy,datetime, sys arcpy.env.overwriteOutput=1 network=r"C:\Users\Reza\Documents\NCSU\2015SUMMER\LUC_Simulation\Datasets\street_centerlines_ND.nd" inOrgins=r"C:\Users\Reza\Documents\NCSU\2015SUMMER\LUC_Simulation\Parcel_points.shp" inDestinations=r"C:\Users\Reza\Documents\NCSU\2015SUMMER\LUC_Simulation\Parcel_points.shp" outLayerFile=r"C:\Users\Reza\Documents\NCSU\2015SUMMER\LUC_Simulation\Parcel_points.lyr" searchTolerance = "1000 Meters" arcpy.CheckOutExtension("Network") g = arcpy.Geometry() geometryList = arcpy.CopyFeatures_management(inOrgins, g) print "Creating OD matrix",datetime.datetime.now().ctime() outNALayer=arcpy.na.MakeODCostMatrixLayer(network,'OD_layer','Minutes','','','','','','','','No_Lines') print "OD matrix created",datetime.datetime.now().ctime() outNALayer = outNALayer.getOutput(0) subLayerNames = arcpy.na.GetNAClassNames(outNALayer) originsLayerName = subLayerNames["Origins"] destinationsLayerName = subLayerNames["Destinations"] print "adding destintions",datetime.datetime.now().ctime() arcpy.na.AddLocations(outNALayer, destinationsLayerName, geometryList,"", searchTolerance) print "destintions added",datetime.datetime.now().ctime() i=0 for inOrgin in geometryList: print "myTable_{0}.dbf".format(i) arcpy.na.AddLocations(outNALayer, originsLayerName, inOrgin, "",searchTolerance,"","","",'CLEAR') print "origin {0} added".format(i),datetime.datetime.now().ctime() arcpy.na.Solve(outNALayer) print "solved for origin {0}".format(i),datetime.datetime.now().ctime() lines_lyr=arcpy.mapping.ListLayers(outNALayer, 'Lines')[0] arcpy.TableToTable_conversion(lines_lyr,r"C:\Users\Reza\Documents\NCSU\2015SUMMER\LUC_Simulation\tables","myTable_{0}.dbf".format(i)) print "dBase for origin {0} creatd".format(i),datetime.datetime.now().ctime() i=i+1
أكثر...