I have planet.osm imported into a database. Now I am trying to set up hourly updates.
I generated a file changes.osc.gz using osmosis. This file contains updates for 1 hour - which seem to be running EXTREMELY slow. I should add out of desperation I am now using an Amazon machine with 16 processors and 122 GB of memory - with everything running off internal SSD drives (not EBS volumes)
I then try to append these changes using osm2pgsql using the following command:
osm2pgsql --append -s -H db.host -d osm -U user -C 64000 --number-processes 16 --cache-strategy dense --hstore --hstore-add-index -K -G -W -p osm --extra-attributes -v changes.osc.gzThe options I used are the same as the ones used for the initial import.
I get stats like this:
Processing: Node(120k 0.4k/s) Way(15k 0.17k/s) Relation(431 3.65/s) parse time: 521sThis looks extremely slow - but then the situation gets worse... overall I waited more than 1 hour and the process was nowhere near finished.
Then pending ways are about 32 million.. is this normal for an hourly update?
I really hope I am doing something very wrong. Does anyone have a setup that is feasible with updates? What parameters are you guys using?
Does anyone know if it is even possible to keep an OSM database with updates?
أكثر...
I generated a file changes.osc.gz using osmosis. This file contains updates for 1 hour - which seem to be running EXTREMELY slow. I should add out of desperation I am now using an Amazon machine with 16 processors and 122 GB of memory - with everything running off internal SSD drives (not EBS volumes)
I then try to append these changes using osm2pgsql using the following command:
osm2pgsql --append -s -H db.host -d osm -U user -C 64000 --number-processes 16 --cache-strategy dense --hstore --hstore-add-index -K -G -W -p osm --extra-attributes -v changes.osc.gzThe options I used are the same as the ones used for the initial import.
I get stats like this:
Processing: Node(120k 0.4k/s) Way(15k 0.17k/s) Relation(431 3.65/s) parse time: 521sThis looks extremely slow - but then the situation gets worse... overall I waited more than 1 hour and the process was nowhere near finished.
Then pending ways are about 32 million.. is this normal for an hourly update?
I really hope I am doing something very wrong. Does anyone have a setup that is feasible with updates? What parameters are you guys using?
Does anyone know if it is even possible to keep an OSM database with updates?
أكثر...