Clustering memory limit?

Post any questions or comments regarding the Desktop PC version of HYSPLIT. This includes the model execution, GUIs, results, or graphics. Be sure to include the operating system you are using (Windows XP, Windows Vista, Windows 7, Windows 8, etc.

Clustering memory limit?

Postby arl.noaa » November 13th, 2012, 9:16 am

During my research project I am running the Hysplit model to calculate back-trajectories at a site for the period 1998-2011. The final goal is to do a cluster analysis of my back-trajectories and to link the obtained clusters with the concentrations of some atmosheric tracers measured at that site. Since I have data only for some periods, the final dataset consists of a total 9020 trajectories. When I start the cluster calculation on my PC-Windows I get this message: " ERROR - allocation cluster mean array: 40675690 "
I think that it comes from the limited RAM or memory disk or both, but I do not know exactly. Could you help me and tell the characteristics for a PC that I need to run the cluster calculation? Moreover, I cannot find a version for a PC Linux-based.
Thank you very much for your help
arl.noaa
 
Posts: 32
Joined: November 9th, 2012, 2:49 pm

Re: Clustering memory limit?

Postby glenn.rolph » November 13th, 2012, 9:18 am

Hello,

Yes, it is a memory issue. We haven't used this cluster program on more than about 1000 trajectories. The limit will depend on your particular PC, but we don't have a simple way to calculate that limit. Try to reduce the number of trajectories and/or number of endpoints along the trajectory that are used in the cluster calculation.

There is a linux version of the HYSPLIT system, however scripts to run the cluster program are not included. This is something we could add in the future. If reducing the memory requirements above is not sufficient, please contact us for the linux version.
glenn.rolph
 
Posts: 102
Joined: November 7th, 2012, 1:39 pm

Re: Clustering memory limit?

Postby egribbin » March 27th, 2013, 12:23 pm

I would definitely agree that it's a memory limit. I've tried clustering a large number of trajectories (about 2000) on a PC [Intel Core 2 Duo E8500 @ 3.16GHz, Windows XP Pro, 4GB RAM] and then the same PC (same processor, Windows 7 Enterprise 64-bit, and 10GB RAM]. With a little experimentation, I found that the clustering on the latter set up tends to "crash" somewhere close to 900 trajectories - your mileage may vary of course.

Are there any hypotheses on whether these limitations can be overcome on a linux system or cluster?

Thanks in advance!

Erik
egribbin
 
Posts: 10
Joined: November 19th, 2012, 6:45 pm
Location: Austin, Texas

Re: Clustering memory limit?

Postby Ramirez » April 7th, 2013, 4:15 pm

Hello,
I have similar issue. I think your answer can be useful for me.
Thank you
Ramirez
 
Posts: 1
Joined: April 7th, 2013, 1:32 pm


Return to HYSPLIT for PCs

Who is online

Users browsing this forum: No registered users and 1 guest

cron