Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

Increasing performance of HB estimation

Hi All,

As the complexity and sample sizes of our studies increase, we have noticed that the amount of time needed to run the HB is increasing exponentially - sometimes taking as long as 4 days,

I was wondering what hardware/software recommendations you can make which will decrease the processing time required for this analysis,

Many Thanks,

Dean
asked Jun 13, 2012 by djtindall Bronze (600 points)
retagged Sep 13, 2012 by Walter Williams

2 Answers

+3 votes
You may want to read the section I wrote for the CBC/HB manual in Appendix L: Hardware Recommendations.  It covers some general performance principles and practices.

In addition, solid state hard drives tend to be much faster than normal hard drives.  If money is no object, then you may consider looking at fusionio.com which has solutions faster than SSD.

In CBC/HB v5 there is also the skip factor for displaying in the graph, which reduces the amount of data sent to the graphing module.  When doing large numbers of variables or iterations, the graph can be storing lots of data, and increasing the skip factor will improve performance.

Performace is also affected by the design.  Some researchers will use a 'kitchen sink' approach and specify many covariates, interactions, and constraints, all of which affect performance.  If including a covariate, interaction, or constraint, you should first look to see if it makes sense to include it by doing some preliminary analysis like counts.  You may also want to look at attribute patterns to see if you can estimate some as linear instead of part-worth, resulting in fewer parameters to estimate.
answered Jun 13, 2012 by Walter Williams Gold Sawtooth Software, Inc. (23,305 points)
Thanks Walter, this appendix file should definitely help in my business case for getting upgraded software,

Dean
If you are using version 4 of CBC/HB, then you are missing out.  The 64-bit abilities of v5 are worth it alone.  Also beginning with v5.2 we are using memory-mapped I/O to increase memory performance and availability.

If you have to make a case to your superiors, you should call our support staff about getting a trial license (a few days) for v5.  You can take the time it took v4 to run a dataset vs the time it takes v5 and show that to them.
Looking at the information available it looks like the biggest improvement will be moving over to a 64bit system - I already use cbc v5.3 so hopefully will see an improvement from that then.
+2 votes
Hi Dean,

a bit more info may help us determine what will work:
 1)What type of hardware are U currently running?
 2) What operating system
3) how big are the files U are modelling.
4) are u including interactions or is it only main effects?
5) Is it 'thrashing" ie is your hard drive making alot of noise as data gets written to and from it?

In general U need as much RAM to reduce the need for any writing to hard drive. And if U are writing to hard drive then a very quick hard drive will help (such as a SSD drive).

A faster processor may help, but sawtooth is usually I/O bound not CPU bound, and the above things help an I/O bound process.

Also I just asked a question on optimisation a few days ago, so there are some helpful hints in there too.
answered Jun 13, 2012 by anonymous
I am stuck on a 32 bit Windows XP machine at the moment, so that leaves me with a maximum of around 2-3Gig Ram,

I notice that when I run the analysis it seems to be more CPU intensive than RAM intensive, and it definitely makes a lot of noise when running,

The biggest file recently was around 6,000 respondents and had around 50 parameters
...