Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

Data cleaner tool for v8?

Since v8 does not use.dat files, will there be a new data cleaner program coming out that will work on these data sets?

asked May 24, 2012 by anonymous

1 Answer

+1 vote
There shouldn't be a need for the data cleaner anymore.  All that functionality is already built into V8.
answered May 24, 2012 by Mike Lodder Gold (22,975 points)
Just to clarify, version 8 of SSI Web has its own data management area.  This let's you view, edit, and delete respondent data all from within SSI Web.  You can also export your data and clean it up using some other piece of software, and then import the cleaned data back into SSI Web via a CSV file.  Finally, when doing things like making CHO files or estimating utilities, you can create your own custom filters based on respondent data, i.e. only include respondents who answered a 2 to the gender question.
Hi Brian,

Please allow me to ask an addition question regarding the new data management area.
We usually use a differnt software for the survey and just redirect to Sawtooth when the survey includes a conjoint. This means that after the end of the fieldwork we have to clean the data of both parts - normal survey + conjoint - to have the same database.
In SSI V7 this was very convenient using the data cleaner. I have taken some time to make myself familiar with the new data management area but I haven't found a possibity to achieve what the data cleaner was able to do.
Example: We are fielding a study right now with a sample of 1200 interviews. At the end of the fieldwork there might be 1250 interviews in the data base. We then check the quality of the data (normal survey + conjoint) and decide that we only want to keep a list of 1200 ids. With the aid of the data cleaner it was very easy to keep only the desired 1200 ids.
Does the new data management area also offer this capability or do you have an idea how I could achieve this?

Thanks for your help,
Hi Thomas,

The data management area does not function like the data cleaner in the sense that you can provide it with a filter file to quickly remove respondents.  There are still some (hopefully) easy ways to do it, though.  

My first approach might be just to simply grab the conjoint data from SSI Web and paste it in to a spreadsheet with the other data.  Then if you decide to remove 50 people, import the cleaned CSV back into SSI Web.  If SSI Web detects new variables that aren't part of your SSI Web survey, it will prompt you to tel it know what the new variables are (text versus integer, for example) and import them in.

Another approach might be to add a column to the Sawtooth data with some sort of binary variable, i.e. 0 and 1 (or really just one or the other).  1 might indicate a "clean" respondent you wish to use for building a cho file or utility analysis.  When you add an export job or run utilities, you can filter so that only respondents with a value of 1 are included in the export/analysis.

Perhaps in the future we could update the data management area to include an import filter option, where you could again import a CSV of things like respondent numbers, if the current functionality doesn't meet the needs of our users.
Hi Brian,

Thanks for your quick reply.
I think option 2 might be a good idea.
When I clean the data of our current survey (perhaps at the beginning of next week), I will try your suggestions.

Thanks for your help,
Since it is a CSV why don't you just filter the bad respondents using excel and not import those respondents in the first place.  The other option is to tag the bad respondents with a merged variable like Brian McEwan is suggesting.  You can do this just like you did with the data cleaner, but just add a second column of data to the bad respondent list.  So in the end you would have a two column CSV, one with the respondent numbers, two with the filter merged variable.
Thanks Mike,

I will let you know how I did the cleaning next week.

Kind regards,
I can't seem to import the new "cleaned" data and overwrite the old...any hints would be appreciated.

I assume that you have already imported your data file.
By importing a new cleaned data file additional rows are added (please check "Datamanagement" / "View/Edit Data").
You need to delete all rows before importing the new cleaned data file.