Sequential ID Query in V8

Question

With v8, we do not have the ability to export completed surveys only via the admin module. The filtering out of completes is now conducted via the Data Management / Export option within the Sawtooth software.

My concern with this change is the incompletes get mixed in with the completes and when using the sys_SequentialRespNum, the IDs are no longer sequential, they contain holes in the numbering over a far wider range of ID numbers.

This is okay for small projects but creates a new problem for larger surveys, like one that I am currently working on where 40,000+ completed surveys are captured and over 100,000 incompletes.

Is it possible to include a sequential ID feature which exports IDs with no gaps in the numbering (like a sequentail start)?

Thanks.

Comments

Hi Paul,

Could you comment a bit more about the benefits and drawbacks of this area in v8?  I can understand having to download a larger file, but I'm not sure if I understand the downside to having your IDs be sequential from 1-40,000 versus 1-100,000 with gaps as long as it's always the same number when you export.

---

Thanks for the comments Brian. If I export 40,000 surveys with gaps so that the last survey may have an ID of say 100,000, I need to initialise (or make room) in our analysis software for up to 100,000 records, not 40,000 records. This causes our data file in our analysis software to be much larger in size and also take longer to process when running outputs.

Our analysis software has a feature called ""sequence start exporting" which exports all the surveys out and renumbers them from 1-40,000 then I import them back in so I get my desired result. This whole process increased my time and effort in getting the data from Sawtooth to our analysis software.

Not sure if you can add in a feature that exports the ID's from 1-40,000 with the gaps removed? I know I would find it handy. Not sure about the remaining Sawtooth world?

---

Thanks, I think I understand the issue.  Do you think it would be a valid workaround for the upcoming project to download the data from the admin module to CSV, filter by respondent status to just get a CSV of completes, and then import the data?

---

That's exactly the procedure I followed but the ID's were spread across 1-100,000 instead of 1-40,000. Because the spread of the additional ID's, I had to setup additional room in my analysis software for 100,000 ID's, not 40,000 ID's. If the sys_SequentialRespNum variable was sequential from 1-40,000, I would have an easier task.

Is it possible to add in a new ID when exporting without the gaps?

---

Sorry, I forgot to add one part.  I was thinking perhaps to download the data to CSV from the admin module, take the existing respondent numbers and move them to another column (if desired), and then in Excel just go 1-40,000 and then import the data file to SSI Web.

I'll talk to the developers about adding back in a download only completes option from the admin module.

---

Thanks Brian. I like the idea of downloading only the completes from the host. I know there is a filtering option via the data management menu option but the host option of downloading completes only adds more flexibility and fixes the issue I raised.