Have an idea?

Visit Sawtooth Software Feedback to share your ideas on how we can improve our products.

Testing surveys in Lighthouse Studio

We have been using SSI Web 8.4.8. With this version, I can append &hid_test_mode=2 to the end of the survey URL to turn on/off the options (e.g. question/variable names, randomization, skip logic, etc). 

I find that adding &hid_test_mode=2 doesn' work with Lighthouse Studio. 9.2. However, I can still go to the Test Mode (with options) from within the Admin page. 

I have installed the Chrome extensions (Random Select Radio Buttons) and (Multicheck Checkbox Checker) to help me speed up the checking/testing process. However, I find these extensions no longer work under the Test Mode (their functionalities are disabled). However, they still work under the Live mode. I will like to use both these Chrome extensions as well as turning on/off question/variable names and randomization during testing at the same time. 

How can I achieve this? Is there a workaround?
asked Jan 11, 2017 by JKincaid Bronze (1,035 points)

2 Answers

0 votes
The value for hid_test_mode to enter test mode is now a little more secure.  There is a randomly generated ID that has to match a value stored in your survey files for test mode to work properly.  If you do Test Survey in Lighthouse Studio, you will see what that ID is for that particular survey in the URL.  

Example:
http://localhost:61501/Test%20Survey/cgi-bin/testmode.pl?studyname=MYSTUDY&url=http://localhost:61501/Test%20Survey/cgi-bin/ciwweb.pl?hid_studyname=MYSTUDY&hid_test_mode=STUDY_ID&hid_pagenum=0

You will need this whole URL to enter Test Mode.  You can also get this URL by entering Test Mode through the Admin Module.

As for your other question, the new Test Mode uses an <iframe> so you can simulate different screen sizes.  My guess is that this is throwing off your extension because there are no radio buttons or checkboxes on the actual html page that is being displayed.  They are all contained in the <iframe> element when in Test Mode.  Maybe there is a way to tell the extension you're using to select things in the <iframe> instead of the containing html page.

Have you tried using the Data Generator in Lighthouse Studio?  You can run it locally on your machine or remotely.  To test remotely and get the random checkbox/radio behavior you want, you would go to Field -> Generate Data.  By default, the Data Generator answers everything randomly but you can specify a certain answer or range of answers for each question if you want to test specific logic.  If you want to walk through some parts of the survey and then randomly generate other sections, you can check Use Browser and then put Stop() in the Defined Values tab of the Data Generator window on the answer you want to walk through manually.  Hopefully this helps.
answered Jan 11, 2017 by Jon Heaton Gold Sawtooth Software, Inc. (10,380 points)
edited Jan 11, 2017 by Jon Heaton
Thanks Jon.

My live test link is https://versiontest.insightiqsurvey.com.au/cgi-bin/ciwweb.pl?studyname=VersionTest&task=test

My local machine test link is http://localhost:49915/Test%20Survey/cgi-bin/testmode.pl?studyname=VersionTest&url=http://localhost:49915/Test%20Survey/cgi-bin/ciwweb.pl?hid_studyname=VersionTest&hid_test_mode=1K6UP39ALUS0JB3T3W9AE7BD4PCN5LLN&hid_pagenum=0

The test link from Test Mode for live survey is https://versiontest.insightiqsurvey.com.au/cgi-bin/testmode.pl?studyname=VersionTest&url=ciwweb.pl?hid_studyname=VersionTest&hid_pagenum=0&hid_test_mode=1K6UP39ALUS0JB3T3W9AE7BD4PCN5LLN&hid_test_db=1

When I append hid_test_mode=1K6UP39ALUS0JB3T3W9AE7BD4PCN5LLN to the live Test link so that the URL becomes https://versiontest.insightiqsurvey.com.au/cgi-bin/ciwweb.pl?studyname=VersionTest&task=test&hid_test_mode=1K6UP39ALUS0JB3T3W9AE7BD4PCN5LLN, it doesn't work - the options do not appear.

The two Chrome extensions are installed from Google Play store , there are no setting/options and even no way to contact the authors.

These two extensions prove to be extremely useful because there may be hundreds of checkboxes/radio buttons (within grids) in a single survey. Due to frequent changes, additions and issues identified, the same survey may be tested dozens of times and it is a pain to manually tick the checkboxes and select the radio buttons each time.

The two Chrome extensions offer an automated, randomized way of quickly making selections and I would say help to cut down the testing time by as much as 90%, thus greatly increasing work productivity and rapidly getting the survey to live.

Is there any way to turn on/off the iframe (screen size simulation functionality) in the Test Mode? So that I can use the two extensions.

Will Sawtooth consider introducing such "random select of radio buttons/check boxes" with click of a button in its software? This will greatly help the survey designers.

The Data Generator is good but we need something that allows us to visually inspect each survey page/screen and have control of what we want to try on each page as we test progressively from start to end of survey.
The benefit of using the data generator is that the random responses still fall within the settings of your Lighthouse Studio question.  For example, if you require between 4 and 6 responses to a specific multi-select question, the data generator will answer the correct number of responses and then submit the page.  In Lighthouse Studio menu, go to Field -> Generate Data and select Use Browser on the first screen.  On the Defined Values tab, check Use Defined Values and then put Stop() on the question that you want to visually inspect as shown in the following screenshot:

https://drive.google.com/file/d/0B00GnyaXzy1BbXUwcloxU2YxbVU/view?usp=sharing

Then you would click Generate Data and you will see the survey run on your live server and answer things properly until your Stop() question as specified in Defined Values.  Next to the Play/Pause button, you will see two right arrows.  The first one will just answer the next question on the page and the second one will answer all the rest of the questions on the page.  If you just press Play again, data generation will continue until your next Stop() specified in Defined Values.  

The only downsides to this method are that it only tests one browser, Internet Explorer, and you don't get the browser developer tools (F12) that you get in a full browser.

You may need to do some extra manual testing in other browsers, but you should be able to test all survey logic through the built-in data generator.

I will speak to my colleague in charge of the Test Mode features and see if an "Answer page randomly" button would be simple and useful for a good number of customers.

Thank you for your feedback and please try the data generator solution I proposed since your other request for a button or to turn off the iframe would only be considered for a future release of Lighthouse Studio.
0 votes
I know it's not as convenient as using the Chrome extensions, but in the mean time you can randomly answer radios / checks using console commands.  Here's how you can do it:

1. Open the survey you want to test.
2. Open the dev tools in your browser by pressing F12, then navigate to a tab labelled "Console."
3. There may be a dropdown with two options: "top" and "display_iframe." You should use the latter.
4. Click into the console, paste the below code, and submit it by pressing Enter:

function answerAllRadiosAndChecks() {
    var radioGroups = {};
    $('#page input[type=radio]').each(function(){
        radioGroups[this.name] = radioGroups[this.name] || [];
        radioGroups[this.name].push(this.id);
    });
    for (var name in radioGroups) {
        var radioGroup = radioGroups[name];
        var selectedRadio = radioGroup[Math.floor(Math.random() * radioGroup.length)];
        SSI_SetSelect(selectedRadio, true);
    }
    
    $('#page input[type=checkbox]').each(function(){
        SSI_SetSelect(this.name, Math.random() > 0.5);
    });
}


5. Continue through your survey.  At any point, you can submit this code through the console to randomly answer all radios and checkboxes:

answerAllRadiosAndChecks()
answered Jan 11, 2017 by Zachary Platinum Sawtooth Software, Inc. (171,525 points)
Hi Zachary,

Thanks for your code.

I find that I need to paste the entire function answerAllRadiosAndChecks() code everytime in addition to the calling command answerAllRadiosAndChecks().

Otherwise this will generate an Uncaught ReferenceError: answerAllRadiosAndChecks is not defined at <anonymous>:1:1

For Select question with checkboxes, can you alter this code to select all checkboxes except for those exclusive option(s)?

This is because we often code new scripts based on similiar past scripts. The no. of checkboxes for a question may have changed in the new script but sometimes we forget to update the max. no. of checks, which we discover through testing.

E.g. we may have 30 checkboxes in the new question and we want the respondent to tick 1 to a max of 29 (the last option is the exclusive None of These). The previous question this new question is based on may have only 25 checkboxes. But I may forget to update the max to 29. During testing, if I check all the 29 checkboxes (except for the exclusive None of These) and click the Next button, I will get an error saying that I can select only up to 25 options. This testing helps me to identify my scripting error.
Sorry you have to paste the whole thing.  If you add "answerAllRadiosAndChecks()" as a new last line to the code, you should end up with just one thing to paste in the console.

If you don't know about it already, the functionality of the up and down arrows in the console may be helpful to you.  You can toggle through previous console inputs using the arrow keys.  So if you need to run this code repeatedly, you can click up arrow once then Enter to submit the same code again.

As for exclusive checkboxes, there are a few options.  I could add something to the code like "[Q1_4, Q2_r5_c10]" that could tell the code certain checkboxes to skip, but that's not ideal when you are dealing with lots of questions.  The better option would be if the JavaScript could determine on its own which checkboxes are exclusive.

I'm looking into this now.  Checkboxes in select questions seem pretty easy to do.  Checkboxes in grid questions are a bit trickier, though.
It required a bit of a hack, but I've got it working for grids now:

function answerAllRadiosAndChecks() {
    // Radio buttons
    var radioGroups = {};
    $('#page input[type=radio]').each(function(){
        radioGroups[this.name] = radioGroups[this.name] || [];
        radioGroups[this.name].push(this.id);
    });
    for (var name in radioGroups) {
        var radioGroup = radioGroups[name];
        var selectedRadio = radioGroup[Math.floor(Math.random() * radioGroup.length)];
        SSI_SetSelect(selectedRadio, true);
    }
    
    // Select checkboxes
    $('#page .question.select').each(function(){
        var qname = this.id.replace(/_div/, '');
        var exclusives = ($('input[name=hid_noneabove_' + qname + ']').val() || '').split(',')
            .map(function(item) { return qname + '_' + item });
        $(this).find('input[type=checkbox]').each(function(){
            if (exclusives.indexOf(this.name) == -1) {
                SSI_SetSelect(this.name, Math.random() > 0.5);
            }
        });
    });
    
    // Grid checkboxes
    $('#page .question.grid').each(function(){
        var qname = this.id.replace(/_div/, '');
        var exclusiveValues = ($('input[name=hid_noneabove_' + qname + ']').val() || '').split(',');
        var exclusiveTest;
        if ($(this).find('.input_cell').attr('class').split(' ').pop().startsWith(qname + '_r')) { // row-oriented
            exclusiveTest = function(name) {
                var ret = false;
                exclusiveValues.forEach(function(exclusiveValue){
                    var regex = new RegExp('^' + qname + '_r([0-9]+)_c' + exclusiveValue + '$');
                    if (regex.test(name)) {
                        ret = true;
                    }
                });
                return ret;
            };
        }
        else { // col-oriented
            exclusiveTest = function(name) {
                var ret = false;
                exclusiveValues.forEach(function(exclusiveValue){
                    var regex = new RegExp('^' + qname + '_r' + exclusiveValue + '_c([0-9]+)$');
                    if (regex.test(name)) {
                        ret = true;
                    }
                });
                return ret;
            };
        }
        $(this).find('input[type=checkbox]').each(function(){
            if (!exclusiveTest(this.name)) {
                SSI_SetSelect(this.name, Math.random() > 0.5);
            }
        });
    });
    
    // Other checkboxes
    $('#page .question:not(.select):not(.grid) input[type=checkbox]').each(function(){
        SSI_SetSelect(this.name, Math.random() > 0.5);
    });
}
Hi Zachary,

Thanks for your help.

I tried with your new code and for multi response select questions, it didn't select all the check boxes.

I had taken a screenshot which you can find here: https://lh3.googleusercontent.com/ZdhF-jkGWPHYyAKUco30DRJM-at9V6Cyg7rhOP_rCYhI5L73PXgGCfr14BWv_q2X0NNj9SsYFZs2lijM8AnhE83TnQcNp-ZGWQ3ShZuj3v3WpDIQWK_9lo8AkIbnHohh9KZ0=w799-h604-no
Sorry.  I had assumed you wanted checkboxes to be checked randomly.  A quick edit to the above code should make it check all non-exclusive options.  In all three locations that this appears:

Math.random() > 0.5


that code must be replaced with this:

true
Hi Zachary,

This code works with Select type of question. But does not work with a MR grid. It doesn't tick any check boxes at all - see this screenshot:
https://lh3.googleusercontent.com/pvwaplI-yv80UXEePcll72kVOqngYUu8whI9UwpqqgWaILD_oGIE66-A0DVTNfYRVNSdgv1ere9OTSk=w1600-h770-rw

Thanks.
Grid checkboxes appear to work on my end.  Is it possible your console is on "top" instead of "display_iframe"?
You are absolutely right. What a silly mistake I make. Thank you so much for your help.
Thanks Zachary, I had created a Chrome extension based on your codes. The only problem is that it selects group headers in MR select questions.

I modify your code by inserting this:
$(this).find('input[type=checkbox]').not(".group_header_row input[type=checkbox]")

This is working for Sawtooth 8.4.8, haven't tested with Lightning Studio 9.2.
...