Randomly show ad to be rated if the quota for that ad is not filled.

Question

I have 4 ads I need to have rated on several questions. Each respondent will see one ad randomly selected provided the quota for that ad is not filled (100 ratings per ad). Looking for ideas on the best way to achieve this. Thanks in advance!

Answer 1

Hi Chris, this type of question appears to be a popular item of late.

Here's a solution I hope you find helpful ...

Create a quota question and call it QTAds with the following settings applied:

Value=1 / Cell Name=Ad1 / Cell Limit=100 / Qualification=ListValue(CompleteConList,1)=1

Value=2 / Cell Name=Ad2 / Cell Limit=100 / Qualification=ListValue(CompleteConList,1)=2

Value=3 / Cell Name=Ad3 / Cell Limit=100 / Qualification=ListValue(CompleteConList,1)=3

Value=4 / Cell Name=Ad4 / Cell Limit=100 / Qualification=ListValue(CompleteConList,1)=4

*Note: I will explain the "Qualification" below.

Create a parent list called AdList. It will look like this ...

1-Ad 1
2-Ad 2
3-Ad 3
4-Ad 4

Now create a constructed list called AdConList which will use AdList as the parent list. Insert this Perl script ...

Begin Unverified Perl 

 my $i=1;

 for($i=1; $i<=4; $i++)
  {
   if (ISQUOTACELLOPEN("QTAds",$i)==1)
    {  
     ADD("AdList",$i);
    }  

  RANDOMIZE();

  }

End Unverified

This constructed list selects all ad's that have an open quota and randomises them.

Now create another parent list and call it CompleteList. It will look like this ...

1-[%QuotaCellCompletes(QTAds,1)%]
2-[%QuotaCellCompletes(QTAds,2)%]
3-[%QuotaCellCompletes(QTAds,3)%]
4-[%QuotaCellCompletes(QTAds,4)%]

This parent list reads in all 4 completes counts for each quota.

Now moving onto the last constructed list. Call it CompleteConList. It will use the CompleteList as the parent list  ...

Mirror(AdConList)
SortByLabel()

This constructed list will select the open quotas only and sort them from lowest to highest based on their completes count. The first item on the list will be the quota (or ad) that requires the most number of competes at the time to reach their target of 100.

In your skips for each ad (or ad block of questions), use the following conditions ...

Ask Ad 1 if: ListValue(CompleteConList,1)=1
Ask Ad 2 if: ListValue(CompleteConList,1)=2
Ask Ad 3 if: ListValue(CompleteConList,1)=3
Ask Ad 4 if: ListValue(CompleteConList,1)=4

So this solution reaches out and selects ad's that have open quotas only and then selects from that list, the ad that requires the most filling at the time.

Good luck!

Comments

Thank you...this is awesome and works perfectly!!!

---

What if each respondent should have to see two ads randomly selected provided the quota for that ad is not filled (100 ratings per ad)??

Many thanks!

---

If there are 2 ads required per respondent, you could split the quotas out separately as 4 separate quota questions: QTAd1, QTAd2, QTAd3, QTAd4.

Quota questions only allow one cell to be true. This method will enable multiple quotas to be catered for.

The qualifications would be as follows ...

Ad1: Qualification=ListValue(CompleteConList,1)=1 or ListValue(CompleteConList,2)=1
Ad2: Qualification=ListValue(CompleteConList,1)=2 or ListValue(CompleteConList,2)=2
Ad3: Qualification=ListValue(CompleteConList,1)=3 or ListValue(CompleteConList,2)=3
Ad4: Qualification=ListValue(CompleteConList,1)=4 or ListValue(CompleteConList,2)=4

The AdConList constructed list would look something like this ...

Begin Unverified Perl 
 
 my $i=1;
 
 for($i=1; $i<=4; $i++)
  {
   if (ISQUOTACELLOPEN("QTAd".$i,1)==1)
    {  
     ADD("AdList",$i);
    }  
 
  RANDOMIZE();
 
  }
 
End Unverified

The CompleteList will look like this ...

1-[%QuotaCellCompletes(QTAd1,1)%]
2-[%QuotaCellCompletes(QTAd2,1)%]
3-[%QuotaCellCompletes(QTAd3,1)%]
4-[%QuotaCellCompletes(QTAd4,1)%]

And your skips for each ad would use the following conditions ...

Ask Ad 1 if: ListValue(CompleteConList,1)=1 or ListValue(CompleteConList,2)=1
Ask Ad 2 if: ListValue(CompleteConList,1)=2 or ListValue(CompleteConList,2)=2
Ask Ad 3 if: ListValue(CompleteConList,1)=3 or ListValue(CompleteConList,2)=3
Ask Ad 4 if: ListValue(CompleteConList,1)=4 or ListValue(CompleteConList,2)=4

Let me know how you go. If you need a working example, I can whip an ssi file up. I am using v9.3.1 at the moment.

---

Many thanks Paul! I will try what you are proposing next week, I will let you know if it works.

---

According to your method, single quota is successful, but multiple quotas are not successful, please help me.
Thanks.

---

This method was something I would use in an earlier version of Sawtooth Software, before least fill quotas were available within a quota question. It works on single quotas only.

It still comes in handy today.

Look to the Question Library on the Sawtooth Software website at the multi quota feature which you can download and import into your software ...

https://sawtoothsoftware.com/resources/question-library/multi-select-quota

This was put together by Zachary Anderson who is an employee at Sawtooth Software. He has put together some very nice features which you should browse through.

If you have any queries with a multi quota least fill type issue, I would recommend starting a new post as this will generate interest from Zachary and others.

Answer 2

As always, Paul's solution is fantastic.  I've found it especially helpful when each quota is defined by an earlier response or some qualifier.  Here are a couple of other very basic ideas you can consider as well depending on your situation.  For your quota question, create the four quota cells but set each to always qualify.  Then on your Advanced tab of the quota question, set Check Cell Membership to Randomize All.  This will keep your quotas "somewhat" even as you progress and once one fills it will just select from the other 3.  I've had this be an issue for me if a study didn't reach the total quota for some reason, then the quotas may not be even at that point.  Another method I use is to define 4 quota as follows.  Following Paul's naming, assume your quota question is called QTAds:
Cell 1 - QuotaCellRemaining(QTAds, 1) > QuotaCellRemaining(QTAds, 2)
Cell 2 - QuotaCellRemaining(QTAds, 2) > QuotaCellRemaining(QTAds, 3)
Cell 3 - QuotaCellRemaining(QTAds, 3) > QuotaCellRemaining(QTAds, 4)
Cell 4 - Always Qualify

This will wind up assigning each quota in sequence (4,3,2,1....4,3,2,1) until they are all closed.  This helps assure they remain even throughout the course of your study.  You can also use QuotaCellCompletes in a similar way.

Now you have several different methods you can use depending on your situation.  Each has it's best used place, you just need to decide what works best for your situation.  Hope this helps.

Comments

Yes, nice approach Jay.

The "Constructed List" method versus the "Cell Membership" method. Both do a great job. And it says a bit about the power of Sawtooth Software in that it has a number of techniques to handle complex problems.

Glad I have the privilege of using this software.

And thanks for chipping in with another neat idea. Top notch!

---

Thanks for the assistance. This is awesome :)