This is a joint post with Michael Clemens.
Michael Clemens recently wrote me, saying that he gets asked this question a lot. I do, too. So I was interested when he brought my attention to a 2007 article in Forbes that discusses a number of companies that do use randomized studies. I wasn’t surprised to see Google in the list, but I never imagined that all the junk mail that I receive from Capital One might be guided by sophisticated research (though it hasn’t convinced me to sign up yet!). Progressive Insurance apparently discovered profitable lines of business (middle-aged motorcycle drivers) by randomly accepting a portion of applicants who would normally be rejected and studying their claims behavior. According to Hunt Alcott, other companies that have used randomized studies include H&R Block, PNC Bank, Amazon, Subway and Harrah’s Casino.