Your Results Are a Joke
Last week I spoke at the Ads4Dough conference in San Diego. When I got invited by Khoa, it sounded like I was being invited to speak at a little party get together of 10 or 20 of his buddies who were also internet marketers.
When I saw Nicky Cakes, the facebook expert, was speaking I realized the game had stepped up. That was when I saw the title of my talk for the first time, facebook Ads Crash Course to Profitability. After doing a little research I realized there would be at least a few heavy hitters coming, and I only had a few days to prepare.
I decided to see if I really was what I was speaking on, so I put together an affiliate campaign in a few days, and tried to make some money. It backed out, made a fistfull of dollars, and I presented it along with some other secret sauce. Turned out in fact to be one of the most concentrated group of successful guys I have ever been with in the internet business world.
Walking that night to go see the fight with Dr. Ngo, Nicky and a bunch of other guys, I was called out on the fact that the results of my case study, were largely irrelevant due to the small sample size. I looked at him and said, “yes”. It is what it is, right?
So when doing stuff for real, and not just trying to fluff something up for a good stage show, here is how I do it.
Feel free to chime in if you see any defect to my logic, but what I do is way simple and as I understand it quite valid.
I’ve gone all the way through calculus in college, but try to keep math to very, very practical, fast and simple applications.
In the work I have been doing with Tim Ash lately of Site Tuners, who by the way hands down is THE expert on landing page design and statistical analysis, (Check out his Conversion Conference and use promo code CCW503 to save some money and make me some! I get a commission.). This is one conference that can make you a lot of money.
Bottom line, is that until you have about 1000 of some action taking place, your statistics could very well just be random chance. By the way, just for the record, I honestly do believe that if I decide something is going to take place strongly enough it will. 🙂 The stats just record it.
Here is how to verify that the numbers you are getting are real, and not just chance.
We are only going to need a few numbers here.
The first one is sample size. How many of the first thing are you looking at? With facebook advertising, it could be how many clicks did you get on your ad, which translates into how many people hit your landing page?
Then next number is the observed result. Say 100 people clicked on your ad and went to your landing page (sample size), then how many of those people bought your product, which is the observed result. 100 people to the landing page, and 10 of them bought. Or it could be another conversion, such as 5433 people landed on your landing page, and 234 of them clicked on the link to your online store. You can figure out if that conversion makes money or not, but was it just luck? or if you roll this out big will the results continue to be the same? How much could it vary?
For any large sample size, such as all the women in the U.S., there is a percentage that will buy when they hit your landing page. Since you are not going to be able to afford to show your ad to all of them just to test, then you will just show a small sample of them your landing page, measure how many convert, and then figure out how reliable the results are.
There will be some variance between what your sample test result shows, and what the entire U.S. will be. I am about to show you what that variance will be, and I will make it easy.
This is a site I found years ago, that makes it really simple for doing this variance analysis.
To keep your interest, I have added 6th grade style drawings to the screen shot to describe this, as well as photoshoping in one of my winning dating ad pictures I have used in Europe in the lower right side of the picture.
Keep one eye on the following picture, and continue reading below.
Where the alligator fish is biting the percentage, you enter in 90%, which means you want the results to be 90% accurate. If you can afford to make a mistake, leave it at 90%. If failing at this will bankrupt you, then switch it to 95% or 99%.
Enter the sample size in step #2. This is by the fish’s lower lip. Fish do have lips you know. The sample size is how many of the first thing you have. So if 45000 people clicked on your facebook ad, and you are going to see how many of them download your game, then the sample size is 45000. If 300 people go to their shopping cart, and 34 of them check out, then 300 people is your sample size.
Next go to step 3 where the arrow is pointing. Here is where you enter the observed result. This how many of the desired action took place. Of the 300 people who hit the shopping cart, how many of them checked out? How many of the girls that you asked out, actually said yes? Of the opportunities you had for the good thing to take place, how many times did it actually take place?
To the right of the frequency that you just entered, the percentage will automatically pop up to its right. This is cause since your math aint so great, and firing up excel would have taken too much time, it is done for you. This is the percentage of the opportunities you had, that the good thing took place.
Then look down at the subliminally out of focused picture of the girl to make her look more approachable, and a realistic option that she might date you, and you will find the calculate button.
Push It. But not on my photoshopped photo, but on the real site!
It will then give you a range.
You will find that range has at its center, the percent that you just looked at.
What this is saying now, (in my example which is real by the way), that of the 209 clicks I got on facebook today, 5 of them did the thing I wanted them to, which was 2.39% of them. In reality, this sample was too small to be of much use, since the bottom line is saying, that if I keep running this ad and have 100,000 people click on it, that the final result will be between .65% (less than a percent) and 4.13% (which would be rad!). That number at the top? Well I am 90% sure this is true which is good enough for this since I can afford to fail at it.
So this lets me know that my results are mostly meaningless, that it is very unlikely my result will be over 4% of the clicks converting (which in this case is o.k.), and that I need to get a lot more clicks before I know if this ad is working.
Here is the stats tool.
The upside of this, is that I should put my energy toward spending more money on clicks, before I even bother wasting my time, trying to test new ads or landing pages.
So tomorrow I am going to crank this up, and if you are an Eastern European interested in dancing and horses, you will be likely to see my ad! Then tomorrow night, with more data, I will run the numbers again.
If this was at all confusing, just leave in the comments below where you got lost and I will try to help.
If you are a math weanie, and saw something I was inaccurate on, please let me know, so other math weanies can not their heads a the accuracy of this post afterwards.
