we are hiring an applied statistical modeler at @getrecast, which will be a good fit for folks with experience doing bayesian statistical modeling at the graduate level. if you or anyone you know might be a good fit, i want to meet them! boards.greenhouse.io/recast/…

I’ve started using a simulation-based approach to research design and analytic workflow. Instead of an a priori power analysis, I draw a causal DAG and then simulate n=3000 or so observations (usually multilevel) to see if my proposed statistical model recovers the trend. (1/n)

I did some A/B tests and holdout tests on various email tactics while at tapestry, and then studied the audience behavior in each cohort for 30-60 days post. If we saw lift at the last click level, we a rarely saw it at the audience level. And when we did see audience lift on day 1 it regressed to the mean by day 30. Most email tactics (SL strat, send times, etc) help you shift revenue into a specific time period or merch category, but don’t drive a lift in LTV. The exception to this was dynamic product recs in each email, but our brands had really big catalogs. If you want to test incrementality of tactics I’d recommend a test where you put two segments into distinct email strategies for 2-4 weeks. Ex one group gets promos, the other doesn’t or one group gets emails only featuring best selling products and the other gets a broader selection of products

Iterating on winning ad angles is killing your growth. Chances are you've found a few winning creatives and you're making a ton of iterations to milk them as much as possible. You try new hooks, new CTAs, the same VO with different visuals, the same visuals with different voiceovers, and so on. Here is an analogy to understand why this is the reason you are capping your scale, without even knowing it: Situation 1: Your current winning angle is probably that yellow dot. When you're iterating on that angle and you're not trying different concepts, the impact you can have is restricted, hence the improvement you can see will be relatively small (green arrow). Situation 2: When you're trying completely different concepts that are targeting completely different audiences from your current ads, the potential to unlock the next level of scale is huge (green arrow). Now, that needs a bunch of creativity and resources to test, and probably the majority of the different concepts that you will create will fail. But then you'll find that one persona with that one messaging that will 2x, 3x your ad spend. The bottom line is that if you're only iterating, you aren't unlocking new personas, which is a huge bottleneck if you want to maintain efficiency while scaling. PS: Of course, you should keep iterating on your winning angles and milk them as much as possible, just don't restrict yourself to that. S/o to @couuor for the idea.

The closer you get to being held accountable to true financial outcomes, the less attribution matters. When money is on the line, incrementality is what’s important.

# on shortification of "learning" There are a lot of videos on YouTube/TikTok etc. that give the appearance of education, but if you look closely they are really just entertainment. This is very convenient for everyone involved : the people watching enjoy thinking they are learning (but actually they are just having fun). The people creating this content also enjoy it because fun has a much larger audience, fame and revenue. But as far as learning goes, this is a trap. This content is an epsilon away from watching the Bachelorette. It's like snacking on those "Garden Veggie Straws", which feel like you're eating healthy vegetables until you look at the ingredients. Learning is not supposed to be fun. It doesn't have to be actively not fun either, but the primary feeling should be that of effort. It should look a lot less like that "10 minute full body" workout from your local digital media creator and a lot more like a serious session at the gym. You want the mental equivalent of sweating. It's not that the quickie doesn't do anything, it's just that it is wildly suboptimal if you actually care to learn. I find it helpful to explicitly declare your intent up front as a sharp, binary variable in your mind. If you are consuming content: are you trying to be entertained or are you trying to learn? And if you are creating content: are you trying to entertain or are you trying to teach? You'll go down a different path in each case. Attempts to seek the stuff in between actually clamp to zero. So for those who actually want to learn. Unless you are trying to learn something narrow and specific, close those tabs with quick blog posts. Close those tabs of "Learn XYZ in 10 minutes". Consider the opportunity cost of snacking and seek the meal - the textbooks, docs, papers, manuals, longform. Allocate a 4 hour window. Don't just read, take notes, re-read, re-phrase, process, manipulate, learn. And for those actually trying to educate, please consider writing/recording longform, designed for someone to get "sweaty", especially in today's era of quantity over quality. Give someone a real workout. This is what I aspire to in my own educational work too. My audience will decrease. The ones that remain might not even like it. But at least we'll learn something.

It never ceases to amaze me how in the google universe, if you’re purely looking at platform or click reported attribution, you’d invest in the exact opposite way that you should be to drive your business. CPA = platform reported CPA from google CPIA = incremental CPIA adjusted for conversions that would’ve happened anyway (measured by comparing to a control group)

"More than half—56%—of all action outcomes, according to Meta, are attributable solely to creative factors."