By Sean Clarke, Founder of PacificIQ and EcomIQ

Every Shopify merchant has a version of this story.

You spend weeks on a redesign. Ship it. Push traffic. Then watch conversion quietly fall off a cliff.

And then you spend the next month trying to figure out which change broke it. Was it the homepage hero? The new collections layout? The PDP template? By the time you work it out, you've already lost the revenue.

I've been testing Shopify's new SimGym app for a few weeks, and frankly, it's the first tool I've seen that actually solves this problem properly. Here's how to use it without wasting credits figuring it out yourself.

What SimGym actually does

SimGym is not an analytics tool. It doesn't tell you what's already broken. It tells you what's likely to break before a real customer ever sees your store.

Here's how it works. The app sends AI-powered shoppers through your storefront in a controlled simulation. These aren't random bots. They're personas built from behavioural data across billions of real Shopify transactions, and they browse the way real customers actually do. Land on the homepage, click through to a collection, view a product, add to cart, hit the same friction points your real shoppers hit.

When the sim finishes, you get a report showing exactly where shoppers dropped off, which nav paths they took, where they hesitated, and if you ran a theme comparison, which theme won on add-to-cart rate. You also get individual session recordings so you can watch it play out and see your store through a shopper's eyes.

Shopify describe it as a flight simulator for your storefront. That's actually the right framing. No pilot learns to handle an engine failure mid-flight, they run it in sim first. Same idea.

Who this is actually for

Any merchant can get something out of SimGym, but there are three situations where it genuinely earns its keep.

You're launching a new theme or major redesign. High stakes, real pressure to ship, and traditional A/B testing needs traffic volume you might not have yet. SimGym gives you directional signal before you commit.

You don't have enough traffic to run meaningful A/B tests. Statistical significance needs volume. A store doing 50K sessions a month can run a clean live test in a couple of weeks. A store doing 8K sessions is waiting two months for the same confidence level. SimGym runs independently of your real traffic, so you skip that problem entirely.

You've restructured navigation, collections, or mobile layout. Button colour tweaks and headline changes are low risk. Restructuring how customers move through your store is high risk. That's exactly the kind of change SimGym was built to pressure-test.

If you're running a mature, high-traffic store that already gets clean A/B test results in a week, SimGym is less critical. If you're tweaking PDP copy, it's overkill. Use it for the big structural stuff.

Before you install: the setup checklist

SimGym has a handful of requirements that need to be in place before the app will do anything. Miss one of these and you'll install it, click around, and wonder why nothing's happening. Run through this first.

Shopify Network Intelligence must be on. This is the data layer SimGym uses to generate store-specific simulations. Without it, the app will not run. Open your Shopify admin settings, find Network Intelligence, confirm it's switched on. Hands down the most common reason SimGym appears broken when it's actually just been installed without SNI turned on first.

You need at least two themes installed. The headline use case is theme comparison, which means you need your published theme plus at least one other in your library. If you've only got one, install or duplicate a second one before you go any further.

Your store needs live product listings. SimGym's shoppers need real product content to simulate meaningful behaviour. Empty store equals meaningless simulation.

Your storefront can't be password protected. The AI shoppers need to access your store the same way a real visitor would. Password block, no access.

Hydrogen and headless storefronts aren't supported. If you're on a headless setup, SimGym isn't available to you in this version.

Step by step: running your first simulation

Step 1: Turn on Network Intelligence. In Shopify admin, go to Settings, find Shopify Network Intelligence, confirm it's active. If it isn't, flip it on and give it a few minutes to initialise before you move on.

Step 2: Install SimGym from the Shopify App Store. Search for it, install it. During the AI Research Preview period, access may require joining a waitlist. Once you're in, open it from Apps, SimGym.

Step 3: Create a new simulation. Click Create Simulation and give it a clear name. Something you'll actually recognise in a month, like "BFCM Theme vs Current" or "Nov Nav Restructure Audit". Future you will thank present you.

Step 4: Pick your simulation type. You've got two options.

Theme comparison runs AI shoppers across two theme versions at once and tells you which one wins on add-to-cart rate and navigation behaviour. Use this when you've got a new theme ready and want to validate before publishing.

Single theme analysis runs shoppers through one theme, live or draft, and surfaces friction points without needing a second theme to compare. Use this when you want a general audit of your current store, or when you're pressure-testing a draft theme that isn't polished enough to go head to head yet.

Step 5: Choose a Focus Area (optional, but do it). If you want to test a specific part of your funnel rather than the whole store, set a Focus Area. Options are Home Page, Products, Collections, Cart, and Search.

Directing the simulation at one area gets you sharper, more actionable results, especially when you already know where the potential problem is. Restructured a collection page last week? Focus the sim there.

Step 6: Launch it and wait. Run the sim and let it do its thing. It's significantly faster than a live A/B test. Minutes, not weeks. Nothing for you to do during this bit.

Step 7: Read the report in the right order. When results come back, work through them like this.

Start with the headline recommendation. If you ran a theme comparison, SimGym will surface a winner based on simulated add-to-cart performance. That's your top-line answer before you dig in.

Then navigation and discovery. What paths did shoppers take after the homepage? Which collections did they hit? Which PDPs got views? This tells you whether your information architecture is doing what you thought it was doing.

Then the friction point summary. These are the moments shoppers hesitated, got stuck, or dropped off. Look for patterns. A single drop-off is probably noise. The same drop-off across multiple shoppers is signal, and that's where you focus.

Finally, watch the individual shopper recordings. This is where you get context the summary can't give you. You'll see the store through a shopper's eyes, where their attention went, what pushed them to add to cart, where they bailed.

Step 8: Prioritise fixes, then validate. Use the output to build a prioritised list of changes before you go live. Fix the highest-impact friction first, particularly anything affecting add-to-cart flow or creating dead ends in navigation. Run a follow-up sim if the changes are significant, then validate with real traffic before full rollout. Crawl, walk, run.

What to watch out for

SimGym is in Research Preview. That means it's not production-ready. Expect bugs, edge cases, and limitations that haven't been fully documented yet.

Credits are pay-per-use. Every sim burns credits. Don't run speculative tests just to see what happens. Be deliberate about what you're testing and why.

Results are directional, not definitive. SimGym gives you signal. It doesn't give you certainty. Treat the output as input to your decision, not the final word.

Always validate with real traffic. Even if a theme wins decisively in simulation, run a controlled live test before full rollout. SimGym reduces risk, it doesn't eliminate it.

When SimGym actually makes sense

This tool solves a specific problem. If you're about to launch a redesign and you don't have the traffic volume or time to wait for a traditional A/B test to hit significance, SimGym fills that gap.

If you've already got high traffic and can run clean live tests quickly, you probably don't need it. If you're tweaking button colours and headline copy, it's overkill.

But if you're restructuring navigation, changing your mobile layout, or shipping a new theme and you need directional signal before it goes live, SimGym does something that didn't exist until recently. Used properly, it'll save you from launching a redesign that tanks conversion for a month before you figure out why.

Eyes wide open, credits well spent.

If you want operator-led strategies, real numbers, and the tactics we're actually testing this week, get on the mailing list. You'll get every article and playbook delivered to your inbox before they go live anywhere else. No fluff, no theory, just what's working right now for Shopify brands in the trenches.