Sidekick now tests store improvements on AI shoppers

Shopify has updated Sidekick, its AI assistant, so that it can suggest improvements to your online store and then test them before you decide whether to use them. Each suggested change is drafted, compared with your current storefront using AI-simulated shoppers, and presented for your review. It is a useful step towards evidence-led storefront decisions, as long as you understand what the testing is, and what it is not.

What it is

Sidekick can already look at your store and point out things worth changing. The new part is what happens next. Rather than simply handing you an idea, Sidekick drafts a version of each suggested change and runs it against your current storefront using AI-simulated shoppers. You then see the outcome and choose whether to apply the change.

Not every idea goes through this process. According to Shopify's changelog entry, Sidekick only selects changes for simulation if they can be tested as storefront theme changes and are expected to affect add-to-cart behaviour. A suggestion about, say, your returns policy or an email flow would not be a candidate, because it cannot be expressed as a theme change.

How it works and how to access it

The flow, as described by Shopify, has three stages:

• Sidekick finds a possible improvement to your online store.
• It drafts a version of that change and compares it with your current storefront using AI-simulated shoppers.
• You review each suggested change and decide whether to apply it.

The changelog entry does not state which plans or regions the feature is available in, or give rollout timing, so it is worth checking in your own admin and in the Shopify Help Center, which the entry links to for more information.

Limits and caveats

The most important point is stated plainly by Shopify: these recommendations are not tested on real shoppers. The results come from AI-simulated shoppers, and they are not live A/B tests. That distinction matters. A simulation can be a quick, cheap way to filter out weak ideas, but it cannot tell you how your actual customers, with their habits, price sensitivity and loyalty, will respond.

The scope is also narrow by design. Only theme-level changes that are expected to influence add-to-cart behaviour are tested. Anything that depends on the wider customer journey, such as delivery promises, stock availability, or what happens after checkout, sits outside this process.

Finally, because you review each change before applying it, the responsibility for the decision stays with you and your team. That is the right design, but it means someone needs to own the review.

What it means for merchants

For a fashion, streetwear or sportswear brand, the storefront is where a lot of sales are won or lost: product page layout, size information, imagery order, and how clearly the add-to-cart button and key product details are presented. A tool that can suggest and pre-test changes in those areas could save time and reduce the number of weak ideas that reach your live store.

It also lowers the barrier for smaller teams. Not every brand has the traffic to run a statistically meaningful A/B test on every idea. A simulated comparison gives you an early signal to help you prioritise, even though it is no substitute for real customer data.

The sensible approach is to treat Sidekick's tested suggestions as a shortlist generator. If an idea has performed well in simulation and also makes sense against what you know about your customers, it is a good candidate for a proper live test.

What it means for agencies and partners

Agencies should expect merchants to arrive with Sidekick suggestions and ask whether to apply them. That creates a useful role: reviewing each proposed theme change for brand fit, performance impact and accessibility, and checking that it does not conflict with custom theme code or other apps. It also reinforces the case for structured experimentation, where simulated results are validated against real shopper behaviour before a change becomes permanent.

How we would approach it

1. Open Sidekick in your admin and see which storefront suggestions it offers for your store.
2. Read each simulated comparison and note what Sidekick expects to change in add-to-cart behaviour.
3. Sense-check each idea against your own analytics, customer feedback and brand standards.
4. Apply only the changes you would be comfortable defending, and keep a note of what you changed and when.
5. Watch conversion and add-to-cart rates in the weeks afterwards, and be ready to reverse a change that does not deliver.

The short version

Sidekick can now draft storefront changes and compare them with your current store using AI-simulated shoppers, then leave the final call to you. It only tests theme changes expected to affect add-to-cart behaviour, and Shopify is clear that the results are not based on real shoppers and are not live A/B tests. Use it to shortlist ideas, then confirm the important ones with real customer data.

ETTA provides fractional ecommerce leadership for fashion, streetwear and sportswear brands. If you want a senior view on how this affects your store, get in touch.