Enhanced Conversion Lift Report: FAQs

[CommunityMod_WM]

When should I use the Enhanced Conversion - Lift report? 

The Enhanced Conversion - Lift report represents the incremental value of having Enhanced Content on retailer websites. This report should be referenced to understand the conversion value which can be directly attributed to having enhanced content and may be used to calculate ROI for an enhanced content program when the minimum sample size and minimum effect size has been observed in the Holdout Experiment. 
When the criteria for calculating the Conversion Rate Lift and Incremental Revenue KPIs is not met, the primary purpose of this report is to provide guidance on how enhanced content may be further optimized with engaging widget types to promote a higher conversion rate.  

What do conversions mean in the context of this report?  

The definition of a conversion depends on the site where the enhanced content was published. Approximately one quarter of the sites track the completed transactions and the remaining track clicks on the add to cart buttons as the conversion metric. The list of sites from which Syndigo is able to obtain conversion data is continues to grow with time.  

Why is there no $ sign for the conversion value? 

The conversion being measured is not always in US Dollars (USD). We track conversions on sites from around the world so we do not place the currency sign in the conversion value metric. Use the locale and/or currency filters to isolate conversion value for a specific region and/or currency. 

Why is Conversion Rate Lift over 100%?  

Conversion Rate Lift is not to be confused with the standard Conversion Rate metric. A lift KPI is a comparison – In the case of Syndigo’s Conversion Rate Lift, the conversion rate of one group is compared to another. It is possible that Treatment Group shoppers have a substantially higher conversion rate than the Holdout Group shoppers in the Holdout Experiment, resulting in a lift calculation that is greater than 100%. The following may be a true statement: “Treatment Group shoppers were 215% more likely to add to cart or purchase than the Holdout Group shoppers visiting the same pages during the same time period.”  

Why is Incremental Revenue on the Lift report a different number than the Value in the Enhanced Conversion – Products report?  

Incremental revenue, sometimes referred to as incremental value, is derived from a formula commonly used across many industries to estimate or project the value of sales that can be directly attributed to a specific strategy or solution.  Syndigo’s Incremental Revenue KPI utilizes the conversion rate of the Treatment Group, including the revenue associated with Treatment Group conversions, to estimate the value of conversions that can be attributed to the presence of enhanced content during the shoppers’ experience. Incremental Revenue is generally a smaller number than the total Value as presented in the Enhanced Conversion – Products report. The Enhanced Conversion – Products report is agnostic of the impact of enhanced content and provides all available conversion information from all sources Syndigo has accumulated, resulting in Value being greater than Incremental Revenue.  

Why are there no Conversion Rate Lift and Incremental Value KPIs displayed in the summary tiles at the top of the report?  Why is there messaging stating  “more data is required” at the top of the report?  

Given the timeframe and other filters applied, all the data available to the report is either unqualified based on criteria enforced to promote accuracy or conveys that there is no story of lift to be reported. 

Refer to the article Troubleshooting the Enhanced Conversion – Lift Report for guidance on how to interpret and analyze this result.  

Why are there no websites displayed in the Retailer Summary? Why are some websites I’m expecting to see not included in the Retailer Summary?  

Given the timeframe and other filters applied, the data associated with each missing website is either unqualified based on criteria enforced to promote accuracy or conveys that there is no story of lift to be reported for the given site(s). 

It is more likely to encounter the scenario where overall Conversion Rate Lift is calculated in the summary tile above, but one or more websites do not qualify for the KPIs, as the data is split up by website – thus reducing the number of data points available. 

For example, 100 URLs across 5 websites may have generated enough insights to calculate overall Conversion Rate Lift and Incremental Revenue. However, if each of the 5 websites is only associated with 25 URLs, no 25 URL data set may be large enough to yield statistically significant findings. In this case, the summary tiles may contain KPI calculations, but the Retailer Summary section may contain no further website KPIs.  

Refer to the article Troubleshooting the Enhanced Conversion – Lift Report for guidance on how to interpret and analyze this result.  

What should I do when data is missing or KPIs are not available?  

The article Troubleshooting the Enhanced Conversion – Lift Report covers in detail the recommended steps to understand the reason KPIs are not calculated and how to improve the likelihood the Holdout Experiment will produce statistically significant results in the future.

What do the values in the “Views by Widget” heatmap table mean? How are they calculated?  

The green-colored table visualization in the Content view of the Lift report can be interpreted as a heatmap or a histogram-like chart. The purpose of this information is to show how frequently each widget type was viewed compared to every other widget type available when the Treatment Group shoppers added products to cart or purchased. This visualization speaks to the widget types that may be influencing shoppers positively. The information may serve to compel your organization to reorder the content types or redesign the content overall to elevate the most compelling widgets when possible.

In order to achieve a comparison like this, the report digs deeper into the specific content types that were brought into the browser viewport during the Treatment Group sessions that resulted in conversions. The count of views on each widget type are stacked together, and then a baseline of “1.0” is calculated (this is also known as a Frequency Index), upon which all widget types are indexed.  
If all widget types are viewed during every session, then “1.0” would be the value assigned to every field in the heatmap. As this is not often the case, “1.0” should be thought of as an abstracted baseline. A quick analysis is all that is necessary to reveal which widget types are being viewed most frequently when conversions occur – the higher the number and darker the shade of green, the greater the frequency that the widget type is viewed by shoppers when they add to cart or purchase.  

Example: The data available to Enhanced Conversion – Lift report encompasses a total of 100 Treatment Group sessions where conversions occurred. There are three distinct widget types present on the pages: Feature Set, Specification Tables, and Document Galleries.  

• Feature Sets were viewed in 40 sessions. 
• Specification Tables were viewed in 10 sessions. 
• Document Galleries were viewed in 80 sessions.

Divide the number of sessions for each widget type by the total number of sessions to calculate the Frequency Index. 

• Feature Set FI: 40 / 100 = 40% 
• Specification Table FI: 10 / 100 = 10%
• Document Gallery FI: 80 / 100 = 80%

Average all of the Frequency Index values to derive the baseline.  

(40% + 10% + 80%) / 3 = 43.33%  

The FI of 43.33% is associated with the baseline value of 1.0. Divide each Frequency Index by 43.33% to calculate the indexed value to display in the heatmap.  

• Feature Set Index: 40% / 43.33% = 0.92
• Specification Table Index: 10% / 43.33% = 0.23
• Document Gallery Index: 80% / 43.33% = 1.84  

The heatmap resulting from the above data may then be interpreted as telling the following story: The Document Gallery widget type is viewed at double the frequency of the Feature Set widget type and significantly more frequently than the Specification Table widget type. This may be due to the Document Gallery having a more prominent, eye-catching location on the URLs or it may indicate that widgets should be reordered to test the efficacy of the less frequently viewed widget types, as well.