How to accurately analyze user engagement across multiple sources

[LookerExperts]

This content, written by Dillon Morrison, was initially posted in Looker Blog on Mar 2, 2016. The content is subject to limited support.

Most businesses face questions about company performance and customer behavior. How well did our product launch go last week? Are customers using our new features the way we intended? How can I take action to encourage more product usage or user engagement?

To answer these questions at Looker, we make strong use of rigorous event tracking and analytics. Many other companies do the same, but most only scratch the surface of this analysis. With the proliferation of analytics tools in the market, businesses often think that high-quality funnels, attribution analysis, user engagement analysis, etc. are simple plug-and-play exercises. Tools like Mixpanel, Google Analytics, and Kissmetrics often advance this belief.

These tools provide wonderful, intuitive user front ends on top of a well-constructed data collection framework, but they incorporate inflexible metrics. These metrics are great for understanding high-level performance, but abstract metrics like Daily Active Users (DAU) and pageviews don’t tell the full story. Ultimately, these tools don’t allow you to drill down and don’t provide enough customization to uncover the meaningful insights you need to share across your organization. Accuracy and impact are sacrificed for ease of use and standardization.

As Looker’s Chief Analytics Officer Colin Zima , most successful companies understand that event collection and event analysis are discrete and independent tasks: strength in one area does not necessarily translate to strength in the other. To resolve issues of drilling and inflexible analysis, there’s a strong trend toward decoupling the tools used for data collection and for data analysis. More and more companies (including Looker and a host of our customers) use a data collection tool to gather the data, then use Looker to perform analysis.

Segment Warehouses and Looker

Segment, a leader in data collection, recently launched a new offering: . With Warehouses, customers have the ability to load their web, mobile, and server user data directly into their centralized Amazon Redshift or Postgres database without building a data pipeline. Segment makes all event data from disparate sources accessible via a single API, which eliminates the burdensome and complicated process of linking and maintaining multiple data streams. Within hours, customers have access to all of their behavioral data in a single location, in a schema that’s pre-optimized for analytics with SQL. All data is collected and structured, simply and effectively.

In this centralized data warehouse, companies combine transactional data with behavioral data, creating a powerful and nimble environment for analysis. Looker connects directly to Segment Warehouses and provides a modeling layer to define metrics and business logic for the data. Looker also recently launched a Looker Block for Segment, which provides Segment customers with a predefined Looker data model, templated to work with directly Segment’s standard schema. After getting started with the Block, users can add or modify any additional business logic they’d like. The result is a robust set of both industry standard and completely customized metrics and dashboards that provide an advanced starting point for event analytics. Looker’s user-friendly interface then enables any end user to explore all this data through an intuitive UI that executes real-time SQL queries based on the data definitions established in the modeling layer. Best of all, with Segment + Looker, the entire setup takes days rather than months.

To demonstrate the difference between our solution and some other tools, let’s look at one of the most universally accepted staples of event analytics: user engagement.

User engagement

Event data allows us to create analysis to examine user / customer engagement within a web page, product, or application. Most tools come with a fairly comprehensive set of high-level engagement metrics: DAU, MAU, average time spent on page, amount of usage of each product feature, and so forth. These prebaked metrics provide a great view of high-level performance. But what actionable insights can these metrics reveal? They offer only a glance at pre-aggregated analysis, so you can’t get down to user-level data and sessions. To make this analysis actionable, you need the ability to drill down and the ability to create custom metrics.

"We were using Google Analytics before we moved to Looker, and that was great for aggregate level. But, when I wanted to break it down, and know how much our average order value is, I want to know a distribution. I want to know who our top customers are and who we should be spending money on and things like that. It just wasn't possible." -Sarah Vigrass, Strategic Project Manager, Lyst

With a modeling layer like Looker’s, we can continuously refine elements like user cohorts based on any attribute or action to help us understand how and why different customer groups are engaging with our product or service differently. The ability to drill down into funnels to look at user sessions tells us why users followed certain event paths, rather than just telling us that users did follow those paths. This process of drilling down to enable data discovery bridges the gap between understanding and action.

A company also needs to see custom metrics to effectively evaluate customer engagement. Since each web page, product, or application is inherently different from that of its competitors, each company may (and in most cases should) want to calculate user or customer behavior differently as well.

“Looker allowed us to design our own custom business rules and really build our business intelligence around the things that we care about. Because data lives in the database and Looker is an analytics layer on top of our database, we had unique flexibility to use LookML, the proprietary language of Looker, to expose the data in a format, in a way, and in a fashion that we wanted to, to our team. That was a really big win, just being able to code those business rules ourselves as opposed to having to ship our data to an outside warehouse and have analysts create custom but mostly boilerplate dashboards.” -Vlad Dubovskiy, Data Scientist, DonorsChoose.org

Another example we’ve used here at Looker to display the importance of custom event metrics is for a gaming company. Companies that produce game apps (such as Candy Crush and FarmVille) put out monthly rankings based on DAU and MAU. Again, these metrics give a clear view into number of users but tell us nothing about how users are navigating the games. They give no guidance about how to improve specific aspects of the game. To solve for this, one company decided to create custom metrics that revealed, for instance, how many users who started a level actually completed it.

The above graphic shows that early users had a tough time beating level one, but once they did, they tended to stick around. However, three different levels later in the game seemed to drive users away. Since mobile gaming companies can quickly modify their products and deploy updates, this company was able to revisit those levels and turn a quick iteration to remove the user hurdles from those levels. After doing so, the company saw more users completing those levels and staying engaged with the product. It even saw some churned users returning.

Stay proactive, not reactive

We strongly believe that event data is integral to the success of any web or mobile-based product or application. Countless examples and our gaming company demonstrate the strategic edge companies gain over competitors who lack the data infrastructure to create comparable, rich analysis. In our world of rapidly evolving industries, companies that fail to take advantage of these risk stagnation.

Accurate data collection across multiple web pages and applications is a complex task, requiring continuous refinement and monitoring of back-end tracking or scripting processes. If your company needs to understand user engagement at a meaningful, actionable level, you should consider a decoupled collection and analytics solutions. Segment Warehouses manages the entire collection process, end to end, for all your customer and user behavior. Plus, setup is quick and intuitive. Looker provides the drilling and customization functionality, in addition to the high-level metrics other tools provide. Both and offer free trials. If you’re interested in the power of deep user analysis, check us out!