Unlock the power of Looker Block for Cortex Framew...

shashanktp · 10-15-2024 09:27 AM

Introduction

Businesses need a robust and scalable platform to derive actionable insights from their data. Looker, combined with the Cortex Framework, offers a powerful solution for building and deploying impactful data applications. This article provides a step-by-step guide for deploying Cortex Framework when using Looker Block, while sharing best practices to ensure a successful implementation.

Understanding Looker Block and Cortex

Looker is a cutting-edge business intelligence (BI) platform that enables users to explore, analyze, and visualize data from various sources. Looker Blocks are pre-built data models designed for common analytical scenarios within Looker. Rather than creating LookML views and exploring from scratch, you can leverage these ready-made building blocks, available on Looker Marketplace. This feature accelerates the delivery of insights with minimal setup, making it a valuable tool for both developers and end-users. Setting up a Looker Block using a BigQuery connection is straightforward and is often the default option available in the Looker Marketplace.

Cortex is a data framework that provides pre-built data models and metrics, accelerating the development of Looker applications. Together, they enable organizations to quickly create and deploy data applications that cater to their specific needs. Cortex Data Foundation is the core architectural component of the Cortex Framework reference architecture and provides packaged analytics artifacts which can be automatically deployed for use with Google Cloud BigQuery.

Steps for Looker Block deployment

These are the step required for the installation of Looker Block Deployment :

Project Setup and Planning
Looker Block Installation
Data Connection and Modeling
Looker Explore and Dashboard Development

We will cover each step in detail:

Project Setup and Planning: Begin by setting up a new Looker project and defining the scope of your deployment. Identify the key business questions you want to address and the data sources you’ll need to integrate. If installing Looker Block from Marketplace make sure you have enabled the Marketplace in the Admin -> Platform -> Marketplace. Note: In order to enable the Marketplace you will need to have admin access in Looker.

Enable Marketplace in Looker

Once enabled you can access Marketplace from the right corner in Looker as seen below:

Marketplace Icon in Looker

Data Connection and Modeling: Connect Looker to your data sources by creating Database Connection and leverage the Cortex data models to structure your data. Customize the models to fit your organization’s unique requirements and create additional metrics or dimensions as needed. Setting up a database connection in Looker involves defining the connection details that allow Looker to interact with your database.

Follow these steps to configure a database connection in Looker:

Click on Admin -> Databases -> Connections -> Add Connection

Enter the connection details by giving the connection name, Dialect (e.g. MySQL, PostgreSQL, BigQuery, etc.), Billing Project ID, Dataset and Authentication (Service Account or OAuth). In order to set up. Persistent Derived Tables (PDTs): Enable this if you need PDTs.

Big Query Connection

Once all fields are filled out, click Test These Settings. Looker will try to connect to the database using the information provided. Once tested successfully save this connection.

Additional Tips:

Service Accounts: If connecting to databases like Google BigQuery(Standard/Legacy), you may need to set up a service account and provide appropriate roles (such as BigQuery Job User and BigQuery Data Editor).
Persistent Derived Tables: You will need to create the Temp Database manually in BigQuery and Specify the Temp Database in PDT Section.

Once the connection is set up, it’s ready to be used for LookML modeling and creating reports in Looker.

Looker Block Installation for Cortex: Install the necessary Cortex blocks from Looker Marketplace or set up the forked Github Repository. These blocks contain pre-built data models, metrics, and dashboards that align with common business use cases.

There are two options for installing Looker Block depending on the Model:

Installing directly from the Marketplace: If the Looker Block is available in the Marketplace, install directly from the UI. Example: we can install Cloud Cost Management: Google Cloud model from the Marketplace. This is the most simple installation using Marketplace. After clicking on the default install you can agree to the license and provide the connection name which contains the billing data and click on install as described below. This will create a Marketplace Project. Click on the Develop Panel -> Projects. You will be able to see the Cloud Cost Management Project.

2. Manually install Looker Block: The Cortex sap block in Looker is not directly available in Marketplace UI. You can still install this using one of the below options.

Option A: Marketplace Install via Git

Refer to the Looker documentation for Installing a Tool from Marketplace. Provide values for the required prompts. The only cons with this installation is you can’t customize the LookML to fit your unique business needs in case of any changes required in the pre-built Dashboards.

Option B: Manual Install via Fork of this Repository

With the Looker project based on your forked repository, you can customize the LookML to fit your unique business needs. Since Option B is the most used method for installing Looker Block to fit your unique business need will be talking about the detail steps required to do this deployment end to end below:

Step 1: Fork the Cortex SAP Block Github Repository by clicking on Fork as shown below:

Fork the Cortex Repository

Step 2: Once you fork the repository, create a blank LookML project with any name (e.g. cortex_sap_block) as described in the link. Once you create the cortex_sap_block you can configure the git as shown in Step 3.
Step 3: Connect the new LookML project to the forked repository

Configure Git

After clicking on Configure Git. Copy the SSH Github url of the repository as seen below:

SSH Github Url

Paste this copied SSH Github (see below image). Copy the Deploy key and paste this in Github when generating the SSH key as modeled in the below step.

For creating the SSH Key in Github: Click on Github Profile -> Settings -> SSH and GPG Keys -> New SSH Key

Step 4: Update the values of constants in the manifest.lkml file and commit the changes by validating the LookML. Once committed, you can deploy to Production which merges the code with the production branch. The Pre-built Dashboards will have a cortex_sap_operational model since that’s the name of the file (see highlighted below).

manifest.lkml

Looker Explore and Dashboard Development: You can find the pre-built Dashboards in Folders -> LookML dashboards (see below). In case you want to fit your unique business needs on top of the pre-built Dashboards, you can change the LookML files or add new LookML files. Commit the changes and deploy to production. After exiting the development mode you will be able to see these Dashboards.

Pre-built LookML Dashboard

Below are examples of pre-built Finance Dashboards which are deployed:

Accounts Payable :
Find financial information such as accounts payable, accounts payable turnover, overdue payables, accounts payable aging, and cash discount utilization.Accounts Payable Dashboard

Accounts Receivable:

Analyze total receivables, overdue receivables, days outstanding, and top companies with highest receivables.

Accounts Receivable

Benefit of using Option B: Manual Install via Fork of this Repository:

One major advantage when using Option B: Manual Install via Fork of this Repository is that you can add another LookML dashboard by creating a separate model file in the same project. Commit and push the changes to the local repository then deploy it to production. The below snippet shows cortex_latest_inventory_operational.model which is another model file if you are looking to fit your unique business needs on top of the pre-built Dashboards.

Additional Model File

Conclusion:

Deploying the Looker Block for the Cortex Framework is a powerful way to enhance your data analysis capabilities. This guide has walked you through the essential steps, from project setup to the final stages of Explore and Dashboard development. While there are multiple ways to install Looker Blocks, the manual installation via a fork of the repository offers the most flexibility and is widely adopted by users. By following this comprehensive guide, you can leverage the full potential of Looker Blocks to drive insightful analytics within your organization.

Thank you for reading! Have questions? Ask me in the comments below!