A global renewable energy leader partnered with us to enhance their pricing capabilities using Databricks, improving revenue efficiency and overcoming data challenges to optimize financial decisions.
How IGS Energy achieved 83% Databricks cost optimization
IGS Energy is a large retail natural gas and electric supplier in the United States with a workforce of more than 1,500 people. IGS is focused on clean energy solutions and has adopted an ambitious goal to be carbon neutral by 2040. To do this, they are using new tools and innovative ideas to re-imagine the future of energy in a way that will benefit their customers, effectively manage resources, and protect energy systems.
Client Snapshot
Who:
IGS Energy
Industry:
Natural gas and electric supply
Technology and software:
Lakehouse Optimizer, Databricks
Work Summary
What we Did:
- Cost optimization through job scheduling
- Pipeline creation to ingest platform and infrastructure consumption data into Delta tables
Results:
- 25% cost optimization in phase one
- 58% additional cost optimization in phase two
The Challenge
IGS Energy provides customers options to source energy that’s right for them, which means allowing them to manage their costs and carbon footprint. To help provide cost-effective product, their data science team leverages Databricks to produce forecasts for future energy demand. Blueprint was tapped to help IGS gain deeper visibility into their Databricks cost and performance data while optimizing their infrastructure. As a result, Blueprint helped IGS maximize performance and reduce total cost of ownership (TCO). Savings gleaned from streamlined infrastructure and lowered operational costs could then be passed along to customers by way of continued affordable energy solutions and investments made into additional green energy solutions that would have long-term and widespread positive impact.
The Blueprint Way
Blueprint initiated a dual-phased approach that bookended a 6-month interim adoption and monitoring period.
Phase 1
Installation & Adoption
Phase 2
Analysis & Optimization
Phase one
Duration:
6 months
Goals:
- Information and feedback gathering
- Installation of the Lakehouse Optimizer into the client’s Databricks environment
- Incorporation of the Lakehouse Optimizer into daily rhythm of business
Following a brief discovery period with IGS it was determined that their goal of gaining clarity into their Databricks environment and finding ways to optimize infrastructure and identify cost-savings opportunities could be met with the installation of Blueprint’s proprietary monitoring software, the Lakehouse Optimizer.
As is the case with many Databricks users, the data available was not easily accessible and not persisted, which makes it difficult to organize, analyze, and action on their own. The Lakehouse Optimizer solves this by presenting the data in a way that is easily accessible and actionable, with real-time monitoring and feedback on results on reconfigurations. The Lakehouse Optimizer also offers Databricks and Azure visibility within one view, removing the need to toggle between platforms and saving both time and effort when reviewing change impacts between the two.
The installation of the Lakehouse Optimizer is a seamless process, allowing IGS to be up and running with the tool quickly. The IGS team was able to access the dashboards and move into the interim phase within two weeks.
During phase one, the IGS team was able to get comfortable using the tool and use their existing Databricks and Spark expertise to begin testing reconfigurations and identifying optimization opportunities immediately. The Lakehouse Optimizer is designed with UX at the forefront, making it user-friendly and intuitive across varying technical abilities. Because of this, the IGS team was able to utilize the Lakehouse Optimizer capabilities to identify anomalies and surface 25 percent in cost-savings—all without sacrificing any business value or needing to employ additional consulting services.
Phase two
Duration:
Two weeks
Goals:
- Analysis and review of data collected during interim phase
- Offer recommendations for additional cost-savings
Phase one allowed the IGS team to explore the benefits of the Lakehouse Optimizer to better manage their data and surface previously unseen savings opportunities. Phase two was focused on building on what had already been accomplished at IGS.
Blueprint began by analyzing the data collected in phase one and reviewing the IGS team’s self-administered modifications for additional optimization opportunities. As a result of this analysis, Blueprint recommended additional actions that would improve performance and contribute to reduced TCO. These recommendations included a proposed solution to a business-critical impediment not previously addressed: Several jobs that produced forecasts for the Supply organization were often coming close to, or exceeding, their threshold for when the reporting was due. Each lapse forced critical business decisions to be made without the timely data that was expected and needed. This was contributing to wasted time and resources across the organization, causing downstream impact on business value and stifling innovation. Blueprint worked to identify the jobs with the highest risk for negative business impact and reconfigured them to run faster and at a lower cost.
Blueprint’s expertise in data analysis along with recommendations that included job configuration and right-sizing instance types for workload and capacity requirements helped IGS to realize an additional 58 percent in cost-savings. Blueprint is a team of Azure cloud and Databricks utilization experts. We know the common pitfalls in implementations and understand cost accountabilities. We guide our clients in identifying what will make the biggest impact on their business—and we do it quickly.
Client Testimonial
Dan Shah
Manager IT Services
“The Lakehouse Optimizer transformed our cost understanding. Through data analysis, we gained unprecedented clarity, empowering us to make informed decisions and prioritize changes effortlessly”
Ready to get full visibility into your Databricks environment?
You may also enjoy
MissionWired partnered with Blueprint for a successful Unity Catalog migration, optimizing Databricks’ features and Lakehouse spend. The project exceeded expectations with 143 refactored notebooks and comprehensive post-migration training.
