A global renewable energy leader partnered with us to enhance their pricing capabilities using Databricks, improving revenue efficiency and overcoming data challenges to optimize financial decisions.
Streamlining data infrastructure and analytics for a leading electrical cooperative
A large organization consisting of multiple electrical cooperatives in the energy industry sought to modernize their data infrastructure by migrating extensive datasets to Azure Databricks. Blueprint collaborated with their engineering and data teams to acquire and organize this data for analytics, leveraging the Lakehouse Optimizer to ensure cost transparency and continuous improvement within their Databricks platform. This transformation facilitated their ability to forecast energy prices and anticipate weather-related outages, ultimately projecting an annual savings of over $170,000. Blueprint’s efforts not only streamlined their data processes but also laid the foundation for future analytics projects and for improved operations, enabling them to better serve millions of customers with reliable and affordable electricity.
Client Snapshot
Who:
A large organization of electrical cooperatives
Industry:
Energy
Stakeholders:
Director of Data Science and Director of Analytics
Work Summary
What we did:
- Migrated large amounts of data to Azure Databricks
- Organized the data for analytics
- Implemented the Lakehouse Optimizer powered by Blueprint to enable cost transparency and offer insights into what they can improve within their Databricks platform
Client background
Our customer is one of the largest generation and transmission electric cooperatives in the nation, providing reliable, affordable electricity to millions of customers. Additionally, each non-profit within the cooperative goes a step further to offer innovative energy solutions that enhance the quality of life for millions within their state. They own power generation assets, purchase electricity through contracts, and coordinate transmission resources for their members. Their data is a valuable resource that aids in forecasting energy prices and understanding weather patterns to better plan for outages.
The Blueprint way
This customer aimed to migrate their data to Azure Databricks, emphasizing data quality and understanding data origins. Blueprint assisted in eliminating disparate data sources, enabling a smooth migration. Post-migration, Blueprint’s Lakehouse Optimizer provided cost transparency and long-term monitoring of Azure Databricks.
"
I finally wrapped my head around this UDF and have to say, this is very impressive.
Software Developer
The challenge
As the leading organization overseeing many other cooperatives, their biggest challenge was the siloed data, which was not easily accessible. Blueprint assisted in unifying the orchestration and processing of disparate data sources into a cohesive system. Despite their ambitious plans for utilizing their data, they were hindered by limited time and resources to complete the migration efficiently and at scale. Additionally, being new to Databricks, they needed someone with high expertise to ensure their data was accessible and unified.
The solution
The Blueprint team successfully migrated 3 large data products that include many pipelines within each. Our client created a metadata framework (MDF) driven approach using Microsoft SQL Server and Synapse Pipeline, which our team altered and enhanced. The primary effort was focused on migrating outdated ETL processes that utilized a variety of tools, ranging from Visual Basic to shell scripting, to their MDF process and Databricks. The majority of the pipelines that were transitioned ingested data from FTP/SFTP endpoints. The Blueprint team saved countless hours of labor and costs by eliminating irrelevant reports and outdated legacy data that had not been used for years.
"
This (Blueprint’s Lakehouse Optimizer) is a very clean and powerful UI. Very intuitive!
Director of Software Services
Impact
- Transformed engineering processes to align with new technology.
- Modernized operations by migrating on-premises workflows and data to Azure.
- Implemented the Lakehouse Optimizer to govern their lakehouse, projecting annual savings of over $170,000
- Blueprint’s engineers set the foundation for all future analytics projects with consolidated data.
- Data is now readily available for analytics, enabling accurate energy price forecasting and weather condition anticipation.
Share with your network
You may also enjoy
MissionWired partnered with Blueprint for a successful Unity Catalog migration, optimizing Databricks’ features and Lakehouse spend. The project exceeded expectations with 143 refactored notebooks and comprehensive post-migration training.
