The problem
In the oil and gas industry, quick, efficient operations and lean processes are critical. Fairly new to the industry, this company didn’t yet have the infrastructure necessary to integrate all its data, which resulted in employees manually pulling raw data from external sources and loading it into Excel spreadsheets. The client had two huge master Excel spreadsheets – one for horizontal wells and one for vertical wells – with complex functions that pulled data from other spreadsheets. The method they were using to combine data sets required an expensive third-party system to run the transformations and serve up a clean dataset. This data wrangling process took time as well as extensive memory and space on the client’s platform, and the master Excel spreadsheets were so large they caused a significant lag time when opened.
The Blueprint Way
When it comes to exploring for and extracting oil, there are many steps involved, including land procurement, construction, mapping and production measurement processes. Before modernizing the client’s data platform, Blueprint operated as a technical adviser in selecting the best vendor to manage all the aspects of exploration and production within the same system.
The first step to any data platform modernization effort is to understand the current state of a company’s infrastructure. Once the client selected a vendor, Blueprint evaluated all of the client’s data and processes, keeping the client’s primary goals in mind: to consolidate all relevant data into one location and automate data integration, so employees no longer need to spend hours manually hunting down data, and to eliminate the need for “citizen data engineers” spending time and resources wrangling data in massive Excel spreadsheets.
Because of Blueprint’s extensive experience in the industry, we recognized a common problem while evaluating the client’s processes. For data sets from multiple sources to flow together effortlessly, wells need unique identifiers that travel with the data across applications and analysis. When building the client’s data platform, Blueprint established unique identifiers to connect all of the client’s current well data and automated the process for any future data ingestion, eliminating the possibility of data entry mistakes.
After validating and indexing the client’s massive Excel Spreadsheets, Blueprint built 14 data pipelines into an Azure data warehouse where we automated the manual data transformation work being done in Excel. The client now has access to 458 tables of data within its Azure data warehouse with easily accessible views in its Spotfire analytics platform, including financials, production forecasts and realized price differentials. What was previously a 7-step process that included an expensive third-party application to combine and analyze data through Excel is now a simple 2-step process of opening Spotfire and viewing the report.
“Excel can be a great tool for analyzing data,” Blueprint Development Manager Allison Williams said. “But when a document gets that complex, it would be hard to notice if something went wrong, and even more difficult to track down where the problem initiated once it is noticed.”
With a modernized data platform, this company will now be able to quickly and easily access the data necessary to make informed decisions about which land and wells it should acquire to positively impact its strategic growth goals.
