Natural Gas Infrastructure Firm Modernises Records Management Processes for Future Growth and Compliance

As a leader in the highly regulated energy industry, this client’s Information Technology (IT) team was struggling with how to design and implement a M365 and Azure solution to centrally store documents, allowing advanced search and discovery of its operations files, from new projects to ongoing maintenance. With tens of millions of existing and new files, the team needed a repository that would consolidate content, integrating it with its line of business systems and its existing asset hierarchy. The primary challenge was scaling the solution to allow upwards of 100 million documents for the future enterprise-wide deployment, which involved handling vast amounts of data securely and ensuring seamless integration with existing systems and identifying multiple advanced migration scenarios to populate the system from existing technical and physical locations. This also included the planning for AI integration via automated metadata extraction and LLM integration with the documents in the repository.

A three-phased strategy

Following discovery, we determined the project would consist of multiple tracks of build and migration, using three different delivery models to successfully analyse, plan, and build the new operations document repository and migration of operations content to Microsoft 365 (M365).

Protiviti designed for M365 and Azure, including a structured Teams environment for governance, integration into their master data system, unified search, and a migration plan to populate the repository with the appropriate documents. The project tracks included:

Track 1 – Operations Teams Workspaces discovery and design

• Bringing Protiviti’s UX/UI and M365 expertise to design modern, effective Teams Workspaces unique to the Operations department requirements.

Track 2 – Migration analysis and planning

• Discovery and analysis of existing operations content in multiple source repositories and recommending tool(s) and processes for migration including AI automated metadata extraction.

Track 3 – Operations repository implementation

• Implementation of the approved design of the operations repository in SharePoint Online and Azure including custom SPFx development and integration with line-of-business systems

Getting to work

We introduced a holistic approach that included strategic planning, technical expertise, and change management:

• Technical execution: We began with a thorough assessment of existing digital infrastructure to ensure compatibility and smooth integrations, developing custom scripts and tools to automate much of the data migration process and minimised errors during data transfer. We built an integration layer on Azure that sat between multiple source systems and SharePoint Online to funnel documents through and perform metadata management both manually and automatically via the new integration layer. This allowed us to handle the scale of uploading large batches of files to SharePoint. We also built a modern end user experience in SPFx on SharePoint for searching and sorting capabilities from such a large repository.  
• Risk management: Understanding the risks associated with large-scale data build and migration, the team implemented robust security measures to protect sensitive information during and after the transfer. This included automated code deployment and strong security within Azure for the integration layer.  
• Training and support: To address the migration’s human aspect, testing and education programmes were tailored for different user groups within the organisation, delivered through webinars, face-to-face workshops, and detailed manuals.  

Separate workstreams were established for each of the three project tracks. 

For Track 1, we:

• Configured and implemented the approved design based on  
• SPFx web parts for Operations Teams collaboration and discovery
• Created testing scripts for end users and admins​ and tracked issues and enhancements
• Finalised site map and navigation​ and established an organisational change management plan for launch

For Track 2, we:

• Performed detailed analysis in preparation for migrations​
• Established migration technical architecture ​across multiple migration servers using automated migration functionality through Azure pipelines
• Performed functional testing​ and production pilot migrations from source data to SharePoint Online

And, for Track 3, we:

• Developed a strong backlog and executed sprint planning​ based on the custom development plan for the new operations document repository
• Configured and implemented the approved design of the repository​ in Azure and SharePoint Online
• Created and executed end-user and admin testing  
• ​Implemented the organisational change management plan​ and launched the new Repository to the operations team
• Maintained backlog for enhancement requests​ which now includes integration with new source systems for more automated document uploads

Mission accomplished

The client now has a new location to maintain documents by their asset hierarchy, improving the accuracy of record metadata tagging, and providing a better employee experience for collaborating in Teams. The solutions delivered ensure the client remains compliant in the highly regulated energy and utility safety industry as it tackles architecture planning for future-state AI enablement. Benefits include:

• Enhanced operational efficiency: With SharePoint Online already fully migrated into its operations, the client experienced noticeable improvements in workflow efficiencies thanks to better discovery tools and streamlined access to information via the new Repository
• Removal of technical debt: SharePoint 2016 was a key repository used by the client with infrastructure and maintenance costs to be maintained. Through data migrations, this platform is on the path to decommissioning.
• Empowered employees: The new repository provides a central place for employees to find materials that they used to have to scour physical and digital locations with the hope of finding anything.

Throughout the project, we were able to alleviate capacity constraints from critical client IT resources, allowing them to focus on other key initiatives within the organisation. The client’s IT team led oversight of the approach and architecture determinations, with our project team executing and driving the migration efforts. The commitment and collaboration from across the client’s organisation provided invaluable insights critical to the success of the project.