How to implement an effective cloud governance framework

What is cloud governance, and why is it important?

Cloud governance is a set of practices that help businesses take full advantage of the cloud while keeping risks in check.

Effective cloud governance helps enable the following:

• Users work in the cloud in the ways they want.
• Cloud operations are more efficient.
• Cloud use is more economical.
• IT departments and security teams can monitor cloud deployments and make corrections as needed.
• The organization minimizes data breach risk.

A cloud governance framework is not a new set of concepts or practices, but simply the application of existing governance practices to cloud operations. Note that cloud governance can apply to any cloud architecture or strategy type. Whether you use only public clouds, such as AWS and Microsoft Azure, rely on a private cloud or use a mix of both models in a hybrid strategy, cloud governance can help achieve an optimal balance between the risks and rewards that come with using the cloud.

How to implement a cloud governance framework

There is no one-size-fits-all approach to implementing cloud governance because every organization's cloud resources, business processes and risk tolerance are different. That said, building a cloud governance framework typically boils down to the following key steps:

Assess cloud risks

Analyze cloud resources and configurations to determine where the risks are.

For example, a review of cloud data storage services might reveal sensitive data assets that the organization should protect to prevent data breaches. The review might also surface issues with data quality, which governance practices could also mitigate.

Likewise, assessing virtual machines (VMs) hosted in the cloud could highlight risks associated with network configurations or insecure operating system environments.

Define risk tolerance levels

After identifying risks, decide which ones to manage through cloud governance policies.

Importantly, this process should focus on balancing acceptable risk with feasible governance practices. Organizations should expect to tolerate some risk because they cannot avoid it entirely. However, an effective cloud governance framework can minimize risks in the areas that matter most to the business.

Define cloud governance policies

Next, write governance policies. Governance policies are formal descriptions of the goals the business wants to achieve in areas such as efficiency, cost-effectiveness and security.

Typically, governance policies focus on what the governance outcome should be, but they don't spell out exactly how to achieve that outcome (this is the purpose of procedures, which I'll get to in a minute). For example, a cloud data governance policy might say that the organization will avoid storing sensitive data without encrypting it. But it won't describe how to find sensitive data or which tools to use to encrypt it.

Decoupling policies from procedures is valuable because overarching governance objectives tend to stay the same, but technologies change quickly. Policies are where you define objectives, while you use procedures to describe the specific technology that achieves those objectives.

In many cases, organizations already have governance policies in place that they implemented to help manage on-premises resources. If that's the case, it might be possible to extend those policies to cover the cloud as well.

But if existing policies don't align well with the unique challenges of the cloud -- such as managing highly complex cloud bills or the performance of workloads hosted on infrastructure that someone else owns -- you'll want to write new policies from scratch.

Set governance procedures

Next, write governance procedures to define the tools and practices necessary to achieve the goals established in the governance policies.

For instance, a policy related to cloud data governance could describe the data discovery tools and processes an organization should use to keep track of sensitive data, as well as how it will encrypt that data.

Review and update policies

To ensure that governance policies and procedures remain up to date as risks, priorities and technologies change, review them regularly. Generally, governance reviews should happen at least annually, although some organizations implement them as often as once a quarter.

What are the principles of cloud governance?

A cloud management strategy must incorporate several key aspects of cloud governance, which are essential to institute proper controls and optimize cloud service use.

Key cloud governance principles include:

• Security and compliance management.
• Financial management.
• Operations management.
• Data management.
• Performance management.
• Asset and configuration management.

Rather than being isolated and independent, these objectives influence and, in some cases, constrain each other. Data management and security are interwoven. Operations management and cost controls overlap and influence each other, while operations management also helps shape how an enterprise implements data lifecycle management policies. Developers and product managers can choose a specialized data loss protection service to boost security, but it can be prohibitively expensive.

Let's walk through each aspect of a cloud governance framework and how to accomplish them.

1. Security and compliance management

Cloud governance includes the same security topics found in any enterprise security effort: risk assessment, identity and access management (IAM), data encryption and key management, application security, contingency planning and many other areas. From a governance perspective, business objectives and regulations shape information security practice objectives.

As you formulate information security practices, understand that you'll need to choose tradeoffs between business expediency and security risks. For example, you can try to eliminate all moderate and severe vulnerabilities in your applications, but you'll have to shift IT resources from developing new features to remediating code vulnerabilities. Balance product development and other business considerations with the government and security regulations that apply to your business.

A governance model should build on existing governance policies and frameworks, including cybersecurity, privacy and risk management. For example, National Institute of Standards and Technology (NIST) cybersecurity resources point to frameworks for those three. Also, take advantage of your public cloud provider's specialized security services to mitigate the risk of data leaks, denial-of-service attacks and other common threats.

2. Financial management

An unwelcome rite of passage in enterprise IT is surviving the first explosively high cloud computing bill. Cloud service providers and advocates rightly argue that cloud services make more financial sense than buying and managing your own infrastructure. That's true, but only if you effectively control your cloud costs with diligent policies and reporting.

Financial management policies provide a framework to make business decisions about cloud resources. For example, one organization might use managed services as much as possible to reduce operational costs. Another business might create a checklist of cost management steps to follow before deploying a new service to a public cloud.

In the context of budgeting, it can be difficult to estimate cloud costs because the detailed information you need is often distributed across multiple services. For example, a billing summary might have subtotals for object storage, but details about the contents stored in those systems might only be available from the storage service itself. To calculate the total cost, a business might need to search across various regions, accounts and cloud services.

Develop a plan to gather information to create and track budgets. Most cloud vendors provide cost-reporting tools. If those do not meet your needs, look to third-party services to fill the gap.

Set up realistic policies around cost alerts. If your cloud environment exceeds 50% of its budget just a few days into the month, an alert gives you time to adjust your infrastructure and service use. Many alerts reflect real-time use and spending, but others might arrive after you've breached a spending threshold, so create a budget and policies that give you extra flexibility.

3. Operations management

Operations management controls how cloud resources deliver services. Consider the following action items:

• Define rules and processes that control how to create new applications or workloads that run in the cloud.
• Set service-level agreements (SLAs) to allocate resources.
• Deploy application code to various environments, particularly production environments.
• Monitor the state of services to ensure SLAs are met.

Perhaps a developer or product manager will ask how best to deliver a new application to customers. The answer should be found in a well-defined operations policy that says how to do the following:

• Coordinate with the operations team.
• Specify IAM requirements.
• Estimate compute, storage and network requirements.
• Meet monitoring and logging requirements.

Moreover, a clear, well-defined operations management practice is one of the best ways to prevent shadow IT operations from creeping into your cloud environment. Good cost and performance monitoring can also help identify when cloud resources are deployed outside of normal operating procedures.

4. Data management

As the ability to collect, store and analyze data expands, so does the difficulty of effectively managing that data. Your governance strategy and practices should include clear guidance on managing the full lifecycle of data in your organization.

Begin with a data classification scheme. Not all data is equally valuable or needs comparable levels of security. Sensitive and confidential data warrants more security controls than public information. The best practice for data in the cloud is to encrypt all data in transit and at rest, and this should be considered the default behavior. Other controls, such as who can access or update particular data types, will vary according to the data classification and functional requirements of how the data is used.

Governance policies help data owners, product managers and application developers understand how to protect data based on its classification. This includes guidance on how to manage the data lifecycle, such as how long to store data and when to move data from high-performance (and high-cost) storage systems to lower-cost archival systems. Manual data lifecycle management does not scale well and is error-prone. Take advantage of cloud providers' data management tools to migrate data automatically to different storage systems or delete data that is no longer useful.

5. Performance management

Performance management in cloud computing focuses on monitoring applications and infrastructure resources to ensure you deliver expected levels of IT services and efficient cloud infrastructure use.

Application performance metrics vary depending on the application. Some common metrics include:

• Latency when retrieving data, loading webpages or calling API functions.
• Number of database transactions per time period.
• Number of connected users.

Additionally, create alerts that notify application managers and support teams when a service doesn't work as expected.

Infrastructure monitoring is particularly important in controlling cloud costs. A key advantage of the cloud is the ability to scale and adjust resources to a workload's level. At any time, you should have enough compute and storage resources to handle the existing workload while minimizing unused resources. Monitoring tools and the cloud provider's autoscaling features can help you allocate cloud resources dynamically and efficiently.

6. Asset and configuration management

A big challenge for organizations is to maintain a dynamic array of cloud infrastructure resources within the bounds of what they expect to deploy. Developers and cloud engineers manually deploying a VM for an ad hoc need and forgetting to shut it down is not a big concern, but teams should rely on controlled processes to deploy large clusters or use high-cost cloud services.

One way to manage infrastructure is to use infrastructure as code (IaC). Rather than having cloud engineers start and stop resources, IaC specifies what to run or deploy in your environment to support the application. The IaC application can then monitor the state of the infrastructure, which is distinct from the state of configuration. If it varies from the desired state -- say, some VMs fail -- it can bring your infrastructure back to the desired state automatically.

Configuration management also helps an organization control the use and storage of secrets, such as credentials and encryption keys. Use centralized repositories to store secrets instead of using insecure practices such as login credentials in scripts or programs, which may be visible to anyone with access to the script.

Cloud governance models and standards

Several governance models and standards are relevant to cloud computing standards, although none are specific to the cloud. Governance models and standards are less about specific technologies and more about people and processes.

• COBIT is a governance standard created by the Information Systems Audit and Control Association to help businesses and other organizations manage IT operations. The model includes a framework of processes and practices, process descriptions, control objectives, management guidelines and maturity models. COBIT is a generic governance approach that works well with other standards, such as ITIL.
• ITIL is a framework with detailed process descriptions to help businesses standardize how they select, deliver and maintain IT services, and strategically plan for new technology initiatives.
• ISO/IEC 38500 is an international standard for corporate IT governance that covers processes, communications and decision-making. The standard addresses how to define responsibilities; support IT operations, technology and related acquisitions; monitor performance; and conform with policies. It also helps businesses understand how users interact with applications and systems, as well as avoid creating incentives for users to bypass policies and procedures.
• NIST Cybersecurity Framework (CSF) provides guidance on managing cybersecurity risks in the cloud and beyond. Because CSF focuses on cybersecurity, it doesn't offer extensive guidance on other aspects of cloud governance, such as controlling costs. Nonetheless, CSF is a good starting point for defining certain aspects of cloud risk management. NIST also offers a similar framework called Risk Management Framework (RMF). While it is a bit broader than CSF, RMF is designed mainly for U.S. federal agencies that have to meet specific compliance mandates for the government sector.

What are the benefits of cloud governance?

Implementing an effective cloud governance strategy is critical for making full use of the cloud. When done right, cloud governance delivers benefits that include the following:

• Lower costs. Cloud governance can help prevent issues such as shadow IT (workloads that employees deploy without permission). In this way, it can reduce cloud costs.
• Lower risk of data breaches. Governance policies and procedures that help protect against cyberattacks reduce the chances of experiencing a data breach.
• Better compliance outcomes. Effective cloud governance reduces risks that could trigger a violation of compliance frameworks. It also offers a way to demonstrate to auditors and regulators that the business is managing cloud risks responsibly.
• Enhanced experience for business users. Well-governed cloud resources often translate to fewer hiccups, such as misconfigured applications or data loss events, that prevent employees from being as productive as possible.
• Better experience for IT employees. Good cloud governance makes the lives of IT employees easier because it reduces the mistakes they must mitigate manually.

What are the challenges of cloud governance?

A major challenge of cloud governance is the breadth of topics to address. Therefore, it is more practical to introduce a comprehensive governance framework incrementally, rather than in a single step. Start with the highest priority items for your organization. For example, in strictly regulated industries such as finance, compliance and security are often top priorities. If your cloud spending is excessive and unsustainable, focus on cost management early in the process.

Automation is essential to governance. Cloud environments are dynamic and can scale to numerous resources, components and services. Take advantage of cloud service features that support governance, such as data lifecycle management policies, which can help ensure data is stored in proper storage services and purged on a defined schedule. Use third-party tools, such as vulnerability scanners to check the contents of code repositories and pinpoint vulnerabilities in your applications.

Finally, bear in mind that governance is an ongoing effort with multiple processes. Governance frameworks such as NIST are useful starting points to help guide your organization's governance practices.

Dan Sullivan, M.Sc., is an author, systems architect and consultant with more than 20 years of IT experience with engagements in advanced analytics, systems architecture, database design, enterprise security and business intelligence. He has worked in a range of industries, including financial services, manufacturing, pharmaceuticals, software development, government, retail, power generation and education.

Chris Tozzi is a freelance writer, research adviser, and professor of IT and society who has previously worked as a journalist and Linux systems administrator.