HashiCorp users got a deeper look this week at long sought-after infrastructure-as-code features in Terraform version 0.13, now in public beta.
HashiCorp Terraform is an open source IaC tool IT ops pros use to automatically provision infrastructure, often in support of Kubernetes deployments. Because Terraform users write code to configure infrastructure, Terraform also fits into increasingly popular GitOps deployment patterns, in which all changes to apps and infrastructure are made in code and applied through CI/CD pipelines.
The major new feature added to Terraform 0.13 in the May 21 beta release is called module count. Previously, Terraform users could specify a count, or number, of lower-level resources such as virtual machines in Terraform's HashiCorp Configuration Language (HCL). But with module count, they can specify a number of modules, or supersets of resources, to be added to an infrastructure build without having to individually replicate each of them.
"[Previously], we might define a VM and say 'count is equal to five' and Terraform will create five VMs with identical configuration," said Armon Dadgar, co-founder and co-CTO of HashiCorp, in a keynote presentation this week at HashiConf Digital. "Now, that same count parameter can be applied to a module, [which] might consist of tens or hundreds of subresources or submodules and Terraform can clone and provide multiple instances of that entire module."
Module count is the most important of the new features, according to one Terraform expert.
"I can write a manifest for a whole application deployment, including network, front-end app, back-end [infrastructure], etc.," said Nathan Bennett, cloud architect at HashiCorp partner Sterling Computers, a value-added reseller in North Sioux City, S.D. "The 'count' feature allows users to deploy multiple full module deployments instead of having to re-add modules per deployment."
Terraform 0.13 will also add support for dependencies between modules with the new parameter "depends_on," and another module parameter, "for_each," will allow for more complex logic to be applied to each module instance -- for example, a Kubernetes cluster instantiation for each resource group in different cloud regions.
HashiCorp co-founder Armon Dadgar introduced Terraform 0.13 features this week in a HashiConf Digital keynote.
Terraform HCL presents users with a learning curve
Terraform 0.12, first released in late 2018, included major changes to the HCL syntax that required a complex migration process for many Terraform users, but the features in version 0.13 make that conversion process worthwhile, said Fernanda Martins, a DevOps engineer at L1nda, a software maker for the hospitality industry in Amsterdam.
"Today, I can do dependency management [with version 0.11] but in a very specific way by chaining modules," Martins said. "I also use an external tool, terragrunt, but we might not need to use multiple tools with 0.13."
Terraform supports translating HCL code into the more familiar JSON format, but some users have avoided taking on complex operations in HCL in favor of other tools that natively use more familiar languages.
"Terraform was simple to learn, but hard to master," said Nikola Stjelja, software-defined infrastructure manager at Philip Morris International (PMI), a tobacco company based in Switzerland, in a HashiConf Digital presentation this week. "A lot of the people in the organization didn't have a high level of enthusiasm for the technology -- people wanted to learn it, but adopting it at work was really hard for them, especially for people without experience in infrastructure."
Terraform is a core component of PMI's infrastructure provisioning strategy, and all deployments to the AWS cloud are performed through Bitbucket code repositories connected to Terraform Enterprise. But certain aspects of those deployments, such as virtual machine configuration management, are done with Red Hat Ansible, in part because it can integrate with more familiar Python scripts.
Terraform providers work out kinks
Writing custom Terraform modules also presented a significant learning curve for PMI, Stjelja said in his presentation. Integrations between Terraform and cloud infrastructure resources is done through providers -- plugins written in the Go programming language that engage infrastructure APIs.
"We looked at also writing Terraform modules for cloud solutions which have a REST API but don't have a quality [Terraform] provider, but this is where we hit a snag organizationally," he said. "You have to have a very good and regularly updated Terraform provider, and we don't have the [Go language] skills in the organization for working with Terraform."
Now, PMI only uses Terraform if there's an already-established provider; if not, it looks to other IaC tools, Stjelja said.
HashiCorp officials conceded the integration between Terraform IaC and the HashiCorp Vault secrets manager could still use work. IT pros often use the two together, and some asked company officials pointed questions about potential security concerns during a panel session about Kubernetes integrations at the virtual conference this week. For example, one user pointed out that the Vault provider for Terraform may store secrets in Terraform state cache.
I can write a manifest for a whole application deployment, including network, front-end app, back-end [infrastructure], etc. The 'count' feature allows users to deploy multiple full module deployments instead of having to re-add modules per deployment.
Nathan BennettCloud architect, Sterling Computers
The company is working to eliminate that potential exposure, HashiCorp Vault ecosystem product manager Narayan Iyengar said in response. In the meantime, users should encrypt Terraform state storage and state itself when using the Vault provider, according to a presentation by Andrey Devyatkin, senior systems engineer at Hippo, New York-based makers of an app that manages prescription drug pricing.
HashiCorp, which joined the Cloud Native Computing Foundation in March, is still working to make its Kubernetes provider for Terraform support the full range of features available for the container orchestrator, including customizable infrastructure patterns.
"We cover many Kubernetes resources in the Kubernetes provider that we've had around for a while," said Phil Sautter, Terraform ecosystem product manager at HashiCorp, during a panel discussion. "One of the things we've clearly been lacking is support for custom resource definitions, which we just added in the Kubernetes alpha provider. We hope to merge these two providers into one."
HashiCorp believes the Terraform provider it developed to link Terraform with Kubernetes is a simpler alternative to utilities produced by the Kubernetes community to integrate third-party tools, such as Helm charts written in YAML format and Kubernetes Operators originally developed by rival CoreOS.
However, HashiCorp is hedging its bets with a Helm chart for Vault Enterprise. Product managers also indicated during the panel discussion that they intend to improve support for Kubernetes Operators, which could be used to directly replace Terraform providers.
"Operators are more in our future, and tighter integrations with Kubernetes, better integrations with multicluster [environments]," Sautter said. "The industry is moving toward a Kubernetes world, and so are we."
Dig Deeper on Systems automation and orchestration
