Terraform Providers

Task 1: Learn and Compare Terraform Providers

Spend time learning about Terraform providers and their significance in managing resources across various cloud platforms or infrastructure services.

🛑Terraform Providers

Terraform providers are plugins or connectors that enable Terraform, an infrastructure-as-code tool, to interact with different cloud platforms or infrastructure services. They serve as a bridge between Terraform and the APIs of these platforms/services.

By using the appropriate provider for your target cloud platform (such as AWS, Azure, or GCP), Terraform can create, update, or delete the resources for you based on the defined configuration. The provider handles all the details of making API calls and managing the resources, so you don't have to do it manually.

Here are the key points about the significance of Terraform providers in managing resources across various cloud platforms or infrastructure services:

Cross-platform resource management: Terraform providers enable managing resources across multiple cloud platforms or infrastructure services using a unified approach.
Infrastructure as Code: Providers allow defining infrastructure configurations as code, facilitating version control, collaboration, and automation.
Declarative resource management: Providers handle the complexity of making API calls and ensure that the actual infrastructure state matches the desired state declared in the configuration.
Automation and consistency: Providers enable automation of resource provisioning, configuration, and management, ensuring consistent infrastructure across different environments.
Resource lifecycle management: Providers facilitate creating, updating, and destroying resources, supporting the entire lifecycle management of infrastructure.
Extensibility and community support: Terraform's rich ecosystem of providers, contributed by the community and cloud service providers, allows for the support of various cloud platforms, infrastructure services, and technologies.

Compare the features and supported resources for each cloud platform's Terraform provider to gain a better understanding of their capabilities.

Each cloud provider typically offers its own Terraform provider, tailored to its specific services and resources.

AWS (Amazon Web Services) Provider

Comprehensive coverage of AWS services.
High-level constructs for common infrastructure patterns (e.g., autoscaling groups).
Support for AWS-specific features like CloudFront distributions, Elastic Beanstalk environments, etc.

Supported resources

Compute: EC2 instances, ECS clusters, Lambda functions.
Networking: VPC, Subnets, Security Groups, Route53 DNS records, Load Balancers.
Storage: S3 buckets, EBS volumes, EFS file systems.
Databases: RDS instances, DynamoDB tables, ElastiCache clusters.
Identity & Access Management: IAM roles, policies, users, groups.

Azure Provider

Management of Azure resources using Azure Resource Manager (ARM) templates.
Support for provisioning and configuring Azure services and features.
Integration with Azure DevOps for continuous deployment and pipeline automation.

Supported resources

Compute: Virtual machines, Virtual machine scale sets, App Services, Functions.
Networking: Virtual networks, Subnets, Load Balancers, Application Gateways.
Storage: Blob storage, File storage, Managed Disks, Storage Accounts.
Databases: Azure SQL Database, Cosmos DB, MySQL, PostgreSQL.
Identity & Access Management: Azure Active Directory, Service Principals.

GCP (Google Cloud Platform) Provider

Management of GCP resources using Google Cloud Deployment Manager.
Support for provisioning and configuring GCP services and features.
Integration with Google Cloud Build for CI/CD pipelines.

Supported resources

Compute: Compute Engine instances, GKE clusters, App Engine applications.
Networking: VPC networks, Subnets, Load Balancers, Cloud DNS.
Storage: Cloud Storage buckets, Persistent Disks.
Databases: Cloud SQL instances, Firestore, Datastore.
Identity & Access Management: Cloud IAM roles, Service Accounts.

IBM Cloud Provider

Management of IBM Cloud resources using IBM Cloud Schematics.
Support for provisioning and managing IBM Cloud services and features.

Supported resources

Compute: Virtual servers, VPC instances.
Networking: VPC networks, Subnets, Security Groups, Load Balancers.
Storage: Block storage volumes, Object storage buckets.
Databases: Databases for MongoDB, PostgreSQL, MySQL.
Identity & Access Management: IAM roles, Access Groups.

DigitalOcean Provider

Provisioning and management of DigitalOcean Droplets (virtual machines).
Support for other DigitalOcean resources like volumes, databases, and load balancers.

Supported resources

Compute: Droplets (virtual machines).
Networking: Floating IPs, Load Balancers.
Storage: Block storage volumes, Spaces (object storage).
Databases: Managed databases (PostgreSQL, MySQL).

Oracle Cloud Infrastructure (OCI) Provider

Management of Oracle Cloud Infrastructure resources and services.
Support for provisioning and managing OCI services and features.

Supported resources

Compute: Compute instances, Virtual Cloud Networks (VCNs).
Networking: Subnets, Load Balancers, Security Lists, DNS.
Storage: Block volumes, Object storage buckets.
Databases: Autonomous Database, VM DB systems, MySQL, NoSQL.

Task 2: Provider Configuration and Authentication

Explore provider configuration and authentication mechanisms in Terraform.

🛑 Provider Configuration

Provider configuration in Terraform involves setting up and specifying the details for interacting with a specific cloud platform or infrastructure service.
A provider block is used to define the configuration for each provider in your Terraform configuration files.
The provider block specifies the provider's name and contains its configuration settings.
Common provider configuration settings include access keys, regions, endpoint URLs, authentication mechanisms, and more.
Provider configuration is specific to each cloud provider and requires provider-specific settings to be provided.
Version constraints can be specified in the provider block to ensure compatibility with a specific version of the provider.
Set up authentication for each provider on your local machine to establish the necessary credentials for interaction with the respective cloud platforms.

🛑 Authentication mechanisms in Terraform

Here are some common authentication mechanisms in Terraform:

Access Keys/Secret Keys
- Many cloud providers use access keys (also called access tokens) and secret keys for authentication.
- These keys are typically obtained from the provider's console and provided in the provider configuration.
- Examples include AWS Access Key ID and Secret Access Key, Azure Client ID and Client Secret, etc.
Environment Variables
- Terraform supports authentication through environment variables.
- You can set environment variables with your access keys and secret keys, and Terraform will automatically use them for authentication.
- Common environment variables include AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY for AWS, AZURE_CLIENT_ID and AZURE_CLIENT_SECRET for Azure, etc.
Shared Credentials File
- Terraform can use a shared credentials file for authentication.
- The file typically resides in your user's home directory and contains access keys and secret keys.
- You specify the profile name in the provider configuration to use the corresponding credentials.
Instance Metadata Service
- Some cloud providers, like AWS, offer an instance metadata service.
- This service provides temporary access credentials to EC2 instances or virtual machines.
- Terraform can use the metadata service to authenticate on behalf of the instance, eliminating the need for explicit access keys.
Managed Identity
- Certain cloud providers, like Azure, support managed identity authentication.
- With managed identity, you can assign an identity to a resource, such as a virtual machine or a function.
- Terraform can use this assigned identity for authentication, eliminating the need for explicit access keys or secrets.
Service Principal
- In Azure, a service principal is a security identity used by applications and services to access Azure resources.
- Terraform can authenticate using a service principal by providing the client ID, client secret, and tenant ID in the provider configuration.

Set up authentication for each provider on your local machine to establish the necessary credentials for interaction with the respective cloud platforms.

To set up authentication for each provider on your local machine, follow these general steps for popular cloud providers:

AWS (Amazon Web Services)

Create an IAM user in the AWS Management Console.
Assign appropriate permissions to the IAM user based on your requirements.
After the user is created click on the security credentials tab of the user.
Click on Create an access key.
Select CLI and click on next.
Access keys and secrets are generated, copy from here and configure locally by using the aws configure command
Configure the AWS CLI or set environment variables (AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY) with the credentials.

Azure (Microsoft Azure)

Create an Azure Active Directory (AAD) application registration.
Obtain the Client ID, Client Secret, and Tenant ID for the application registration.
Assign the necessary roles and permissions to the application registration.
Configure the Azure CLI or set environment variables (AZURE_CLIENT_ID, AZURE_CLIENT_SECRET, and AZURE_TENANT_ID) with the credentials.

GCP (Google Cloud Platform)

Create a GCP service account and generate a key file (JSON format).
Assign the appropriate roles and permissions to the service account.
Set the GOOGLE_APPLICATION_CREDENTIALS environment variable to the path of the key file.

Task 3: Practice Using Providers

Create a Terraform configuration file named main.tf and configure the chosen provider within it.

You can find the GitHub code here
Create a main.tf file in the directory. Define VPC, Internet Gateway, Subnet, Route Table, and EC2 Instance. Add the following Terraform resource blocks to create a VPC, subnet, route table, and internet gateway.
Create a user data file with the following code.

Deploy a simple resource using the chosen provider. For example, if using AWS, you could provision a Virtual Private Cloud (VPC), Subnet Group, Route Table, Internet Gateway, or a virtual machine.

Run the terraform init command to initialize Terraform and run terraform plan command to see what resources will be created.
Run the terraform apply command to apply the Terraform configuration and create the EC2 instance. You can check the public IP in the output.
Here we can see VPC, Route Table, Subnets, Internet Gateway and EC2 instance are created.
Run the Public IP in the browser and you can see the nginx is installed and running.

Experiment with updating the resource configuration in your main.tf file and apply the changes using Terraform. Observe how Terraform intelligently manages the resource changes.

I have changed the name of EC2 instance and index file also.
Again run the terraform apply command and you can see it is replacing the EC2 instance.
The instance is created with a different name.
Paste the changed public IP in browser and you can see the changed content as well.