Automate Google Cloud Infrastructure using Terraform
This blog is for those who works on scalable cloud infrastructure and automating repetitive tasks.We have tools like Terraform, Ansible etc. that allow us to turn a little bit of code into something that can plan, deploy, modify, and destroy all of our systems. If we’re able to get it working, we’ll also need to make some changes to each system, such as modifying the disk size and memory, so that our client isn’t wasting money on unused resources.
Instead of modifying an existing system using SSH, which is a mutable process, your systems are rebuilt from a well-reviewed template, validated for correctness, and then deployed if they pass all the required checks. This is what’s called “immutable infrastructure”.
Let’s get started with defining some terms and technology:
- Terraform: a tool used to turn infrastructure development into code.
- Google Cloud SDK: command line utility for managing Google Cloud Platform resources.
- Google Cloud Platform: cloud-based infrastructure environment.
- Google Compute Engine: resource that provides virtual systems to Google Cloud Platform customers.
So, why do we want to use Terraform? Because doing things manually is inefficient, sometimes boring, and could also lead to misconfigurations and costly mistakes. We also want to be able to spend more time focusing on more important things, such as the security of our services, our product’s features, and things of that nature.
Installing Terraform
#########################
[root@localhost ~]# wget https://releases.hashicorp.com/terraform/0.11.7/terraform_0.11.7_linux_amd64.zip
[root@localhost ~]# unzip terraform_0.11.7_linux_amd64.zip
Archive: terraform_0.11.7_linux_amd64.zip
inflating: terraform
You will get the binary file after extraction
##################################
Now move the binary file to /usr/bin/
[root@localhost ~]# mv terraform /usr/bin/
Check the terraform version
#######################
[root@localhost ~]# terraform --version
Terraform v0.11.7
Downloading and configuring Google Cloud SDK
Now that we have Terraform installed, we need to set up the command line utility to interact with our services on Google Cloud Platform. This will allow us to authenticate to our account on Google Cloud Platform, and subsequently use Terraform to manage our infrastructure.
[root@localhost ~]# curl https://sdk.cloud.google.com | bash
Restart shell
##################
[root@localhost ~]# exec -l $SHELL
Check whether google cloud sdk is installed or not
#############################################
[root@localhost ~]# yum install google-cloud-sdk -y
Loaded plugins: langpacks
Package google-cloud-sdk-209.0.0-1.el7.noarch already installed and latest version
Nothing to do
Check the project list
######################
Initialize the gcloud environment & create project
##########################################
[root@localhost ~]# gcloud init [root@localhost ~]# yum install google-cloud-sdk -y
Loaded plugins: langpacks
Package google-cloud-sdk-209.0.0-1.el7.noarch already installed and latest version
Nothing to do
Check the project list
######################
[root@localhost ~]# gcloud projects list
PROJECT_ID NAME PROJECT_NUMBER
intricate-web-209010 My First Project 725095609278
peppy-booth-210808 My Project 1 201223669212
storage-210808 storage 833640504892
Initialize the gcloud environment & create project
##########################################
Welcome! This command will take you through the configuration of gcloud.
Settings from your current configuration [default] are:
core:
account: allahabad.shashank@gmail.com
disable_usage_reporting: 'False'
Pick configuration to use:
[1] Re-initialize this configuration [default] with new settings
[2] Create a new configuration
Please enter your numeric choice: 1
Your current configuration has been set to: [default]
You can skip diagnostics next time by using the following flag:
gcloud init --skip-diagnostics
Network diagnostic detects and fixes local network connection issues.
Checking network connection...done.
Reachability Check passed.
Network diagnostic (1/1 checks) passed.
Choose the account you would like to use to perform operations for
this configuration:
[1] allahabad.shashank@gmail.com
[2] Log in with a new account
Please enter your numeric choice: 1
You are logged in as: [allahabad.shashank@gmail.com].
Pick cloud project to use:
[1] intricate-web-209010
[2] peppy-booth-210808
[3] storage-210808
[4] Create a new project
Please enter numeric choice or text value (must exactly match list
item): 4
Enter a Project ID. Note that a Project ID CANNOT be changed later.
Project IDs must be 6-30 characters (lowercase ASCII, digits, or
hyphens) in length and start with a lowercase letter. sampleinfra
Your current project has been set to: [sampleinfra].
Not setting default zone/region (this feature makes it easier to use
[gcloud compute] by setting an appropriate default value for the
--zone and --region flag).
See https://cloud.google.com/compute/docs/gcloud-compute section on how to set
default compute region and zone manually. If you would like [gcloud init] to be
able to do this for you the next time you run it, make sure the
Compute Engine API is enabled for your project on the
https://console.developers.google.com/apis page.
Your Google Cloud SDK is configured and ready to use!
* Commands that require authentication will use allahabad.shashank@gmail.com by default
* Commands will reference project `sampleinfra` by default
Run `gcloud help config` to learn how to change individual settings
This gcloud configuration is called [default]. You can create additional configurations if you work with multiple accounts and/or projects.
Run `gcloud topic configurations` to learn more.
Some things to try next:
* Run `gcloud --help` to see the Cloud Platform services you can interact with. And run `gcloud help COMMAND` to get help on any gcloud command.
* Run `gcloud topic -h` to learn about advanced features of the SDK like arg files and output formatting
Check the project list now
##########################
[root@localhost ~]# gcloud projects list
PROJECT_ID NAME PROJECT_NUMBER
intricate-web-209010 My First Project 725095609278
peppy-booth-210808 My Project 1 201223669212
sampleinfra sampleinfra 842662222827
storage-210808 storage 833640504892
Configuring our Service Account on Google Cloud Platform
Create a project, set up a service account and set the correct permissions to manage our project’s resources.
Now Create a project directory
############################
[root@localhost ~]# mkdir GCPterra
[root@localhost ~]# cd GCPterra/
[root@localhost GCPterra]# ls
[root@localhost GCPterra]# touch .gitignore
Copy the key file to the project directiory
######################################
[root@localhost ~]# cp /root/Downloads/sampleinfra-d322fcbcfb12.json /root/GCPterra/
[root@localhost GCPterra]# ls
sampleinfra-d322fcbcfb12.json
sampleinfra-d322fcbcfb12.json
Now create a provider.tf file for provider details
##########################################
[root@localhost GCPterra]# vim provider.tf
# Specify the provider (GCP, AWS, Azure)
provider "google" {
credentials = "${file("sampleinfra-d322fcbcfb12.json")}"
project = "sampleinfra"
region = "us-east-1"
}
Now create a compute.tf file for instance details
###########################################
[root@localhost GCPterra]# vim compute.tf
# Create a new instance
resource "google_compute_instance" "ubuntu-xenial" {
name = "ubuntu-xenial"
machine_type = "f1-micro"
zone = "us-west1-a"
boot_disk {
initialize_params {
image = "ubuntu-1604-lts"
}
}
network_interface {
network = "default"
access_config {}
}
service_account {
scopes = ["userinfo-email", "compute-ro", "storage-ro"]
}
}
[root@localhost GCPterra]# ls
compute.tf provider.tf sampleinfra-d322fcbcfb12.json
Now initialize terraform
#####################
[root@localhost GCPterra]# terraform init
Initializing provider plugins...
- Checking for available provider plugins on https://releases.hashicorp.com...
- Downloading plugin for provider "google" (1.16.2)...
The following providers do not have any version constraints in configuration,
so the latest version was installed.
To prevent automatic upgrades to new major versions that may contain breaking
changes, it is recommended to add version = "..." constraints to the
corresponding provider blocks in configuration, with the constraint strings
suggested below.
* provider.google: version = "~> 1.16"
Terraform has been successfully initialized!
You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.
If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
[root@localhost GCPterra]# terraform plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.
------------------------------------------------------------------------
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
+ google_compute_instance.ubuntu-xenial
id: <computed>
boot_disk.#: "1"
boot_disk.0.auto_delete: "true"
boot_disk.0.device_name: <computed>
boot_disk.0.disk_encryption_key_sha256: <computed>
boot_disk.0.initialize_params.#: "1"
boot_disk.0.initialize_params.0.image: "ubuntu-1604-lts"
boot_disk.0.initialize_params.0.size: <computed>
boot_disk.0.initialize_params.0.type: <computed>
can_ip_forward: "false"
cpu_platform: <computed>
create_timeout: "4"
deletion_protection: "false"
guest_accelerator.#: <computed>
instance_id: <computed>
label_fingerprint: <computed>
machine_type: "f1-micro"
metadata_fingerprint: <computed>
name: "ubuntu-xenial"
network_interface.#: "1"
network_interface.0.access_config.#: "1"
network_interface.0.access_config.0.assigned_nat_ip: <computed>
network_interface.0.access_config.0.nat_ip: <computed>
network_interface.0.access_config.0.network_tier: <computed>
network_interface.0.address: <computed>
network_interface.0.name: <computed>
network_interface.0.network: "default"
network_interface.0.network_ip: <computed>
network_interface.0.subnetwork_project: <computed>
project: <computed>
scheduling.#: <computed>
self_link: <computed>
service_account.#: "1"
service_account.0.email: <computed>
service_account.0.scopes.#: "3"
service_account.0.scopes.1632638332: "https://www.googleapis.com/auth/devstorage.read_only"
service_account.0.scopes.2428168921: "https://www.googleapis.com/auth/userinfo.email"
service_account.0.scopes.2862113455: "https://www.googleapis.com/auth/compute.readonly"
tags_fingerprint: <computed>
zone: "us-west1-a"
Plan: 1 to add, 0 to change, 0 to destroy.
------------------------------------------------------------------------
Note: You didn't specify an "-out" parameter to save this plan, so Terraform
can't guarantee that exactly these actions will be performed if
"terraform apply" is subsequently run.
Now apply the configuration to create/provision instances
###############################################
[root@localhost GCPterra]# terraform apply
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
+ google_compute_instance.ubuntu-xenial
id: <computed>
boot_disk.#: "1"
boot_disk.0.auto_delete: "true"
boot_disk.0.device_name: <computed>
boot_disk.0.disk_encryption_key_sha256: <computed>
boot_disk.0.initialize_params.#: "1"
boot_disk.0.initialize_params.0.image: "ubuntu-1604-lts"
boot_disk.0.initialize_params.0.size: <computed>
boot_disk.0.initialize_params.0.type: <computed>
can_ip_forward: "false"
cpu_platform: <computed>
create_timeout: "4"
deletion_protection: "false"
guest_accelerator.#: <computed>
instance_id: <computed>
label_fingerprint: <computed>
machine_type: "f1-micro"
metadata_fingerprint: <computed>
name: "ubuntu-xenial"
network_interface.#: "1"
network_interface.0.access_config.#: "1"
network_interface.0.access_config.0.assigned_nat_ip: <computed>
network_interface.0.access_config.0.nat_ip: <computed>
network_interface.0.access_config.0.network_tier: <computed>
network_interface.0.address: <computed>
network_interface.0.name: <computed>
network_interface.0.network: "default"
network_interface.0.network_ip: <computed>
network_interface.0.subnetwork_project: <computed>
project: <computed>
scheduling.#: <computed>
self_link: <computed>
service_account.#: "1"
service_account.0.email: <computed>
service_account.0.scopes.#: "3"
service_account.0.scopes.1632638332: "https://www.googleapis.com/auth/devstorage.read_only"
service_account.0.scopes.2428168921: "https://www.googleapis.com/auth/userinfo.email"
service_account.0.scopes.2862113455: "https://www.googleapis.com/auth/compute.readonly"
tags_fingerprint: <computed>
zone: "us-west1-a"
Plan: 1 to add, 0 to change, 0 to destroy.
Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: yes
google_compute_instance.ubuntu-xenial: Creating...
boot_disk.#: "" => "1"
boot_disk.0.auto_delete: "" => "true"
boot_disk.0.device_name: "" => "<computed>"
boot_disk.0.disk_encryption_key_sha256: "" => "<computed>"
boot_disk.0.initialize_params.#: "" => "1"
boot_disk.0.initialize_params.0.image: "" => "ubuntu-1604-lts"
boot_disk.0.initialize_params.0.size: "" => "<computed>"
boot_disk.0.initialize_params.0.type: "" => "<computed>"
can_ip_forward: "" => "false"
cpu_platform: "" => "<computed>"
create_timeout: "" => "4"
deletion_protection: "" => "false"
guest_accelerator.#: "" => "<computed>"
instance_id: "" => "<computed>"
label_fingerprint: "" => "<computed>"
machine_type: "" => "f1-micro"
metadata_fingerprint: "" => "<computed>"
name: "" => "ubuntu-xenial"
network_interface.#: "" => "1"
network_interface.0.access_config.#: "" => "1"
network_interface.0.access_config.0.assigned_nat_ip: "" => "<computed>"
network_interface.0.access_config.0.nat_ip: "" => "<computed>"
network_interface.0.access_config.0.network_tier: "" => "<computed>"
network_interface.0.address: "" => "<computed>"
network_interface.0.name: "" => "<computed>"
network_interface.0.network: "" => "default"
network_interface.0.network_ip: "" => "<computed>"
network_interface.0.subnetwork_project: "" => "<computed>"
project: "" => "<computed>"
scheduling.#: "" => "<computed>"
self_link: "" => "<computed>"
service_account.#: "" => "1"
service_account.0.email: "" => "<computed>"
service_account.0.scopes.#: "" => "3"
service_account.0.scopes.1632638332: "" => "https://www.googleapis.com/auth/devstorage.read_only"
service_account.0.scopes.2428168921: "" => "https://www.googleapis.com/auth/userinfo.email"
service_account.0.scopes.2862113455: "" => "https://www.googleapis.com/auth/compute.readonly"
tags_fingerprint: "" => "<computed>"
zone: "" => "us-west1-a"
google_compute_instance.ubuntu-xenial: Still creating... (10s elapsed)
google_compute_instance.ubuntu-xenial: Creation complete after 16s (ID: ubuntu-xenial)
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
Now check for the compute engine instances
#########################################
[root@localhost GCPterra]# gcloud compute instances list
NAME ZONE MACHINE_TYPE PREEMPTIBLE INTERNAL_IP EXTERNAL_IP STATUS
ubuntu-xenial us-west1-a f1-micro 10.138.0.2 35.233.186.126 RUNNING
ReplyDeleteThanks for Posting this information.
Google Cloud Platform Training
GCP Online Training
Google Cloud Platform Training In Hyderabad
Thanks
Delete