24 KiB
An Example Terraform(Opentofu) setup "packed" as a (docker)compose application
This repo contains a compose.yml file. With such a compose.yml which
sometimes can be named also docker-compose.yml we setup an application, defined by services
# this is an example compose.yml content
services:
a_service:
image: name/of-container-image:tag
another_service:
image: name/of-container-image:tag
[...]
The compose.yml in this repo has only a single service that is the container/service "terraform"
Usage Part 1: the docker compose part
Requirements
After having docker compose installed (it should be a versoin 2.XX, given version 1 is outdated)
which can be checked via:
#> docker compose version
Docker Compose version 2.29.2
Build the application
A first step is to docker compose build the application
#> docker compose build
This will build the image for the container. The service terraform inside compose.yml
uses the this information to have an inline Dockerfile/recipe:
services:
terraform:
hostname: container-for-tf
volumes:
- ./terraformdata:/terraformdata
build:
dockerfile_inline: |
FROM alpine:latest
RUN apk update
RUN apk add aws-cli-bash-completion aws-cli aws-cli-doc bash bash-completion
RUN apk add man-db man-pages
RUN apk add opentofu
RUN apk add vim jq less
RUN <<EOF
cat >> /etc/bash/bashrc <<BASHRC
complete -C '$(which aws_completer)' aws
tofu() {
while ! aws sts get-caller-identity
do
echo "no valid aws credentials setup, running 'aws configure'"
aws configure
done
unset tofu
command tofu "$@"
}
alias terraform='tofu'
alias terra='tofu'
BASHRC
EOF
ENTRYPOINT ["/bin/bash"]
WORKDIR /terraformdata
Use/Run the terraform/service to do opentofu/terraform stuff
After the compose application terrafrom was build one can run it.
(to be fair, if having skipped the previous docker compose build step it would be automaticly
build when running anyways, as clearly the contaner image is required to run the container service
hence docker compose run -it terraform would automatically build it with the inline Dockerfile
information in the services.terraform.build.dockerfile_inline information.
To now run the terraform service use:
docker compose run -it terraform
this will drop the user into a shell inside the container with the following utilities setup:
aws(the Amazon web services aws command line tool)jq(a tool to handle JSON on the command line)less(less command)vim(to have an editor if needed inside the container)- a feels like
bashshell and completion for theawscommand line (given manmanual pagestofu(aliased also to be run asterraandterraform), the opentofu tool
Usage Part 2: the AWS part
Once dropped into the shell in the container of compose.yml's terraform service.
the main command one can interact with is tofu.
The first stell should be to run just tofu which will check if the container
is already setup to interact with an AWS account (which if it is a fresh container
it will most likely not)
this will happen:
container-for-tf:/terraformdata# tofu
Unable to locate credentials. You can configure credentials by running "aws configure".
no valid aws credentials setup, running 'aws configure'
AWS Access Key ID [None]: AK..............
AWS Secret Access Key [None]: hereYourAWSSecretAccessKeyPasted
Default region name [None]: eu-central-1
Default output format [None]:
{
"UserId": "AID........",
"Account": ".........",
"Arn": "arn:aws:iam::.......:user/user....."
}
Usage: tofu [global options] <subcommand> [args]
The available commands for execution are listed below.
The primary workflow commands are given first, followed by
less common or more advanced commands.
Main commands:
init Prepare your working directory for other commands
validate Check whether the configuration is valid
plan Show changes required by the current configuration
[...]
as is visible in the above this required to specify a amazon IAM user via the crediatals of a) AWS Access Key ID (i.e. alike an ID/username) b) AWS Secret Access Key (i.e. kind of a password, indeeed a base64 encoded key)
hence to successfuly go through the dialog on needs to setup the users this can be done in the amazon web gui for IAM IAM is the user service/permissions part of AWS.
It makes much sense to setup a new user that is dedicated to EC2 (aws instances). The process to do so is somewhat challenging because of the sheer number of stuff that AWS has stuffed into AWS such as
- users
- roles
- policies
- permissions
- identify provides......
indeed we need only users. Such a user should have those Permission Policies set
- AmazonEC2FullAccess (since we want to have the use be able to do all EC2 stuff)
- a "inline persmission" allowing the read of STS -> get-caller-identiy (required to use the
awscli tool)
This is an exmaple user screenshoted
Once the user is created it is required to generate the credentials to be used in the dialog above. This can be done in the here:
since they are more complicatred, disregard the suggested alternatives:
also we need no tag to be set (AWS really makes it a point to strech out and prolong simple stuff):
lastly get the credintials can be retrieved:
with those credentials the above mask should have completed successfully
Usage Part 3: the Opentofu/Terraform parts
Somewhat boiled down the idea of terraform is infrastructure as code meaning one has files
that defined resource(s) which then can be created or torn-down more easily then the it might
otherwise be able via the web-guis of AWS etc....
The files used for that are *.tf files and there are two:
- terraformdata/aws.tf which is sets up the modules needed to handle AWS (after all opentofu can handle other cloud providers...)
- terraformdata/ec2-instance.tf (is the file that defines the resource(s) we seek to create
After the previous step(Usage Part2) one should have reached the point where one can proceed to initialize opentofu
container-for-tf:/terraformdata# tofu init
Initializing the backend...
Initializing provider plugins...
- Reusing previous version of hashicorp/aws from the dependency lock file
- Using previously-installed hashicorp/aws v4.67.0
OpenTofu has been successfully initialized!
You may now begin working with OpenTofu. Try running "tofu plan" to see
any changes that are required for your infrastructure. All OpenTofu commands
should now work.
If you ever set or change modules or backend configuration for OpenTofu,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
this basically downloads the stuff needed to handle aws (i.e most stuff refered to in the aws.tf file)
After that we can then proceed to plan the creation of the ec2-instance via
setting up the ssh-key in (terraformdata/ec2-instance.tf)
One step is to setup our ssh public key inside the file terraformdata/ec2-instance.tf
# use vim to edit the ssh-key in ec2-instance.tf (you can get your key from by runnning `ssh-add -L`, or maybe creating a new ssh keypair via `ssh-keygen`)
vim ec2-instance.tf
The we can use tofu plan
container-for-tf:/terraformdata# tofu plan -out ourplan
OpenTofu used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
+ create
OpenTofu will perform the following actions:
# aws_instance.ubuntu-on-t4g-nano will be created
+ resource "aws_instance" "ubuntu-on-t4g-nano" {
+ ami = "ami-07034695835d8f3bd"
+ arn = (known after apply)
+ associate_public_ip_address = (known after apply)
+ availability_zone = (known after apply)
+ cpu_core_count = (known after apply)
+ cpu_threads_per_core = (known after apply)
+ disable_api_stop = (known after apply)
+ disable_api_termination = (known after apply)
+ ebs_optimized = (known after apply)
+ get_password_data = false
+ host_id = (known after apply)
+ host_resource_group_arn = (known after apply)
+ iam_instance_profile = (known after apply)
+ id = (known after apply)
+ instance_initiated_shutdown_behavior = (known after apply)
+ instance_state = (known after apply)
+ instance_type = "t4g.nano"
+ ipv6_address_count = (known after apply)
+ ipv6_addresses = (known after apply)
+ key_name = "our_public_ssh_key"
+ monitoring = (known after apply)
+ outpost_arn = (known after apply)
+ password_data = (known after apply)
+ placement_group = (known after apply)
+ placement_partition_number = (known after apply)
+ primary_network_interface_id = (known after apply)
+ private_dns = (known after apply)
+ private_ip = (known after apply)
+ public_dns = (known after apply)
+ public_ip = (known after apply)
+ secondary_private_ips = (known after apply)
+ security_groups = (known after apply)
+ source_dest_check = true
+ subnet_id = (known after apply)
+ tags_all = (known after apply)
+ tenancy = (known after apply)
+ user_data = (known after apply)
+ user_data_base64 = (known after apply)
+ user_data_replace_on_change = false
+ vpc_security_group_ids = (known after apply)
}
# aws_key_pair.our_public_ssh_key will be created
+ resource "aws_key_pair" "our_public_ssh_key" {
+ arn = (known after apply)
+ fingerprint = (known after apply)
+ id = (known after apply)
+ key_name = "our_public_ssh_key"
+ key_name_prefix = (known after apply)
+ key_pair_id = (known after apply)
+ key_type = (known after apply)
+ public_key = "ssh-rsa AAAAB3NzaC-your-public-ssh-key"
+ tags_all = (known after apply)
}
# aws_security_group.our_security_group will be created
+ resource "aws_security_group" "our_security_group" {
+ arn = (known after apply)
+ description = "Managed by Terraform"
+ egress = [
+ {
+ cidr_blocks = [
+ "0.0.0.0/0",
]
+ description = ""
+ from_port = 0
+ ipv6_cidr_blocks = [
+ "::/0",
]
+ prefix_list_ids = []
+ protocol = "-1"
+ security_groups = []
+ self = false
+ to_port = 0
},
]
+ id = (known after apply)
+ ingress = [
+ {
+ cidr_blocks = [
+ "0.0.0.0/0",
]
+ description = ""
+ from_port = 22
+ ipv6_cidr_blocks = []
+ prefix_list_ids = []
+ protocol = "tcp"
+ security_groups = []
+ self = false
+ to_port = 22
},
]
+ name = (known after apply)
+ name_prefix = (known after apply)
+ owner_id = (known after apply)
+ revoke_rules_on_delete = false
+ tags_all = (known after apply)
+ vpc_id = (known after apply)
}
Plan: 3 to add, 0 to change, 0 to destroy.
Changes to Outputs:
+ instance_ip = (known after apply)
───────────────────────────────────────────────────────────────────────────────────────────────────────────────
Saved the plan to: ourplan
To perform exactly these actions, run the following command to apply:
tofu apply "ourplan"
we finnaly can use tofu apply to realize said plan
container-for-tf:/terraformdata# time tofu apply "ourplan"
aws_key_pair.our_public_ssh_key: Creating...
aws_security_group.our_security_group: Creating...
aws_key_pair.our_public_ssh_key: Creation complete after 0s [id=our_public_ssh_key]
aws_security_group.our_security_group: Creation complete after 3s [id=sg-0719d547d9498a51b]
aws_instance.ubuntu-on-t4g-nano: Creating...
aws_instance.ubuntu-on-t4g-nano: Still creating... [10s elapsed]
aws_instance.ubuntu-on-t4g-nano: Creation complete after 13s [id=i-0c7083c9f94aaf739]
Apply complete! Resources: 3 added, 0 changed, 0 destroyed.
Outputs:
instance_ip = "3.122.104.160"
real 0m18.633s
user 0m3.743s
sys 0m0.359s
in a mere 18 seconds the whole ec2 was setup all as defined in ec2-instance.tf we can test access
[alex@thinkbox terraform]$ ssh ubuntu@3.122.104.160
The authenticity of host '3.122.104.160 (3.122.104.160)' can't be established.
ED25519 key fingerprint is SHA256:IsPr12o6nBpbvQktStDJHiyTR52A5lkQnoE0sd3mP9E.
This key is not known by any other names.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '3.122.104.160' (ED25519) to the list of known hosts.
Welcome to Ubuntu 22.04.3 LTS (GNU/Linux 6.2.0-1018-aws aarch64)
* Documentation: https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support: https://ubuntu.com/pro
This system has been minimized by removing packages and content that are
not required on a system that users do not log into.
To restore this content, you can run the 'unminimize' command.
Expanded Security Maintenance for Applications is not enabled.
0 updates can be applied immediately.
Enable ESM Apps to receive additional future security updates.
See https://ubuntu.com/esm or run: sudo pro status
The list of available updates is more than a week old.
To check for new updates run: sudo apt update
The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.
Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.
To run a command as administrator (user "root"), use "sudo <command>".
See "man sudo_root" for details.
ubuntu@ip-172-31-9-206:~$
The joy of destruction (i.e. cloud resources AWS would make money on )
since our setup of the instance is by now largy a command at our fingertip. It becomes sensible to remove the EC2 instance when we do not need it (after all we can recreate it and thus save costs in the meanwhile)
this is done with tofu destroy
container-for-tf:/terraformdata# time tofu destroy
aws_key_pair.our_public_ssh_key: Refreshing state... [id=our_public_ssh_key]
aws_security_group.our_security_group: Refreshing state... [id=sg-0719d547d9498a51b]
aws_instance.ubuntu-on-t4g-nano: Refreshing state... [id=i-0c7083c9f94aaf739]
OpenTofu used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
- destroy
OpenTofu will perform the following actions:
# aws_instance.ubuntu-on-t4g-nano will be destroyed
- resource "aws_instance" "ubuntu-on-t4g-nano" {
- ami = "ami-07034695835d8f3bd" -> null
- arn = "arn:aws:ec2:eu-central-1:000000000000:instance/i-0c7083c9f94aaf739" -> null
- associate_public_ip_address = true -> null
- availability_zone = "eu-central-1c" -> null
- cpu_core_count = 2 -> null
- cpu_threads_per_core = 1 -> null
- disable_api_stop = false -> null
- disable_api_termination = false -> null
- ebs_optimized = false -> null
- get_password_data = false -> null
- hibernation = false -> null
- id = "i-0c7083c9f94aaf739" -> null
- instance_initiated_shutdown_behavior = "stop" -> null
- instance_state = "running" -> null
- instance_type = "t4g.nano" -> null
- ipv6_address_count = 0 -> null
- ipv6_addresses = [] -> null
- key_name = "our_public_ssh_key" -> null
- monitoring = false -> null
- placement_partition_number = 0 -> null
- primary_network_interface_id = "eni-075acde855df2488b" -> null
- private_dns = "ip-172-31-9-206.eu-central-1.compute.internal" -> null
- private_ip = "172.31.9.206" -> null
- public_dns = "ec2-3-122-104-160.eu-central-1.compute.amazonaws.com" -> null
- public_ip = "3.122.104.160" -> null
- secondary_private_ips = [] -> null
- security_groups = [
- "terraform-20240913111314866800000001",
] -> null
- source_dest_check = true -> null
- subnet_id = "subnet-0d1704e550107e0b3" -> null
- tags = {} -> null
- tags_all = {} -> null
- tenancy = "default" -> null
- user_data_replace_on_change = false -> null
- vpc_security_group_ids = [
- "sg-0719d547d9498a51b",
] -> null
- capacity_reservation_specification {
- capacity_reservation_preference = "open" -> null
}
- cpu_options {
- core_count = 2 -> null
- threads_per_core = 1 -> null
}
- credit_specification {
- cpu_credits = "unlimited" -> null
}
- enclave_options {
- enabled = false -> null
}
- maintenance_options {
- auto_recovery = "default" -> null
}
- metadata_options {
- http_endpoint = "enabled" -> null
- http_put_response_hop_limit = 1 -> null
- http_tokens = "optional" -> null
- instance_metadata_tags = "disabled" -> null
}
- private_dns_name_options {
- enable_resource_name_dns_a_record = false -> null
- enable_resource_name_dns_aaaa_record = false -> null
- hostname_type = "ip-name" -> null
}
- root_block_device {
- delete_on_termination = true -> null
- device_name = "/dev/sda1" -> null
- encrypted = false -> null
- iops = 100 -> null
- tags = {} -> null
- throughput = 0 -> null
- volume_id = "vol-0739466f7213740f1" -> null
- volume_size = 8 -> null
- volume_type = "gp2" -> null
}
}
# aws_key_pair.our_public_ssh_key will be destroyed
- resource "aws_key_pair" "our_public_ssh_key" {
- arn = "arn:aws:ec2:eu-central-1:000000000000:key-pair/our_public_ssh_key" -> null
- fingerprint = "23:4a:bd:cb:3f:da:a2:1b:e4:a7:1d:9e:d1:67:0b:55" -> null
- id = "our_public_ssh_key" -> null
- key_name = "our_public_ssh_key" -> null
- key_pair_id = "key-0c1810505afa4b27f" -> null
- key_type = "rsa" -> null
- public_key = "ssh-rsa AAAAB3Nyour-ssh-pub-key"
- tags = {} -> null
- tags_all = {} -> null
}
# aws_security_group.our_security_group will be destroyed
- resource "aws_security_group" "our_security_group" {
- arn = "arn:aws:ec2:eu-central-1:000000000000:security-group/sg-0719d547d9498a51b" -> null
- description = "Managed by Terraform" -> null
- egress = [
- {
- cidr_blocks = [
- "0.0.0.0/0",
]
- description = ""
- from_port = 0
- ipv6_cidr_blocks = [
- "::/0",
]
- prefix_list_ids = []
- protocol = "-1"
- security_groups = []
- self = false
- to_port = 0
},
] -> null
- id = "sg-0719d547d9498a51b" -> null
- ingress = [
- {
- cidr_blocks = [
- "0.0.0.0/0",
]
- description = ""
- from_port = 22
- ipv6_cidr_blocks = []
- prefix_list_ids = []
- protocol = "tcp"
- security_groups = []
- self = false
- to_port = 22
},
] -> null
- name = "terraform-20240913111314866800000001" -> null
- name_prefix = "terraform-" -> null
- owner_id = "000000000000" -> null
- revoke_rules_on_delete = false -> null
- tags = {} -> null
- tags_all = {} -> null
- vpc_id = "vpc-06a85af8b5132190d" -> null
}
Plan: 0 to add, 0 to change, 3 to destroy.
Changes to Outputs:
- instance_ip = "3.122.104.160" -> null
Do you really want to destroy all resources?
OpenTofu will destroy all your managed infrastructure, as shown above.
There is no undo. Only 'yes' will be accepted to confirm.
Enter a value: yes
aws_key_pair.our_public_ssh_key: Destroying... [id=our_public_ssh_key]
aws_instance.ubuntu-on-t4g-nano: Destroying... [id=i-0c7083c9f94aaf739]
aws_key_pair.our_public_ssh_key: Destruction complete after 0s
aws_instance.ubuntu-on-t4g-nano: Still destroying... [id=i-0c7083c9f94aaf739, 10s elapsed]
aws_instance.ubuntu-on-t4g-nano: Still destroying... [id=i-0c7083c9f94aaf739, 20s elapsed]
aws_instance.ubuntu-on-t4g-nano: Still destroying... [id=i-0c7083c9f94aaf739, 30s elapsed]
aws_instance.ubuntu-on-t4g-nano: Destruction complete after 33s
aws_security_group.our_security_group: Destroying... [id=sg-0719d547d9498a51b]
aws_security_group.our_security_group: Destruction complete after 0s
Destroy complete! Resources: 3 destroyed.
real 0m44.331s
user 0m7.067s
sys 0m0.566s
as such in a 44 seconds the ec2 is downed (there might be a way to kill the instance more quickly, even though one has to say that AWS gains of course the longer a instance is up, so I would not count on there being a real instantly up/down of instances)
however the real great thing with this opentofu/terraform setup is clearly that reusability and documentation etc has been achieved with the tf files. The whole process to setup machines/instances has become an automized thing from being a slow manual click-fiesta before