Are you a guy who loves to work on Command Line and wanted to interact and use various AWS Services/Resources from the command line? Yes, We can fully Operate Our AWS Cloud Services using the command line arguments CLI). Here in this post, I will let you know how we can use AWS CLI to manage our entire IT Infrastructure On Amazon Web Services.
Most of the peoples prefer to work graphically on AWS management console and that’s definitely a simple and interactive way to Work, but even from the AWS management console you can not manage your Infrastructure say up to 100%, you would definitely need to Switch to CLI in Some cases.
In Some of the Scenarios, we need to Automate our task and Infrastructure Automation is the preferable things nowadays, So in order to automate the tasks we need to use the AWS CLI Commands in a form of Script to get rid of the repetitive or scheduled tasks. Another example, like if you want to configure a Custom metrics monitoring (memory and Disk) then a better way to switch to CLI and Configure Custom metrics to monitor the disk and memory utilization of your servers or what we call instances here on AWS.
So before starting with the AWS CLI, we need to perform some basic simple steps to set it up. I will show you few methods and a way how we can interact with your AWS resources. Let’s get started.
Prerequisite
- You must have an AWS Account if you don’t have, you can create one. Amazon provides 1 year of free tier account for learning and testing purposes where you can use various AWS services and you don’t need to pay anything. All you need is a Credit card (Just for registration purpose) and an email account.
- We need an IAM user with Access Key Id and Secret access key with which we will be interacting with AWS using Command line.
- A server where you will be Configuring CLI or if you are a Linux user you can configure there itself. (you can configure even if you have a virtual machine).
- And of course a fast internet connection.
Method 1: Install AWS CLI Using Python PIP.
- Log on to your machine or open the terminal. start with updating the repository.
- First, switch to root user to get the full privilege.
[testuser@devopsage ~]# sudo su -
- Now update the repository.
##For Redhat/CentOS
[root@devopsage ~]# yum update -y
## For Ubuntu/Debain
root@devopsage:~# apt-get update
- Once you are done with updating your repository then go ahead and install python-pip. # pip is package manager especially used for installing python packages.
## For Redhat/CentOS
[root@devopsage ~]# yum install python-pip -y
## For Ubuntu/Debian
root@devopsage:~# apt-get install python-pip -y
- As we have installed the python-pip package manager, now we will install AWS CLI using pip. so lets use the below command to install AWS CLI.
## For Redhat/CentOS
[root@devopsage ~]# yum install awscli
## For Debian/Ubuntu
root@devopsage:~# apt-get install awscli
- So okay, now we are done with installing AWS CLI, you can also check the version of your AWS CLI by using the below command.
root@devopsage:~# aws --version
aws-cli/1.11.13 Python/3.5.2 Linux/4.4.0-1049-aws botocore/1.4.70
- The package for Cli is installed, but we are still are not done with the setup of Cli, at this stage if we will use any of the AWS Cli commands, it will get failed as we did not configure our credentials yet. Let me show you how it will fail at this point in time. We will try to list our S3 buckets in AWS Account.
root@devopsage:~# aws s3 ls
An error occurred (AccessDenied) when calling the ListBuckets operation: Access Denied
- So let’s start with configuring the credentials now,
root@devopsage:~# aws configure
AWS Access Key ID [None]: Your_access_key_id_here
AWS Secret Access Key [None]: Your_secret_acceess_key_here
Default region name [None]: us-east-1
Default output format [None]: json
- So we are all set to use the AWS command line, before we will fire any command let me quickly show you were does these credential resides.
- Once we configured the credentials, it creates a hidden directory “.aws” in your home directory. which has 2 files, “config” and credentials whch usually contains your credentials and other details. you can see it in the below sample output.
root@devopsage:~# cd .aws/
root@devopsage:~/.aws# ls -l
total 8
-rw------- 1 root root 43 Feb 20 14:56 config
-rw------- 1 root root 116 Feb 20 14:56 credentials
root@devopsage:~/.aws# cat config
[default]
region = us-east-1
output = json
root@devopsage:~/.aws# cat credentials
[default]
aws_access_key_id = your access_key_Id_here
aws_secret_access_key = your_secret_access_key
root@devopsage:~/.aws#
- Config files basically contain your profile, region, and the output format and credentials file contains your credentials (access key id and secret access key).
- Now to get started we will use some CLI commands, first, we will start with listing the S3 Buckets.
root@devopsage:~# aws s3 ls
2017-12-13 15:30:00 sample-bucket-123231
- It list down the bucket, Currently I have only one bucket in my account. If you are new to aws cli you will have to check commands from AWS Website, simply google for aws s3 cli and you first link will be of AWS or click here to visit Amazon Cli section.
- You can also use # aws s3 help to check the proper command examples with all its parameters in the command line terminal itself.
- Now we will create one more bucket and will check whether it works or not.
root@devopsage:~# aws s3 ls
2017-12-13 15:30:00 sample-bucket-123231
root@devopsage:~# aws s3 mb s3://test-bucket-123321
make_bucket: test-bucket-123321
root@devopsage:~# aws s3 ls
2017-12-13 15:30:00 sample-bucket-123231
2018-02-20 15:27:19 test-bucket-123321
So It Works!!
Note: S3 bucket name should be unique globally, so give some random bucket name which you think should not exist in the entire world.
Method 2: Installation of AWS CLI Using Bundled Installer
- Here we will be doing exactly the same thing, Our requirement is to install AWS CLI, In method 1 we have used python-pip to install the CLI, here we will be using AWS Bundled installer to setup the CLI.
- Download awscli-bundle.zip file.
root@devopsage:~# curl "https://s3.amazonaws.com/aws-cli/awscli-bundle.zip" -o "awscli-bundle.zip"
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 9154k 100 9154k 0 0 674k 0 0:00:13 0:00:13 --:--:-- 1665k
root@devopsage:~#:~# ls -l
total 9156
-rw-r--r-- 1 root root 9373841 Feb 20 16:37 awscli-bundle.zip
root@devopsage:~#
- Unzip the file, below is the sample output.
root@devopsage:~# unzip awscli-bundle.zip
Archive: awscli-bundle.zip
inflating: awscli-bundle/install
inflating: awscli-bundle/packages/docutils-0.14.tar.gz
inflating: awscli-bundle/packages/python-dateutil-2.6.1.tar.gz
inflating: awscli-bundle/packages/jmespath-0.9.3.tar.gz
inflating: awscli-bundle/packages/argparse-1.2.1.tar.gz
inflating: awscli-bundle/packages/pyasn1-0.4.2.tar.gz
inflating: awscli-bundle/packages/awscli-1.14.42.tar.gz
inflating: awscli-bundle/packages/ordereddict-1.1.tar.gz
inflating: awscli-bundle/packages/colorama-0.3.7.zip
inflating: awscli-bundle/packages/six-1.11.0.tar.gz
inflating: awscli-bundle/packages/simplejson-3.3.0.tar.gz
inflating: awscli-bundle/packages/PyYAML-3.12.tar.gz
inflating: awscli-bundle/packages/botocore-1.8.46.tar.gz
inflating: awscli-bundle/packages/virtualenv-15.1.0.tar.gz
inflating: awscli-bundle/packages/futures-3.2.0.tar.gz
inflating: awscli-bundle/packages/s3transfer-0.1.13.tar.gz
inflating: awscli-bundle/packages/rsa-3.4.2.tar.gz
root@devopsage:~# ls -l
total 9160
drwxr-xr-x 3 root root 4096 Feb 20 16:42 awscli-bundle
-rw-r--r-- 1 root root 9373841 Feb 20 16:42 awscli-bundle.zip
root@devopsage:~# rm -rf awscli-bundle.zip
- Now Install the AWS CLI using the Installer Script.
root@devopsage:~# ./awscli-bundle/install -b ~/bin/aws
Running cmd: /usr/bin/python virtualenv.py --no-download --python /usr/bin/python /root/.local/lib/aws
Running cmd: /root/.local/lib/aws/bin/pip install --no-index --find-links file:///root/awscli-bundle/packages awscli-1.14.42.tar.gz
You can now run: /root/bin/aws --version
root@devopsage:~# /root/bin/aws --version
aws-cli/1.14.42 Python/2.7.12 Linux/4.4.0-1049-aws botocore/1.8.46
- We have successfully installed the AWS CLI using the Bundled Installer.
- Note: It will install the AWS CLI to the default location (~/.local/lib/aws) and will create a symbolic link at ~/bin/aws. Make sure that ~/bin is in your PATH environment variable for the symbolic link to work:
root@devopsage:~# export PATH=~/bin:$PATH
- Also, do not forget to make this entry in ~/.bashrc or ~./profile. So that you don’t need to keep on running the export command all the time one you login and logout of the system.
root@devopsage:~# echo "export PATH=~/bin:$PATH" >> ~/.bash.rc
- Now, we will quickly configure the credentials and will check whether our command line works or not.
root@devopsage:~# aws configure
AWS Access Key ID [None]: your_Access_key_id_here
AWS Secret Access Key [None]: your_secret_access_key_here
Default region name [None]: us-east-1
Default output format [None]: json
root@devopsage:~# aws s3 ls
2017-12-13 15:30:00 sample-bucket-123231
2018-02-20 15:27:19 test-bucket-123321
root@devopsage:~#
It Works!!
- Now in some cases, there will be a scenario where you would be manging multiple AWS Accounts using the CLI, so for that you have to configure multiple credentials which we can call multiple profile here. So let’s see how we can configure multiple credentials.
root@devopsage:~# vim ~/.aws/config
### This part already exists as we did setup before
[default]
region = us-east-1
output = json
### Now add another profile and name it as "devopsage"
[devopsage]
region = us-west-1
output = json
- Also update the credentials section for this profile.
root@devopsage:~# vim ~/.aws/credentials
### this already exists, default profile
[default]
aws_access_key_id = your access_key_Id_here
aws_secret_access_key = your_secret_access_key
### Add credentials for devopsage profile
[devopsage]
aws_access_key_id = access_key_id_for_devopsage_profile_different_account_credentials
aws_secret_access_key = secret_access_key_for_devopsage_profile_different_account_credentials
- Now, as you are done with the setup of multiple profile, we can use these profile (credentials) to manage our 2 different AWS accounts. Only the thing is that, we have to add our profile name with our Cli Commands.
root@devopsage:~# aws s3 ls // this will use the default profile (credentials)
root@devopsage:~# aws s3 ls --profile devopsage // this will be executed in our 2nd AWS Account as it's using our different profile.