Logging Docker Container Application logs to AWS Cloudwatch Logs

Logging Plays a very important role for any kind of the application you are running as it helps to debug any issues to you running applications or your system. Use of Docker Container is very popular nowadays and there is a lot of infrastructure running application on top of the Docker Container.

For this particular blog, we will see how we can add the container logs to the AWS cloudwatch logs using the AWS log driver. Using the log driver, the application logs are automatically pushed to the Cloudwatch logs which are very easy to visualize and identify the exceptions. We Assume that you have already have an AWS Account.

Prerequisite,

1. AWS Account
2. ec2 Instance, where you will be running your application on top of the docker container.
3. An IAM Role, which has sufficient permission to push logs to the Cloudwatch log Stream.
4. Existing Cloudwatch log group to which you will be pushing your logs.
5. Dockerized Environment and Docker Compose.

Note: You can refer our previous blogs to,
1. Install Docker on Ubuntu
2. Install Docker On CentOS
3. Install Docker Compose
4. Working With Docker Compose
You can have a look at any of the above blog post to have a better understanding.
So let’s get Started,
Check whether the docker and docker compose are installed or not, if not install it using the above link.

ubuntu@devopsage:~$ docker version
ubuntu@devopsage:~$ docker-compose version

Now Clone a Sample docker-nginx repository from the devopsage.

ubuntu@devopsage:~$ git clone https://github.com/devopsage/docker-compose-nginx.git

ubuntu@devopsage:~$ cd docker-compose-nginx/

Now add the log driver to the services in the docker compose, the format will be like below,

logging:
  driver: "awslogs"
  options:
    awslogs-group: "web-logs"
    awslogs-region: "ap-south-1"
    awslogs-stream: web1

So the complete docker compose file will look like as below, Just add the logging Section to the docker compose.

version: '3'
 services:
   web1:
     image: nginx:1.0
     logging:
         driver: "awslogs"
         options:
           awslogs-group: "web-logs"
           awslogs-region: "ap-south-1"
           awslogs-stream: web1
     ports:
       - 80:80
 web2:
     image: nginx:1.2
     logging:
         driver: "awslogs"
         options:
           awslogs-group: "web-logs"
           awslogs-region: "ap-south-1"
           awslogs-stream: web2
     ports:
       - 90:80

Now, Once you have updated the logging Section in the compose file then go ahead and create the Log group in cloudwatch logs. For this particular example, web-logs is the log group name. docker compose file itself will create the log stream and it does not need to be created manually.
Now Run the Docker compose file and check in the cloudwatch logs whether the log stream has been created or not.

ubuntu@devopsage:~/docker-compose-nginx$ docker-compose -f docker-compose.yml up -d
ubuntu@devopsage:~/docker-compose-nginx$ docker ps
 CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                NAMES
 4a96c310ad4b        nginx:1.0           "nginx -g 'daemon of…"   17 seconds ago      Up 15 seconds       0.0.0.0:80->80/tcp   docker-compose-nginx_web1_1
 9c2499c55cc3        nginx:1.2           "nginx -g 'daemon of…"   17 seconds ago      Up 15 seconds       0.0.0.0:90->80/tcp   docker-compose-nginx_web2_1

Both the Containers are up and running, just verify the logs. Go to the Cloudwatch Section in AWS, Click on Logs.

Now Click on web-logs then you will see 2 logs streams for both the application in the docker compose. Click on it and check whether you can see the incoming logs or not.

Note: You may Not see any logs as this web service does not have any logs, Just to create some log, try to hit the Ip:port on the web browser and then see the incoming logs.

So here we can see the logs generated by the application running on the container are pushed to the cloudwatch logs. You Can easily filter out your logs in case you need to perform any troubleshooting.

---

If you Like Our Content here at Devopsage, then please support us by sharing this post.

Please Like and follow us at, LinkedIn, Facebook, Twitter, and GitHub

Also, Please comment on the post with your views and let us know if any changes need to be done.

Thanks!!
You May Also Need to Look Into,

Working With Docker Compose and Its Basic Commands.

Install Docker Compose on Linux – Ubuntu/CentOS.

How to Install Docker on Redhat/CentOS 7

How to Install Docker on Ubuntu 16.04