Brilliant Guide to Check all possible ways to view Disk usage on Ubuntu Machine

Monitoring of application or system disk utilization has always remained a top most and crucial responsibility of any IT engineer. In the IT world with various software’s , automation and tools it is very important to keep a track of disk utilization regularly.

Having said that, In this tutorial we will show you best commands and tools to work with your disk utilization. Please follow me along to read and see these commands and their usage.

Table of content

  1. Check Disk Space using disk free or disk filesystems command ( df )
  2. Check Disk Space using disk usage command ( du )
  3. Check Disk Usage using ls command
  4. Check Disk Usage using pydf command
  5. Check Disk Usage using Ncdu command( Ncurses Disk Usage )
  6. Check Disk Usage using duc command
  7. conclusion

Check Disk Space using disk free or disk filesystems command (df)

It stands for disk free. This command provides us information about the available space and used space on a file system. There are multiple parameters which can be passed along with this utility to provide additional outputs. Lets look at some of the commands from this utility.

  • To see all disk space available on all the mounted file systems on ubuntu machine.
df
  • To see all disk space available on all the mounted file systems on ubuntu machine in human readable format.
    • You will notice a difference in this command output and a previous. The difference is instead of 1k-blocks you will see size which is human readable.
df -h
  • To check the disk usage along with type of filesystem
df -T
  • To check disk usage of particular Filesystem
df /dev/xvda1
  • To check disk usage of multiple directories.
df -h  /opt /var /etc /lib
  • To check only Percent of used disk space
df -h --output=source,pcent
  • To check data usage based on filesystem wise
df -h -t ext4

Check Disk Space using disk usage command ( du )

du command provides disk usage information. This command provides file and directories space utilization. Lets see some of the example .

  • To check disk usage of directory
du /lib # Here we are taking lib directory
  • To check disk usage of directory with different block size type .
    • M for MB
    • G for GB
    • T for TB
du -BM /var
  • To check disk usage according to the size
    • Here s represents summarize
    • Here k represents size in KB , you can use M, G or T and so on
    • Here sort represents sort
    • Here n represents in numerical order
    • Here r represents in reverse order
du -sk /opt/* | sort -nr

Check Disk Usage using ls command

ls command is used of listing of files but also provides information about disk utilized by directories and files. Lets see some of these command.

  • To list the files in human readable format.
ls -lh
  • To list the file in descending order of size of files.
ls -ls

Check Disk Usage using pydf command

pydf is a python based command-line tool which is used to display disk usage with different colors. Lets dive into command now.

  • To check the disk usage with pydf
pydf -h 

Check Disk Usage using Ncdu command (Ncurses Disk Usage)

Ncdu is a disk utility for Unix systems. This command is text-based user interface under the [n]curses programming library.Let us see a command from Ncdu

ncdu

Check Disk Usage using duc command

Duc is a  command line utility which queries the disk usage database and also create, maintain and the database.

  • Before we run a command using duc be sure to install duc package.
sudo apt install duc
  • duc is successfully installed , now lets now run a command
duc index /usr
  • To list the disk usage using duc command with user interface
duc ui /usr

Conclusion

There are various ways to identify and view disk usage in Linux or ubuntu operating system. In this tutorial we learnt and showed best commands and disk utilities to work with . Now are you are ready to troubleshoot disk usage issues or work with your files or application and identify the disk utilization.

Hope this tutorial gave you in depth understanding and best commands to work with disk usage . Hoping you never face any disk issues in your organization. Please share if you like.

How to Launch AWS Elastic beanstalk using Terraform

Working on Amazon web service itself is a amazing thing as there are lots of services which are provided by Amazon to make you free from hassle of setting up entire infrastructure step by step . Suppose you want to create a 3 instance and align a load balancer in front of them and hosting a website and store all data in database.

No doubt we have amazon provided AWS EC2 service, for database we have AWS RDS service in case of relational database and for load balancer amazon provides ELB i.e.. elastic load balancers. But what if we have can have a common platform to work with all these services together and work ? Wouldn’t that be much easier for all of us. Yes that’s correct it will make things much easier, so does the Amazon Elastic beanstalk and is one of the best service one can use Amazon Web Service.

In this tutorial, we will learn how to step up Amazon Elastic beanstalk using terraform on AWS step by step and then upload the code to run one of the simple application.

Table of content

  1. What is Elastic beanstalk?
  2. Prerequisites
  3. How to Install Terraform on Ubuntu 18.04 LTS
  4. Terraform Configuration Files and Structure
  5. Launch Elastic beanstalk on Amazon Web Service using Terraform
  6. Conclusion

What is Elastic beanstalk?

AWS Elastic beanstalk is one of the best service one can use on amazon web service tool. It is service which can work with variety of languages such as python , go , ruby, java , .net , PHP for hosting the application. The only thing you need to do in elastic beanstalk is just create your code with any of these high level languages and upload your code in AWS elastic beanstalk, then rest of the things will be taken care by elastic beanstalk itself such as scaling, load balancing , monitoring and so on .

Elastic beanstalk makes the life of developer as well as for cloud admins or sysadmins so easy compared to setting each service individually and interlinking each other.

  • Some of the key benefits of AWS Elastic beanstalk are
    1. It scales the applications up or down as per the required traffic.
    2. As infrastructure is managed and taken care by AWS Elastic beanstalk developers working with admins don’t need to spend much time.
    3. It is fast and easy to setup
    4. You can interlink with lots of other AWS services of your own choice or you can skip it such as linking of application or classic or network load balancer.
Listen to What is AWS Elastic beanstalk?

Prerequisites

  • Ubuntu machine to run terraform preferably 18.04 version + , if you don’t have any machine you can create a ec2 instance on AWS account
  • Recommended to have 4GB RAM
  • At least 5GB of drive space
  • Ubuntu machine should have IAM role attached with AWS Elastic beanstalk creation permissions or it is always great to have administrator permissions to work with demo’s.

You may incur a small charge for creating an EC2 instance on Amazon Managed Web Service.

How to Install Terraform on Ubuntu 18.04 LTS

  • Update your already existing system packages.
sudo apt update
  • Download the latest version of terraform in opt directory
wget https://releases.hashicorp.com/terraform/0.14.8/terraform_0.14.8_linux_amd64.zip
This image has an empty alt attribute; its file name is image-163.png
  • Install zip package which will be required to unzip
sudo apt-get install zip -y
  • unzip the Terraform download zip file
unzip terraform*.zip
  • Move the executable to executable directory
sudo mv terraform /usr/local/bin
  • Verify the terraform by checking terraform command and version of terraform
terraform               # To check if terraform is installed 

terraform -version      # To check the terraform version  
This image has an empty alt attribute; its file name is image-164.png
This image has an empty alt attribute; its file name is image-165.png
  • This confirms that terraform has been successfully installed on ubuntu 18.04 machine.

Terraform Configuration Files and Structure

Let us first understand terraform configuration files before running Terraform commands.

  • main.tf : This file contains code that create or import other AWS resources.
  • vars.tf : This file defines variable types and optionally set the values.
  • output.tf: This file helps in generating of the output of AWS resources .The output is generated after the terraform apply command is executed.
  • terraform.tfvars: This file contains the actual values of variables which we created in vars.tf
  • provider.tf: This file is very important . You need to provide the details of providers such as AWS , Oracle or Google etc. so that terraform can make the communication with the same provider and then work with resources.

Launch AWS Elastic beanstalk on AWS using Terraform

  • Now you will create all the configuration files which are required for creation of Elastic beanstalk on AWS account .
  • Create a folder in opt directory and name it as terraform-elasticbeanstalk-demo and create all the files under this folder.

main.tf

# Create elastic beanstalk application


resource "aws_elastic_beanstalk_application" "elasticapp" {
  name = var.elasticapp
}

# Create elastic beanstalk Environment

resource "aws_elastic_beanstalk_environment" "beanstalkappenv" {
  name                = var.beanstalkappenv
  application         = aws_elastic_beanstalk_application.elasticapp.name
  solution_stack_name = var.solution_stack_name
  tier                = var.tier

  setting {
    namespace = "aws:ec2:vpc"
    name      = "VPCId"
    value     = var.vpc_id
  }
  setting {
    namespace = "aws:autoscaling:launchconfiguration"
    name      = "IamInstanceProfile"
    value     =  "aws-elasticbeanstalk-ec2-role"
  }
  setting {
    namespace = "aws:ec2:vpc"
    name      = "AssociatePublicIpAddress"
    value     =  "True"
  }

  setting {
    namespace = "aws:ec2:vpc"
    name      = "Subnets"
    value     = join(",", var.public_subnets)
  }
  setting {
    namespace = "aws:elasticbeanstalk:environment:process:default"
    name      = "MatcherHTTPCode"
    value     = "200"
  }
  setting {
    namespace = "aws:elasticbeanstalk:environment"
    name      = "LoadBalancerType"
    value     = "application"
  }
  setting {
    namespace = "aws:autoscaling:launchconfiguration"
    name      = "InstanceType"
    value     = "t2.medium"
  }
  setting {
    namespace = "aws:ec2:vpc"
    name      = "ELBScheme"
    value     = "internet facing"
  }
  setting {
    namespace = "aws:autoscaling:asg"
    name      = "MinSize"
    value     = 1
  }
  setting {
    namespace = "aws:autoscaling:asg"
    name      = "MaxSize"
    value     = 2
  }
  setting {
    namespace = "aws:elasticbeanstalk:healthreporting:system"
    name      = "SystemType"
    value     = "enhanced"
  }

}

vars.tf

variable "elasticapp" {
  default = "myapp"
}
variable "beanstalkappenv" {
  default = "myenv"
}
variable "solution_stack_name" {
  type = string
}
variable "tier" {
  type = string
}

variable "vpc_id" {}
variable "public_subnets" {}
variable "elb_public_subnets" {}

provider.tf

provider "aws" {
  region = "us-east-2"
}

terraform.tfvars

vpc_id              = "vpc-XXXXXXXXX"
Instance_type       = "t2.medium"
minsize             = 1
maxsize             = 2
public_subnets     = ["subnet-XXXXXXXXXX", "subnet-XXXXXXXXX"] # Service Subnet
elb_public_subnets = ["subnet-XXXXXXXXXX", "subnet-XXXXXXXXX"] # ELB Subnet
tier = "WebServer"
solution_stack_name= "64bit Amazon Linux 2 v3.2.0 running Python 3.8"
  • Now your files and code are ready for execution . Initialize the terraform
terraform init
  • Terraform initialized successfully , now its time to see the plan which is kind of blueprint before deployment. We generally use plan to confirm if correct resources is going to provisioned or deleted.
terraform plan
  • After verification , now its time to actually deploy the code using apply.
terraform apply

Great Job, terraform commands execution was done successfully. Now we should have AWS Elastic beanstalk launched in AWS.

  • Now go to AWS account and search for AWS Elastic beanstalk service and after it gets open click on myenv and check the URL

Conclusion

In this tutorial, we demonstrated some benefits of Amazon Elastic beanstalk and learnt how to set up Amazon Elastic beanstalk using terraform on AWS step by step .

Hope this tutorial will help you in understanding the terraform and provisioning the elastic beanstalk on Amazon cloud. Please share with your friends

How to Setup Docker step by step on Ubuntu machine

I am sure most of you have installed lots of application on single Virtual machine and faced challenges while maintaining them and facing memory or system issues and things become more ugly when you troubleshoot each application to fix the problem.

What should one do ? Cannot we isolate our apps so that we can easily find the problem ? Cant one app which is not working should be down and others should run as it is ? Yes, that correct that’s very much possible with docker and containers. They provide you isolated environment and are your friend for deploying many applications with each taking its own container.

In this tutorial, you’ll install and use Docker on Ubuntu 18.04. You’ll then work with containers , images and push an image to a Docker Repository. So lets get started.

Table of Content

  1. What is Docker ?
  2. Prerequisites
  3. How to Install Docker on ubuntu machine
  4. How to run Docker commands using Non Root user
  5. Working with docker images and containers
  6. Pushing docker images to docker hub
  7. Conclusion

What is Docker ?

🐟Docker is an open source tool for developing , shipping and running applications. It has ability to run applications in loosely isolated environment using containers. Docker is an application which helps in management of containers in a very smooth and effective way. In containers you can isolate your applications. Docker is quite similar to virtual machine but it is light weighted and can be ported easily.

Containers are light weighted as they are independent of hypervisors load and configuration. They directly connect with machines ie. hosts kernel.

Prerequisites

  • Ubuntu machine preferably 18.04 version + , if you don’t have any machine you can create a ec2 instance on AWS account
  • Recommended to have 4GB RAM
  • At least 5GB of drive space
  • An account on Docker Hub if you wish to create your own images and push them to Docker Hub.

You may incur a small charge for creating an EC2 instance on Amazon Managed Web Service.

How to Install Docker on Ubuntu 18.04 LTS

sudo apt update
  •  Install some required packages for apt so that it can use over https
sudo apt install apt-transport-https ca-certificates curl software-properties-common
  • Add the GPG key for the official Docker repository to your system
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key --keyring /etc/apt/trusted.gpg.d/docker.gpg add -
  • Add the Docker repository to APT sources
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
sudo apt update
  • Now, Install docker
sudo apt-get update && sudo apt-get install -y \
  containerd.io=1.2.13-2 \
  docker-ce=5:19.03.11~3-0~ubuntu-$(lsb_release -cs) \
  docker-ce-cli=5:19.03.11~3-0~ubuntu-$(lsb_release -cs)
  • Docker should be started now, verify the status and version of docker.
  • Check the docker version
  • Docker is successfully installed and now you are reading to work with docker and containers.

How to run Docker commands using Non Root user

To enable users other than root and users with sudo access to be able to run Docker commands:

  1. Create the docker group
groupadd docker
  1. Add the users that should have Docker access to the docker group
sudo usermod -a -G docker jenkins   # To add Jenkins user to docker group
sudo usermod -a -G docker user1     # To add user1 to docker group
  1. Restart the docker service by following command
# The UNIX socket /var/run/docker.sock is now writable by members of the docker group.

service docker restart 

Working with docker images and containers

Most important part in docker is docker images which contains your code , environment to run it and several parameters. Docker images are stored on docker repository known as docker hub , though you can store it on your local machine as well.

  • To pull docker image ubuntu on your system
docker pull ubuntu
  • Now check the downloaded image.
docker images
  • In this tutorial we are providing the demo so we will use the same pulled docker image and run our first container , but you can update the same image with more configurations or as per requirements as well.
  • Run the first container
docker run -it ubuntu                # Here i and t provides you interactive shell access and ubuntu is image name
  • We used exit to come out of shell , so container has been exited. To check the containers details.
 docker ps -a

Pushing docker images to docker hub

  • Firstly you will need to login to docker hub with your credentials and then login.
docker login -u docker-registry-user
  • Before you push your docker image to docker hub , its highly recommended to to tag your image with your docker hub username .
docker tag ubuntu:latest  <dockerhub-username>/ubuntu:latest
  • After successful login in docker hub now push your docker image
docker push <dockerhub-username>/ubuntu:latest

Conclusion

In this tutorial you installed Docker, worked with images and containers, and pushed a image to Docker Hub

Hope this tutorial will help you in understanding and setting up Docker on ubuntu machine. Please share with your friends.

Fixing an Ubuntu System that Will Not Boot

How can you fix an ubuntu machine if its causing issues? There could be various reasons and sometimes your machine even don’t boot, that’s a huge problem !! Why not fix it or learn how to fix it permanently? Lets begin!

Steps to be performed when system will not boot.

  1. Checking BIOS
  • If your system is unable to boot even with LIVE DVD or USB Drive , there could be two possibilities.
    • You accidently deleted the boot device or
    • Its an Hardware issue.

2. Checking GRUB

  • If you are able to turn on the computer on and get past the initial BIOS startup then bring up the GRUB menu .It could be that few things might have overwritten which can be solved by using the recovery mode. Lets learn how to recover all the things.
    • Press Shift after the BIOS is done to access the GRUB menu.
    • Select Advanced Options for Ubuntu.
    • From the new menu, select an entry with the words recovery mode. This boots into a recovery menu with options to automatically fix several possible problems, or at least it lets you boot into a minimal recovery-mode version of Ubuntu with only the most necessary processes loaded. From here, you may be able to fix disks, check file systems, drop to a root prompt to fix file permissions, and so on.
  • In case you are unable to bring up GRUB menu then reinstalling GRUB is the only option. Lets learn how to reinstall GRUB.
    • Boot Ubuntu from a live DVD or bootable USB drive that has the same Ubuntu
      release as your system, such as 16.04.
    • Determine the boot drive on your system:
      a. Open a terminal and use sudo fdisk -l to list the drives attached to the system.
      b. Look for an entry in the output with an * in the Boot column. This is your boot
      device. It will look something like /dev/sda1.
    • Mount the Ubuntu partition at /mnt by using this command, replacing /dev/sda1
      with the information you just found:
      sudo mount /dev/sda1 /mnt
    • Reinstall GRUB with this command, again replacing /dev/sda1 with what you found
      earlier:
      sudo grub-install --boot-directory=/mnt/boot /dev/sda1
    • Restart the computer, and Ubuntu should boot properly.

Conclusion

In this small and very important post you learnt how to bring back ubuntu machine that doesn’t boot and learnt ways to recover them in future !!