Search for a command to run...
I am a Data Engineer I love Python! I am based in Singapore I have good knowledge on cloud I have some knowledge on building Machine Learning Models I have helped 30+ companies use data effectively I have worked with Petabytes scale of data I write @ TowardsDataScience Nowadays I am focusing on learning Natural Language Processing Ask me about anything, I'll be happy to help.
No comments yet. Be the first to comment.
Great Expectations Tutorial
Data Engineering vs. Data Science: Choosing Your Data Path
Future-Proof Your Success: Unveiling the Must-Have Certifications for 2024 and Beyond!
In today's fast-paced digital world, keeping up with the latest advancements in data engineering is crucial to stay ahead of the competition. With the amount of data collected every day increasing, data engineering plays an important role in guarante...
Data Engineering Fundamentals with Python
Ansible is a universal language, unraveling the mystery of how work gets done. — Ansible
When it comes to automation, having a multipurpose and hassle-free option for ease and agility becomes crucial. This approach is applied when it comes to choosing an IT automation engine. Who would want to have separate engines for performing different tasks, after all? That is when Ansible comes into the picture.
So what is Ansible? Why is it an important tool in automation? How does it work? I will be answering all these questions through a data engineer’s perspective in this blog today.
Ansible is a radically simple IT automation engine that automates cloud provisioning, configuration management, application deployment, intra-service orchestration, and many other IT needs.
Ansible allows you to write the configuration files in YAML in a certain format, and they work cohesively to start a server, build a network, deploy the application, add configuration files, and restart the server for you; all of this is done order-wise.
It essentially delivers simple IT automation that ends repetitive tasks and frees up DevOps teams for more strategic work.
Automation: All it takes is to run a single command to automate all the steps, which is a key to efficient production deployment that is done end-to-end. Therefore, it reduces the need for bigger DevOps teams.
Fewer Errors: Due to fewer codes and more configuration, Ansible is prone to fewer errors. The deployment is done using YAML files instead of heavy SSH files.
Less Time Consuming: Writing YAML files takes less amount of time, making Ansible an agile tool.
Universal Code for all Infrastructure Platforms: One of the reasons I recommend Ansible is its sheer multi-utility. You just need to write one code, be it for bare-metal machines, virtual machines, or clouds, and it works everywhere.
Baremetal Machines: Ansible integrates with multiple datacenter management tools to both invoke and enact the provisioning steps required.
Virtualized: With Ansible, you can simplify the experience of cross-platform management. It gives you the flexibility and choice to manage your diverse environment.
Network Automation: You can configure, validate, and ensure continuous compliance for physical network devices. Ansible is the only language that can easily provision across multi-vendor environments, replacing the need for manual processes.
Storage: From software-defined storage, cloud-based storage, or even hardware storage appliances, you can find a module to leverage Ansible’s common and powerful language.
Cloud: For Public Cloud, Ansible is packaged with hundreds of modules supporting services on the largest public cloud platforms. If using Private Cloud, one of the easiest ways to deploy, configure, and orchestrate OpenStack private cloud is using Ansible.
The core Ansible project manages systems by connecting to them over existing transport mechanisms already in use for your machines. For Linux and Unix machines, this means using SSH, either using OpenSSH or in constrained environments, paramiko (a Python library). For Windows hosts, this means using Windows Remote Management via PowerShell remoting.
It works by connecting to nodes (servers, clients, etc.) on a network, and then transferring Ansible Modules, which are small Ansible programs containing baked-in arguments, to these transport mechanisms to a temporary directory on the remote machine, executed, and then removed in one action. The modules return JSON over standard output, and this return data is processed by the Ansible program on the controlling machine.
These modules are executed by Ansible over SSH and are removed once finished. This only requires login access to managed nodes at the control node to take place. There are various forms of authentication that are supported, however, SSH keys are the most used way to grant access.
The result of this is that a very large amount of remote activity can occur with a minimum of traffic interchange. Modules manage “idempotent resources” and are not simply commands or scripts. For instance, a module can decide that a package should be installed at a particular version and knows not to execute any commands if the system is already in the proper working state.
The term ‘Ansible Modules’ may sound complex, but the biggest complexity is handled by Ansible, and not by the user. Each module defines what holds on any managed node. For example, if an administrator decides that each workstation should have LibreOffice version X.Y, then it is Ansible’s duty to find out if any workstation has a different version installed. If so, it detects the OS and runs the necessary routine to update it to the desired version. In this way, all the workstations in an organization can be updated overnight with the software that is supported by the IT department. However, maintaining infrastructure is more than just checking versions of the software.
Ansible Modules gives you the ability to automate tasks across numerous computers. If you’re a programmer, it grants you the freedom to write your very own custom modules to perform specialized tasks.
While the modules provide the means to accomplish a task, Ansible Playbook provides you the way to use them. A playbook is basically a configuration file written in YAML which provides instructions about what needs to be done to bring a managed node into the desired state. They are meant to be simple, self-documenting, and human-readable. They are also idempotent, which means it can be run on a system at any time without any negative effect. If you run a playbook on a system that is already properly configured in its desired state, even then that system should still be properly configured after it runs.
A playbook can be fairly simple, such as this particular one that installs, as a privileged user, the Apache HTTP server on any node in an IT department’s webservers group:
 on[ Unsplash](https://unsplash.com/s/photos/tech?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText)***](https://cdn-images-1.medium.com/max/3200/0*s0V5qVd7EAHkXE6i)
Photo by**- name: Apache server installed
hosts: webservers
become: yes**
Playbooks can also be quite complex, with variables and conditionals. However, Ansible modules do most of the real work, playbooks stay readable, brief, and clear even though they can arrange entire networks of the managed nodes.
If you want to play around with playbooks, some full sets are illustrating a lot of these techniques in the ansible-examples repository.
You can learn Ansible at work or by using it at home. If you have just started with YAML, take some time out to learn it and then write your first playbook. If you wish to start small, you can get Ansible installed on your PC and use it to manage itself or a few computers on your home network. Most importantly, practicing is the key. Try out different modules to get comfortable using and configuring the new ones. After all, they are your gateway to making your nodes to conform to your infrastructure designs. Ansible is the engine that makes it all possible.
(Part 1) was more around introduction on key concepts of Ansible As a (Part 2) of the tutorial, I will cover an example deployment from scratch on GCP
If you are interested in similar content do follow me on Twitter and Linkedin