How to Automate Code

Hey guys,

One of the first big steps you take when learning to code is going from doing tutorials to working on actual projects. There’s a lot of steps you have to take, such as writing the code, learning how to deploy that code to a platform or cloud host, automating that code to run, etc.

Creating and working on production-level projects is going to be the best way to continue increasing your skills and showcase your abilities.

In this article let’s talk about how to automate your code and why you should be trying to automate as much as possible.

Why automation?

Imagine you work for Google.

In this job, Google decides to move away from automation and have people manually run the code every time someone does a Google search. So now your job is to manually run a piece of code every time someone clicks search to retrieve the results and deliver them back to them.

Hopefully, you can see how terrible that would be!

Automation helps with scaling, running code pieces, gathering data, running ML models, and more.

You want to automate your code so that you don’t have to manually run it every single time. It just runs and you can work on other tasks.

The two types of automation

With automation of code, you are going to see mainly two different types of automation.

1. Event-driven automation

2. Schedule-based automation

Event-driven happens every time an event is triggered such as in the Google search example. Every time something happens, it triggers a piece of code. This piece of code could be rendering a piece of frontend, it could be storing information in the database, running an ML model, etc.

Schedule-based means that you are going to run a piece of code on a defined schedule. This is very common in things such as data collection, web scraping, and analytics. To do this something called a cron job is used, which is just a syntax that allows you to say how often you want to run the code.

It looks like how in this picture each of the stars represents a different element of time.

So if you set it up to run * * * * *, you would execute the code every minute of every day.

If you changed it to 0 23 * * *, you would run it once a day at 11 pm (23:00).

How to automate

We now know why we should automate and different types of automation, so let’s go through a quick example of how to easily automate a data collection pipeline.

There’s a couple of ways to do this such as by using a cloud provider such as AWS, you can automate it on a server you own, you can set up something locally on a computer or something like a Raspberry Pi.

The simplest and cheapest way I’ve found is to use GitHub Actions which is how I power the data collection in my analytics site.

You’ll need a GitHub account, and after you have a GitHub account you’ll also need to have a repository created with some code.

This is what my repository looks like for my analytics site:

As you can see I have all of my code stored in this repository.

There are essentially two ways to create an automation action,

1. Set up and store the code in a .yml file in the .github/workflows folder you can see at the top of my repository

   This is what my code file looks like:

   

   There are some key things here:

   • name → the name of the workflow

   • schedule → the cron job trigger, mine runs at 23:55 every day

   • jobs → a series of events you want to happen

     • Mine first gets the code, then installs Python, then installs all my dependencies, then runs the script I tell it

   The great thing is that it runs every day, and if it fails I get an email saying it failed. If you go to the actions tab you can also see all of my recent runs:

   

2. Create or choose an automation by going to the actions tab, and then “new workflow”

   You can see they have a lot of options to choose from:

   

   This can be a simple way to get started and they have templates to choose from that can make it easy. This will also eventually create a .yml file where the action description will be stored.

   In my opinion, I just like to do the .yml file myself and it becomes pretty easy to configure by utilizing ChatGPT.

This is how you can automate code which is an important step in building production-ready systems!