How important is that (CRON) job anyway?

For decades, we have had to run jobs on our servers at specific intervals. This includes creating daily backups, reorganizing databases, sending files to other servers or other platforms.

You name it, we have (had) a job for it. In the meantime, life has changed. We still run the type of jobs we ran 30 years ago. But we’ve added a few – and then some more.

We don’t just run a lot of jobs; we run them in the most complex way ever. Consider the inter-dependencies between applications, platforms and databases that are all depending on each other, the time of the day, the day of the week or the person who submitted it. We have secured them to prevent the wrong people from running the wrong jobs and we have connected different platforms, schedulers and other tools to make this all possible.

A glorified alarm clock

So you would expect that a utility like CRON (the default UNIX/Linux scheduler) has evolved as well to help you cope with all this complexity, right? Well, it didn’t…

But despite the fact that today’s CRON is 95 percent the same as the CRON we started using decades ago, it’s still used heavily in many data centers. And the managers of these same datacenters tell me how extremely complex their infrastructure, workload architecture, security and automation is.

So why still use a tool that is nothing more than an alarm clock really? You set it to go off at a certain time, it goes off, and does whatever it’s told to do, no matter if the file that is supposed to be there never arrived, or the back-up that is supposed to start only once the database reorg is done starts backing up, even though it shouldn’t.

The reality of today’s integrated data centers

I realize that many UNIX and Linux experts will now tell me that all this can be solved with scripts; some simple, some complex and some highly complex – all of them written by people with more technical knowledge than I will ever have, and all of them refined and improved in the past decades.

But unfortunately, hardly anybody just uses Linux and UNIX these days. We have different Windows flavors, cloud services, Java applications and databases that have to kick off workload activities. All of this has to work across multiple servers, platforms and operating systems.

And this brings me to the conclusion that no matter how intelligent the people who wrote these shell scripts are, there comes a time for everybody to admit that this is not the most risk-free, efficient, smart way of running complex workloads.

When things go wrong, they really go wrong

We all know that jobs fail for many reasons. And they never fail when all the experts are just sitting there, waiting for things to go wrong. So when things go wrong, you need insight that not only shows the job that fails, but why it fails, what will fail after it and what the impact will be on your SLA’s, including all dependencies across platforms and applications.

Back to square CRON

So let’s go back to CRON. I have done crazy things with it and so have many people who know UNIX and Linux 100 times better than I do. But that was then, and this is now.

Today, people simply expect more from a scheduler than the fact that it starts doing things when I tell it to. The amount of shell scripts needs to be reduced to an absolute minimum if we want to offer real SLA’s and maintainability. CRON is simply not up to the task of enterprise workload management anymore.

Take a look at the chart below for proof.

And the list goes on, and on…

Now, for those CRON lovers amongst you, I’m not saying CRON is bad, but the world has changed and to deal with the complexity that this has caused, we need different solutions. Just like the DBA’s who have gone from one database to 100 with 1000 times more data, or the network folks that have seen their environment grown exponentially.

New times require new challenges and new solutions.

Your organization should be looking for a workload management solution that offers everything under the sun and more, including complex things like simulation, good visualization and above all, predictability. Because that should be the dream of everyone working with job-scheduling solutions, to know in advance that things will go wrong, so you can fix them before they actually do go wrong.

For more information take a look at a quick reference guide about the potential pitfalls of CRON.