Elastic Beanstalk crons on multi-instance environments

Wow. What a lot of pain.

So let me first confirm what won’t work. Whether Apache 1, Apache 2, leader_only or EB_IS_COMMAND_LEADER testing, you cannot consistenty get exactly one (not zero, or 2+) instances to run a cron job over a long period of time.

The issue is the the leader information isn’t safe. It’s only available during deployments, when you usually first set up the cron job. But when an instance replacement happens—for whatever reason—there is no related deployment, the test to check if the new instance is the leader fails, and you end up with no crons.

So for anything you do at the moment of deployment—database migrations, logging, whatever— that you only want to run once, even on multi-instance environments in an Elastic Beanstalk Application, you can use and trust the leader tests available to you.

But for anything you want to run regularly, on a schedule across weeks and months, you cannot trust those options.

After a lot of research, and trial and error there are two options available to you.

Worker environments

The ‘official’ way is probably via a worker environment. Essentially an instance set up specifically to schedule and run short or long lives processes.

You can learn more about them via the links below, but for the rest of this article we’ll be talking about the alternative option of ‘runtime leader testing’.

https://medium.com/@joelennon/running-cron-jobs-on-amazon-web-services-aws-elastic-beanstalk-a41d91d1c571

Runtime leader testing

This option means running the cron on every instance of an environment, but the initial section of the code being run checks the if the instance id that it’s running on is the first to be returned in the list of all instances. That is, only the first instance id returned by the aws elastic beanstalk api will continue to run the task.

Below the examples are in PHP, but it’s a valid method for other languages too.

We’re assuming you’ve already set up a cron and know how to alter it to make it run on all instances of your environment.

The steps in this guide:

  • Create an IAM user just for this process and give it limited read permissions
  • Add the credentials to EB configuration
  • Write code to check the instance id from the api against the current instance id

Adding the user

  • Open IAM in AWS console
  • Click Users
  • Click Add users
  • Type your User name – we used “[applicationCode]-eb-read-user”
  • Click the ‘Access Key’ access type
  • Click Next: Permissions
  • Click the ‘Attach existing policies directly’ tab
  • Type ‘AWSElasticBeanstalkReadOnly’ and select it
    • NB: This permission actually gives more access than is required and you could trim it further
  • Click Next: Tags
  • Click Next: Review
  • Click Create user
  • Take a copy of the KEY and SECRET

Storing the key and secret in the environment

  • Open your environment
  • Click configuration
  • Click ‘edit’ on the ‘Software’ section
  • Add two new items, one for the key (examples use AWS_KEY) and one for the secret (examples use AWS_SECRET)

The code

So this is php, but the basic idea is:

  • Install the aws library – here we use composer
  • Add some code to call the library
  • Compare the first returned instance id against the id of the instance running the code

Install the aws library

$composer require aws/aws-sdk-php

Add the code to do the check

$client = ElasticBeanstalkClient::factory([
    'credentials' => [
        'key' => $_ENV['AWS_KEY'],
        'secret' => $_ENV['AWS_SECRET'],
    ],
    'region'  => '[[your region]]',
    'version' => 'latest'
]);

Replace [[your_region]] above with your own

$result = $client->describeEnvironmentResources([
    'EnvironmentName' => '[[your_environment_name]]'
]);

Then load in the current instance’s id

$currentId = file_get_contents("http://instance-data/latest/meta-data/instance-id");

Then compare the API’s first result to the current one. If the same, so something, otherwise don’t.

if ($currentId == $result['EnvironmentResources']['Instances'][0]['Id'])
{
    // do something. Only one instance will get here
}
else
{
    // don't do anything. All other instances will be here
}

Other useful resources:

https://rotaready.com/blog/scheduled-tasks-elastic-beanstalk-cron

Check AWS Lightsail Bitnami WordPress PHP version

Before you spend time upgrading to a new lightsail instance in the hope of your PHP version meeting WordPress minimum requirements, it’s useful to know if the version available is high enough.

You can check which PHP version you’re going to end up with using the following steps:

  • Open your lightsail home page
  • Click the Create Instance button
  • Check the ‘Linux/Unix’ option is selected
  • Look for the ‘WordPress’ option and make a note of the number displayed underneath (Something like a.b.c-d)

Ok, next we’ll check the Bitnami WordPress changelog

  • Open the changelog file
  • Search for a matching number from your note above
  • Check for a bullet which begins Updated php to
  • If no match, keep reading down the file until you find the first match for Updated php to
  • That’s the current PHP version you’ll get if you spin up a new lightsail wordpress install

If it’s the same or higher than the message being shown inside WordPress you can go ahead. If it’s not then you need to set a reminder to check back regularly on the lightsail creation page to see when the version updates.

AWS don’t use the very latest version available from Bitnami.

The final thing to consider… strangely is if WordPress supports the PHP version you’ll be upgrading to. WordPress maintain a list of which versions of PHP they support (properly and in beta).

WHM: List Largest Emails In An Account With Subject

I recently needed to find the largest emails in various email accounts and share the size and subject. Not if it was a single email account which I had access too there are likely much simpler options using webmail or an email client to get the same outcome.

It looks like this hasn’t been written about before so I pieced together various tutorials to end up with the final solution.

This guide assumes you know what you’re doing on the command line prompt and that you have the permissions to view the files and folders below. It also assumes you have the permissions to go poking around in people’s email accounts, so be sure to ask for that before you start outputing private email subjects from their accounts

The first step is to check you have the following results for a list of files in the chosen CPanel account’s mail folder:

$ls -a /home/[account]/mail

The above should show you a folder for each domain, but hopefully also a symlink for each account. In my case:

.email@example_com

The above actually points to example.com/email but we’ll be using it in our code.

This code also expects emails to be stored in files ending with ,Sab or ,RSab or ,S or ,RS so check that too.

So here is the full example which I’ll then break down below. You’ll need to update it to suit your needs:

find /home/[account]/mail/.email\@example_com/ -type f ( -iname "*,Sab" -or -iname "*,RSab" -or -iname "*,S" -or -iname "*,RS" ) -size +1M -exec grep "Subject: " {} \; -printf '%s B - ' | sort -nr | head -10

First we use the find command to list files which match our email file name (end with 4 options) and have a file size of greater than 1M:

find /home/[account]/mail/.email\@example_com/ -type f ( -iname "*,Sab" -or -iname "*,RSab" -or -iname "*,S" -or -iname "*,RS" ) -size +1M

We then trigger an exec on that which will eventually return their subject as part of the output, and also print their file size in bytes (%s) followed by a B and a hyphen with -printf

-exec grep "Subject: " {} \; -printf '%s B - '

We then sort the list:

sort -nr

and finally only return the first 10:

head -10

Setting up Google 2FA using an app not SMS

Google appear to be intentionally hiding the option to use an authenticator app such as Google Authenticator, 1Password and LastPass to store and generate 2FA codes for your Google Account login.

You can use them, but it’s not clear how, and you have to temporarily submit a mobile number you have access to before you’ll find the option available to you.

  • Head to manage your account.
  • Choose to add 2FA
  • Select to do it via SMS
  • Submit your number
  • Enter the code they send you
  • On the next screen select to add an authenticator app
  • Follow the steps and submit the code you can now access
  • View the backup codes and store them somewhere safely
  • Finally you can remove your mobile number from the list of option

So to not use your mobile number and SMS (which is vulnerable to hijacking) you first have to submit your mobile number ¯\_(ツ)_/¯