Quickly get memcached working in Python Django

As with most frameworks, the Django framework for Python can make use of caching to greatly improve performance for many common requests. Here we will look at using memcached as it enjoys good Django support and production use although there is also Redis support which definitely improves on memcached in some aspects such as data persistence.

  1. The first step is to install memcached on your server:
  2. RedHat Linux:

    yum install memcached

    Ubuntu / Debian Linux:

    apt-get install memcached
  3. Let Django know how to access memcached:
  4. In Django’s settings.py file, add the following line:

    'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache'
  5. Load the cache within your application
  6. from django.core.cache import cache
  7. Save the value to the cache
  8. cache.set('exampleValue',exampleValue)
  9. Retrieve the value from the cache
  10. exampleValue = cache.get('exampleValue')

The beauty being that exampleValue can be anything from a computed / database retrieved value to large blocks of static text or a URL etc.

The only problem with caches is they don’t always contain the data you expect, what if the value got flushed or hasn’t yet been stored? Lets rewrite step 5 to handle the event of the value not being available in the cache:

exampleValue = cache.get('exampleValue')
if not exampleValue:
     exampleValue = exampleValueLookup
     cache.set('exampleValue',exampleValue)

Here we see the value exampleValue being retrieved with a backup regeneration if the value has not been set. In a real application this would usually be encapsulated in a getExampleValue function or somewhere appropriate.

Advertisements

Symfony2 custom exceptions

When using the Symfony2 Framework it is often better to create custom exceptions to handle unique circumstances rather than incorrectly utilising one of the many pre-existing Symfony exception types. Using an interface improves code cohesion, allows the adding of custom functionality as needed and is super easy to setup.

The first step is to create a custom exception interface.

namespace AppBundle\Exception;

interface AppBundleExceptionInterface
{
}

Then create a custom exception class (perhaps in an Exception directory if it suits your project’s structure):

namespace AppBundle\Exception;

class NewTypeOfException extends \Exception implements AppBundleExceptionInterface
{
}

This exception can now be thrown and caught as desired, just be sure to name your exceptions well. You may be throwing them long after they were originally created!

The many different syntaxes of PHP’s if statement

The PHP language is blessed with many possible syntax structures for all sorts of language constructs. Some better than others but all have their place . Here we will be examining flexible usage of some of the styles available for use with if statements.

First, let’s look at the styles for the following statement in the standard c style:

$value = 1;
if ($value == 1) {
    print "if expression run";
} elseif ($value == 2) {
    print "elseif (1st) expression run";
} elseif ($value == 3) {
    print "elseif (2nd) expression run";
} else {
    print "else expression run";
}

(note: elseif has identical else if snytax in above and below examples)

In semi-colon syntax:

$value = 1;
if ($value == 1):
    print "if expression run";
elseif ($value == 2):
    print "elseif (1st) expression run";
elseif ($value == 3):
    print "elseif (2nd) expression run";
else:
    print "else expression run";
endif;

In switch statement format:

$value = 1;
switch (true) {
    case ($value == 1):
        print "if expression run";
        break;
    case ($value == 2):
        print "elseif (1st) expression run";
        break;
    case ($value == 3):
        print "elseif (2nd) expression run";
        break;
    default:
        print "else expression run";
};

The following are recursive language constructs.
In inline recursive format:

$value = 1;
($value == 1?
    (
        print "if expression run"
     ) :
    ($value == 2?
        (
            print "elseif (1st) expression run"
        ) :
        ($value == 3?
            (
                print "elseif (2nd) expression run"
            ):
            (
                print "else expression run"
            )
        )
    )
);

Using logical operators:

$value = 1;
$value == 1 and (print "if expression run")
or $value == 2 and (print "elseif expression run")
or $value == 3 and (print "elseif expression run")
or (print "else expression run");

 

Any other syntaxes missed?

Doctrine DateTime format() errors

A pesky and difficult to debug doctrine problem I’ve been encountering is the following occurring after a flush() call:

PHP Fatal error:  Call to a member function format() on a non-object in /var/www/myapp/vendor/doctrine/dbal/lib/Doctrine/DBAL/Types/DateTimeType.php on line 53

Obviously this gives no info on the actual cause of the error, which it turns out is because of a mismatch between doctrine expecting a DateTime object and instead encountering a string object, which does not have a format() function to call.

The solution is to remove the default string value that doctrine’s orm tools sometimes wrongly attaches to objects and instead use the correct DateTime() object, e.g.:

/**
 * MyObject
 *
 * @ORM\Table(name="my_object_table")
 * @ORM\Entity
 */
class MyObject
{
...
    protected $timestamp = '0000-00-00 00:00:00';
...
}

The first problem if you have a lot of model files is actually finding where the problem is. Whilst there are a host of debug tools which can help, a quick and easy hack is to simply add a stacktrace to the offending doctrine vendor file.

First go to your project’s main folder and vim the relevant file, if you are not a vim file, the substitute it for another one of the many text editors out there:

vim vendor/doctrine/dbal/lib/Doctrine/DBAL/Types/DateTimeType.php

Then go to line 53 (or whatever line number appears in the error message) by typing :53 (i.e. a semi-colon followed by the line number) where you should see a line reading:

        return ($value !== null)
            ? $value->format($platform->getDateTimeFormatString()) : null;

Go up one line above this and press the a key to enter insert mode and enter:

        if(is_string($value)) {
                debug_print_backtrace();
        }

A good tip is to save the file using :w, but don’t actually close the file. Running your application again you will then print out the usual – hideously ugly – stacktrace. This can be filtered down but even just loooking at the raw ouput should reveal the reflection classes inspecting the model file a couple of steps down which has the incorrectly set line exampled above. This is where you then remove the changes from the above file, by pressing u to undo the changes. Followed by :q to quit the file.

This string needs to be changed to a DateTime() object and you will need to set a default value in the constructor, e.g.:

public function __construct()
    {
        $this->timestamp = new \DateTime();
    }

Leaving out the parameter will be set it to the current system time. This will lead to doctrine correctly interpreting and saving timestamps to the database upon flush.

A more complete strategy to automating timestamp & date generation for data using Doctrine is present in my article:

Automating timestamps with Doctrine ORM

Stopping favicon 404 spam in Apache error logs

Favicons display many elements of the dark, early days of the Internet. Typically a 16×16 pixel image in ICO format (wtf is that?), they were introduced back in 1999 with Internet Explorer 5. More recently mobile browsers have followed in this unholy tradition by introducing more favicon files, bringing the list of files needing to be created / handled to:

  • favicon.ico
  • apple-touch-icon-precomposed.png
  • apple-touch-icon.png

Whilst they do provide a nice little visual logo space for your website or company, you may sometimes think they aren’t worth the bother. One unexpected problem: An Apache error log full of 404 errors from web browsers trying to locate /favicon.ico, /apple-touch-icon-precomposed.png and /apple-touch-icon.png files in your website’s root directory.

A simple way to prevent this is to add the following code to your apache virtualhost configuration:

Redirect 404 /favicon.ico

<Location /favicon.ico>
    ErrorDocument 404 "favicon does not exist"
</Location>

Redirect 404 /apple-touch-icon-precomposed.png

<Location /apple-touch-icon-precomposed.png>
    ErrorDocument 404 "apple-touch-icon-precomposed does not exist"
</Location>

Redirect 404 /apple-touch-icon.png

<Location /apple-touch-icon.png>
    ErrorDocument 404 "apple-touch-icon does not exist"
</Location>

Whilst this solution seems the most technically correct, it still means your server is going to be hit with constant 404 requests for the non-existant files on every page and frame load, although thankfully they should no longer be clogging up error logs. A more pragmatic approach is to create 3 blank files, 1 in ICO format and 2 in PNG format, placing them in your website’s root public directory as favicon.ico, apple-touch-icon-precomposed.png and apple-touch-icon.png. A good tip is to set the Expires header to sometime in the distant future to avoid any further favicon requests.

You will also need to add the following code to your page:

<link rel="shortcut icon" href="#" />

Now finally you can live in peace from bombardment of favicon requests …until browser makers start adding new favicon types.

libssh2 vs phpseclib

As tempting as it can be to make use of the simple include library which is phpseclib, it is better (if possible) to install the libssh2 module. Libssh2 it grants PHP access to your system’s OpenSSL implementation rather than relying on phpseclib’s own version which is reason alone to use libssh2 despite phpseclib being undeniably more portable, faster and offering enhanced debug facilities (there’s nothing to stop you switching to phpseclib purely to debug pesky issues or writing code first in phpseclib then porting to libssh2).

Whilst OpenSSL has come under attack recently with exploits such as Heartbleed, it remains one of the best tested and trusted security suites around. Major exploit discoveries like Heartbleed and Shellshock (with its openSSH attack vector) demonstrate the need for systems to be patched as soon as possible.

By using libssh2, any patch to the system’s OpenSSL implementation will be automatically applied to your PHP applications. On a related note, unattended-upgrades / yum-cron should always be enabled to ensure you are patched against exploits as they are released with a seemingly increasing regularity.

On a Ubuntu / Debian libssh2 can be installed via the command:

sudo apt-get install libssh2-php

On Red Hat based systems, use:

yum install libssh2

You then need to add the module to your php.ini file with the following line (place it after all the other extension loading calls):

extension=ssh2.so

Then finally restart apache.

In Javascript, the “equal and of the same type” definition only applies to values

The colloquial definition of === is “equal and of the same type” however this is only true for values. For objects, both x == y and x === y comparison will return false even if the x & y objects are identical in every way, it will only return true if x & y are both references to the same object. In the case of == this can be overridden with the use of a toString() / valueOf() functions, but not ===.

The actual definition of equality operators is that == tests for “loose equality” and === tests for “strict equality”. This rather ambiguous definition gives opening to the strange and somewhat inconsistent results which can arise during intricate equality comparisons.

The Mozilla Developer Network has a good rundown on how this loose comparison works out.