Switching databases in Symfony upon application events

We’re big fans of the PHP Symfony2 framework and Doctrine2 ORM but a recent project involved a REST API for a multi-client system where each client wanted their database fully separated from each other, but we only knew which client database to connect to once a user had logged in (so no easy config.yml and parameters.yml configuration).

The login gets handled via Java Web Tokens using the excellent Lexik JWT Authentication bundle. So the first call to /auth/login_check with a username and password retrieves a token along with a id of the client (added using the onAuthenticationSuccessResponse listener documented here. Subsequent requests supply a HTTP Authorization header with the token and a custom “client” header specifying the id of the client (also documented in the above link).

This gave us stateless authentication for our API, but we still had to actually switch the Doctrine dbal connection to point to the right database. The solution (inspired by this Stack Overflow question) involved an onKernelRequest() event listener to switch the database from a base database (holds login / authentication data and is referenced in the default connection settings – or whichever connection you’re overriding – of your config.yml file) to the desired database and update the tokenStorage object accordingly. As a tip though, it’s best to keep the base and individual databases schematically identical as possible for reasons explained later.

First create the Event Listener:

Then add this to the services.yml file:

Then add the client databases to the end of the parameters-dist.yml file:

The security.yml file looks something like:

Conclusion:
This solution works fairly well in providing multi-database connectivity based on a site event (in this case user login) with the occasional bit of peering underneath’s Symfony’s hood. As mentioned above, keeping the base database schematically identical (i.e. same table structure, differing data) to your individual databases is very highly advisable. Some Symfony logic occurs before the onKernalRequest event gets fired (looking at your forms) so functionality limitations can otherwise occur there. Also parsers like the Nelmio Api Documentation bundle typically build pre-authentication (so whilst the base database is still selected).

Doctrine Migrations can also be enabled with a little tinkering (I may write a new post on how to do this if there is demand) but again, it helps if the schemas are identical (although isn’t vital).

Advertisements

Saving a root owned file in vim when not opened using sudo

A very common Linux administration annoyance can be opening a file in vim, making loads of changes, going to save and quit the file (using :wq) only to receive the message:
E45: 'readonly' option is set (add ! to override)
But as only root can write to the file, this won’t work either. Before pulling hair out, try this handy little command:

:wq !sudo tee %

Deleting folders beginning with a dash “-” on Linux

If you have a folder or file which starts with a dash (e.g. “-someFolder”) then simply running rmdir (or rm for a file) will produce the following error:

> rmdir -someFolder
rmdir: illegal option -- s
usage: rmdir [-p] directory ...

A neat way to delete such a folder is by referencing the directory first:

> rmdir ./-someFolder

The folder should now be deleted. Files are the exact same, just replace rmdir with rm.

Doctrine createQueryBuilder() – EntityManager vs. EntityRepository

EntityRepository and EntityManager have slightly different versions of the createQueryBuilder() function. Whereas the EntityManager’s version takes no arguments, EntityRepository’s version expects an ‘$alias‘ parameter. What is going on?!

The EntityRepository class wraps the EntityManager’s call, as such:

From /lib/Doctrine/ORM/EntityRepository.php

The function’s phpdoc reveals the reasoning behind the parameter count differences, EntityRepository returns a version of createQueryBuilder() customised for itself. We no longer need to specify the primary table we’re selecting from. Instead we must supply an $alias parameter which would usually be later supplied to the from() function.

Also note the above means all columns are selected from the entity by default.

Apt-get not finding packages

On a freshly installed Ubuntu system, apt-get may not have bothered to download its list of packages. This can lead to the confusing position of ‘E: Unable to locate package X‘ on almost everything. Fix this by entering:

sudo apt-get clean
sudo apt-get update

Packages should then be visible and installable via the usual command:

sudo apt-get install X

If you’re not sure on the package name, search for it using:

apt-cache search X

If you still cannot find the software you’re after you may of course need to add additional repositories.

Fun with counting in PHP

PHP’s loose typing can throw up some amusing bugs, take using scalars with the count() function. The function isn’t designed for scalars sure, but it doesn’t complain about them and often PHP makes use of global functions like count() rather than the more object orientated $i.count style, creating a nasty little minefield (one of many) for those unfamiliar with PHP’s peculiarities regarding proper objects vs. native types.

So the following looks valid enough:

$i = 5;
if (count($i) > 0) print "true";

But the code is bugged and whilst true will be printed as expected (maybe satisfying the programmer’s initial testing), guess what happens here:

$i = 0;
if (count($i) > 0) print "true";

We get an output of true !?#!??

What happens is the count() function evaluates one scalar value (the 0 integer) and so returns an integer of 1, which is greater than 0.

The following will print true:

$i = false;
if (count($i) > 0) print "true";

Whilst the following will not print anything:

$i = null;
if (count($i) > 0) print "true";

So remember, unless you want weirdness, only use the count() with arrays and objects implementing a Countable interface.

Quickly get memcached working in Python Django

As with most frameworks, the Django framework for Python can make use of caching to greatly improve performance for many common requests. Here we will look at using memcached as it enjoys good Django support and production use although there is also Redis support which definitely improves on memcached in some aspects such as data persistence.

  1. The first step is to install memcached on your server:
  2. RedHat Linux:

    yum install memcached

    Ubuntu / Debian Linux:

    apt-get install memcached
  3. Let Django know how to access memcached:
  4. In Django’s settings.py file, add the following line:

    'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache'
  5. Load the cache within your application
  6. from django.core.cache import cache
  7. Save the value to the cache
  8. cache.set('exampleValue',exampleValue)
  9. Retrieve the value from the cache
  10. exampleValue = cache.get('exampleValue')

The beauty being that exampleValue can be anything from a computed / database retrieved value to large blocks of static text or a URL etc.

The only problem with caches is they don’t always contain the data you expect, what if the value got flushed or hasn’t yet been stored? Lets rewrite step 5 to handle the event of the value not being available in the cache:

exampleValue = cache.get('exampleValue')
if not exampleValue:
     exampleValue = exampleValueLookup
     cache.set('exampleValue',exampleValue)

Here we see the value exampleValue being retrieved with a backup regeneration if the value has not been set. In a real application this would usually be encapsulated in a getExampleValue function or somewhere appropriate.

Symfony2 custom exceptions

When using the Symfony2 Framework it is often better to create custom exceptions to handle unique circumstances rather than incorrectly utilising one of the many pre-existing Symfony exception types. Using an interface improves code cohesion, allows the adding of custom functionality as needed and is super easy to setup.

The first step is to create a custom exception interface.

namespace AppBundle\Exception;

interface AppBundleExceptionInterface
{
}

Then create a custom exception class (perhaps in an Exception directory if it suits your project’s structure):

namespace AppBundle\Exception;

class NewTypeOfException extends \Exception implements AppBundleExceptionInterface
{
}

This exception can now be thrown and caught as desired, just be sure to name your exceptions well. You may be throwing them long after they were originally created!

The many different syntaxes of PHP’s if statement

The PHP language is blessed with many possible syntax structures for all sorts of language constructs. Some better than others but all have their place . Here we will be examining flexible usage of some of the styles available for use with if statements.

First, let’s look at the styles for the following statement in the standard c style:

$value = 1;
if ($value == 1) {
    print "if expression run";
} elseif ($value == 2) {
    print "elseif (1st) expression run";
} elseif ($value == 3) {
    print "elseif (2nd) expression run";
} else {
    print "else expression run";
}

(note: elseif has identical else if snytax in above and below examples)

In semi-colon syntax:

$value = 1;
if ($value == 1):
    print "if expression run";
elseif ($value == 2):
    print "elseif (1st) expression run";
elseif ($value == 3):
    print "elseif (2nd) expression run";
else:
    print "else expression run";
endif;

In switch statement format:

$value = 1;
switch (true) {
    case ($value == 1):
        print "if expression run";
        break;
    case ($value == 2):
        print "elseif (1st) expression run";
        break;
    case ($value == 3):
        print "elseif (2nd) expression run";
        break;
    default:
        print "else expression run";
};

The following are recursive language constructs.
In inline recursive format:

$value = 1;
($value == 1?
    (
        print "if expression run"
     ) :
    ($value == 2?
        (
            print "elseif (1st) expression run"
        ) :
        ($value == 3?
            (
                print "elseif (2nd) expression run"
            ):
            (
                print "else expression run"
            )
        )
    )
);

Using logical operators:

$value = 1;
$value == 1 and (print "if expression run")
or $value == 2 and (print "elseif expression run")
or $value == 3 and (print "elseif expression run")
or (print "else expression run");

 

Any other syntaxes missed?

Doctrine DateTime format() errors

A pesky and difficult to debug doctrine problem I’ve been encountering is the following occurring after a flush() call:

PHP Fatal error:  Call to a member function format() on a non-object in /var/www/myapp/vendor/doctrine/dbal/lib/Doctrine/DBAL/Types/DateTimeType.php on line 53

Obviously this gives no info on the actual cause of the error, which it turns out is because of a mismatch between doctrine expecting a DateTime object and instead encountering a string object, which does not have a format() function to call.

The solution is to remove the default string value that doctrine’s orm tools sometimes wrongly attaches to objects and instead use the correct DateTime() object, e.g.:

/**
 * MyObject
 *
 * @ORM\Table(name="my_object_table")
 * @ORM\Entity
 */
class MyObject
{
...
    protected $timestamp = '0000-00-00 00:00:00';
...
}

The first problem if you have a lot of model files is actually finding where the problem is. Whilst there are a host of debug tools which can help, a quick and easy hack is to simply add a stacktrace to the offending doctrine vendor file.

First go to your project’s main folder and vim the relevant file, if you are not a vim file, the substitute it for another one of the many text editors out there:

vim vendor/doctrine/dbal/lib/Doctrine/DBAL/Types/DateTimeType.php

Then go to line 53 (or whatever line number appears in the error message) by typing :53 (i.e. a semi-colon followed by the line number) where you should see a line reading:

        return ($value !== null)
            ? $value->format($platform->getDateTimeFormatString()) : null;

Go up one line above this and press the a key to enter insert mode and enter:

        if(is_string($value)) {
                debug_print_backtrace();
        }

A good tip is to save the file using :w, but don’t actually close the file. Running your application again you will then print out the usual – hideously ugly – stacktrace. This can be filtered down but even just loooking at the raw ouput should reveal the reflection classes inspecting the model file a couple of steps down which has the incorrectly set line exampled above. This is where you then remove the changes from the above file, by pressing u to undo the changes. Followed by :q to quit the file.

This string needs to be changed to a DateTime() object and you will need to set a default value in the constructor, e.g.:

public function __construct()
    {
        $this->timestamp = new \DateTime();
    }

Leaving out the parameter will be set it to the current system time. This will lead to doctrine correctly interpreting and saving timestamps to the database upon flush.

A more complete strategy to automating timestamp & date generation for data using Doctrine is present in my article:

Automating timestamps with Doctrine ORM