Pomm NextGen is in the pipe

Posted 2 years ago.

It has been quiet around this website lately. In the quietness of a cathedral a new version of Pomm was emerging.

What will be Pomm 2.0 has been written from scratch relying on almost four years of Pomm experience. As expected there are tons of improvements, as expected, migrating a project from Pomm 1.x will be hard. Pomm NextGen is still under developments and no beta has been released yet, so it is very likely API will change.

Let's get aboard the A-train

The most visible change is that the project has been split in 3 different packages:

  • Foundation
  • ModelManager
  • Cli

Yes, you have well read: there is a CLI with Pomm NextGen. It can generate model classes but it can also inspect the database to quickly display information you are looking for without having to switch to the psql console.

$ ./vendor/bin/pomm inspect:database my_database
Found 2 schemas in database.
+--------+-------+-----------+------------------------+
| name   | oid   | relations | comment                |
+--------+-------+-----------+------------------------+
| blog   | 24587 | 8         | Schema for blog module |
| public | 2200  | 13        | standard public schema |
+--------+-------+-----------+------------------------+

Pomm NextGen also puts database comments on the front line. Comments in the database schema are so useful, they are displayed with the inspector and they are set as PHPDoc annotations in the generated structure files.

Prelude to foundation

The service and session part have been entirely rethought to get use and framework integration more simple it was with Pomm 1.x:

<?php

$pomm = new Pomm(
        [
        'database_a' => [ 'dsn' => 'pgsql://user:pass@host:port/db_a', … ],
        'database_b' => [ 'dsn' => 'pgsql://user:pass@host:port/db_b', … ],
        …
        ]
        );

$session = $pomm['database_a']; // ← build, store and return a session

Session instances are more or less the equivalent of Pomm 1's connections. Pomm NextGen's sessions share a «physical» database connection to clients. Clients are all the classes that need to access a connection. Here are different ways to perform queries using the client mechanism:

<?php
//…
$pomm['database_a']
->getConnection()
->executeAnonymousQuery("select * from my_table where name = 'my_name'")
;

The call above returns a ResultHandler that needs to be traversed to get raw results, parameters are not escaped, it is not really the best way to use the database.

<?php
//…
$pomm['database_a']
->getPreparedQuery('select * from my_table where name = $*')
->execute(['my_name'])
;

This is a bit better since the parameters are properly escaped but it still returns a ResultHAndler instance. On the other side, the session pooler will keep this prepared query instance and if this query is issued twice, the prepared statement will be re-used automatically.

Let's call the query manager instead:

<?php
//…
$pomm['database_a']
->getQuery() // ← returns a query manager client
->query('select * from my_table where name = $*', ['my_name'])
;

The query method returns a ConvertedResultsIterator that triggers the converter mechanism so each line will be a field name indexed array with converted results as values. It is still possible to ask the query manager to use the prepared statement client:

<?php
//…
$pomm['database_a'] // ↓ summon prepared_query query manager
->getQuery('\PommProject\Foundation\PreparedQuery\PreparedQueryQuery')
->query('select * from my_table where name = $*', ['my_name'])
;

The Model service

The ModelManager package is composed of two complementary services: the model manager and the model layer. The model manager is the place where application developers write queries, the model layer is where model calls are grouped into transaction if needed.

<?php
//…
class CarModel extends Model
{
    use WriteQueries; // ← Since the relation is a table, import read & write queries.

    protected function __construct()
    {
        $this->structure = new CarStructure; // ← Defines the structure
        $this->flexible_entity_class = "\Model\Rental\VehicleSchema\Car";
        // ↑ associated flexible entity
    }
}

Queries are imported to the model classes through traits. Here are the different queries:

  • findAll()
  • findWhere(Where)
  • countWhere(Where)
  • findByPK([primary_key])
  • paginateFindWhere(…)
  • paginateQuery(…)

  • insertOne(FlexibleEntity)

  • updateOne(FlexibleEntity, [changes])
  • deleteOne(FlexibleEntity)
  • createAndSave([fields])
  • updateByPK([primary_key], [changes])
  • deleteByPK([primary_key], [changes])

Note that Pomm2 supports Postgresql multiple inheritance.

<?php
//…
class CarModel extends Model
{
    use WriteQueries; // ← Since the relation is a table, import read & write queries.
    use VehicleQueries; // ← set vehicle dependent queries in a trait

    protected function __construct()
    {
        $this->structure = (new CarStructure())
            ->inherits(new VehicleStructure)            // ←┬ inherit structures
            ->inherits(new EnginePropulsedStructure)    // ←┘
            ;
        $this->flexible_entity_class = "\Model\Rental\Vehicle\Car";
        // ↑ associated flexible entity
    }
}

Using model classes from a controller is pretty straightforward:

<?php
//…
function deleteCarController(Request $request)
{
    $car = $this->getService('pomm')['rental']
        ->getModel('\Model\Rental\VehicleSchema\CarModel')
        ->deleteByPK(['plate_number' => $request['plate_number']])
        ;

    if (!$car) {
        throw new NotFoud404Exception(
                sprintf("Unknown car, plate_number = '%s'.", $request['plate_number'])
                );
    }

    return json_encode($car->extract());
}

Projection and custom queries

The philosophy behind Pomm NextGen is the same as it was with Pomm 1.x, models propose a default projection used by every queries and even though there are already queries using their sole relation, Pomm will not generate any queries joining related tables. Pomm proposes to write native SQL queries using placeholders for all the changing or tedious parts.

<?php
//…
public function createProjection()
{
    return parent::createProjection()
        ->unsetField('unused_field')
        ->setField('new_field', '2 * %field_a + %field_b', 'int4')
        ;
}

The createProjection method is the direct equivalent of Pomm1's getSelectFields but things have improved:

  • Projection is an instance
  • it is possible to compose fields by using the % sign that is expanded with table alias if needed.

All the queries coming with the traits now return instances hydrated like described above. Let's see how to create a custom query using this projection:

<?php
//…
public function findBySlugWithNeighbours($slug)
{
    // 1 - define query
    $sql = <<<SQL
        with
        neighbour as (
                select
                slug,
                lag(slug)  over published_at_wdw as next_slug,
                lead(slug) over published_at_wdw as prev_slug
                from
                :news_table
                window
                published_at_wdw as (order by published_at desc)
                )
        select
        :news_fields
        from
        :news_table
        left join neighbour n using (slug)
        where
        news.slug = $*
        SQL;

    // 2 - define projection
    $projection = $this
        ->createProjection()
        ->setField('next_slug', 'n.next_slug', 'varchar')
        ->setField('prev_slug', 'n.prev_slug', 'varchar')
        ;

    // 3 - expand placeholders
    $sql = strtr(
            $sql,
            [
            ':news_fields' => $projection,
            ':news_table'  => $this->structure->getRelation(),
            ]
            );

    // 4 - issue the query
    return $this->query($sql, [$slug], $projection)->current();
}

The query above is the actual query used to display this blog article with links to the previous and next articles. Few comments about it, the :news_fields placeholder is expanded by calling Projection's method __toString that returns formatFieldWithFieldAlias() with no table alias. The normal return for the query method is a CollectionIterator instance but since slug is the primary key, the expected result of this method is a single instance of FlexibleEntity or null, that is why the method current() is called.

Model layer

Using model methods directly from controllers works but applications with complex business rules often need an extra code layer: the model layer (sometimes called Service Layer in the SOA world or Application Layer in the DDD world). This layer is responsible of:

  • Group model method calls in transactions.
  • Drive model method calls using condition composition.
  • Turn technical exceptions into meaningful exceptions.

Here is an example of transaction using the ModelLayer class:

<?php
//…
class BlogModelLayer extends ModelLayer
{
    public function archiveBlogPost($slug)
    {
        try {
            $this->startTransaction();
            $post = $this
                ->getModel('\Model\PommProject\BlogSchema\PostModel')
                ->deleteByPK(['slug' => $slug])
                ;

            if (!$post) {
                $this->rollbackTransaction();

                return null;
            }

            $archived_post = $this
                ->getModel('\Model\PommProject\BlogSchema\ArchivedPostModel')
                ->archivePost($post)
                ;
            $this->sendNotify(
                    'blog_post:archive',
                    json_encode($archived_post->extract())
                    );
            $this->commitTransaction();
        } catch (PommException $e) {
            $this->rollbackTransaction();
            throw new \RuntimeException('error', null, $e);
        }

        return $archived_post;
    }
}

Note the call to sendNotify method that takes advantage from Postgresql queue messaging system to launch asynchronous jobs behind the scene. There are other features like savepoints in transaction or constraint deferring. Using this class from a controller is as easy as using a model class:

<?php
                                                                                                                                                                                                                                                                                                                                       //…
function archiveBlogPostController(Request $request)
{
    $archived_post = $this->getService('pomm')['pomm_project']
        ->getModelLayer('\Model\PommProject\BlogSchema\BlogModelLayer')
        ->archiveBlogPost($request['slug'])
        ;

    if (!$archived_post) {
        throw new NotFoud404Exception(
                sprintf(
                    "No such blog post '%s'.",
                    $request['slug']
                    )
                );
    }
    …
}

Back to Foundation

Even through Foundation looks like a simple piece of software, it does more that just holding a Connection. A Session is a complex client holder. Clients are all classes that need to access to the Connection. They are managed by classes named poolers. By example, the following code:

<?php
…
$pomm['my_connection']
->getPreparedQuery('select * from my_table where my_field = $*')
->execute(['myself'])
;

is in fact the equivalent of:

<?php
…
$pomm['my_connection']
->getClientUsingPooler('prepared_statement', 'select …')
->execute(['myself'])
;

So:

  1. The pooler checks if the client does exist.
  2. As it does not, it creates the new client with the sql query, the query is then prepared.
  3. The client executes the query with the given parameters and return a ResultHandler.

Let's have a look at the logs generated to display this page:

 myapp.INFO: Matched route "blog_show" (parameters: "_controller": "[{},"show"]", "slug": "pomm-1-2-is-available", "_route": "blog_show") [] []
myapp.INFO: > GET /news/pomm-1-2-is-available.html [] []
myapp.DEBUG: Pomm: Registering new client {"type":"model","identifier":"Model\\PommProject\\PommSchema\\NewsModel"} []
myapp.DEBUG: Pomm: Registering new client {"type":"query","identifier":"PommProject\\ModelManager\\Model\\CollectionQuery"} []
myapp.INFO: Pomm: ListenerPooler: notification received. {"receivers":"query:pre"} []
myapp.INFO: SQL query …
myapp.DEBUG: Pomm: Registering new client {"type":"prepared_query","identifier":"14f806b7048e349a6e185492ed9b0a7d"} []
myapp.INFO: Pomm: ListenerPooler: notification received. {"receivers":"query:post"} []
myapp.INFO: SQL query {"result_count":1,"time_ms":"17.9","flexible_entity":"\\Model\\PommProject\\PommSchema\\News"} []
myapp.DEBUG: Pomm: Registering new client {"type":"converter","identifier":"varchar"} []
myapp.DEBUG: Pomm: Registering new client {"type":"converter","identifier":"timestamp"} []
myapp.DEBUG: Pomm: Registering new client {"type":"converter","identifier":"text"} []
myapp.DEBUG: Pomm: Registering new client {"type":"converter","identifier":"bool"} []
myapp.DEBUG: Pomm: Registering new client {"type":"converter","identifier":"\\Model\\PommProject\\PommSchema\\News"} []
myapp.INFO: < 200 [] []

The model client type is registered to the session and the model's method is then executed. As it calls the query method, the query pooler registers the CollectionQuery query manager that returns hydrated flexible entity instances. When preforming the given query, it notifies the listener pooler that forwards the notification to the query listener client which logs the query (not displayed here, too large). The CollectionQuery uses the prepared_query pooler to actually perform the query, the client is then created and registered. A new notification is sent, logging the query duration and results. When data are fetched from the iterator, they are converted on the fly using the converter pooler.

Foundation is meant to be used to develop model layers. It can integrates legacy code -- where Pomm model manager may not be suitable -- to offer a performing yet easily extendible connection sharing layer.

What's up now ?

Pomm NextGen is the result of almost two months full time work and even though it is now functional, it is still under development with a stable release planned in early 2015. Pomm NextGen is easy to use and is aiming at being simple to extend so it can be adapted to a lot of different cases. The model manager kept the same philosophy Pomm 1 had but with enhancements.

Now Pomm needs you. Use it, test it, send us feedback so we will be able to create one of the best tool for using Postgresql in web developments out there.