beberlei.de https://beberlei.de/ en-us Thu, 11 Mar 2021 00:00:00 +0100 https://beberlei.de/2021/03/11/running_shopware_platform_tests_on_a_linux_dev_machine.html https://beberlei.de/2021/03/11/running_shopware_platform_tests_on_a_linux_dev_machine.html <![CDATA[Running Shopware Platform tests on a Linux Dev-Machine]]> Running Shopware Platform tests on a Linux Dev-Machine

Participating in the Shopware Boost Day #4 today I tried the most simple approach to get it running on a Linux developer machine, avoiding the Docker setup. With some help from Shopware team I got it down to:

  1. Checkout the shopware/development repository
  2. Checkout the shopware/platform repository inside development repository
  3. Create a database “shopware” in MySQL (this might not be needed?)
  4. Run the bin/setup command

These steps in the terminal are:

git git@github.com:shopware/development.git
cd development/
composer install
git clone git@github.com:shopware/platform.git platform
php bin/setup
php vendor/bin/phpunit -c platform

Note: These are the steps in March 2021 (during Shopware 6.4).

]]>
Thu, 11 Mar 2021 00:00:00 +0100
https://beberlei.de/2021/02/26/free_software_not_free_support_my_reply_template.html https://beberlei.de/2021/02/26/free_software_not_free_support_my_reply_template.html <![CDATA[Free Software, Not Free Support: My Reply Template]]> Free Software, Not Free Support: My Reply Template

Over the last weeks I have seen a few people tweet or write about the burden of open-source maintainership or being a public person in a programming community.

The topic of open source maintenance responsibilites has become bigger over the last years and having previously burned out on OSS work myself, I find it important that the stress of demands on people contributing their free time is something that needs to be talked about.

As an open source maintainer myself I get my fair share of e-mails and Twitter direct messages asking for support with specific problems.

For many years I usually reply along the same lines that I provide only free software and not free support. Maybe this is helpful to others and an encouragement to stand your ground:

Thank you for your message, unfortunately my limited time doesn’t allow me to help with support requests for my open-source projects, especially when they are done in private and my answer doesn’t benefit others as well.

Please ask your question in a public forum such as Stackoverflow, Doctrine or Symfony Slack or Mailinglists where community members can help you out and both question and answer will help others.

This is intentionally formulated to avoid talking the questioner down for asking an open-source maintainer for help in private, because they might not be aware of my time constraints or about open-source and support. That is ok, no one can be all knowing.

Instead my main argument is to move the support to a public forum, so that the community of free users of my software can help each other and provide something “free” in return.

Additionally this “nice and defensive” phrasing is robust for almost any kind of message and I rarely need to adjust it after copying it into an e-mail or a Twitter DM.

]]>
Fri, 26 Feb 2021 00:00:00 +0100
https://beberlei.de/2021/02/19/when_to_use_empty_in_php_i_say_never.html https://beberlei.de/2021/02/19/when_to_use_empty_in_php_i_say_never.html <![CDATA[When to use empty in PHP? I’d say never]]> When to use empty in PHP? I’d say never

I realized that I am very strict about the use of PHPs empty function in code review. There isn’t really any reason to use it in my opinion:

  • it tests both for the existance and the non-falsy value and as a reader the writers intent is not clear.
  • it does not communicate what type a variable has, seeing empty($value) does not narrow down what value is enough.
  • it hides errors caused by typos in the variable name passed to empty.

This is a list of alternatives that I recommend to use instead of empty.

testing that the string has length 0

// Replace
if (empty($string)) {
}

// With
if (strlen($string) === 0) {
}

Using strlen communicates that the variable is a string.

testing that array has no elements

// Replace
if (empty($array)) {
}

// With
if (count($array) === 0) {
}

Using count communicates that the variable is an array.

testing if array has key

// Replace
if (empty($array[$key])) {
}

// With
if (array_key_exists($array, $key)) {
}

// or if you know that existance also means a non-empty value
if (isset($array[$key])) {
}

Both array_key_exists and isset more clearly communicate that the code cares about the existance of a key and not that it assumes existance and tests only for a non-empty value.

testing if a variable was declared in the scope

// Replace
if (empty($var)) {
}

// With
if (isset($var)) {
}

In this case you can also use isset, but in general if you need to check this there is something wrong with the code anyways. A variable should always be or not be declared in the current scope, there should not be uncertainty about it.

testing if a variable is null

// Replace
if (empty($var)) {
}

// With
if ($var === null) {
}

Code is more cleary a test for the variable being null and not accidently also true when 0, false or empty string.

testing if a variable is 0

// Replace
if (empty($var)) {
}

// With
if ($var === 0) {
}

Communicates that variable is integer.

testing if a variable is false

// Replace
if (empty($var)) {
}

// With
if ($var === false) {
}
if (!$var) {
}

Communicates that variable is boolean.

What if?

Now what if I want to test for existance and non emptiness for an array keys? Using two conditions makes this more clear.

Now what if I care about performance of empty being a language construct vs calling an internal function such as strlen or count? No worries, both strlen and count are optimized by Opcache if you prefix them with a backslash to directly call them in the global namespace.

Automatic Enforcement

Armed with PHP Code Sniffer and Psalm I am happy to enforce rules with my own custom plugins for either one tool.

We have a custom Psalm plugin in our Tideways code base and I have added an automatic tests for empty including a suggest for the right replacement based on the type. Its not perfect and does not yet catch all different kinds of empty usages, but its a good enough start. See this Gist for the code and how to include it in your psalm.xml.

]]>
Fri, 19 Feb 2021 00:00:00 +0100
https://beberlei.de/2020/07/05/what_to_look_out_for_when_testing_php_jit.html https://beberlei.de/2020/07/05/what_to_look_out_for_when_testing_php_jit.html <![CDATA[What to look out for testing PHP8 JIT]]> What to look out for testing PHP8 JIT

PHP 8 is coming with a Just In Time Compiler (JIT) and people are starting to test it in more detail. Here are a few things you should be careful about when testing the JIT:

  • Make sure sure test with different trigger modes. Trigger is the 3rd number in the line: 1205 (JIT everything), 1235 (JIT hot code based on relative usage), 1255 (trace hot code for JITability).

    Best results seem to be with “opcache.jit=1255” at the moment.

  • Do not run your examples in the CLI. Use “php-cgi” with the -T flag for reptitions. This allows Opcache and JIT to optimize on the first request, and then you can see in the following repetitions how the optimized performance is. Just for experimenting it makes sense to have “php-cgi -T 2” for running one warmup and one hot request against the same script.

    Clarification: JIT works with the CLI, but I find its harder to make the exact effects visible and comparable and requires more configuration.

  • Make sure the JIT is actually running by looking at opcache_get_status(false). It has a key “jit” and it should have a key “on” set to true, and the “buffer_free” should be lower than “buffer_size”, meaning the JIT actually uses some of the allocated memory for something.

  • To test if code gets faster with JIT, don’t run it under stress/concurrency. JIT optimizes CPU time, so the improvement must already be visible when running one jitted request against one unjitted one. Repeat those runs multiple times to find out if the result is “stable” or fluctuates a lot.

  • If you want to test if JIT increases throughput, that means more requests can be handled at the same time than without JIT, then you should ramp-up the number of concurrent requests and not start with a high number. Start with 10, and go up to 100, 1000, … - At some point “regular” PHP will fall over and you can then find how much more JIT takes before it also falls over.

Testing JIT performance for Doctrine ORM I setup this Github repository with more detailed examples, especially the php-cgi call, which I want to repeat here:

/opt/php/php-src/bin/php-cgi -T 10 -dopcache.enable=1 -dopcache.jit=1255 -dopcache.jit_buffer_size=64M index.php

The -T 10 here means 10 reptitions of the index.php script. This triggers OPCache and JIT optimizations on the first request and allows testing performance of already optimized code in subsequent requests.

The ini settings explained:

  • opcache.enable=1 enables OPCache.
  • opcache.jit=1255 sets various flags, including the triggering mode, of the JIT. Defaults to 1205, which is probably inefficient JITing everything.
  • opcache.jit_buffer_size=64M must be set, because otherwise JIT is not enabled. By default this is zero. At the moment setting the buffer size is the switch to enable/disable JIT. It is planned to change INI settings slightly before the PHP 8.0 GA to simplify this step.

Looking forward to read more about the JIT experience from all of you!

Updates:

  • July 5th, clarified JIT works with CLI, added detailed explanation about JIT INI settings
]]>
Sun, 05 Jul 2020 00:00:00 +0200
https://beberlei.de/2020/02/25/clean_code_object_calisthenics_rules_i_try_to_follow.html https://beberlei.de/2020/02/25/clean_code_object_calisthenics_rules_i_try_to_follow.html <![CDATA[Clean Code and Object Calisthenics Rules I try to Follow]]> Clean Code and Object Calisthenics Rules I try to Follow

I do a lot of Code Reviews, and without proper automation of most of the low level items that you are usually “remarking” to colleagues it is a frustrating experience for everyone involved and takes more time that needed.

My natural tendency is towards action over planning, so I tend to find arguments over coding style are a waste of time if sufficient standards exist that you can just adopt. In my opinion this is is something team leads and managers need to rule on to avoid that their teams become unproductive in little coding-standard and style discussions.

Choosing a set of strict rules is very important and enforcing them helps reducing time spent on unimportant details during code review and allows more junior developers to prepare their changes in a way that most feedback can be automated.

Coding Styles alone are not enough in most cases. There should also be rules with respect to what type of language constructs and building blocks are preferred over others.

Equipped with a fixed ruleset, you can automate the low-level parts of code review in such a way that every developer can perform it themselves and fix violations before you even get to see the code in the first place.

In this post I give an (probably incomplete) overview of rules and patterns we follow for Tideways code base.

A strict coding style

The first thing I put in place when Tideways hired its first engineer was PHP Code Sniffer with a slightly adopted Doctrine Standard (PSR-2 + a lot more), coupled with static analysis using Psalm. Combined with Github Checks Run API and Scotts diff filter tool all the violations are shown directly within Github commits and Pull Requests.

../../../_images/github-check1.png

Using the phpcbf command and a lot of manual work we are down to zero violations. If you start using a standard from the beginning this won’t happen, but strict enforcement of coding standard is not something I was used to before.

Violation check runs on Github include a message with a command to copy paste and auto-fix violations if available for PHP Code Sniffer, ESLint and TSLint.

We also have PHPStorm configurations to integrate each tool into the IDE.

In my opinion PSR-2 is a good first step, but it leaves a lot of details open that the Doctrine Coding Standard locks further down. A lot those rules are very strict and we had to disable them on our code base. If you are interested our phpcs rule definitions are in this Gist.

Besides “simple” code style checks, we have some clean code guidelines that we don’t check with phpcs, because they are not black and white but require some deliberation.

For Javascript we enforce the strict defaults of ESLint/TSLint and for Go the default coding standard that go fmt produces.

A static code analyzer, but not at highest level

We use Psalm for static code analysis. It automatically tries to find out all types of all variables in your application and then checks hundrets of different rules that might indicate bugs or improve the design of your application.

While type safety is awesome, without generics forcing Psalm or PHP-Stan at the highest level leads to a lot of clutter and boilerplate code that is only necessary to make the tool happy.

This is why I recommend to only fail the build on the obvious rules and leave the other ones as Warnings for the developer visible from the IDE.

We use a lenient psalm.xml and the Baseline feature to get down to zero errors. Every error fails the build.

Take very good care of API Information Hiding and Encapsulation

A lot of accidental complexity in code comes from exposing implementation details to the outer layers of your application. This is mostly due to function arguments or return values that expose the implementation details.

This concept is called Information Hiding.

When adding arguments or return values think about the client calling the function:

  • Do they only get the information required to work, or does it return additional information (extra array keys, objects) that are unncessary?
  • Does a function argument expose the internal implementation of a datastructure? Can you turn it into a simpler input that is converted to the desired variable internally?

This is extremely difficult skill to master and takes a lot of practice.

To New or Not to New?

One of the most influential blog posts on me has been To new or not to new by Misko Hevery.

It clearly defines which types of objects can be constructed with new in place (data objects, entities, value objects) and which ones should also be injected and centrally constructed by a factory object or a dependency injection container.

Separate Statements and Declartions by empty lines

Blocks of statements and blocks of declarations usually get separated by an empty line, to make a very visual distinction. Also statements with a different context tend to be split by empty lines as well. See this example from one our controllers, where each statement and declaration is in its own paragraph.

<?php

 public function deleteAction(PageContext $context, $serverName)
 {
     $server = $this->findServer($serverName, $context);

     $this->serverRepository->remove($server);

     $this->jobQueue->queue(new BalanceServerTask(['applicationId' => $context->application->getId()]));

     return $context->redirectRoute('XhprofProfilerBundle.Server.environment');
 }

I would define paragraphs are statements or declarations that belong to each other. This concept closely follows paragraphs in natural languages and is easy to grasp and follow.

Use early exits and don’t use else

Code almost never needs the else keyword. It can be written in a way to use early exits with return or continue instead. This following code could have been written with an else. This rule keeps the main/happy path of the code on the main level of the method.

<?php

 public function formatQueryString(string $query): string
 {
     if (strlen($query) === 0) {
         return '';
     }

     parse_str($query, $queryArgs);

     $queryArgs = $this->truncateArrayValues($queryArgs);

     return json_encode($queryArgs, JSON_PRETTY_PRINT);
 }

For loops we continue to the next iteration for early exits:

<?php

 private function parseErrorTrace(string $trace, $removeArguments = false)
 {
     $traceResult = [];

     $parts = explode("\n", $trace);
     foreach ($parts as $line) {
         if (strpos($line, '{main}') !== false) {
             continue;
         }

         if (strpos($line, '#') !== 0) {
             continue;
         }

         // more
     }
 }

Only one level of indentation per method and a second for early exits

Another rule I try to follow is “only one level of indentation per method” from the Object Calisthenics rulebook.

Guilherme from the Doctrine team introduced me to Object Calisthenics many years ago and I found it a very good guidestick to improve my code without having to think too much.

The deleteAction in the controller example above inhibits this rule, having no additional level of indentation. This second example shows a controller with one level of indentation, using the “Guard clause” pattern instead of nested conditions (See the refactoring)

<?php

 public function process(Task $task)
 {
     $application = $this->organizationRepository->findApplication($task->applicationId);

     if (!$application) {
         return;
     }

     $retentionDays = $application->getRetentionDays();
     $daysDiff = (new \DateTime('now'))->diff($task->date)->days;

     if ($daysDiff > $retentionDays) {
         return;
     }

     $this->historicalDataAction->compute($task->applicationId, $task->date, $task->overwrite);
 }

When methods use loops two levels of indention are ok, if the second level is used only for guard clauses.

<?php
 $lastProductionCommunication = null;

 foreach ($servers as $server) {
     if ($server->getEnvironment() !== 'production') {
         continue;
     }

     if (!$server->getLastCommunication()) {
         continue;
     }

     if ($lastProductionCommunication !== null && $lastProductionCommunication > $date)
         continue;
     }

     $lastProductionCommunication = $date;
 }

It is not easy to follow this rule to the dot, and we do have a lot of code in Tideways that doesn’t follow this rule as we don’t enforce it. But I view the violation of the one level rule as an automtaic, immediate signal of both complexity and potential for refactoring.

Don’t abbreviate names (too much)

This rule is easier in PHP than in Go (by design). As we use both languages, we force ourselves to write out what a variable, class, constant or function is doing, and avoid abbreviations.

func (previous PreviousSpans) GetLastIndex(category, summary string) int {
    if entry, ok := previous[category]; ok {
        if entry.Summary == summary {
            return entry.Index
        }
    }
    return -1
}

Certain names still get abbreviated though, for example i, idx, j and other common ones.

Prefer Null Coalesce Operator over if isset

We still work with a lot of arrays, because PHP makes it much easier to use them than complex types (objects). The risk of using arrays is not knowing if a key exists, and code being littered with if (isset($data['key'])) { statements all over the place. It is much better to use the null coalesce operator ?? to define default values for keys that you don’t know exist on an array:

<?php
 $hostName = $serverNames[$currentOccurrance['sId']] ?? 'unknown';

Don’t allow a variable to have two types

If something doesn’t exist it is NULL or false right? Wrong, if you care about simplicity of your code.

  • If a method is supposed to return an object but can’t, it may be better to throw an exception.
  • If an array is empty, then it is empty, not null or false. Iterating over an empty array can correctly do nothing.

I prefer methods written in a way, where they either only handle the default (happy path) by throwing exceptions otherwise, or by defining all variables in a way where even the “unhappy/error” paths can run successfully through the happy code path and yield the exactly right result.

Don’t re-use a variable with a different type

PHP allows this, but it can lead to very confusing code if you re-use the same variable with different types in a method or function. Avoid at all costs.

<?php
 public function deleteAction(PageContext $context, string $serverName)
 {
     $server = $this->findServer($serverName, $context);

In the previous block the argument $serverName could easily have also be called $server and therefore be both a string and an object of type Server within different parts of the method.

Typehint Collection Items with assert

PHP has no way to typehint the items of collections (for example arrays, iterators). You either typehint array, \Doctrine\Common\Collections\Collection or any other class in your methods, with no way to specify type of the key or value once you start iterating with foreach.

There are various docblock supported styles, to define these types, which I recommend using on class properties and return values such as:

<?php
 class XhprofReport extends Struct
 {
     /** @var array<string> \*/
     public $namespaces = [];
 }

But to help both PHPStorm and Psalm (our static code analyser) to understand items of a collection, we use assert() with instanceof, if we can’t provide the types ourselves on return values where we cannot define the typehint, for example when using Doctrine repositories:

<?php
 $organizations = $organizationRepository->findAll();

 foreach ($organizations as $organization) {
     assert($organization instanceof Organization);

     // ..
 }

Use structs to wrap arrays (especially from SQL results)

To avoids typical complex array problems such as not knowing what keys exist or what types values have, arrays should always be wrapped in objects extending from a Struct base class.

The struct base class maps an array to properties of the class in the constructor:

<?php
 abstract class Struct
 {
     public function __construct(array $data = [])
     {
         foreach ($data as $property => $value) {
             if (!property_exists($this, $property)) {
                 throw new \RuntimeException();
             }

             $this->$property = $value;
         }
     }
 }

Often data in Tideways is best represented in domain objects that doesn’t fit the entity or database model 100%. In those cases we manually map the database to the desired objects using this Struct pattern for simplicity.

Take this ServerConfiguration struct which is a combination of data from three Doctrine entities and is assembled in a repository from custom SQL.

<?php
class ServerConfiguration extends Struct
{
    /** @var int */
    public $id;
    /** @var string */
    public $environment;
    /** @var int */
    public $tracesPerMinute;
    /** @var bool */
    public $disabled;
    /** @var bool */
    public $newlyCreated = false;
    /*\*/
}

I am really looking forward when we deploy PHP 7.4 with typed properties where this pattern will become much more powerful and will allow us to write a “mini orm” for mapping SQL results to typed objects.

You can read more about Struct objects on the Qafoo blog and look into Kore’s DataObject package that implements this pattern in a reusable Composer package.

]]>
Tue, 25 Feb 2020 00:00:00 +0100
https://beberlei.de/2020/02/09/firegento_hackathon_mainz_2020.html https://beberlei.de/2020/02/09/firegento_hackathon_mainz_2020.html <![CDATA[Firegento Hackathon Mainz 2020]]> Firegento Hackathon Mainz 2020

This weekend I attended the Firegento Hackathon at Netz98 in Mainz. Due to the storm I had to leave early and skip presentation of my project, so I want to post a short summary of my work here on the blog.

I want to thank everyone from the Firegento organizer team, the hosts and sponsors. The event was really great and I was happy to get to know new people from the community.

Magento 2 Performance and Profiling

I proposed a project to work on Magento 2 Performance and Profiling and we started working on it on and off with a group of 6-7 people, which is quite an awesome crowd!

The biggest achievement was that we could introduce PHP profiling to a few people that have never used Profilers before. After a general overview on how different profiling approaches work we looked at Tideways and Xdebug as examples. It was really awesome to see the reaction of what is actually possible with PHP and that its possible to gather this data.

We looked at some data from production shops with performance problems on the category page with a lot of N+1 Product loads and Andreas explained how they refactored this bottleneck away.

With input from several people we poked into Magento 2 (demo shop) and found two things that seem worth investigating more:

  1. The Redis cache implementation could use a more efficient implementation for the app/config cache, where items are quite static over time. We saw 80-100ms of Redis calls for these caches across different shops (not only demo) that could be cut down by a more specialized cache implementation.

    I attempted a prototype working on this but relaized it needs probably 2-3 days of concentrated work to get a prototype working.

    One thing that Symfony + Doctrine benefit from is caches that write and execute PHP files so that Opcache can make use of interned strings, packed and immutable arrays internally.

  2. There is a ProductViewCounter UI block that renders JSON into the output of each page. This widget is used for the feature to show the last viewed/compared products and also is responsible for the view counter in the backend. This features are often not used, but the output of this UI block is still rendered into every page anyways.

    The serialization takes about 60-80ms, so there is potential here for a performance gain if there was a setting to disable this feature completly.

While working with Magneto 2 shops in Tideways, we realized there are a lot of small optimization that we can do to the callgraph profiler. There is room to add more specialized instrumentation and improve Magento 2 results. With the way Magento 2 uses interceptors (AOP) the profiling call stacks are sometimes really hard to understand, so I have written down around 10 improvement ideas we can add to the Profiler. Expect the results to go into Tideways over the next weeks and months.

This affects things like how Closures are rendered, or skipping them in the callgraph. And for a lot of calls in Magento, the actual function name should be augmented with more information, or changed to a different name to make it easier to grasp without knowing Magneto2 interceptor magic details.

]]>
Sun, 09 Feb 2020 00:00:00 +0100
https://beberlei.de/2019/08/20/simple_responsive_feature_per_plan_table_using_css_grids.html https://beberlei.de/2019/08/20/simple_responsive_feature_per_plan_table_using_css_grids.html <![CDATA[A simple, responsive feature per plan table using CSS Grids]]> A simple, responsive feature per plan table using CSS Grids

For a long time I was extremely unhappy with the unresponsiveness of the features list in Tideways. The landingpage is using Bootstrap 3 since the beginning and its table-responsive class is supposed to help here, but requires visitors to scroll and loose context. See a screenshot here of the mobile version of our features table.

https://beberlei.de/_static/cssgrid1.png

For a large table I can’t see which feature and plan a cell belongs to while scrolling around. At the size of Tideways feature table is relatively useless in my opinion.

I came across the Webflow Pricing Page a few days ago and got inspired to redesign the page to use CSS Grids and Sticky Positioning. It took me a while to get around the CSS to understand the concepts and then started from scratch to try to come up with a bare bones solution.

In this blog post I try to explain the solution in my own words. I am by no means a CSS expert, so take all the explanations with a grain of salt. I am linking as many Mozilla Developer docs as possible for reference.

First, I want to use an ordered list for semantic reasons. I then need to group the feature name and its availability in different plans by splitting each list-item into several cells.

<section class="features">
   <ol>
      <li class="header">
         <div>Features</div>
         <div>Free</div>
         <div>Pro</div>
      </li>
      <li>
         <div>Simple A</div>
         <div>Yes</div>
         <div>Yes</div>
      </li>
      <li>
         <div>Fancy B</div>
         <div>No</div>
         <div>Yes</div>
      </li>
   </ol>
</section>

First we look at the style of the list item:

section.features ol li {
   /* hide the ordered list item numbers */
   list-style-type: none;

   /* set element to grid mode rendering */
   display: grid;

   /* grid has 3 columns with a pre-defined width */
   grid-template-columns: 50% 25% 25%;
}

The magic here is the grid-template-columns directive that can be thought of similar to defining the number and width of table columns.

Next we modify the .header class such that it always scrolls to the top of the screen as long as the whole features table is visible on the screen using the sticky position.

section.features ol li.header {
   position: sticky;

   /* this must be modified if you have a static top navigation for example */
   top: 0px;

   /* hide feature cells when header is "over" them */
   background-color: #fff;

   /* some styling to make the header stand out a little from the features */
   border-bottom: 1px solid #666;
   font-weight: bold;
}

Lastly we align all text to center in all divs:

section.features ol li div {
    text-align: center;
}

This makes the feature table work nicely on desktop browsers. The white background is necessary to that the features that scroll under the header will not be visible anymore.

Now to the responsive part, we use the CSS grid to change the three column row into two rows, with the feature label spanning the size of both cells that indiciate feature availability per plan.

@media(max-width: 672px) {
    section.features ol li {
        /* redefine the grid to have only two columns */
        grid-template-columns: 50% 50%;
        /* define two template "rows per grid" (this is my murky understanding) */
        grid-template-rows: auto auto;
    }

    section.features ol li div:nth-child(1) {
            /* define first div (cell) to be 3 columns wide and span a whole row */
            grid-column-start: 1;
            grid-column-end: 3;
            grid-row-start: 1;
            grid-row-end: 2;

            border-bottom: 1px solid #000;
        }
    }
}

The magic is in the grid-column-start (Mozilla Docs) and grid-column-end directives that sort of act like colspan in tables. In addition the possibility to change a grid from one to two rows just with CSS does the rest of the trick here.

You can see a full code example of this blog posts feature table and the re-designed Tideways feature table in action.

https://beberlei.de/_static/cssgrid3.png

Let me know if there are mistakes in my CSS or ways to simplify even further by contacting me under kontakt@beberlei.de.

]]>
Tue, 20 Aug 2019 00:00:00 +0200
https://beberlei.de/2019/08/14/pplusplus_is_a_bad_idea.html https://beberlei.de/2019/08/14/pplusplus_is_a_bad_idea.html <![CDATA[P++ is a bad idea for non-technical reasons]]> P++ is a bad idea for non-technical reasons

Last week the idea of changing PHP to include two languages “PHP” (Classic) and “P++” was proposed on the internals mailing list by Zeev and in more detail by a FAQ answering questions. For context, Zeev is one of the original authors of the Zend Engine in PHP 3 and 4 and co-founder of the Zend company, so the proposal got a lot of responses and additional discussions on Reddit and HackerNews.

The goal of the propsal is find a way to evolve the language PHP using a new dialect (P++) and stay backwards compatible by continuing to support the old dialect (PHP).

Zeev proposes a new tag would <?p++ that sets the PHP compiler into a different mode than <?php now does, providing a “clean start” with BC breaks and more strict typing syntax.

tl;dr: The proposal for P++ is at the core a proposal to have the PHP runtime support multiple versions of the PHP language at the same time. Other languages already have this distinction for ages (C with 89, 99, 11, …). By going with a language version number instead of a new language name, we can avoid a lot of non-technical issues with P++.

I will start with a few non-technical arguments why I think this is a bad idea to introduce a distinct language called “P++” or any other name, with its own name and brand:

  • From a “governing” perspective, introducing P++ is like a big bang that would force the community on a path without knowing all the implementation details up front. This goes against the current governing model of PHP where each larger technical decision is made democratically using the RFC process. At this point we have to respect and accept the fact that without a benelovant dictator, you cannot make these big bang changes in an open source project anymore. Improvements have to be made in incremental steps and P++ would not fit into this model.
  • From an evolutionary perspective, the premise that the PHP community and internal teams can design a new language from the ivory tower, and get the details right the first time is pretence of knowledge fallacy. It is much more likely mistakes are made and then in 5 years we are back with the same problem. The P++ proposal sounds like a perpetual experimental version. It would be better to find a long term strategy to cope with incremental change to the language instead of a big bang change every 10 years.
  • From a marketing perspective, introducing a new brand “P++” is going to be extremely hard to bring to the market. With “the PHP company” Zend swallowed by larger companies, there is no company with the primary goal of bringing forward the language anymore. PHP is truely a community effort now, without even a foundation. There is no centralized body that can effectively lead the marketing effort for this new P++ brand. We are not in 1979 anymore when C++ was invented, the language market is highly fought for and we as the PHP community are protected by PHPs enormous market share that we should not give up by fragmenting.
  • I recognize “P++” is just a working name right now, a name without special characters is certainly a better idea. But a name different from PHP introduces even more problems w.r.t to SEO/Google and the way the PHP project is organized right now there isn’t even a good process defined that would lead to a great outcome.
  • From a documentation perspective, one of PHPs unique selling points is its awesome docs living on “php.net”. As both dialects PHP and P++ would run on the same engine, it becomes much harder to represent this on the website. Here the argument that the P++ project is feasible even with few internal developers falls apart. It would require a completly overhauled new website, an approach to represent both dialects sufficiently without confusing users, new mailing lists, new everything.
  • From a documentation perspective, assuming P++ were to break BC on Core APIs compared to PHP. Would php.net/strpos show the PHP and the P++ function body with haystack and needle switched? Or Would we need to copy the entire documentation? This would be a huge documentation team effort whose time hasn’t been accounted for by the P++ FAQ/proposal.
  • From a teaching perspective, Code examples in the wild on blogs, mailing lists and other sources often would need to make an extra effort to target either PHP, P++ or both. Knowledge would become clustered into two groups.
  • From an ecosystem perspective, a second name/brand would complicate everything for third party vendors, conferences, magazines. Examples: “PHPStorm, the lightning smart PHP & P++ IDE”, “Xdebug - Debugger and Profiler for PHP and P++”, “Dutch PHP and P++ conference”, “PHP and P++ Magazine”. We would probably need to introduce another name for the runtime, say PVM, to allow to make a precise distiction. This adds even more confusion.
  • From a SEO perspeective, Google and other search engines are a primary tool for software developers. If PHP and P++ now start fragmenting the communtiy it becomes much harder for developers to find solutions to problems, because “PHP sort array” will not find the articles “P++ sort array” that offer the same solution.
  • A long time ago, PHP was described to me as the Borg of programming languages. Assimilating APIs, features, paradigms from everywhere. This is still a very good analogy. And today it supports even more paradigms than 15 years ago and gives users extreme freedom to choose between dynamic or strict typing. This has been done in a way with as few BC breaks as possible. Python 3 and Perl 6 are examples of languages that made it much much harder for users to upgrade. I don’t see why suddenly now this approach is not possible anymore and requires two separate dialects.

The P++ proposal makes a few analogys to arrive at the P++ idea, but they are both flawed in my opinion:

  • The analogy that P++ is to PHP what C++ is to C is wrong. C++ introduced a completly new paradigm (object oriented programming). P++ as proposed is PHP with some BC breaks. Its more comparable to Python 2 to 3.
  • The analogy that P++ is to PHP what ES6 is to ES5 is wrong. ES6 and ES5 are versions like PHP 5 and PHP 7 are. EcmaScript is much better in not breaking backwards compatibility than PHP is, but the language by design makes this easier. You can still write Javascript with just ES5 syntax on every ES6 and ES7 compiler. The same is true of PHP 7, where you can still write code that would also run on PHP 3, PHP 4 and PHP 5.

With my arugments I have hopefully established enough non-technical arguments why separating PHP into two separate dialects is not a good idea.

An Alternative Approach

But what are the alternatives to evolve the PHP language?

PHP could avoid all the non-technical problems that P++ would introduce by going with an approach like C, C++, ECMAScript or Rust have: Define different versions of the language that the Runtime/Compiler can all support. Currently PHP combines runtime and language and upgrading to PHP 7 runtime requires you to update your code to PHP 7 semantics.

In C you specify to the compiler according to which version of the standard the file should be compiled.:

gcc -std=c89 file.c
gcc -std=c99 file.c

And then you can combine their output to a new binary which includes code compiled with both versions.

Rust has a smiliar concept named editions. In ECMAscript you use a third party compiler (like Babel) to compile one version down into another.

Essentially the proposed semantics of P++ boil down to defining a new version of PHPs language, they don’t warrant a new language.

If we allow the PHP runtime to support several standards at the same time we can avoid fragmentation of the community, avoiding all the non-technical issues listed above.

PHP already uses declare for this kind of decisions at the moment, so it would be natural to introduce a construct to PHP and make it responsible for switching the Compiler between different versions. Example with made up option name and version:

<?php declare(std=20);

This could be defined to automatically include strict_types=1, but also include some cleanup to type juggling rules for example. The sky is the limit. If we improve the language for the next version, we can introduce the next standard version, but the compiler could still support the old ones for a few years.

PHP users could upgrade to the latest version of the PHP runtime, get security patches, bugfixes, and performance improvements, but can keep the semantics of the version their software was written against. This would simplify the process of keeping backwards compatibility.

Deciding on the actual naming and syntax would be a minor technical problem.

]]>
Wed, 14 Aug 2019 00:00:00 +0200
https://beberlei.de/2019/03/25/the_jit_in_relation_to_php_extensions.html https://beberlei.de/2019/03/25/the_jit_in_relation_to_php_extensions.html <![CDATA[The JIT in relation to PHP extensions]]> The JIT in relation to PHP extensions

A few days ago I posted about Playing with the PHP JIT and included some simple benchmarking with the react-php-redis server project, which involves a lot of parsing but is ultimately still bound by I/O even when running async.

I got some questions on Twitter that are around some misconcetions of what the JIT really an do for PHP applications and what it cannot do.

So to show what the JIT is good for, I wanted to have truly CPU bound problem that was realistic from my POV.

Inside Tideways we use a datatype called HDRHistogram (high dynamic rrange histogram), a statistical datatype to calculate exact percentiles in monitoring data. For each minute and server we might have a histogram and when rendering a chart, we merge and aggregate this data in large numbers.

At the moment we use a PHP Extension interfacing with a C library to use this datatype.

I have ported the necessary code to PHP to test this with the JIT, without the JIT and against the PHP extension.

<?php

function simulate_hdr() {
    $hdr = hdr_init(1, 1000, 1);
    for ($i = 1; $i < 1000; $i++) {
        for ($j = 0; $j < 1000; $j++) {
            hdr_record_value($hdr, $i);
        }
    }
    hdr_value_at_percentile($hdr, 95);
}


for ($i = 0; $i < 5; $i++) {
    $time = microtime(true);
    simulate_hdr();

    echo number_format(microtime(true) - $time, 4) . "\n";
}

Again take the numbers with a grain of salt, these are just here to show the approximate relationships:

Runs PHP nojit PHP jit C/PHP Ext
1 0.5916 0.3671 0.0775
2 0.6322 0.4038 0.0775
3 0.6025 0.3866 0.0799
4 0.6010 0.3892 0.0829
5 0.6137 0.3947 0.0828
Average 0.6082 0.3883 0,0801
% 100,00% 63,84% 13,17%

As you can see, the JIT code runs at roughly 2/3 (63,84%) of the original non-jitted code and gets into the region of twice as fast that the RFC claims for PHPs internal benchmark. The improvement is much better than with the react-php-redis server example from a few days ago, where the improvement was only in the 5-20% region.

But compared to implementing this code directly in C as a PHP extension, even the jitted code is still 5 times slower.

Yes, with the JIT there is a massive improvement of this CPU bound problem, but it doesn’t mean we can now re-implement all PHP extensions in pure PHP and rely on the JIT to make them perform.

What the JIT does improve:

  • Make the parts of CPU bound problems that are written in PHP (!) faster.

What the JIT does not improve:

  • It does not improve performance of already fast internal functions written in C, for example hashing, encryption functions.
  • It does not improve performance (by a lot) for I/O bound problems.

To close the gap between JIT and C, we could look at PHP 7.4 including the FFI extension. It allows interfacing with C code more easily from PHP. Anthony Ferrara is building his “php-compiler” project on top FFI that would allow compiling a subset of PHP code directly to an FFI C extension.

]]>
Mon, 25 Mar 2019 00:00:00 +0100
https://beberlei.de/2019/03/23/playing_with_the_php_jit.html https://beberlei.de/2019/03/23/playing_with_the_php_jit.html <![CDATA[Playing with the PHP JIT]]> Playing with the PHP JIT

The PHP JIT RFC is a hot topic on the internals list right now and the voting has started for it to be included in PHP 8.0 and as experimental feature in 7.4.

I wanted to test it out myself, here are the steps necessary to get started on a Linux (Ubuntu) server (or desktop):

git clone https://github.com/php/php-src.git
cd php-src
git remote add zendtech https://github.com/zendtech/php-src.git
git checkout zendtech/jit-dynasm-7.4
./buildconf
./configure  --prefix=/opt/php/php-7.4 --enable-opcache --enable-opcache-jit --with-zlib --enable-zip --enable-json --enable-sockets --without-pear
make -j4
sudo make install

For testing I needed a more realistic problem that was bit more complex than PHPs internal benchmark (which the JIT doubles in speed).

Luckily I came across a good one at Symfony User Group in Cologne this week: The react-php-redis server by @another_clue re-implements redis server in PHP with almost zero PHP extension dependencies. That means the vanilla build from above with no dependencies is enough to get it running.

In addition it fully works with the redis-benchmark command that the original redis-server package includes, so it takes no effort to make some tests. The benchmark pegs the PHP redis server to 100% making it a good candidate for testing JIT.

The code is doing async I/O so the Redis protocol parsing and internal handling should play a significant role in this code that might be optimizable by the JIT.

git clone https://github.com/clue/php-redis-server.git
cd php-redis-server/
composer install

I ran it without the JIT:

/opt/php/php-7.4/bin/php bin/redis-server.php --port 6380

And with the JIT use these flags:

/opt/php/php-7.4/bin/php -dopcache.enable_cli=1 -dopcache.jit_buffer_size=50000000 -dopcache.jit=1235 bin/redis-server.php --port 6380

The -dopcache.jit=1235 only jits the HOT functions that are called often.

Then to check their performance in relation to each other, I ran redis-benchmark -p 6380 -q against the servers (from the redis-tools package).

Don’t take my numbers for gold (I ran them on my busy desktop machine), but you can see a 4-23% improvement depending on the benchmarked command.

Benchmark Nojit Jit % change
PING_INLINE 30674.85 31877.59 3.92%
PING_BULK 87873.46 95969.28 9.21%
SET 81766.15 87336.24 6.81%
GET 81433.22 91575.09 12.45%
INCR 77881.62 83682.01 7.45%
LPUSH 71275.84 79617.83 11.70%
RPUSH 67294.75 79239.3 17.75%
LPOP 73529.41 84530.86 14.96%
RPOP 76103.5 80450.52 5.71%
SADD 84745.77 89686.1 5.83%
HSET 82712.98 91074.68 10.11%
SPOP 87260.03 99700.9 14.26%
LPUSH (needed to benchmark LRANGE) 68493.15 83822.3 22.38%
LRANGE_100 (first 100 elements) 21743.86 26759.43 23.07%
LRANGE_300 (first 300 elements) 9825.11 11923.21 21.35%
LRANGE_500 (first 450 elements) 6819.42 8272.67 21.31%
LRANGE_600 (first 600 elements) 5120.33 5707.11 11.46%
MSET (10 keys) 45998.16 52631.58 14.42%

Looking at the results in a system profiler (perf) I can see that the PHP process is spending a lot of time in I/O functions, so these numbers are not showing the full potential of the JIT with CPU bound code.

]]>
Sat, 23 Mar 2019 00:00:00 +0100
https://beberlei.de/2019/01/05/integrate_ansible_vault_with_1password_commandline.html https://beberlei.de/2019/01/05/integrate_ansible_vault_with_1password_commandline.html <![CDATA[Integrate Ansible Vault with 1Password Commandline]]> Integrate Ansible Vault with 1Password Commandline

We are using Ansible to provision and deploy Tideways in development and production and the Ansible Vault feature to unlock secrets on production. Since we recently introduced 1Password I integrated them both and unlock the Ansible Vault using 1Password.

This way we can centrally change the Ansible Vault password regularly, without any of the developers with access to production/deployment needing to know the actual password.

To make this integration work, you can setup 1Password CLI to query your 1Password vault for secrets after logging in with password and two factor token.

Then you only need a bash script to act as an executable Ansible Vault password file.

First, download and install the 1Password CLI according to their documentation.

Next, you need to login with your 1Password account explicitly passing email, domain and secret key, so that the CLI can store this information in a configuration file.

$ op signin example.1password.com me@example.com
Enter the Secret Key for me@example.com at example.1password.com: A3-**********************************
Enter the password for me@example.com at example.1password.com:
Enter your six-digit authentication code: ******

After this one-time step, you can login more easily by just specifiying op signin example, so I create an alias for this in ~.bash_aliases (I am on Ubuntu).

alias op-signin='eval $(op signin example)'
alias op-logout='op signout && unset OP_SESSION_example'

The eval line makes sure that an environment variable OP_SESSION_example is set for this terminal/shell only with temporary access to your 1Password vault in subsequent calls to the op command. You can use op-logout alias to invalidate this session and logout.

Then I create the bash script in /usr/local/bin/op-vault that is used as Ansible Vault Password File. It needs to fetches the secret and print it to the screen.

#!/bin/bash
VAULT_ID="1234"
VAULT_ANSIBLE_NAME="Ansible Vault"
op get item --vault=$VAULT_ID "$VAULT_ANSIBLE_NAME" |jq '.details.fields[] | select(.designation=="password").value' | tr -d '"'

This one liner uses the command jq to slice the JSON output to print only the password. The tr command trims the double quotes around the password.

Make sure to configure the VAULT_ID and VAULT_ANSIBLE_NAME variables to point to the ID of your vault where the secret is stored in, and its name in the list. To get the UUIDs of all the vaults type op list vaults in your CLI.

Afterwards you can unlock your Ansible Vault with 1Password by calling:

ansible-playbook --vault-password-file=/usr/local/bin/op-vault -i inventory your_playbook.yml

This now only works in the current terminal/shell, when you called op-signin before to enter password and 2 factor token.

]]>
Sat, 05 Jan 2019 00:00:00 +0100
https://beberlei.de/2018/10/28/unslacking_tideways_company.html https://beberlei.de/2018/10/28/unslacking_tideways_company.html <![CDATA[Unslacking Tideways Company]]> Unslacking Tideways Company

We have moved away from Slack at Tideways over the last three months, because I found Slack is already annoying, even with just a four person team (plus the occasional freelancer). For me, it disrupts deep work phases and knowledge lost in the depth of chat history.

As an engineer, I have learned to be productive when I have a quiet space and can tinker on a problem without getting interrupted. Slack makes this very difficult and at least for me, is a primary cause of anxiety and fear of missing out (FOMO).

While at first sight it seems chat is asynchronous it really is not.

  1. If you wait for a long time to reply to a message on chat, then the discussion thread is already spread in the history of the chat room and messages back and forth interleaved with many additional messages that are not related is not helpful.
  2. In addition chats online status indicators exist just so that you know if someone can answer any question directly, increasing communication anxiety.
  3. If you set yourself into some kind of do not disturb mode or mute a channel, then its easy to miss important conversations and get left out of discussions or decisions. Reading up on long conversations you have missed is hard with chat tools.
  4. By also sending Github commit messages, OpsGenie and Tideways alerts, excerpts of HelpScout ticket updates and other “events” of the business into Slack through various integrations we made Slack the primary tool to check if anything is going on. While it is important to see what is going on, this almost never has to happen in realtime and by connecting it to the chat, we excuberated the previous points and everyone is checking chat even more frequently.

Jason Fried summed it up much better than I could in this blog post.

We are now using Github (issues and pull requests) and Basecamp (Messages, Todolists) to replace Slack. Both tools allow us to have context sensitive, asynchronous discussions on specific topics.

Since work never happens in a vacuum, I would be happy to have only a single realtime notification tool (OpsGenie/PagerDuty) that sends notifications to poeple currently on-call, and only about problems that require realtime attention. Everything else, including chat, can be part of a daily summary e-mail or screen.

My ultimate goal is to get longer stretches of uninterrupted time to work on features, customer support or operational issues. Under time pressure the last 3 years I realized that productive and concentrated tinkering on projects is my number one driver of happiness at work. I consider this a primary value of my company, and it takes work and distance to the current status quo to make it happen.

If you want to follow up on this ideas on your own, I can recommend the books Deep Work by Cal Newport, The Entrepreneur’s Guide to Keeping Your Sh*t Together by Sherry Walling and It doesn’t have to be crazy at work by Jason Fried and David Heinemeier Hansson.

]]>
Sun, 28 Oct 2018 00:00:00 +0200
https://beberlei.de/2018/10/20/how_we_paid_the_hidden_costs_of_bootstrapping_tideways.html https://beberlei.de/2018/10/20/how_we_paid_the_hidden_costs_of_bootstrapping_tideways.html <![CDATA[How we paid the hidden costs of bootstrapping Tideways]]> How we paid the hidden costs of bootstrapping Tideways

Justin Jackson started an extremely interesting discussion about the Bootstrappers paradox and the hidden costs of bootstrapping with replies from many others.

In this series of posts (there are more) he is discussing how hard bootstrapping a SaaS business is in the first few years, when you are essentially investing a lot of personal time and money until you can finally get a return on this investment and pay yourself a decent salary.

All bootstrappers have to deal with the same tension: can I get this to
scale, while paying my bills, without burning out?

This is something I am thinking about a lot to understand the journey and history of Tideways and by writing about my thoughts I can share my unconventional route for “bootstrapping” a SaaS.

In his post Jason projects the growth to sustainable $ 20.000 Monthly Reccurring Revenue (MRR) to take 5 years, which is much too long for my taste. We projected to reach 20.000€ MRR (~ $23.000) after three years, and missing this target would probably have meant a re-evaluation, maybe even shutdown or sale of Tideways.

We reached this goal this summer, after almost exactly three years.

What went into funding Tideways instead of money?

  • I was able to invest 50+ hours/week and live on 50-75% of my previous salary for two years only because I don’t have credits to pay; my wife and I don’t have kids, no cars and no other large monthly expenses. We saved up a lot of money that we partially invested into the company and partially used for personal expenses during the first two years. Trading your own personal time for bootstrapping a business is probably the largest, most hidden cost and I for myself am pretty sure that I am never going to do this again.
  • My then employer Qafoo allowed me to work on Tideways a few days per month and invested a lot of their own time after we decided to found a company together and test product market fit. In return they now own shares of the business.
  • We struck a deal with our first business partner SysEleven to trade hosting for Tideways in return for highly discounted licenses for almost all their customers. This helped us with the large four-digit monthly hosting costs that we had from day one, because as a monitoring company you need more servers than other SaaS, In return for investing in us, they now have us as a happy paying customer and they still get a good deal on licenses for their customers.
  • With over 7000 Twitter followers from my Doctrine open source project days and my reach within the European and German PHP community I accidentily already built a large audience that would be potential customers for Tideways. We didn’t need to invest more time into building an audience and we luckily never had to pay for customer acquisition. However realistically I invested hundreds of unpaid hours into my open source work since 2009 that allowed me to build this audience.

I consider points 2 & 3 “unconventional” ways for bootstrapping Tideways and already having an audience (4) as a very long term investment that I didn’t plan and got lucky with.

Roughly summing up these investments now, they amount to costs of around 400.000-500.000€ that Tideways would have required funding for.

Maybe raising 500.000€ would be possible in hindsight, but without previous founding experience and no connections to investors, I don’t know if I would have succeeded.

I consider our approach similar to Fundstrapping but instead of money you raise freedom, time and resources from your investors.

Most specifically my advice is to find strategic early business parterns that either become your first large customer from day one, or make your product available to their large customer base.

]]>
Sat, 20 Oct 2018 00:00:00 +0200
https://beberlei.de/2018/09/24/proofing_the_pythagorean_theorem_and_what_this_means_for_domain_driven_design.html https://beberlei.de/2018/09/24/proofing_the_pythagorean_theorem_and_what_this_means_for_domain_driven_design.html <![CDATA[Proofing the Pythagorean theorem and what this means for Domain-Driven Design]]> Proofing the Pythagorean theorem and what this means for Domain-Driven Design

According to Wikipedia the Pythagoraen theorem is the theorem with the highest number of known proofs, citing a book containing over 370 of them.

What does this mean for modelling under the Domain-Driven Design paradigm?

In discussions with other developers I often hear the opinion, that you should develop the model of a software entirely disconnected from technical decisions as long as possible and fully abstracted from the choices.

I held this opinion myself for a long time.

Discussing this with a colleague today I found a nice analogy to describe my growing pain with this approach, something I wasn’t able to formulate before.

Maybe the analogy is a bit of a stretch.

The Pythagoraen theorem shows that you can find an entirely different solution for the same problem by using a different toolbox. Finding one or more proofs for the theorem depends on your knowledge of different fields of mathematics. And depending on the toolbox, the solution is simple and elegant, or complex and theoretical.

The reverse conclusion could be, that if you develop a model (theorem) entirely abstracted from the future implementation and toolbox, then it must increase in generality and complexity to take into account the potential implementation with any kind of toolbox. Or the problem space is simple like the Pythagoraen theorem that its description allows many implementations.

Consequently, if you make technical decisions and restrict yourself to fewer approaches/toolboxes before building the model, then the resulting model can be simpler under the assumption that you only need it to be solved under the given technical constraints.

For example, take Redis as a choice of database. Its built in support for more complex data-types allows to implement extremely simple abstractions directly on top of Redis. If you know you use Redis up front for a problem that can be solved with lists, sets or maps, then the model and code describing this domain logic could be extremely simplified compared a solution that makes no assumptions about the data storage.

]]>
Mon, 24 Sep 2018 00:00:00 +0200
https://beberlei.de/2017/07/29/hide_slack_unread_and_highlight_tray_icon_on_ubuntu_linux.html https://beberlei.de/2017/07/29/hide_slack_unread_and_highlight_tray_icon_on_ubuntu_linux.html <![CDATA[Hide Slack unread and highlight Tray Icon on Ubuntu/Linux]]> Hide Slack unread and highlight Tray Icon on Ubuntu/Linux

The Slack Ubuntu/Linux application doesn’t provide a way to disable the tray icon and that means the unread or highlight notification bubbles will distract you whenever somebody writes something in ANY channel that you are in, even when the messages are not even interesting for you.

I wrote to Slack support to make a feature request and they suggested to make clever use of an upstream Electron bug that makes the tray icon disappear in 17.04 as a workaround. This gave me the idea for a more stable workaround.

I poked around in the list of files of the slack-desktop package and found out that you can just exchange the files for the tray icons , so that the Slack app is fooled into thinking it is showing the unread or highlight icons.

sudo -i
cd /usr/lib/slack/resources/app.asar.unpacked/src/static
mv slack-taskbar-highlight.png slack-taskbar-highlight-backup.png
mv slack-taskbar-unread.png slack-taskbar-unread-backup.png
cp slack-taskbar-rest.png slack-taskbar-highlight.png
cp slack-taskbar-rest.png slack-taskbar-unread.png

Since I probably need to re-run this after every update to the Slack app, I packed this into a shell script slack-quiet.

]]>
Sat, 29 Jul 2017 00:00:00 +0200
https://beberlei.de/2017/03/29/why_does_the_php_extension_fail_to_load_with_zval_ptr_dtor_wrapper_error.html https://beberlei.de/2017/03/29/why_does_the_php_extension_fail_to_load_with_zval_ptr_dtor_wrapper_error.html <![CDATA[Why does the PHP extension fail to load with _zval_ptr_dtor_wrapper error]]> Why does the PHP extension fail to load with _zval_ptr_dtor_wrapper error

I sometimes come across this error when loading an extension in PHP

PHP Warning:  PHP Startup: Unable to load dynamic library
'/usr/lib64/php-zts/5.5/modules/myext.so' -
/usr/lib64/php-zts/5.5/modules/myext.so: undefined symbol:
_zval_ptr_dtor_wrapper in Unknown on line 0

From the Google results it is not very easy to find out what this means.

Technically the extension was compiled with a PHP version using the -enable-debug flag, which wraps all calls to zval_ptr_dtor in a wrapper macro that does not exist in a regular, non-debug build.

But a debug extension and non-debug PHP are incompatible.

PHP extensions are only compatible between different installations if the following constraints are the same for the compiling and the executing PHP binary:

  • OS (Linux, Mac)
  • libc (glibc or musl for Alpine)
  • PHP version up to the minor level (5.5, 5.6, 7.0, 7.1)
  • extension version (20151012 for PHP 7.0)
  • thread-safety (ZTS) or non-thread-safety (NTS)
  • debug or non-debug (--enable-debug flag during compile).
]]>
Wed, 29 Mar 2017 00:00:00 +0200
https://beberlei.de/2017/03/13/how_to_complete_a_side_project_and_turn_it_into_a_business_level1.html https://beberlei.de/2017/03/13/how_to_complete_a_side_project_and_turn_it_into_a_business_level1.html <![CDATA[How to complete a side project and turn it into a business (Level 1)]]> How to complete a side project and turn it into a business (Level 1)

Almost four years ago I wrote about Lessons learned: A failed side project and when I stumbled across the post How to *never* complete anything several days ago I felt it was time for an update. Ewan’s post covers similar lessons learned that I wrote about in my post and I heard about many side projects that failed because of similar mistakes.

There was an argument on HackerNews what a side project is, so I want to clarify this up front. I am talking about side-projects written with some intend of turning them into a business, as opposed to side-projects that are started for learning new technical skills.

Three years ago I have started working on a PHP profiling and monitoring side-project, which I turned into a business called Tideways. In this post I wanted to share some of the reasons why I think this project was successful.

The idea for “XHProf as a Service” was on my “Side Project Ideas” trello board, where I write everything down that I want to try or experiment with. I have picked it out for implementation as my new side project, because that month I would have needed this for a large scale load-testing project at my job with Qafoo. It was not the first time I needed this for myself, I felt a regular pain that a product like this didn’t exist.

Not wanting to make the same mistakes, I have applied all the lessons learned from 2013:

  • I picked a small scope: The first version contained very few features, re-used the existing xhprof PHP extension (Tideways now has its own) and instead of a daemon collecting data in realtime used a cronjob to collect data from a directory every minute. After six months of development I removed half of the features I experimented with to reduce the scope again.
  • I did not compete with the established competition on features, instead I focussed on two single features that I thought where the most important: Response time monitoring and Callgraph profiling. By focussing on a niche (PHP intead of all languages) I was able to provide a better solution than existing generic tools (obviously biased).
  • I did not work all alone on the project, instead I have immediately enlisted alpha-users that gave valuable feedback after 1 month of work and some of them are still our most active customers; brought in my employer Qafoo who is now a shareholder in the new business; formed business partnerships with companies that are now our biggest customers and re-sellers.
  • I did not keep the idea secret, when I announced our beta program publically in June 2014 the launch newsletter list received 250 sign ups and 60 Twitter followers in a few hours.
  • I choose boring technology, using a monolithic Symfony, PHP, MySQL backend and jQuery frontend stack allowed me to iterate very fast on new features with technology that I already mastered. I spent three innovation tokens on using Golang for the daemon, using Elasticsearch as TimeSeries database and later on learning C for the PHP extension.

Making these decisions has not magically turned my side project into a profit-generating business. However I have avoided all the problems that I have written about in my failed side project lessons post from four years ago.

The fast iterations on a small scope combined with early user feedback showed that this idea will work as a business and it was worthwhile to keep pushing for two years, until the project generated enough revenue after all other costs to pay my full time salary.

This felt like passing level 1 in getting a bootstrapped side project of the ground (there are many more levels after that).

]]>
Mon, 13 Mar 2017 00:00:00 +0100
https://beberlei.de/2017/03/12/explicit_global_state_with_context_objects.html https://beberlei.de/2017/03/12/explicit_global_state_with_context_objects.html <![CDATA[Explicit Global State with Context Objects]]> Explicit Global State with Context Objects

Global State is considered bad for maintainability of software. Side effects on global state can cause a very nasty class of bugs. Context objects are one flavour of global state. For example, I remember that Symfony1 had a particularly nasty context object that was a global singleton containing references to very many services of the framework.

As with every concept in programming, there are no absolute truths though and there are many use-cases where context objects make sense. This blog posts tries to explain my reasons for using context objects.

What is a Context?

Context is defined as all the information and circumstances in which something can be fully understood. In daily programming this is mostly related to the state of variables and databases. Some examples in the world of PHP include the superglobals $_GET and $_POST.

Any piece of code is always running in some context and the question is how much of it is explicit in the code and how much is hidden.

A context object is a way to make the existing context explicit for your code.

Context Objects

Lets take a look at the Definition of a context object

A context object encapsulates the references/pointers to services and configuration information used/needed by other objects. It allows the objects living within a context to see the outside world. Objects living in a different context see a different view of the outside world.

Besides the obvious use of encapsulating services and config variables, this definition mentions two important properties:

  1. Allows to see the outside world, which does not mean it can change it. In my opinion it is essential that context objects are immutable, to avoid side effects.
  2. The possibility of objects living in different contexts, seeing different context objects suggets that a context object should never be a singleton.

By using objects instead of global variables for context, we can use encapsulation to achieve immutability.

Context already exists in PHP through the existance of superglobals. Frameworks usually wrap them to achieve the properties mentioned above: Immutability and Not-Singleton.

The Symfony Request object is one good example, where these properties (almost) hold. Wrapping the superglobals with this object even allowed creating Subrequests inside PHP requests.

Application Context

Now that we have defined context we should make use of context objects in your applications.

Anything that is an important global information in your application is relevant for the context:

  1. Building a wiki? I actually have the idea for this approach from Fitnesse, a testing tool based on the wiki idea maintained by Uncle Bob and his team. Their context object provides access to the current and the root page nodes.
  2. Building a shop? The users basket id, selected language/locale, current campaign (newsletter, google, social media?) can be information that should be available all the time.
  3. Building an analytics software? The currently selected date/time-range is probably important for all queries.
  4. Building a CMS/blog? The current page/post-id, the root page id, user language/locale seem to be good candidates for an application context. Wordpress does this although their context is global and encapsulated in an object.
  5. Building a multi-tenant app? The tenant-id and configuration for this tentant (selected product plan, activated features, …) are good candidates for the context.

Real World Example: Context in Tideways

How to introduce such a context? We could create an object in our application, for example how we did it in Tideways with selected tenant (organization), application, date-range and server environment:

<?php

class PageContext
{
    /**
     * @var \Xhprof\Bundle\OrganizationBundle\Entity\Organization
     */
    private $organization;

    /**
     * @var \Xhprof\Bundle\OrganizationBundle\Entity\Application
     */
    private $application;

    /**
     * @var \Xhprof\Common\Date\DateRange
     */
    private $selectedDateRange;

    /**
     * @var \Xhprof\Bundle\ProfilerBundle\View\EnvironmentView
     */
    private $selectedEnvironment;

    // constructor and getters
}

This object is created during request boot, in my case with a framework listener. The listener checks for access rights and security constraints, showing the 403/access denied page when necessary. This make 90% of access control checks unneeded that are usually cluttering the controller.

The context is then made available for the application by using a Symfony parameter converter, every controller-action can get access to the context by type-hinting for it:

<?php

class ApplicationController
{
    public function showAction(PageContext $pageContext)
    {
        return array('application' => $pageContext->getApplication());
    }
}

The beauty of this approach is avoiding global state and passing the context around in a non-singleton way. Depending on the framework you use, it might be hard to achieve this kind of context injection.

Now when I build lightweight Symfony2 controllers in my applications, using a context object allows me to use even less services and move repetitive find and access control code outside of the controllers.

I have also written a Twig extension that gives me access to the context object, so I don’t have to return it from every controller and created a wrapper for the URL Generation that appends context information to every URL (current date range + environment):

<h1>{{ pageContext.application.name }}</h1>

<a href="{{ page_path("some_route") }}">Link with Context query arguments</a>

Conclusion

A context object can help you make global state explicit and control access to it. Good requirements for a context object are immutability and not being a singleton.

When used correctly this pattern can save you alot of redundant code and simplify both controllers and views massively.

The pattern has its drawbacks: You have to be careful not put too powerful objects into the context and if you can modify the context, then you will probably introduce nasty side effets at some point. Additionally if you don’t make sure that creating the context is a very fast operation then you will suffer from performance hits, because the context is created on every request, maybe fetching expensive data that isn’t even used.

]]>
Sun, 12 Mar 2017 00:00:00 +0100
https://beberlei.de/2016/05/28/composer_monorepo_plugin_previously_called_fiddler.html https://beberlei.de/2016/05/28/composer_monorepo_plugin_previously_called_fiddler.html <![CDATA[Composer Monorepo Plugin (previously called Fiddler)]]> Composer Monorepo Plugin (previously called Fiddler)

I have written about monorepos in this blog before, presented a talk about this topic and released a standalone tool called “Fiddler” that helps integrating Composer with a monolithic repository.

At the beginning of the year, somebody in #composer-dev IRC channel on Freenode pointed me in the direction of Composer plugins to use with Fiddler and it was an easy change to do so.

With the help of a new Composer v1.1 feature to add custom commands from a plugin, Fiddler is now “gone” and I renamed the repository to the practical beberlei/composer-monorepo-plugin package name on Github. After you install this plugin, you have the possibility to maintain subpackages and their dependencies in a single repository.

$ composer require beberlei/composer-monorepo-plugin

To use the plugin add monorepo.json files into each directory of a subpackage and use a format similar to the composer.json to add dependencies to a.) external composer packages that you have listed in your global Composer file b.) other subpackages in the current monorepo. See this example for a demonstration:

{
    "deps": [
        "vendor/symfony/http-foundation",
        "components/Foo"
    ],
    "autoload": {
        "psr-0": {"Bar": "src/"}
    }
}

This subpackage here defined in a hypothetical file components/Bar/monorepo.json has dependencies to Symfony HTTP foundation and another subpackage Foo with its own components/Foo/monnorepo.json. Notice how we don’t need to specify versions (they are implicit) and import other dependencies using the relative path from the global composer.json.

The monorepo plugin is integrated with Composer, so every time you perform install, update or dump-autoload commands, the subpackages will be updated as well and each get their own autoloader that can be included from vendor/autoload.php relative to the subpackages root directory as usual.

]]>
Sat, 28 May 2016 00:00:00 +0200
https://beberlei.de/2016/02/21/how_i_use_wordpress_with_git_and_composer.html https://beberlei.de/2016/02/21/how_i_use_wordpress_with_git_and_composer.html <![CDATA[How I use Wordpress with Git and Composer]]> How I use Wordpress with Git and Composer

I maintain two Wordpress blogs for my wife and wanted to find a workflow to develop, update, version-contol and maintain them with Git and Composer, like I am used to with everything else that I am working on.

The resulting process is a combination of several blog posts and my own additions, worthy of writing about for the next person interested in this topic.

It turns out this is quite simple if you re-arrange the Wordpress directory layout a little bit and use some fantastic open-source projects to combine Wordpress and Composer.

Initialize Repository

As a first step, create a new directory and git repository for your blog:

$ mkdir myblog
$ cd myblog
$ git init

Create a docroot directory that is publicly available for the webserver:

$ mkdir htdocs

Place the index.php file in it that delegates to Wordpress (installed later):

<?php
// htdocs/index.php
// Front to the WordPress application. This file doesn't do anything, but loads
// wp-blog-header.php which does and tells WordPress to load the theme.

define('WP_USE_THEMES', true);
require( dirname( __FILE__ ) . '/wordpress/wp-blog-header.php' );

Create the wp-content directory inside the docroot, it will be configured to live outside the Wordpress installation.

$ mkdir htdocs/wp-content -p

And then create a .gitignore file with the following ignore paths:

/htdocs/wordpress/
/htdocs/wp-content/uploads
/htdocs/wp-content/plugins
/htdocs/wp-content/themes

If you want to add a custom theme or plugin you need to use git add -f to force the ignored path into Git.

Don’t forget to include the uploads directory in your backup, when deploying this blog to production.

You directory tree should now look like this:

.
├── .git
├── .gitignore
└── htdocs
    ├── index.php
    └── wp-content

In the next step we will use Composer to install Wordpress and plugins.

Setup Composer

Several people have done amazing work to make Wordpress and all the plugins and themes on Wordpress.org available through Composer. To utilize this work we create a composer.json file inside our repository root. There the file is outside of the webservers reach, users of your blog cannot download the composer.json.

{
    "require": {
        "ext-gd": "*",
        "wpackagist-plugin/easy-media-gallery": "1.3.*",
        "johnpbloch/wordpress-core-installer": "^0.2.1",
        "johnpbloch/wordpress": "^4.4"
    },
    "extra": {
        "installer-paths": {
            "htdocs/wp-content/plugins/{$name}/": ["type:wordpress-plugin"],
            "htdocs/wp-content/themes/{$name}/": ["type:wordpress-theme"]
        },
        "wordpress-install-dir": "htdocs/wordpress"
    },
    "repositories": [
        {
            "type": "composer",
            "url": "http://wpackagist.org"
        }
    ]
}

This Composer.json is using the execellent Wordpress Core Installer by John P. Bloch and the WPackagist project by Outlandish.

The extra configuration in the file configures Composer for placing Wordpress Core and all plugins in the correct directories. As you can see we put core into htdocs/wordpress and plugins into htdocs/wp-content/plugins.

Now run the Composer install command to see the intallation output similar to the next excerpt:

$ composer install
Loading composer repositories with package information
Installing dependencies (including require-dev)
  - Installing composer/installers (v1.0.23)
    Loading from cache

  - Installing johnpbloch/wordpress-core-installer (0.2.1)
    Loading from cache

  - Installing wpackagist-plugin/easy-media-gallery (1.3.93)
    Loading from cache

  - Installing johnpbloch/wordpress (4.4.2)
    Loading from cache

Writing lock file
Generating autoload files

The next step is to get Wordpress running using the Setup Wizard.

Setup Wordpress

Follow the Wordpress documentation to setup your Wordpress blog now, it will create the neccessary database tables and give you wp-config.php file to download. Copy this file to htdocs/wp-config.php and modify it slightly, it is necessary to adjust the WP_CONTENT_DIR, WP_CONTENT_URL and ABSPATH constants:

<?php

// generated contents of wp-config.php, salts, database and so on

define('WP_CONTENT_DIR',    __DIR__ . '/wp-content');
define('WP_CONTENT_URL',    WP_HOME . '/wp-content');

/** Absolute path to the WordPress directory. */
if ( !defined('ABSPATH') ) {
    define('ABSPATH', dirname(__FILE__) . '/wordpress');
}

/** Sets up WordPress vars and included files. */
require_once(ABSPATH . 'wp-settings.php');

Voila. You have Wordpress running from a Git repository and maintain the Wordpress Core and Plugins through Composer.

Different Development and Production Environments

The next step is introducing different environments, to allow using the same codebase in production and development, where the base urls are different, without having to change wp-config.php or the database.

Wordpress relies on the SITEURL and HOME configuration variables from the wp_options database table by default, this means its not easily possible to use the blog under http://myblog.local (development) and https://myblog.com` (production).

But working on the blog I want to copy the database from production and have this running on my local development machine without anything more than exporting and importing a MySQL dump.

Luckily there is an easy workaround that allows this: You can overwrite the SITEURL and HOME variables using constants in wp-config.php.

For development I rely on the built-in PHP Webserver that is available since PHP 5.4 with a custom router-script (I found this on a blog a long time ago, but cannot find the source anymore):

<?php
//htdocs/router.php

$root = $_SERVER['DOCUMENT_ROOT'];
chdir($root);
$path = '/'.ltrim(parse_url($_SERVER['REQUEST_URI'])['path'],'/');
set_include_path(get_include_path().':'.__DIR__);

if(file_exists($root.$path)) {
    if(is_dir($root.$path) && substr($path,strlen($path) - 1, 1) !== '/') {
        $path = rtrim($path,'/').'/index.php';
    }

    if(strpos($path,'.php') === false) {
        return false;
    } else {
        chdir(dirname($root.$path));
        require_once $root.$path;
    }
} else {
    include_once 'index.php';
}

To make your blog run flawlessly on your dev machine, open up htdocs/wp-config.php and add the following if statement to rewrite SITEURL and HOME config variables:

<?php
// htdocs/wp-config.php

// ... salts, DB user, password etc.

if (php_sapi_name() === 'cli-server' || php_sapi_name() === 'srv') {
    define('WP_ENV',        'development');
    define('WP_SITEURL',    'http://localhost:8000/wordpress');
    define('WP_HOME',       'http://localhost:8000');
} else {
    define('WP_ENV',        'production');
    define('WP_SITEURL',    'http://' . $_SERVER['SERVER_NAME'] . '/wordpress');
    define('WP_HOME',       'http://' . $_SERVER['SERVER_NAME']);
}

define('WP_DEBUG', WP_ENV === 'development');

You can now run your Wordpress blog locally using the following command-line arguments:

$ php -S localhost:8000 -t htdocs/ htdocs/router.php

Keep this command running and visit localhost:8000.

]]>
Sun, 21 Feb 2016 00:00:00 +0100
https://beberlei.de/2015/08/08/monolithic_repositories_with_composer_and_relative_autoloading.html https://beberlei.de/2015/08/08/monolithic_repositories_with_composer_and_relative_autoloading.html <![CDATA[Monolithic Repositories with Composer and Relative Autoloading]]> Monolithic Repositories with Composer and Relative Autoloading

Just was reminded on Twitter by Samuel that there is a way for monolithic PHP repositories with multiple components that I haven’t mentioned in my previous post.

It relies on a new composer.json for each component and uses the autoloading capabilities of Composer in a hackish way.

Assume we have two components located in components/foo and components/bar, then if bar depends on foo, it could define its components/bar/composer.json file as:

{
    "autoload": {
        "psr-0": {
            "Foo": "../foo/src/"
        }
    }
}

This approach is very simple to start with, however it has some downsides you must take into account:

  • you have to redefine dependencies in every composer.json that relies on another component.
  • if foo and bar depend on different versions of some third library baz that are not compatible, then composer will not realize this and your code will break at runtime.
  • if you want to generate deployable units (tarballs, debs, ..) then you will have a hard time to collect all the implicit dependencies by traversing the autoloader for relative definitions.
  • A full checkout has multiple vendor directories with a lot of duplicated code.

I think this approach is ok, if you are only sharing a small number of components that don’t define their own dependencies. The Fiddler approach however solves all these problems by forcing to rely on the same dependencies in a project globally and only once.

]]>
Sat, 08 Aug 2015 00:00:00 +0200
https://beberlei.de/2015/06/12/the_containertest.html https://beberlei.de/2015/06/12/the_containertest.html <![CDATA[The ContainerTest]]> The ContainerTest

This is a short post before the weekend about testing in applications with dependency injection container (DIC). This solution helps me with a problem that I occasionally trip over in environments with large amounts of services connected through a DIC.

The problem is forgetting to adjust the DIC configuration when you add a new or remove a dependency to a service. This can easily slip through into production if you rely on your functional- and unit-tests to catch the problem.

I can avoid this problem by adding a functional test in my application that instantiate all the various services and checks if they are created correctly. The first time I saw this pattern was during development of some of the early Symfony2 bundles, most notably DoctrineBundle.

<?php

namespace Acme;

class ContainerTest extends \PHPUnit_Framework_TestCase
{
    use SymfonySetup;

    public static function dataServices()
    {
        return array(
            array('AcmeDemoBundle.FooService', 'Acme\DemoBundle\Service\FooService'),
            array('AcmeDemoBundle.BarController', 'Acme\DemoBundle\Controller\BarController'),
        );
    }

    /**
     * @test
     * @dataProvider dataServices
     */
    public function it_creates_service($id, $class)
    {
        $service = $this->getContainer()->get($id);
        $this->assertInstanceOf($class, $service);
    }
}

Whenever you create or modify a service check the ContainerTest if its already guarded by a test. Add a test if necesary and then make the change. It’s as easy as that.

The SymfonySetup trait provides access to the Symfony DIC using getContainer() as you can see in the test method. See my blog post on traits in tests for more information.

]]>
Fri, 12 Jun 2015 00:00:00 +0200
https://beberlei.de/2015/04/11/monolithic_repositories_with_php_and_composer.html https://beberlei.de/2015/04/11/monolithic_repositories_with_php_and_composer.html <![CDATA[Monolithic Repositories with PHP and Composer]]> Monolithic Repositories with PHP and Composer
tl;dr Monolithic repositories can bring a lot of benefits. I prototyped Fiddler that complements Composer to add dependency management for monolithic repositories to PHP.

Thanks to Alexander for discussing this topic with me as well as reviewing the draft of this post.

As Git and Composer are more ubiquitous in open-source projects and within companies, monolithic repositories containing multiple projects have become a bit of a bad practice. This is a similar trend to how monolithic applications are out of fashion and the recent focus on microservices and Docker.

Composer has made it possible to create many small packages and distribute them easily through Packagist. This has massively improved the PHP ecosystem by increasing re-usability and sharing.

But it is important to consider package distribution and development seperate from each other. The current progress in package manager tooling comes at a cost for version control productivity, because Composer, NPM, Bower force you to have exactly one repository for one package to benefit from the reusability/distribution.

This blog post compares monolithic repositories with one repository per package approach. It focuses on internal projects and repositories in organizations and companies. I will discuss open source projects in a follow-up post.

Workflow at Facebook, Google, Twitter

The move towards smaller repositories is called into question by three extremely productive organizations that work at incredible scale.

  • Facebook mentioned in their talk “Big Code: Developer Infrastructure at Facebook’s Scale” that they are going to merge their three big code repositories Server, iOS and Android into a single big repository over the course of 2015.
  • Google open-sourced Bazel, the build tool behind a huge chunk of their codebase managed in a single Perforce repository with over 20 million commits (Reference).
  • Twitter, Foursquare and Square are working on their clone of Google’s Bazel build system called Pants. It is also designed for monolithic repositories.

All three companies cite huge developer productivity benefits, code-reusability, large-scale refactorings and development at scale for choosing this approach. The Facebook talk even mentions how all their development infrastructure efforts focus on keeping this workflow because of the benefits it brings.

Downsides of having many Repositories

In contrast working with ever smaller repositories can be a huge burden for developer mental models: I have seen this in open-source projects such as Doctrine and several customer projects:

  1. Cross repository changes require certain pull-requests on Github/Gitlab to be merged in order or in combination yet the tools don’t provide visibility into these dependencies. They are purely informal, leading to high error rates.
  2. Version pinning through NPM and Composer package managers is great for managing third party dependencies as long its not too many of them and they don’t change too often. For internal dependencies its a lot of work to update dependencies between repositories all the time. Time gets lost by developers that don’t have the correct dependencies or because of mistakes in the merge process.
  3. Changing code in core libraries can break dependencies without the developer even realizing this because tests don’t run together. This introduces a longer feedback cycle between code that depends on each other, with all the downsides.

One important remark about monolithic repositories: It does not automatically lead to a monolithic code-base. Especially Symfony2 and ZF2 are a very good example of how you can build individual components with a clean dependency graph in a single big repository.

At Qafoo we have always preferred monolithic project repositories containing several components over many small independent ones. We advised many customers to choose this approach except in some special cases where going small was economically more efficient.

Benefits of Monolithic Repositories

Even if you are not at the scale of Facebook or Google, a single repository still provides the mentioned benefits:

  • Adjusting to constant change by factoring out libraries, merging libraries and introducing new dependencies for multiple projects is much easier when done in a single, atomic VCS commit.
  • Discoverability of code is much higher, if you have all the code in a single place. Github and Gitlab don’t offer powerful tools like find, grep, sed over more than one repository. Hunting down dependencies, in specific versions can cost alot of time.
  • Reusability increases as it is much easier to just use code from the same repository than from another repository. Composer and NPM simplify combining repositories at specific versions, however one problem is actually knowing that the code exists in the first place.
  • From an operational perspective it is much easier to get a new developer up to speed setting up projects from a single repository. Just practically its easier to add his public key to only one Team/Repository/Directory than to hundreds. On top of that setting up many small repositories and familiarizing with each of them costs a lot of time.

This is why I have been struggling with how Packagist and Satis force the move to smaller repositories through the technical constraint “one repository equals one composer.json file”. For reusable open source projects this is perfectly fine, but for company projects I have seen it hurt developer productivity more often than is acceptable.

Introducing Fiddler

So today I prototyped a build system that complements Composer to manage multiple separate projects/packages in a single repository. I call it Fiddler. Fiddler introduces a maintainable approach to managing dependencies for multiple projects in a single repository, without losing the benefits of having explicit dependencies for each separate project.

In practice Fiddler allows you to manage all your third-party dependencies using a composer.json file, while adding a new way of managing your internal dependencies. It combines both external and internal packages to a single pool and allows you to pick them as dependencies for your projects.

For each project you add a fiddler.json file where you specify both your third-party and internal dependencies. Fiddler will take care of generating a specific autoloader for each project, containing only the dependencies of the project. This allows you to have one repository, while still having explicit dependencies per project.

Keeping explicit dependencies for each project means it’s still easy to find out which components are affected by changes in internal or third-party dependencies.

Example Project

Say you have three packages in your application, Library_1, Project_A and Project_B and both projects depend on the library which in turn depends on symfony/dependency-injection. The repository has the following file structure:

projects
├── components
│   ├── Project_A
│   │   └── fiddler.json
│   ├── Project_B
│   │   └── fiddler.json
│   └── Library_1
│       └── fiddler.json
├── composer.json

The fiddler.json of Library_1 looks like this::

{
    "autoload": {"psr-0": {"Library1\\": "src/"}},
    "deps": ["vendor/symfony/dependency-injection"]
}

The fiddler.json of Project_A and Project_B look similar (except the autoload)::

{
    "autoload": {"psr-0": {"ProjectA\\": "src/"}},
    "deps": ["components/Library_1"]
}

The global composer.json as you would expect::

{
    "require": {
        "symfony/dependency-injection": "~2.6"
    }
}

As you can see dependencies are specified without version constraints and as directory paths relative to the project root. Since everything is in one repository, all internal code is always versioned, tested and deployed together. Dropping the need for explicit versions when specifying internal dependencies.

With this setup you can now generate the autoloading files for each package exactly like Composer would by calling:

$ php fiddler.phar build
Building fiddler.json projects.
 [Build] components/Library_1
 [Build] components/Project_A
 [Build] components/Project_B

Now in each package you can require "vendor/autoload.php"; and it loads an autoloader with all the dependencies specified for each component, for example in components/Library_1/index.php

<?php

require_once "vendor/autoload.php";

$container = new Symfony\Component\DependencyInjection\ContainerBuilder;

This is an early access preview, please test this, provide feedback if you see this as a valuable or not and about possible extensions. See the README for more details about functionality and implementation details.

The code is very rough and simple right now, you will probably stumble accross some bugs, please report them. It is stable enough so that we could actually port Tideways to it already which is a multi package repository.

]]>
Sat, 11 Apr 2015 00:00:00 +0200
https://beberlei.de/2015/02/26/integrate_symfony_and_webpack.html https://beberlei.de/2015/02/26/integrate_symfony_and_webpack.html <![CDATA[Integrate Symfony and Webpack]]> Integrate Symfony and Webpack

Asset Management in Symfony2 is handled with the PHP based library Assetic by default, however I have never really connected to this library and at least for me it usually wastes more time than it saves.

I am also not a big fan of the Node.JS based stack, because it tends to fail alot for me as well. With teams that primarily consist of PHP developers and web-designers the transition to use Node.JS tools should be very conservative in my opinion. Each team member should not feel overburdend by this new technology stack.

Frontend development is really not my strong suit, so these first steps I document here may seem obvious to some readers.

While researching about React.JS I came across a tool called Webpack which you could compare to Symfony’s Assetic. It is primarily focussing on bundling Javascript modules, but you can also ship CSS assets with it.

The real benefits for Webpack however are:

  1. the builtin support for AMD or CommonJS style module loaders
  2. a builtin development web-server that runs on a dedicated port, serving your combined assets.
  3. a hot reloading plugin that automatically refreshes either the full page or just selected code when the assets change.
  4. module loaders that allow instant translation of JSX or other languages with Javascript transpilers (CoffeeScript, …)

Let’s have a look at a simple example javascript application in app.js requiring jQuery. The code is part of the Symfony2 document root in web/:

web/
   css/
     page.css
   js/
     vendor/
       jquery.js
     app.js

Then we can use AMD-style modules to resolve the dependencies in our code:

// app.js
define(['./vendor/jquery.js'], function($) {
    $(document).ready(function() {
        $("#content").html("Webpack Hello World!");
    });
});

You can compare this to PHPs require() and autoloading functionality, something that Javascript has historically been lacking and usually leads to javascript files with many thousands lines of code. You can also use CommonJS-style module loading if your prefer this approach.

The downside of adding this functionality is that your code always has to run through Webpack to work on the browser. But Webpack solves this geniously by including a web-server that does the translation for you in the background all the time. With a little help of a configuration file called webpack.config.js

// webpack.config.js
module.exports = {
    entry   : "./web/js/app.js",
    output: {
        filename: "bundle.js",
        path : 'web/assets/',
        publicPath : '/assets/',
    }
}

we can start our assets development server by calling:

$ webpack-dev-server --progress --colors --port 8090 --content-base=web/

This will start serving the combined javascript file at http://localhost:8090/assets/bundle.js as well as the asset page.css at http://localhost:8090/css/page.css by using the --content-base flag. Every change to any of the files that are part of the result will trigger a rebuild similar to the --watch flag of Assetic, Grunt or Gulp.

Webpack can be installed globally so it is easy to get started with. I find this a huge benefit not having to require a package.json and Node+npm workflow for your PHP/Symfony project.

$ sudo npm install -g webpack

For integration into Symfony we make use of some Framework configuration to change the base path used for the {{ asset() }} twig-function:

# app/config/config.yml
framework:
  templating:
    assets_base_url: "%assets_base_url%"

# app/config/parameters.yml
parameters:
  assets_base_url: "http://localhost:8090"

This adds a base path in front of all your assets pointing to the Webpack dev server.

The only thing left for integration is to load the javascript file from your twig layout file:

<html>
    <body>
        <div id="content"></div>

        {% if app.environment == "dev" %}
        <script src="{{ asset('webpack-dev-server.js') }}"></script>
        {% endif %}
        <script type="text/javascript" src="{{ asset('assets/bundle.js') }}"></script>
    </body>
</html>

The webpack-dev-server.js file loaded only in development environment handles the hot module reload exchanging, adding, or removing modules while an application is running without a page reload whenever possible.

For production use the assets_base_url parameter has to be adjusted to your specific needs and you use the webpack command to generate a minified and optimized version of your javascript code.

$ webpack
Hash: 69657874504a1a1db7cf
Version: webpack 1.6.0
Time: 329ms
    Asset   Size  Chunks             Chunk Names
bundle.js  30533       0  [emitted]  main
   [2] ./web/js/app.js 1608 {0} [built]
   [5] ./web/js/vendor/jquery.js 496 {0} [built]

It will be placed inside web/assets/bundle.js as specified by the output configuration in the Webpack configuration. Getting started in production is as easy as seting the assets base url to null and pushing the bundle.js to your production server.

I hope this example shows you some of the benefits of using Webpack over Assetic, Grunt or Gulp and the simplicity using it between development and production. While the example is Symfony2 related, the concepts apply to any kind of application.

Back to why I stumbled over Webpack in the first place: React.JS. I have been circling around React for a while with the impression that is extremly well-suited for frontend development. The problems I had with React where purely operation/workflow based:

  1. React encourages modular design of applications, something that you have to get working first using require.js for example.
  2. Differentation between development (refresh on modify) and production assets (minified).
  3. React uses a template language JSX that requires cross-compiling the *.jsx files they are written in into plain javascript files.

Now this blog post has already shown that Webpack solves points one and two, but it also solves the JSX Transformation with some extra configuration in webpack.config.js:

// webpack.config.js
module.exports = {
    entry: './web/js/app.jsx',
    output: {
        filename: 'bundle.js',
        path: 'web/assets/',
        publicPath: '/assets'
    },
    module: {
        loaders: [
            { test: /\.jsx$/, loader: 'jsx-loader?insertPragma=React.DOM&harmony' }
        ]
    },
    externals: {'react': 'React'},
    resolve: {extensions: ['', '.js', '.jsx']}
}

Now it is trivally easy to use React, just create a file with the *.jsx extension and Webpack will automatically load it through Facebooks JSX transformer before serving it as plain javascript. The only requirement is that you have to install the NPM package jsx-loader.

So far I have used webpack only for two playground projects, but I am very confident integrating it into some of my production projects now.

]]>
Thu, 26 Feb 2015 00:00:00 +0100
https://beberlei.de/2015/02/19/vagrant_nfs_and_npm.html https://beberlei.de/2015/02/19/vagrant_nfs_and_npm.html <![CDATA[Vagrant, NFS and NPM]]> Vagrant, NFS and NPM

I have ranted about Node.JS and NPM on Twitter before, costing me lots of time, so I have to make up for this now and offer some solutions.

One problem I regularly have is the following: I have a Vagrant/Virtualbox using NFS and want to run NPM inside of that. Running it inside the box is necessary, because I don’t want everyone using the box have to setup the node stack.

However running npm install on an NFS share doesn’t work as per issue #3565 because a chmod fails and apparently from the ticket, this is not going to be fixed.

I finally got it working with a workaround script by Kevin Stone that mimics NPM, but moves the package.json to a temporary directory and then rsyncs its back:

#!/bin/bash
# roles/nodejs/files/tmpnpm.sh

BASE_DIR=${TMPDIR:-/var/tmp}
ORIG_DIR=$PWD
HASH_CMD="md5sum"

DIR_NAME=`echo $PWD | $HASH_CMD | cut -f1 -d " "`

TMP_DIR=$BASE_DIR/$DIR_NAME
mkdir -p $TMP_DIR

pushd $TMP_DIR

ln -sf $ORIG_DIR/package.json
npm $1

# Can't use archive mode cause of the permissions
rsync --recursive --links --times node_modules $ORIG_DIR

popd

Integrating this into my Ansible setup of the machine it looked like this:

# roles/nodejs/tasks/main.yml
# More tasks here before this...
- name: "Install npm workaround"
  copy: >
      src=tmpnpm.sh
      dest=/usr/local/bin/tmpnpm
      mode="0755"

- name: "Install Global Dependencies"
  command: >
      /usr/local/bin/tmpnpm install -g {{ item }}
  with_items: global_packages

- name: "Install Package Dependencies"
  command: >
      /usr/local/bin/tmpnpm install
      chdir={{ item }}
  with_items: package_dirs

Where global_packages and package_dirs are specified from the outside when invoking the role:

# deploy.yml
---
- hosts: all
  roles:
    - name: nodejs
      global_packages:
        - grunt-cli
      package_dirs:
        - "/var/www/project"

This way the Ansible Node.JS role is reusable in different projects.

]]>
Thu, 19 Feb 2015 00:00:00 +0100
https://beberlei.de/2015/02/14/phpunit_before_annotations_and_traits_for_code_reuse.html https://beberlei.de/2015/02/14/phpunit_before_annotations_and_traits_for_code_reuse.html <![CDATA[PHPunit @before Annotations and traits for code-reuse]]> PHPunit @before Annotations and traits for code-reuse

I have written about why I think traits should be avoided. There is a practical use-case that serves me well however: Extending PHPUnit tests.

The PHPUnit TestCase is not very extendable except through inheritance. This often leads to a weird, deep inheritance hierachy in testsuites to achieve code reuse. For example the Doctrine ORM testsuite having OrmFunctionalTestCase extending from OrmTestCase extending from PHPUnits testcase.

Dependency Injection is something that is not possible easily in a PHPUnit testcase, but could be solved using an additional listener and some configuration in phpunit.xml.

This leaves traits as a simple mechanism that doesn’t require writing an extension for PHPUnit and allows “multiple inheritance” to compose different features for our test cases.

See this simple example that is adding some more assertions:

<?php

trait MyAssertions
{
    public function assertIsNotANumber($value)
    {
        $this->assertTrue(is_nan($value));
    }
}

class MathTest extends \PHPUnit_Framework_TestCase
{
    use MyAssertions;

    public function testIsNotANumber()
    {
        $this->assertIsNotANumber(acos(8));
    }
}

When you have more complex requirements, you might need the trait to implement setUp() method. This will prevent you from using multiple traits that all need to invoke setUp(). You could use the trait conflict resolution, but then the renamed setup methods do not get called anymore.

Fortunately PHPUnit 3.8+ comes to the rescue with new @before and @beforeClass annotations.

See this trait I use for making sure my database is using the most current database version by invoking migrations in @beforeClass

<?php

namespace Xhprof;

use Doctrine\DBAL\DriverManager;

trait DatabaseSetup
{
    /**
     * @var bool
     */
    private static $initialized = false;

    /**
     * @beforeClass
     */
    public static function initializeDatabase()
    {
        if (self::$initialized) {
            return;
        }

        self::$initialized = true;

        $conn = DriverManager::getConnection(array(
            'url' => $_SERVER['TEST_DATABASE_DSN']
        ));

        $dbDeploy = new DbDeploy($conn, realpath(__DIR__ . '/../../src/schema'));
        $dbDeploy->migrate();
        $conn->close();
    }
}

I could mix this with a second trait SymfonySetup that makes the DIC container available for my integration tests:

<?php

namespace Xhprof;

trait SymfonySetup
{
    protected $kernel;
    protected $container;

    /**
     * @before
     */
    protected function setupKernel()
    {
        $this->kernel = $this->createKernel();
        $this->kernel->boot();

        $this->container = $this->kernel->getContainer();
    }

    protected function createKernel(array $options = array())
    {
        return new \AppKernel('test', true);
    }

    /**
     * @after
     */
    protected function tearDownSymfonyKernel()
    {
        if (null !== $this->kernel) {
            $this->kernel->shutdown();
        }
    }
}

The Symfony setup trait uses @before and @after to setup and cleanup without clashing with the traditional PHPUnit setUp method.

Combining all this we could write a testcase like this:

<?php

class UserRepositoryTest extends \PHPUnit_Framework_TestCase
{
    use DatabaseSetup;
    use SymfonySetup;

    public function setUp()
    {
        // do setup here
    }

    public function testNotFindUserReturnsNull()
    {
        $userRepository = $this->container->get('user_repository');
        $unusedId = 9999;
        $user = $userRepository->find($unusedId);
        $this->assertNull($user);
    }
}

Sadly the @before calls are invoced after the original setup() method so we cannot access the Symfony container here already. Maybe it would be more practical to have it work the other way around. I have opened an issue on PHPUnit for that.

]]>
Sat, 14 Feb 2015 00:00:00 +0100
https://beberlei.de/2015/02/10/a_case_for_weak_type_hints_only_in_php7.html https://beberlei.de/2015/02/10/a_case_for_weak_type_hints_only_in_php7.html <![CDATA[A case for weak type hints only in PHP7]]> A case for weak type hints only in PHP7

TL;DR: I was one voice for having strict type hints until I tried the current patch. From both a library and application developer POV they don’t bring much to the table. I think PHP would be more consistent with weak type hints only.

These last weeks there have been tons of discussions about scalar type hints in PHP following Andrea Faulds RFC that is currently in voting. Most of them were limited to PHP Internals mailinglist but since the voting started some days ago much has also been said on Twitter and blogs.

This post is my completly subjective opinion on the issue.

I would have preferred strict type hints, however after trying the patch, I think that strict type hints

  • will cause considerable problems for application developers, forcing them to “replicate weak type hinting” by manually casting everywhere.
  • are useless for library developers, because they have to assume the user is in weak type mode.
  • are useless within a library because I already know the types at the public API, weak mode would suffice for all the lower layers of my library.

Neither group of developers gets a considerable benefit from the current RFCs strict mode.

The simple reason for this, request, console inputs and many databases provide us with strings, casting has to happen somewhere. Having strict type hints would not save us from this, type juggling and casting has to happen and PHP’s current approach is one of the main benefits of the language.

Real World Weak vs Strict Code Example

Lets look at an example of everyday framework code Full Code to support my case:

<?php

class UserController
{
    public function listAction(Request $request)
    {
        $status = $request->get('status'); // this is a string

        return [
            'users' => $this->service->fetchUsers($status),
            'total' => $this->service->fetchTotalCount($status)
        ];
    }
}

class UserService
{
    const STATUS_INACTIVE = 1;
    const STATUS_WAITING = 2;
    const STATUS_APPROVED = 3;

    private $connection;

    public function fetchUsers(int $status): array
    {
        $sql = 'SELECT u.id, u.username FROM users u WHERE u.status = ? LIMIT 10';

        return $this->connection->fetchAll($sql, [$status]);
    }

    public function fetchTotalCount(int $status): int
    {
        $sql = 'SELECT count(*) FROM users u WHERE u.status = ?';

        return $this->connection->fetchColumn($sql, [$status]); // returns a string
    }
}

See how the code on UserService is guarded by scalar typehints to enforce having the right types inside the service:

  • $status is a flag to filter the result by and it is one of the integer constants, the type hint coerces an integer from the request string.
  • fetchTotalCount() returns an integer of total number of users matching the query, the type hint coerces an integer from the database string.

This code example only works with weak typehinting mode as described in the RFC.

Now lets enable strict type hinting to see how the code fails:

  • Passing the string status from the request to UserSerice methods is rejected, we need to cast status to integer.
  • Returning the integer from fetchTotalCount fails because the database returns a string number. We need to cast to integer.
Catchable fatal error: Argument 1 passed to UserService::fetchUsers() must
be of the type integer, string given, called in /tmp/hints.php on line 22
and defined in /tmp/hints.php on line 37

Catchable fatal error: Return value of UserService::fetchTotalCount() must
be of the type integer, string returned in /tmp/hints.php on line 48

The fix everybody would go for is casting to (int) manually:

public function listAction(Request $request)
{
    $status = (int)$request->get('status'); // this is a string

    return [
        'users' => $this->service->fetchUsers($status),
        'total' => $this->service->fetchTotalCount($status)
    ];
}

and:

public function fetchTotalCount(int $status): int
{
    $sql = 'SELECT count(*) FROM users u WHERE u.status = ?';

    return (int)$this->connection->fetchColumn($sql, [$status]);
}

It feels to me that enabling strict mode completly defeats the purpose, because now we are forced to convert manually, reimplementing weak type hinting in our own code.

More important: We write code with casts already, the scalar type hints patch is not necessary for that! Only a superficial level of additional safety is gained, one additional check of something we already know is true!

Strict mode is useless for library developers, because I always have to assume weak mode anyways.

EDIT: I argued before that you have to check for casting strings to 0 when using weak typehints. That is not necessary. Passing fetchTotalCount("foo") will throw a catchable fatal error in weak mode already!

Do we need strict mode?

In a well designed application or library, the developer can already trust the types of his variables today, 95% of the time, without even having type hints, using carefully designed abstractions (example Symfony Forms and Doctrine ORM): No substantial win for her from having strict type hints.

In a badly designed application, the developer is uncertain about the types of variables. Using strict mode in this scenario she needs to start casting everywhere just to be sure. I cannot imagine the resulting code to look anything but bad. Strict would actually be counterproductive here.

I also see a danger here, that writing “strict mode” code will become a best practice and this might lead developers working on badly desigend applications to write even crappier code just to follow best practices.

As a pro strict mode developer I could argue:

  • that libraries such as Doctrine ORM and Symfony Forms already abstract all the nitty gritty casting from request or database today. But I don’t think that is valid: They are two of the most sophisticated PHP libraries out there, maybe used by 1-5% of the userbase. I don’t want to force this level of abstraction on all users. I can’t use this level myself all the time. Also if libraries already abstract this for us, why need to duplicate the checks again if we can trust the variables types?
  • that I might have complex (mathematical) algorithms that benefit from strict type hinting. But that is not really true: Once the variables have passed through the public API of my fully typehinted library I know the types and can rely on them on all lower levels. Weak or strict type hinting doesn’t make a difference anymore. Well designed libraries written in PHP5 already provide this kind of trust using carefully designed value objects and guard clauses.
  • that using strict type in my library reduce the likelihood of bugs, but that is not guaranteed. Users of my library can always decide not to use strict type hints and that requires me as a library author to consider this use-case and prevent possible problems. Again using strict mode doesn’t provide a benefit here.
  • to write parts of the code in strict and parts in weak mode. But how to decide this? Projects usually pick only one paradigm for good reason: E_STRICT compatible code yes or no for example. Switching is arbitrary and dangerously inconsistent. As a team lead I would reject such kind of convention because it is impractible. Code that follows this paradigm in strict languages such as Java and C# has an aweful lot of converting methods such as $connection->fetchColumnAsInteger(). I do not want to go down that road.

Would we benefit from only strict mode?

Supporters of strict mode only: Make sure to understand why this will never happen!

Say the current RFC gets rejected, would we benefit from a strict type hinting RFC? No, and the current RFC details the exact reasons why. Most notably for BC reasons all the PHP API will not use the new strict type hinting.

This current RFC is the only chance to get any kind of strict hinting into PHP. Yet with the limited usefullness as described before, we can agree that just having weak mode would be more consistent and therefore better for everyone.

Conclusion

I as PHP developer using frameworks and libraries that help me write type safe code today, strict typing appeals to me. But put to a test in real code it proves to be impractical for many cases, and not actually much more useful than weak type hinting in many other cases.

Weak types provide me with much of the type safety I need: In any given method, using only typehinted parameters and return values, I am safe from type juggling. As a library developer I have to assume caller uses weak mode all the time.

Having strict type hints suggests that we can somehow get rid of type juggling all together. But that is not true, because we still have to work with user input and databases.

The current RFC only introduced the extra strict mode because developers had a very negative reactions towards weak type hints. Strike me from this list, weak type hints are everything that PHP should have. I will go as far that others strict-typers would probably agree when actually working with the patch.

I would rather prefer just having weak types for now, this is already a big change for the language and would prove to be valuable for everyone.

I fear Strict mode will have no greater benefit than gamification of the language, the winner is the one with the highest percentage of strict mode code.

]]>
Tue, 10 Feb 2015 00:00:00 +0100
https://beberlei.de/2015/02/01/running_hhvm_webserver.html https://beberlei.de/2015/02/01/running_hhvm_webserver.html <![CDATA[Running HHVM with a Webserver]]> Running HHVM with a Webserver

I haven’t used HHVM yet because the use-case for the alternative PHP runtime didn’t came up. Today I was wondering if our Qafoo Profiler would run out of the box with HHVM using the builtin XHProf extension (Answer: It does).

For this experiment I wanted to run the wordpress blog of my wife on HHVM locally. It turns out this is not very easy with an existing LAMP stack, because mod-php5 and mod-fastcgi obviously compete for the execution of .php files.

Quick googling didn’t turn up a solution (there probably is one, hints in the comments are appreciated) and I didn’t want to install a Vagrant Box just for this. So I decided to turn this into a sunday side project. Requirements: A simple webserver that acts as proxy in front of HHVMs Fast-CGI. Think of it as the “builtin webserver” that HHVM is missing.

This turns out to be really simple with Go, a language I use a lot for small projects in the last months.

The code is very simple plumbing starting with a HTTP Server that accepts client requests, translates them to FastCGI requests, sending them to HHVM and then parsing the FastCGI Response to turn it into a HTTP Response.

As a PHP developer I am amazed how Go makes it easy to write this kind of infrastructure tooling. I prefer PHP for everything web related, but as I tried to explain in my talk at PHPBenelux last week, Go is a fantastic language to write small, self-contained infrastructure components (or Microservices if you want a buzzword).

Back to playing with HHVM, if you want to give your application a try with HHVM instead of ZendEngine PHP it boils down to installing a prebuilt HHVM package and then using my hhvm-serve command:

$ go get github.com/beberlei/hhvm-serve
$ hhvm-serve --document-root /var/www
Listening on http://localhost:8080
Document root is /var/www
Press Ctrl-C to quit.

The server passes all the necessary environment variables to HHVM so that catch-all front-controller scripts such as Wordpress index.php or Symfony’s app.php should just work.

If you don’t have a running Go Compiler setup this few lines should help you out on Ubuntu:

$ sudo apt-get install golang
$ GOPATH=~/go
$ mkdir -p ~/go/{src,bin,pkg}
$ PATH="$GOPATH/bin:$PATH"

You should put the $GOPATH and $PATH changes into your bashrc to make this a permanent solution.

Starting to run HHVM, a Wordpress installation is a good first candidate to check on, as I knew from HHVM team blog posts that Wordpress works. Using a simple siege based benchmark I was able to trigger the JIT compiler and the Profiler charts showed a nice performance boost minute after minute as HHVM replaces dynamic PHP with optimized (assembler?) code.

]]>
Sun, 01 Feb 2015 00:00:00 +0100
https://beberlei.de/2014/10/26/symfony_all_the_things_web.html https://beberlei.de/2014/10/26/symfony_all_the_things_web.html <![CDATA[Symfony All The Things (Web)]]> Symfony All The Things (Web)

My Symfony Hello World post introduced the smallest possible example of a Symfony application. Using this in trainings helps the participants understand of just how few parts a Symfony application contains. Sure, there are lots of classes participating under the hood, but I don’t care about the internals only about the public API.

We use microservice architectures for the bepado and PHP Profiler projects that Qafoo is working on at the moment. For the different components a mix of Symfony Framework, Silex, Symfony Components and our own Rest-Microframework (RMF) are used. This zoo of different solutions sparked a recent discussion with my colleague Manuel about when we would want to use Symfony for a web application.

We quickly agreed on: Always. I can’t speak for Manuel, but these are my reasons for this decision:

  • I always want to use a technology that is based on Symfony HttpKernel, because of the built-in caching, ESI and the Stack-PHP project. I usually don’t need this at the beginning of a project, but at some point the simplicity of extending the Kernel through aggregation is incredible.

    This leaves three solutions: Silex, Symfony Framework and Plain Components.

  • I want a well documented and standardized solution. We are working with a big team on bepado, often rotating team members for just some weeks or months.

    We can count the hours lost for developers when they have to start learning a new stack again. Knowing where to put routes, controllers, templates, configuration et al is important to make time for the real tasks.

    This leaves Symfony Framework and Silex. Everything built with the components is always custom and therefore not documented well enough.

  • I want a stable and extendable solution. Even when you just use Symfony for a very small component you typically need to interface with the outside world: OAuth, REST-API, HTTP Clients, Databases (SQL and NoSQL). There is (always) a bundle for that in Symfony.

    Yes, Silex typically has a copy-cat Provider for their own DIC system, but it is usually missing some configuration option or advanced use-case. In some cases its just missing something as simple as a WebDebug Toolbar integration that the Symfony Bundle has.

    My experience with Silex has been that its always several steps behind Symfony in terms of reusable functionality. One other downside with Silex in my opinion is its missing support for DIC and Route caching. Once your Silex application grows beyond its initial scope it starts to slow down.

  • I want just one solution if its flexible enough.

    It is great to have so many options, but that is also a curse. Lukas points out he is picking between Laravel, Silex or Symfony depending on the application use-case.

    But the web technology stack is already complex enough in my opionion. I rather have my developers learn and use different storage/queue or frontend technologies than have them juggle between three frameworks. If my experience with Symfony in the last 4 years taught me anything: Hands-on exposure with a single framework for that long leads to impressive productivity.

    And Symfony is flexible. The Dependency Injection based approach combined with the very balanced decoupling through bundles allows you to cherry-pick only what you need for every application: APIs, RAD, Large Teams. Everything is possible.

The analysis is obviously biased because of my previous exposure to the framework. The productivity gains are possible with any framework as long as it has a flourishing ecosystem. For anyone else this reasoning can end up to choose Laravel, Silex or Zend Framework 2.

So what is the minimal Symfony distribution that would be a starting point. Extending on the Symfony Hello World post:

  1. composer.json
  2. index.php file
  3. A minimal AppKernel
  4. A minimal config.yml file
  5. routing files
  6. A console script
  7. A minimal application bundle

You can find all the code on Github.

Start with the composer.json:

{
    "require": {
        "symfony/symfony": "@stable",
        "symfony/monolog-bundle": "@stable",
        "vlucas/phpdotenv": "@stable"
    },
    "autoload": {
        "psr-0": { "Acme": "src/" }
    }
}

The index.php:

<?php
// web/index.php

require_once __DIR__ . "/../vendor/autoload.php";
require_once __DIR__ . "/../app/AppKernel.php";

use Symfony\Component\HttpFoundation\Request;

Dotenv::load(__DIR__ . '/../');

$request = Request::createFromGlobals();
$kernel = new AppKernel($_SERVER['SYMFONY_ENV'], (bool)$_SERVER['SYMFONY_DEBUG']);
$response = $kernel->handle($request);
$response->send();
$kernel->terminate($request, $response);

We are using the package vlucas/phpdotenv to add Twelve Factor app compatibility, simplyfing configuration. This allows us to get rid of the different front controller files based on environment. We need a file called .env in our application root containing key-value pairs of environment variables:

# .env
SYMFONY_ENV=dev
SYMFONY_DEBUG=1

Add this file to .gitignore. Your deployment to production needs a mechanism to generate this file with production configuration.

Our minimal AppKernel looks like this:

<?php
// app/AppKernel.php

use Symfony\Component\HttpKernel\Kernel;
use Symfony\Component\Config\Loader\LoaderInterface;

class AppKernel extends Kernel
{
    public function registerBundles()
    {
        $bundles = array(
            new Symfony\Bundle\FrameworkBundle\FrameworkBundle(),
            new Symfony\Bundle\TwigBundle\TwigBundle(),
            new Symfony\Bundle\MonologBundle\MonologBundle(),
            new Acme\HelloBundle\AcmeHelloBundle()
        );

        if (in_array($this->getEnvironment(), array('dev', 'test'))) {
            $bundles[] = new Symfony\Bundle\WebProfilerBundle\WebProfilerBundle();
        }

        return $bundles;
    }

    public function registerContainerConfiguration(LoaderInterface $loader)
    {
        $loader->load(__DIR__ . '/config/config.yml');

        if (in_array($this->getEnvironment(), array('dev', 'test'))) {
            $loader->load(function ($container) {
                $container->loadFromExtension('web_profiler', array(
                    'toolbar' => true,
                ));
            });
        }
    }
}

It points to a configuration file config.yml. We don’t use different configuration files per environment here because we don’t need it. Instead we use the closure loader to enable the web debug toolbar when we are in development environment.

Symfony configuration becomes much simpler if we don’t use the inheritance and load everything from just a single file:

# app/config/config.yml
framework:
    secret: %secret%
    router:
        resource: "%kernel.root_dir%/config/routing_%kernel.environment%.yml"
        strict_requirements: %kernel.debug%
    templating:
        engines: ['twig']
    profiler:
        enabled: %kernel.debug%

monolog:
    handlers:
        main:
            type:         fingers_crossed
            action_level: %monolog_action_level%
            handler:      nested
        nested:
            type:  stream
            path:  "%kernel.logs_dir%/%kernel.environment%.log"
            level: debug

We can set the parameter values for %secret% and %monolog_action_level% by adding new lines to .env file, making use of the excellent external configuration parameter support in Symfony.

# .env
SYMFONY_ENV=dev
SYMFONY_DEBUG=1
SYMFONY__SECRET=abcdefg
SYMFONY__MONOLOG_ACTION_LEVEL=debug

Now add a routing_prod.yml file with a hello world route:

# app/config/routing_prod.yml
hello_world:
    pattern: /hello/{name}
    defaults:
        _controller: "AcmeHelloBundle:Default:hello"

And because our routes are dependent on the environment in config.yml also a routing_dev.yml containing the WebDebug toolbar and profiler routes:

# app/config/routing_dev.yml
_wdt:
    resource: "@WebProfilerBundle/Resources/config/routing/wdt.xml"
    prefix:   /_wdt

_profiler:
    resource: "@WebProfilerBundle/Resources/config/routing/profiler.xml"
    prefix:   /_profiler

_main:
    resource: routing_prod.yml

We now need a bundle AcmeHelloBundle that is referenced in routing.yml and in the AppKernel. When we follow Fabiens best practice about adding services, routes and templates into the app/config and app/Resources/views folders adding a bundle just requires the bundle class:

<?php
// src/Acme/HelloBundle/AcmeHelloBundle.php

namespace Acme\HelloBundle;

use Symfony\Component\HttpKernel\Bundle\Bundle;

class AcmeHelloBundle extends Bundle
{
}

And the controller that renders our Hello World:

<?php
// src/Acme/HelloBundle/Controller/DefaultController.php

namespace Acme\HelloBundle\Controller;

use Symfony\Bundle\FrameworkBundle\Controller\Controller;

class DefaultController extends Controller
{
    public function helloAction($name)
    {
        return $this->render(
            'AcmeHelloBundle:Default:hello.html.twig',
            array('name' => $name)
        );
    }
}

Now we only put a template into app/Resources:

{# app/Resources/AcmeHelloBundle/views/Default/hello.html.twig #}
Hello {{ name }}!

As a last requirement we need a console script to manage our Symfony application. We reuse the vlucas/phpdotenv integration here to load all the required environment variables:

#!/usr/bin/env php
<?php
// app/console

set_time_limit(0);

require_once __DIR__.'/../vendor/autoload.php';
require_once __DIR__.'/AppKernel.php';

use Symfony\Bundle\FrameworkBundle\Console\Application;
use Symfony\Component\Console\Input\ArgvInput;

Dotenv::load(__DIR__ . '/../');

$input = new ArgvInput();
$kernel = new AppKernel($_SERVER['SYMFONY_ENV'], (bool)$_SERVER['SYMFONY_DEBUG']);
$application = new Application($kernel);
$application->run($input);

Voila. The minimal Symfony distribution is done.

Start the php built in webserver to take a look

$ php -S localhost:8080 web/index.php

I personally like this simplicity of that, the only thing that annoys me are the two routing files that I need to conditionally load the web profiler routes and the closure loader for the web_profiler extension. I suppose the nicer approach would be a compiler pass that does all the magic behind the scenes.

From this minimal distribution you can:

  1. Add new services to app/config/config.yml.
  2. Add new routes to app/config/routing_prod.yml.
  3. Add controllers into new bundles and templates into app/Resources.
  4. Add third party bundles or Stack-PHP implementations when you need existing, reusable functionality such as OAuth, Databases etc.
  5. Add configuration variables to .env file instead of using the app/config/parameters.yml approach.

This scales well, because at every point you can move towards abstracting bundles and configuration more using Symfony’s built in functionality. No matter what type of application you build, it is always based on Symfony and the building blocks are always the same.

I suggest to combine this minimal Symfony with the QafooLabsFrameworkExtraBundle that I blogged about two weeks ago. Not only will the Symfony be lightweight also your controllers. You can built anything on top this foundation from simple CRUD, APIs, hexagonal- or CQRS-architextures.

]]>
Sun, 26 Oct 2014 00:00:00 +0200
https://beberlei.de/2014/10/14/lightweight_symfony2_controllers.html https://beberlei.de/2014/10/14/lightweight_symfony2_controllers.html <![CDATA[Lightweight Symfony2 Controllers]]> Lightweight Symfony2 Controllers

For quite some time I have been experimenting how to best implement Symfony 2 controllers to avoid depending on the framework. I have discussed many of these insights here in my blog.

There are three reasons for my quest:

Simplicity: Solutions to avoid the dependencies between framework and your model typically introduce layers of abstraction that produce complexity. Service layers, CQRS and various design patterns are useful tools, but developing every application with this kind of abstraction screams over-engineering.

While the Domain-Driven Design slogan is “Tackling complexity in software”, there are many abstractions out there that can better be described as “Causing complexity in software”. I have written some of them myself.

Testability: There is a mantra “Don’t unit-test your controllers” that arose because controllers in most frameworks are just not testable. They have many dependencies on other framework classes and cannot be created in a test environment. This lead many teams to use slow and brittle integration tests instead.

But what if controllers were testable because they don’t depend on the framework anymore. We could avoid testing all the many layers that we have removed for the sake of simplicity and also reduce the number of slow integration tests.

Refactorability: I found that when using service layer or CQRS, there is a tendency to use them for every use-case, because the abstraction is in place. Any use-case that is not written with those patterns is coupled against the framework again. Both development approaches are very different and refactoring from one to the other typically requires a rewrite.

A good solution should allow refactoring from a lightweight controller to a service layer with a small number of extract method and extract class refactorings.

While working on Tideways Profiler product I went to work on a solution that allowed for Simplicity, Testability and Refactorability and came up with the NoFrameworkBundle.

The design of the bundle is careful to extend Symfony in a way that is easy for Symfony developers to understand. To achieve this it heavily extends upon the FrameworkExtraBundle that is bundled with Symfony.

The design goals are:

  • Favour Controller as Services to decouple them from the Framework.
  • Replace every functionality of the Symfony Base controller in a way that does not require injecting a service into your controller.
  • Never fetch state from services and inject it into the controller instead.
  • Avoid annotations

The concepts are best explained by showing an example:

<?php

use QafooLabs\MVC\TokenContext;

class TaskController
{
    private $taskRepository;

    public function __construct(TaskRepository $taskRepository)
    {
        $this->taskRepository = $taskRepository;
    }

    public function showAction(TokenContext $context, Task $task)
    {
        if (!$context->isGranted('ROLE_TASK', $task)) {
            throw new AccessDeniedHttpException();
        }

        return array('task' => $task);
    }
}

This example demos the following features:

  • The TokenContext wraps access to the security.context service and is used for checking access permissions and retrieving the current User object. It is passed to the controller with the help of ParamConverter feature.

    TokenContext here is just an interface and for testing you can use a very simple mock implementation to pass an authenticated user to your controller.

  • View parameters are returned from the controller as an array, however without requiring the @Template annotation of the SensioFrameworkExtraBundle.

The next example demontrates the abstraction for form requests that help writing very concise form code:

<?php

use QafooLabs\MVC\TokenContext;
use QafooLabs\MVC\RedirectRouteResponse;
use QafooLabs\MVC\FormRequest;

class TaskController
{
    private $taskRepository;

    public function newAction(FormRequest $formRequest, TokenContext $context)
    {
        $task = new Task($context->getUser());

        if ($formRequest->handle(new TaskType(), $task)) {
            $this->taskRepository->save($task);

            return new RedirectRouteResponse('Task.show', array('id' => $task->getId()));
        }

        return array('form' => $formRequest->createFormView());
    }
}
  • The RedirectRouteResponse is used to redirect to a route without a need for the router service.

  • Usage of the FormRequest object that is a wrapper around FormFactory and Request object. It is passed by using a ParamConverter. The method $formRequest->handle combines binding the request and checking for valid data.

    Again there is a set of mock form request that allow you to simulate valid or invalid form requests for testing.

Writing controllers in this way addresses my requirements Simplicity, Testability and Refactorability. For simple CRUD controllers they only ever need access to a repository service. If one of your controllers grows too big, just refactor out its business logic into services and inject them.

Check out the repository on Github for some more features that we are using in Tideways.

Update 1: Renamed FrameworkContext to TokenContext as done in new 2.0 version of the bundle.

]]>
Tue, 14 Oct 2014 00:00:00 +0200
https://beberlei.de/2014/08/11/spotting_featureenvy_and_refactoring.html https://beberlei.de/2014/08/11/spotting_featureenvy_and_refactoring.html <![CDATA[Spotting Feature Envy and Refactoring]]> Spotting Feature Envy and Refactoring

I was following Aki’s on the SoCraTes2014 conference last week about Legacy Code and Refactoring. In the session a piece of real-world code was shown that contained one of most common code smells in LegacyCode: Feature Envy.

Martin Fowler describes this smell as “a method that seems more interested in a class other than the one it is in. The most common focus of the envy is the data.”

This code smell causes two very similar problems in a code base:

  • Duplication of rules related to their data throughout the code base.
  • Spreading domain concepts throughout the whole code-base.

A simple example can show this problem with DateTime is wide-spread in many PHP projects. We often see code such as the following computation to create a range of dates for a reporting function:

<?php

function calculateReport(DateTime $start, $days)
{
    if ($days < 0) {
        throw new InvalidArgumentException($days);
    }

    $endDate = clone $start;
    $endDate->modify('+' . $days . ' day');

    // calculation code here
}

This is a really simple example for Feature Envy: The computation and validation of creating an end date some days in the future of a start date can easily be duplicated throughout the whole code base and implements business rules that belong to the class DateTime itself:

<?php

class DateTime
{
    // ... other methods

    public function getDateDaysInFuture($days)
    {
        if ($days < 0) {
            throw new InvalidArgumentException($days);
        }

        $endDate = clone $this;
        $endDate->modify('+' . $days . ' day');

        return $endDate;
    }
}

function calculateReport(DateTime $start, $days)
{
    $endDate = $start->getDateDaysInFuture($days);

    // calculation code here.
}

(Note: You cant extend the builtin DateTime this way and need to subclass).

This is a great way to improve your code base massively and achieve what Object-Oriented and Domain-Driven Design is about: Modelling concepts of the domain by encapsulating data and behavior.

Learning to spot this kind of Feature Envy is a very good way to incrementally improve your legacy code base towards better models. I find it very interesting that the focus on fixing technical code-smells duplication and feature envy actually leads to a better domain model. As a side effect it massively deemphasizes DDD building blocks like Entity or Value Object, which in my opinion are a big cause for confusion. Testability increases as well, because data objects with behavior are much simpler to setup for testing.

That is why I recommend learning about Feature Envy. For me it is a very simple approach to find improvements.

]]>
Mon, 11 Aug 2014 00:00:00 +0200
https://beberlei.de/2014/04/24/symfony_hello_world.html https://beberlei.de/2014/04/24/symfony_hello_world.html <![CDATA[Symfony2: A Simple Hello World]]> Symfony2: A Simple Hello World

Reading Simplyfing Django I was reminded of how we organize the Symfony2 Training at Qafoo to start with a very simple “Hello World” example.

The Symfony2 Standard Edition already contains a huge burden of concepts, assumptions and conventions that can be confusing for a beginner.

  • YAML
  • Twig
  • Environments and Dependency Injection Container
  • Various weird and magic syntaxes for bundles, _controller and template names
  • Various files to open and poke around in
  • Composer
  • Maybe even Doctrine

As a trainer it is very hard to explain all this at the same time. It makes a lot of sense to avoid as many of them as possible.

That is why the Standard Edition is not the best way to start teaching Symfony, but one should actually consider a much lighter version of the framework and then gradually add more and more features until you end up with the standard edition.

Fabien discussed the simplification of the Symfony Standard Edition from another angle before, considering the minimum necessary code to use when reproducing bugs in his blog post “Packing a Symfony Full-Stack Application in one file”.

This blog post is about a truely minimal Symfony edition to start learning about the framework. The first thing I like to show is a really simple Hello World controller:

<?php

namespace Acme\TrainingBundle\Controller;

use Symfony\Component\HttpFoundation\Response;

class HelloController
{
    public function helloAction()
    {
        return new Response('Hello World!');
    }
}

What is the necessary framework glue code to support this Hello World? Lets start from the index file web/index.php:

<?php

require_once __DIR__ . "/../vendor/autoload.php";
require_once __DIR__ . "/../app/AppKernel.php";

use Symfony\Component\HttpFoundation\Request;

$request = Request::createFromGlobals();
$kernel = new AppKernel('dev', true);
$kernel->handle($request)->send();

We are using Composer here and the AppKernel class, lets take a peak at them:

{
    "require": {
        "symfony/symfony": "@stable"
    },
    "autoload": {
        "psr-0": { "Acme": "src/" }
    }
}

The composer.json file installs the latest stable Symfony release and adds autoloading for classes starting with Acme in the folder src.

We need to define a Kernel for our application in app/AppKernel.php and the minimal version of this looks like:

<?php

use Symfony\Component\HttpKernel\Kernel;
use Symfony\Component\Config\Loader\LoaderInterface;

class AppKernel extends Kernel
{
    public function registerBundles()
    {
        return array(
            new Symfony\Bundle\FrameworkBundle\FrameworkBundle(),
            new Symfony\Bundle\TwigBundle\TwigBundle(),
        );
    }

    public function registerContainerConfiguration(LoaderInterface $loader)
    {
        $loader->load(function ($container) {
            $container->loadFromExtension('framework', array(
                'secret' => 'some secret here',
                'router' => array(
                    'resource' => '%kernel.root_dir%/config/routing.yml'
                ),
                'templating' => array('engines' => array('twig'))
            ));
        });
    }
}

In trainings we are starting with a single config.yml file but that is technically not necessary by using the ClosureLoader. The routing component has no such closure loader, so its not possible to avoid YAML in the first minutes of Symfony2 exposure and we reference the app/config/routing.yml:

hello:
  pattern: /
  defaults:
    _controller: "Acme\TrainingBundle\Controller\HelloController::helloAction"

Did you know that you can specify controllers in Symfony by using the static callback syntax? It has disadvantages when considering bundle inheritance, but why bother a beginner with this feature and confuse him with the convention of "AcmeTrainingBundle:Hello:hello".

There are still lots of trip wires to stumble upon in this simple piece of routing.yml:

  • Tabs are not allowed in YAML, in every training at least one person gets bitten by that.
  • 2 vs 4 spaces ambiguity.
  • What is this magic _controller key?
  • I carefully avoid the {} inline syntax of YAML here as well as it introduces yet another ambiguity.

Run this application with the build in webserver:

$ php -S localhost:8000 web/index.php

Hello World!

The next step for a beginner is his first twig template to print the Hello World. To avoid another confusing convention at the beginning, the "AcmeTrainingBundle:Hello:hello.html.twig" template syntax, you can make use of the app/Resources/views/ folder that is automatically registered in the Twig Bundle. Just create a template app/Resources/views/hello.html.twig and change the controller:

<?php

namespace Acme\TrainingBundle\Controller;

use Symfony\Component\HttpFoundation\Response;
use Symfony\Bundle\FrameworkBundle\Controller\Controller;

class HelloController extends Controller
{
    public function helloAction()
    {
        return $this->render('hello.html.twig');
    }
}

The total number of files is 5 (6 with template) for this example. Note that we haven’t actually created a bundle for AcmeWorkshopBundle yet, because with the static syntax we don’t need it.

This is a very good starting point to start exploring more features of Symfony and gradually add the required configuration and files for this.

]]>
Thu, 24 Apr 2014 00:00:00 +0200
https://beberlei.de/2014/01/31/soap_and_php_in_2014.html https://beberlei.de/2014/01/31/soap_and_php_in_2014.html <![CDATA[SOAP and PHP in 2014]]> SOAP and PHP in 2014
“The SOAP stack is generally regarded as an embarrassing failure these days.” – Tim Bray

While the quote by Tim Bray is still true to some degree, the toolstack and possiblities behind SOAP are still superior to REST in my opinion. REST still requires alot of manual coding, where RPC with SOAP allows much automation with tools.

These last years REST has gotten all the buzz, everybody seems to be using it and there are lots of talks about REST on conferences. SOAP used to be big for building APIs, but everybody seems to hate it for various reasons. I have used SOAP in several projects over the last 10 years and this blog post is a random collection of information about the state of SOAP in PHP 2014, as a reminder to myself and to others as well.

Why care about SOAP in 2014? For server to server (RPC) communication it still has massive time to market and stability benefits over REST. The REST toolchain is just not well developed enough, it lacks:

  • a standard to describe the input/output formats of endpoints
  • a way to “just do it”
  • ways to automatically generate clients in any language

While solutions exist for these problems in the REST space, they are often not standardized and don’t serve the full stack and different languages.

WSDLs for SOAP however allow you to generate servers and clients from datastructures by the click of a button or execution of a script. I can get two servers communicating over SOAP, exposing all service methods in literally minutes.

Basics

SOAP is a protocol for Remote-Procedure Calls. HTTP is used as mechanism to talk between client and servers by sending POST requests with XML request and response bodies.

The SOAPServer and SOAPClient objects ship with PHP core by default.

You can expose functions, classes or objects as server by registering service callbacks with SOAPServer#addFunction, SOAPServer#setClass or SOAPServer#setObject and then calling the handle() method, which handles request and response generation and sending in one step. You can normally exit the request directly after handle.

The client is even simpler, it overwrites the __call magic method. Any call to the client therefore looks exactly like the same call on the server in your code.

PHPs Non-WSDL mode

One argument against SOAP is the requirement to define WSDL documents for both server and client to allow communication. This is not true for Clients and Servers both written in PHP. You can use the non-WSDL mode to expose an object from the server and use a non-wsdl client to talk to it. The PHP SOAPClient and SOAPServer have a common exchange format that is used in this case and replaces WSDL entirely.

<?php
// server.php
class MyService
{
    public function add($x, $y)
    {
        return $x + $y;
    }
}

$options = array(
    'uri' => 'http://server/namespace',
    'location' => 'http://server/location',
);

$server = new SOAPServer(null, $options);
$server->setObject(new MyService());
$server->handle();

The client is as simple as the following lines of code:

<?php
// client.php
$options = array(
    'uri' => 'http://server/namespace',
    'location' => 'http://server/location',
);
$client = new SOAPClient(null, $options);
echo $client->add(10, 10);

This kind of services work with flawlessly all datatypes except with objects, which get converted to stdClass with public properties on the Client.

If you are developing internal APIs between components in your own system, then using SOAP in non-WSDL mode is a massive time saver. You can expose remote services for Remote procedure calls this way very easily.

The only downside is that you have no documentation what methods exist on the client and ALL public methods of the server object can be called, so make sure they only have those methods you want to be called remotely.

Debugging SOAP

When something goes wrong in either the client or server, it sucks to debug these problems. In general there are several mechanisms to debug SOAP in PHP:

  1. Enable 'trace' => true option on the SOAPClient. This allows you to call the methods $client->__getLastResponse() and $client->__getLastRequest() to take a look at the interaction between server and client.
  2. When failures happen, SOAPClient throws a SOAPFault exception. You can even throw this exception yourself from the SOAPServer code, and the client can then read this failure. However you must know that the $faultcode variable in the constructor of new SOAPFault($faultcode, $faultmsg) is NOT an integer error code like in normal Exceptions. Instead its either a value SERVER or CLIENT, with the component of the interaction that failed.
  3. If you throw non SOAPFault exceptions from the server, then you need to catch them and recast them to SOAPFault, otherwise the client only sees “Internal Server Error” messages.

You can easily solve the SOAPFault problem by decorating your service with an exception handler, and also logging the errors yourself.

<?php

class SoapExceptionHandler
{
    private $exposeExceptionMessages = array(
        'MyProject\DomainException',
    );

    private $service;

    public function __construct($service)
    {
        $this->service = $service;
    }

    public function __call($method, $args)
    {
        try {
            return call_user_func_array(
                array($this->service, $method),
                $args
            );
        } catch (\Exception $e) {
            // log errors here as well!
            if (in_array(get_class($e), $this->exposeExceptionMessages)) {
                throw new SOAPFAult('SERVER', $e->getMessage());
            }

            throw new SOAPFault('SERVER', 'Application Error');
        }
    }
}

$server = new SOAPServer(null, $options);
$server->setObject(new SoapExceptionHandler(new MyService()));
$server->handle();

Generating WSDLs

SOAP uses a service description format called WSDL to describe the input and output of the server and what methods exist. WSDL are formatted with XML and use XMLSchema to describe the input/output messages. The format is very complex, however tools for any languages allow you to autogenerate WSDLs from code.

There are several reasons to introduce WSDLs for your SOAP service:

  • Your SOAP clients will not be written in PHP, which prevents use of the non-WSDL mode.
  • Clients of the service are used and written by other teams or companies.
  • You want to use the WSDL as a validation mechanism for input from clients.

While you should have some understanding of how a WSDL looks like, you should never write it manually. I use Zend Frameworks SOAP Autodiscovery for this. By default it uses the docblocks @param and @return to generate the correct WSDL for a service:

<?php
$autodiscover = new Zend\Soap\AutoDiscover();
$autodiscover->setClass('MyService')
             ->setUri('http://server/namespace') // same as server 'uri'
             ->setLocation('http://server/soap.php') // same as server 'location'
             ->setServiceName('MyService');
$wsdl = $autodiscover->generate();
$wsdl->dump("/path/to/file.wsdl");

You can now place that WSDL file in any public location and then point both SOAPServer and SOAPClient at the file using the first constructor argument:

<?php
$server = new SOAPServer('http://server/path/wsdl', $options);
$client = new SOAPClient('http://server/path/wsdl', $options);

To make the WSDL generation work with objects and object graphs, you have to use objects in your service API that have only public properties. If you dont do it this way, you will need to convert the objects in a seperate step, something to avoid.

Sometimes you want to use other metadata than docblocks. When using tools like Doctrine you already now much better what datatypes an object has. You can write your own ComplexTypeStrategy to generate the metadata for your WSDL files. This is more advanced topic, but can be understood and automated in a reasonable amount of time.

Generating Objects from WSDL

If you implement a client, you want to generate objects for the datastructures of a WSDL file. You can use those objects instead of the stdClass objects which are used by default.

For this task I use the XSD-TO-PHP library. I normally hack around in the code a little to adjust for correct namespace generation and code-style adjustments, but it works quite well by default. Here is an example of a generated class for the DHL Intraship SOAP API:

<?php
namespace DHL\Intraship;

class Person extends ComplexType
{
  /**
   *
   * @var salutation $salutation
   * @access public
   */
  public $salutation;

  /**
   *
   * @var title $title
   * @access public
   */
  public $title;

  /**
   *
   * @var firstname $firstname
   * @access public
   */
  public $firstname;

  /**
   *
   * @var middlename $middlename
   * @access public
   */
  public $middlename;

  /**
   *
   * @var lastname $lastname
   * @access public
   */
  public $lastname;
}

The next thing you can generate is a classmap, that maps every WSDL Type to your newly generated code, in the above example:

<?php

$client = new SOAPClient($wsdl, array(
    'classmap' => array(
        'Person' => 'DHL\Intraship\Person',
        // all the other types
    )
));

SOAP with different Languages

As long as you stay within the PHP world, SOAP is rather easy with both WSDL and non-WSDL modes. Once you want to talk to Java or C# you need solve some more problems.

The first thing to understand is that SOAP can actually talk in 4 different modes. You can use ‘document’ or ‘rpc’ style, ‘literal’ or ‘encoded’ use. This post on the IBM website describes all the different modes in much detail and I recommend everybody having to work with SOAP to read it.

The essence from that article is, that you will always want to use document/literal for your SOAP services, to be compliant with all languages, wrapping each method call and response in its own Message Document.

However using this style is rather complicated in PHP itself, because for every input and output message you need to create a wrapper object (or array) with a specific structure.

You can fix this problem on the Server by using this DocumentLiteralWrapper class in Zend Framework 2. It has no external dependencies, so you can just copy it into your project if you want.

To generate a WSDL for document/literal mode, use the following methods on Zend Autodiscovery:

<?php
$autodiscover = new Zend\Soap\AutoDiscover();
$autodiscover->setBindingStyle(array('style' => 'document'))
             ->setOperationStyle(array('use' => 'literal'));

Then use the wrapper like such:

<?php

$server = new SOAPServer($wsdl, $options);
$server->setObject(
    new \Zend\Soap\Server\DocumentLiteralWrapper(
        new SoapExceptionHandler(
            new MyService()
        )
    )
);
$server->handle();

SOAP Servers generated this way can be converted into a C# SOAP Client with a bunch of button clicks from Visual Studio. It will generate both the Client object and all the data transfer objects for you. Truely amazing.

Testing SOAP Interaction

Because SOAP is very painful about the exact format of messages and rejects invalid messages in the client already when they do not match the WSDL you certainly want to Integration test your clients and servers.

You can do that in PHPUnit by using a client, that wraps a Server directly and doesn’t require a Webserver. Zend Framework 2 already has such an object, named ZendSoapClientLocal. Its usage is simple:

<?php

$server = new SOAPServer($wsdl, $options);
$server->setObject(
    new \Zend\Soap\Server\DocumentLiteralWrapper(
        new SoapExceptionHandler(
            new MyService()
        )
    )
);
$client = new \Zend\Soap\Client\Local($server, $wsdl);
$client->add(10, 10);

This will pass through the complete SOAP marshalling and unmarshalling process and allow you test SOAP interaction.

If you want to take a look at the code of the Local client, its very easy to achieve this.

Versioning with SOAP/WSDL

If you want to version your SOAP Service, you will need to provide versioned WSDL files on different URLs. You should never change the WSDL at a location, because languages like C# statically create clients from the WSDL, never talking to the WSDL again.

If you take care of your Service objects, then you can design them in a way that you can use the same PHP service object for many different versions of the WSDL file in a backwards compatible way. If your API changes alot, you might need to implement different PHP service classes to allow for versioned APIs.

Conclusion

While the full extent of SOAP and WSDL can be scary, they allow you to write servers and clients for RPC communication between servers and languages very easily. If you don’t need to expose your API to the webbrowser via REST/JSON, then using SOAP is a very good alternative to most of the handcrafting that is necessary for REST APIs.

]]>
Fri, 31 Jan 2014 00:00:00 +0100
https://beberlei.de/2014/01/26/assert_v2_0__fluent_api_and_lazy_assertions.html https://beberlei.de/2014/01/26/assert_v2_0__fluent_api_and_lazy_assertions.html <![CDATA[Assert v2.0: Fluent API and Lazy Assertions]]> Assert v2.0: Fluent API and Lazy Assertions

Almost two years ago I started developing a small library Assert (blog post) which contained a set of assertion methods to use in production code. With 18.000 installations from Packagist this is my most successful piece of open source software outside of the Doctrine universe. There is a fair number of contributors and I know several companies using the library in production.

The API however didn’t make me too happy, using the static method calls made it impossible to collect multiple errors and also made the code more verbose than necessary when validating the same value with multiple assertions.

Several weeks ago I stumbled across the Java library AssertJ that gave me the idea how to fix these problems. The last two days I had some time to implement those new functionalities and I am releasing a new version 2.0 of Assert today.

Fluent API

Instead of having to use the static assertion methods, there is now a new fluent API, invoked by calling the function Assert\that($value) and then all the assertions you want to call on that value.

Here are some examples:

<?php

\Assert\that(10)->notBlank()->integer()->range(0, 100);
\Assert\that(array('foo', 'bar'))->isArray()->all()->string();
\Assert\that(null)->nullOr()->boolean();

This new API allows for much shorter and compact assertions.

Lazy Assertions

Using Assert with webforms was never possible unless you wanted to show the user only exactly one error message. Because every assertion fails with an Exception, there was no way to execute multiple assertions and collect the errors. This has changed with the new lazy assertion API, that is similar to the Fluent API:

<?php
\Assert\lazy()
    ->that(10, 'foo')->string()
    ->that(null, 'bar')->notEmpty()
    ->that('string', 'baz')->isArray()
    ->verifyNow();

The method that($value, $propertyPath) requires a property path (name), so that you know how to differentiate the errors afterwards.

On failure verifyNow() will throw an exception Assert\\LazyAssertionException (this does not extend AssertionFailedException) with a combined message:

The following 3 assertions failed:
1) foo: Value "10" expected to be string, type integer given.
2) bar: Value "<NULL>" is empty, but non empty value was expected.
3) baz: Value "string" is not an array.

You can also call the method getErrorExceptions() to retrieve all the underyling AssertionFailedException objects and convert them something useable for the frontend.

Error Messages, Values and Constraints

In version 1.0 Assert did not have default error messages when failures occured. This has changed and now every assertion has a default failure message, as well as access to the value and constraints of an exception:

<?php

use Assert\AssertionFailedException;

try {
    \Assert\that(10)->range(100, 1000);
} catch (AssertionFailedException $e) {
    $e->getMessage(); // Value "10" is not between 100 and 1000.
    $e->getValue(); // 10
    $e->getConstraints(); // array('min' => 100, 'max' => 1000)
}

This helps during development when the assertion exceptions occur and also allows to provide error messages to the user, by using the exception code and the value and constraint data for translating into human readable messages.

You can find more information in the README of Assert.

]]>
Sun, 26 Jan 2014 00:00:00 +0100
https://beberlei.de/2013/12/31/2013_and_2014.html https://beberlei.de/2013/12/31/2013_and_2014.html <![CDATA[2013 and 2014]]> 2013 and 2014

This year was amazing with my new job, various talks on conferences and also for this blog. A total of 15 new posts attracted more than 30.000 unique visitors. Most amazing was all the positive feedback that I got at SymfonyCon in Warsaw with regard to my posts and the projects I have been working on in 2013. Thank you all very much for reading and discussing with me. It has been a very insightful year for me!

The three most viewed posts have been on Symfony with Vagrant (5470), Traits (4450) and Doctrine Repositories (3850).

I am always interested in end of the year reading lists to give me an idea what I might want to read the next year. This years reading list of Matthias Meyer of TravisCI provides many ideas for my next years reading list. My own sad list contains only one technical book that I have read this year, but I think you will appreciate the hint if you havent seen this before: Implementing Domain-Driven Design by Vaughn Vernon. That book has influenced many of my blog posts, discussions and experiments this year and you will see effects of it in 2014 as well.

For 2014 I want to keep up the pace of blogging and complement all those ideas with a sample Symfony application that I want to write. The idea for this project spawned from discussions with readers at various events, mostly at the SymfonyCon in Warsaw. My initial plans were to write an ebook on this issue, but various attempts throughout 2013 ended with motivation failures. This is one reason why I have explicitly put the item “Do NOT write a book” in my 2014 todo list.

Most of my blog posts on architecture are very general and even with the practical Symfony and Doctrine posts it is complicated to see the benefits without a good open source sample. That is why I want to write one and discuss the development on this blog.

To support this effort I would be very grateful if you consider supporting me with a regular Gittip donation. Gittip is a fairly new service that allows you to make very small microdonations to others on a weekly basis.

There is also a roadmap of ideas in my mind for 2014 blog posts, specifically about:

  • Doctrine and Domain-Driven Design
  • Hexagonal Architecture
  • Domain Events

I hope to see you again in 2014 and am very eager for the learnings and discussions that the new year will bring.

]]>
Tue, 31 Dec 2013 00:00:00 +0100
https://beberlei.de/2013/12/05/feature_flags_and_doctrine_entities.html https://beberlei.de/2013/12/05/feature_flags_and_doctrine_entities.html <![CDATA[Feature Flags and Doctrine Entities]]> Feature Flags and Doctrine Entities

I was in a discussion with some of the EasyBib developers this week about how to use feature flags with Doctrine Entities, Metadata Mapping and their database.

The problem of feature flags with Doctrine is easily explained: If you add properties for a new feature that is disabled in the Doctrine metadata, then you need to upgrade the database before deployment, even when the feature is not being rolled out for some days/weeks. Doctrine requires the database to look exactly like the metadata specifies it.

You can get around this problem by using the loadClassMetadata event quite easily:

<?php
namespace MyProject\Listener;

use MyProject\Config\FeatureFlagConfiguration;
use Doctrine\ORM\Event\LoadClassMetadataEventArgs;

class FeatureFlagMetadataListener
{
    private $featureFlagConfiguration;

    public function __construct(FeatureFlagConfiguration $config)
    {
        $this->featureFlagConfiguration = $config;
    }

    public function loadClassMetadata(LoadClassMetadataEventArgs $eventArgs)
    {
        $classMetadata = $eventArgs->getClassMetadata();

        if ($this->featureFlagConfiguration->isEnabled('super_feature2')) {

            if ($classMetadata->name === "MyProject\\Entity\\User") {
                $classMetadata->mapField(array(
                    'fieldName' => 'superFeatureProperty',
                    'type' => 'string'
                ));
            }

            if ($classMetadata->name === "MyProject\\Entity\\Something") {
                // more mapping
            }
        }
    }
}

You can find documentation on the ClassMetadata in the documentation or API.

Then if the feature is enabled fore some time and accepted into the pool of permanent features, you can remove the listener and move the mapping onto the Entities metadata.

]]>
Thu, 05 Dec 2013 00:00:00 +0100
https://beberlei.de/2013/11/19/setting_up_development_machines_ansible_edition.html https://beberlei.de/2013/11/19/setting_up_development_machines_ansible_edition.html <![CDATA[Setting up development machines: Ansible edition]]> Setting up development machines: Ansible edition

A little over a year ago I posted on using Puppet for setting up development machines. This approach prooved useful as during that time I did setup a new machine several times and benefited from the mostly automatic installation with Puppet. Github is thinking the same on a bigger scale. Earlier this year they introduced Boxen which is using Puppet to setup mac-development machines.

The space of deployment- and configuration management tools is exploding these last years and it is not always feasible to evaluate all of them. The benefit of Puppet over all these tools is its huge userbase and online resources. However the steep learning curve can be a huge pain coupled with the fact that most of the modules online are not reusable at all and mostly serve a documentation for “how not to work with Puppet”. On top of that Puppet is only for configuration management and not suitable for deployment of applications.

Introduction to Ansible

I came across a blog post on Ansible by the awesome guys at Qandidate.com which lead me to investigate Ansible. It is essentially a hybrid between Capistrano and Puppet, solving both the deployment and configuration management problems with a focus on simplicity. In fact the learning curve of Ansible is much more flat than that of Puppet and you can read the whole documentation and understand even the advanced concepts in a few hours.

Ansible is used from a developer- or continuous-integration-machine, which executes tasks on hosts from an inventory. You only need SSH servers running and private keys to connect to them to get it working. With the inventory of hosts to operate on, you can chose to execute ad-oc commands using the ansible command or playbooks using the ansible-playbook command. Playbooks are files written in YAML.

Read Sanders post on the Qandidate blog to get an introduction to Ansible.

Development-Machine Setup

Using Ansible to setup a development machine is a bit pointless, given that Ansible excels at remote task execution and distribution. The benefit in development-machine setup over Puppet is the simplicity of configuration compared to Puppet modules + DSL.

For demonstration I have converted the puppet examples from my blog post on Puppet to Ansible. To start we have to create a folder and create a new file inventory in it containing our local machine:

localhost   ansible_connection=local

This is necessary to convince the remote task execution part of Ansible that localhost exists and there is no SSH necessary to get into that machine.

Now I can create playbooks for the different tools I might want to install and setup, such as Vim in a vim.yml:

---
- hosts: localhost
  tasks:
    - name: Install VIM
      apt: pkg=vim state=present
      sudo: yes
    - name: Install Dotfiles
      git: >
        repo=git@github.com/beberlei/vim-dotfiles.git
        dest=/home/${ansible_user_id}/.vim
    - name: Create .vimrc Symlink
      file: >
        src=/home/${ansible_user_id}/.vim/vimrc
        dest=/home/${ansible_user_id}/.vimrc
        state=symlink
    - name: Compile and Install Command-T plugin
      command: >
        rake make
        chdir=/home/${ansible_user_id}/.vim/bundle/Command-T
        creates=/home/${ansible_uesr_id}/.vim/bundle/Command-T/command-t.recipe

You can see here, tasks are a list of commands to execute. They are always executed in order and stop when the first task in the chain fails. Like in puppet you try to make those tasks idempotent through flags such as creates, signaling when a task needs to be or was already executed. I can optionally use sudo to run commands such as installing packages.

To execute the playbook I call:

$> ansible-playbook -K -i inventory vim.yml

With the -i flag I define the pool of hosts to run on, in our case the local machine and the playbook applies the hosts filter to that inventory. The -K parameter prompts me to enter the sudo password for my Ubuntu machine, otherwise the tasks will fail.

Conclusion

Compared to Puppet, the configuration management of Ansible simplifies a lot of the assumptions for example on ordering and reusability of tasks. For a part-time “devop” like me this is great news, as I don’t need to understand a complex tool and can focus on solving problems.

On top of that the deployment features of Ansible with parallel remote execution over SSH make Ansible a powerful tool that I hope to use much more in the future.

]]>
Tue, 19 Nov 2013 00:00:00 +0100
https://beberlei.de/2013/09/04/decoupling_from_symfony_security_and_fosuserbundle.html https://beberlei.de/2013/09/04/decoupling_from_symfony_security_and_fosuserbundle.html <![CDATA[Decoupling from Symfony Security and FOSUserBundle]]> Decoupling from Symfony Security and FOSUserBundle

In this blog post I will show how to decouple your core application from the Symfony Security component and User bundles such as the FOSUserBundle.

In my opinion a framework has to be evaluated by how much it allows you to hide it from your actual application. This blog post will add another perspective on how to achieve decoupling from Symfony user and security with a very simple approach. With this puzzle piece and others I wrote about before (Controllers as Service, Param Converters) and other pieces I still need to write about, I would classify Symfony2 as a good framework.

A number of other authors have written blog posts on decoupling Symfony and your model code as well, such as William Durand on project structure , Matthias Verraes on project structure as well and on Decoupling Forms from Entities.

User management breaks the isolation of your business model open, by introducing the UserInterface from the Security component into your code base. In combination with the FOSUserBundle this can cause a dependency on quite a bit of code already because I wouldn’t call the FOS\UserBundle\Model\User object leightweight. However this and other FriendsOfSymfony bundles are very helpful to get annoying features of your application done in a matter of hours.

You can decouple the FOSUserBundle from our own entities and code by introducing a second object that representes the user concept in our model. The UserInterface entities need fields for username, password, enabled and some more. Except the username and email, the business model rarely needs other properties.

Take an ecommerce system as example, where customers can register as users. In this example we could create two entities MyProject\UserBundle\Entity\User and MyProject\ShopBundle\Entity\Customer. Two strategies exist now:

  1. Use the same table for both entities with some shared and other exclusive columns
  2. Use different tables in the database

I haven’t tried the first option yet, so I cannot say much about the feasibility.

For case two, the User entity looks like this:

<?php
// src/MyProject/UserBundle/Entity/User.php;

namespace MyProject\UserBundle\Entity;

use FOS\UserBundle\Model\User as BaseUser;
use Doctrine\ORM\Mapping as ORM;

/**
 * @ORM\Entity
 * @ORM\Table(name="fos_user")
 */
class User extends BaseUser
{
    /**
     * @ORM\Id
     * @ORM\Column(type="integer")
     * @ORM\GeneratedValue(strategy="AUTO")
     */
    protected $id;
}

The Customer entity is a completly new object, not depending on the UserInterface and containing some duplicated properties.

You can either use exactly the same ID or plant the $userId as correlation ID in the Customer object.

<?php
// src/MyProject/ShopBundle/Entity/Customer.php

namespace MyProject\ShopBundle\Entity;

use Doctrine\ORM\Mapping as ORM;

/**
 * @ORM\Entity
 * @ORM\Table(name="customer")
 */
class Customer
{
    /**
     * @ORM\Id
     * @ORM\Column(type="integer")
     * @ORM\GeneratedValue(strategy="NONE")
     */
    protected $id;

    /**
     * @ORM\Column(type="string")
     */
    protected $username;
}

Now you need to extend the application to synchronize all changes from the User entity to the Customer entity. You can achieve this relatively easy by overwriting the ModelManager or when finally supported by FOSUserBundle listening to events. Your own code can exclusively work with Customer objects now, void of any reference to the Symfony Security component or the huge FOSUserBundle dependency.

]]>
Wed, 04 Sep 2013 00:00:00 +0200
https://beberlei.de/2013/08/19/speedup_symfony2_on_vagrant_boxes.html https://beberlei.de/2013/08/19/speedup_symfony2_on_vagrant_boxes.html <![CDATA[Speedup Symfony2 on Vagrant boxes]]> Speedup Symfony2 on Vagrant boxes
  • Update: Added requirements to configure Vagrant and Virtualbox before trying my change.
  • Update 2: Added link to Virtualbox Guest Additions plugin for Vagrant, thanks Peter Kruithof for the hint.

Using Symfony2 inside a Vagrant box is considered to be very slow (here, here), even when using NFS. I can confirm the experience and know many others that reported the same.

Before doing this changes make sure:

  • You are using Vagrant 1.2 (not sure that makes a difference though)
  • You are using NFS with Vagrant (More). If you are on Windows then this can already be the problem (Virtualbox FS is slow with Symfony, because of the large number of files.)
  • You have the Vbox Guest Additions on the guest that match your system (Vagrant Plugin, Manual Update)
  • You have an opcode cache installed, either apc or opcache.
  • You disable xdebug or xhprof.

Without looking at an actual benchmark (mistake!) I always considered the problem to be related to the huge number of files that a Symfony project normally ships with and the I/O that is generated from filemtime calls to check if the configuration files have changed.

This is just a small part, there are other bottlenecks that I have found after finally doing some benchmarking with XHProf, leading to a simple fix:

  1. Monolog logs to the NFS share and the huge number fwrite take their toll.
  2. Writing compiled Twig templates to the NFS share using file_put_contents.
  3. Assetic: Scanning for stylesheets and javascripts within templates is very slow, causing lots of I/O on the NFS share and CPU and changing it to using explicit bundling in the app/config/config.yml helps alot. You can use a cronjob deployed by Puppet/Chef that invokes the assetic:dump command with a –watch flag in the background.
  4. JMSDiExtraBundle scans for Services to check for changes on service objects.
  5. ReplaceAliasWithActualDefinitionPass in Symfony DIC uses a set of recursive algorithms to replace aliases with the real services. That takes a huge amount of time in bigger applications including amounting calls to methods inside the pass over 100.000 times.

The slowest bottlenecks listed (1-4) are I/O bound, directly related to NFS. To fix this just change the cache AND the log directory to sys_get_temp_dir() or shared memory (/dev/shm) instead of writing to the NFS share. I actually tried this before, but since I forgot the log directory, this felt equally slow and I reverted the change.

Here is the code you should add to your AppKernel to give you a considerable performance boost on a Vagrant box:

<?php

class AppKernel extends Kernel
{
    // ...

    public function getCacheDir()
    {
        if (in_array($this->environment, array('dev', 'test'))) {
            return '/dev/shm/appname/cache/' .  $this->environment;
        }

        return parent::getCacheDir();
    }

    public function getLogDir()
    {
        if (in_array($this->environment, array('dev', 'test'))) {
            return '/dev/shm/appname/logs';
        }

        return parent::getLogDir();
    }
}

This brings the page rendering speeds down to between 0.5 and 1.5 seconds, which is quite normal for the development environment even outside a virtual machine.

This tip obviously works for any kind of application that has a I/O intensive development environment: Don’t perform this operations on the NFS share unless you wan’t to suffer.

]]>
Mon, 19 Aug 2013 00:00:00 +0200
https://beberlei.de/2013/08/09/your_service_layer_benefits_from_a_dispatcher.html https://beberlei.de/2013/08/09/your_service_layer_benefits_from_a_dispatcher.html <![CDATA[Your Service Layer benefits from a Dispatcher]]> Your Service Layer benefits from a Dispatcher

One benefit of recent applications over old 90s, early 2000 style PHP is central dispatching of requests through one Application or Dispatcher object. This is true for dispatching objects such as PHPs SoapServer or Zend\XmlRpc\Server as well. Old style applications use the webserver (Apache, Nginx) for the dispatching of routes to executed PHP scripts.

Central dispatching allows developers to hook into central events during the request processing to implement security, routing, caching, debugging, logging, localization and many things more transparently.

If you check this list of features, these are all cross-cutting concerns to the real purpose of your application and nicely abstracted from the core of your application using central dispatching.

When building a future proof system focussing on the domain and starting test-driven without dependency on a framework and UI, we often use the Service Layer Pattern to decouple our business logic from technical implementation details. The service layer however is affected by cross-cutting concerns again and this blog-post is about finding a maintainable solution to handle them.

Service Layer without Dispatching

As a practical example I will pick the classical user registration example, starting with a service-layer that implements the cross-cutting concerns inside the service method:

<?php

class UserService
{
    public function create($email, $password)
    {
        $this->security->assertNotLoggedInAlready();

        $user = new User();
        $user->setEmail($email);
        $user->encodePassword($password);

        $violations = $this->validator->validate($user);

        if (count($violations) > 0) {
            throw new \IllegalArgumentException(((string)$violations);
        }

        $this->entityManager->persist($user);
        $this->entityManager->flush();

        $message = "Thanks for registering! Please click here: ...";
        $this->mailer->sendMessage($email, $message);

        $this->logger->info("New user registration: " . $email);
    }
}

This is quickly getting out of hands with the number of services needed and you will not have fun testing this for example. This approach violates the Single-Responsibility-Princple (SRP) doing way too much non-related operations. Think about having to do this in every method of your service layer and you can already see this leads to lots of spagetti-code.

Using the Decorator Pattern

We could use the decorator pattern to extract some of the cross-cutting concerns into their own services. This requires to introduce an interface for UserService and then building decorators for the different concerns. Then the service can be built as a decorator chain:

<?php

$userService =
    new LogUserService(
        new SecurityUserService(
            new DoctrineTransactionUserService(
                new UserService(), ...
            ), ...
        ), ...
    );

The real UserService will slimmer and more closely handling just the business logic related information. Using decorators requires us to write the same delegation code for every method on every service, introducing many decorators and interfaces that duplicate lots of code. Something we want to avoid to keep our own sanity.

Introducing a Dispatcher

Lets refactor from decorator towards a dispatching approach instead, to keep more reusable. The CommandDispatcher object accepts a service name and method and calls it. We can wrap every call with cross-cutting concerns.

Starting with a very simple dispatcher possible that just delegates the calls to registered services, we add all the cross-cutting concerns our application needs. Note: This is deliberatly written without further abstraction, just to show the concept. The real thing would propably seperate the responsibilities from each other.

<?php

class CommandDispatcher
{
    private $services;

    public function registerService($serviceName, $service)
    {
        $this->services[$serviceName] = $service;
    }

    public function execute($serviceName, $method, array $params)
    {
        $service = $this->services[$serviceName]; // make lazy
        $callback = array($service, $method);

        if ($serviceName === "user" && $method === "create") {
            $this->assertNotLoggedInAlready();
        }

        $this->entityManager->beginTransaction();

        try {
            $result = call_user_func_array($callback, $params);
            $this->entityManager->commit();

            $this->mailer->sendQueuedMails(); // "deferred commit" of mails
            $this->logger->info("Called $serviceName.$method");

        } catch (\Exception $e) {
            $this->entityManager->rollBack();
            throw $e;
        }

        return $result;
    }
}

The dispatcher handles transactions around all the commands and also makes sure that when they send emails, those only get send when the transaction was successful. It checks if the user has the correct access controls/authentication and performs some generic logging.

And using the dispatcher in your code looks like this:

<?php
$dispatcher = new CommandDispatcher();
$dispatcher->registerService('user', new UserService());

$dispatcher->execute('user', 'create', array($email, $password));

Like the front controller in MVC or PHPs SOAPServer you register services/functions with the dispatcher. Registration of services can be done by convention, via some DependencyInjection Container Service name or any other way you prefer. The dispatcher then handles ALL commands by wrapping them inside some generic logic.

Compared to the Decorator approach, you can now easily reuse this code with many commands. Except registering new services, no new code is necessary when adding a new method or service.

A better API for the Dispatcher

So far the API of the dispatcher is tedious, so lets work a little bit on how you actually call methods on the service-layer.

There are two ways to make this call nicer. The first is use magic __call and some clever duck-typing to create an API similar to this:

<?php

$dispatcher = new CommandDispatcher();
$dispatcher->registerService('user', new UserService());

$dispatcher->user()->create($email, $password);

The second approach does not require magic __call, but requires you to write a class for each command. We map the command class name to a callback:

<?php

$userService = new UserService();

$dispatcher = new CommandDispatcher();
$dispatcher->registerCommand('CreateUserCommand', array($userService, 'create'));

$dispatcher->handle(new CreateUserCommand($email, $password));

The naming is very techincal here, but since the dispatcher also acts as a facade to the application, we could give it better names like PayrollApplication, Shop, TrackingSystem, any name the application has inside your organization.

Discussion

Now that I have shown the implementation of a dispatcher a small discussion is necessary to evaluate it. The cross-cutting concerns could be nicely wrapped in the dispatcher, so we achieved a considerable improvement over the first example with all the concerns nicely seperated from each other.

The benefits are:

  • Services themself don’t need access to the cross-cutting concerns anymore, reducing the number of dependencies and increasing maintainability and testability.
  • Handling cross-cutting concerns, that can make the service layer code very complex otherwise, in a clean way
  • All the concerns are easily composable and the result is a SOLID approach towards them.
  • The dispatcher also allows us to add or remove concerns later at one central location without having to change all the service layer code.
  • The framework we use can be very simple as long it fullfils the major requirement to be easily compatible to the dispatcher approach.
  • Compared to an MVC frameworks dispatcher, our service dispatcher can tailored to the application itself and end up being very small and easily understandable.

How do we use this dispatcher in our MVC framework though? Instead of using controllers/actions a REST or SOAP API could just use the dispatching and services directly and map the HTTP request to it based on convention. This would be a real win and simplify the framework-glue code considerably.

In a web-application however this is not so simple. We need to send redirects, manage session state and handle request and response data, which often requires one specific controller-action for each command. With some experimentation it might be possible to achieve a much higher re-use here, but it might fail as well.

That brings us to the downside of the dispatcher approach:

  • We need some additional code and extra classes, which might be too much for small applications and the indirection of handling cross-cutting concerns might confuse teammates.
  • Having the dispatcher object inside controllers feels strange from the MVC point of view, it doesn’t really fit. It also still may require implementing one action for every command, not simplifying this part of the development.
  • While other languages don’t need this because of their support for AOP and annotations (Spring for Java for example) this is necessary in PHP only, because we don’t have this features.
  • Unless we use the explicit command object approach, there is no auto-completion for commands on the dispatcher in the IDEs.

My conclusion from working with both kind of service layers: If you decided for such a service layer, then my experience shows it is a mistake not to use a dispatcher, because the benefits outweigh the downsides.

]]>
Fri, 09 Aug 2013 00:00:00 +0200
https://beberlei.de/2013/07/24/doctrine_and_domainevents.html https://beberlei.de/2013/07/24/doctrine_and_domainevents.html <![CDATA[Doctrine and Domain Events]]> Doctrine and Domain Events

I have written about the Domain Event Pattern before and want to focus on this pattern again by explaining its integration into the Doctrine ORM on a very technical level.

The Domain Event Pattern allows to attach events to entities and dispatch them to event listeners only when the transaction of the entity was successfully executed. This has several benefits over traditional event dispatching approaches:

  • Puts focus on the behavior in the domain and what changes the domain triggers.
  • Promotes decoupling in a very simple way
  • No reference to the event dispatcher and all the listeners required except in the Doctrine UnitOfWork.
  • No need to use unexplicit Doctrine Lifecycle events that are triggered on all update operations.

This blog post will introduce a very simple implementation for Domain Events with Doctrine2. You should be able to easily extend it to be more flexible, reliable or optionally even asynchronuous. I skip some of the glue code in this blog post, but you can try the full code by checking out this Gist and running composer install and then php domain_events.php.

Lets look at the following entity, an InventoryItem tracking the number we have this item in stock:

<?php
namespace MyProject\Domain;

use Doctrine\ORM\Mapping as ORM;
use MyProject\DomainSuperTypes\AggregateRoot;

/**
 * @Entity
 */
class InventoryItem extends AggregateRoot
{
    /**
     * @Id @GeneratedValue @Column(type="integer")
     */
    private $id;
    /**
     * @Column
     */
    private $name;
    /**
     * @Column(type="integer")
     */
    private $counter = 0;

    public function __construct($name)
    {
        $this->name = $name;
        $this->raise('InventoryItemCreated', array('name' => $name));
    }

    public function checkIn($count)
    {
        $this->counter += $count;
        $this->raise('ItemsCheckedIntoInventory', array('count' => $count));
    }
}

We want this entity to raise two domain events using the raise($eventName, array $properties) method. Our preferred use case for this code looks like this:

<?php
$item = new InventoryItem('Cookies');
$item->checkIn(10);

$entityManager->persist($item);
$entityManager->flush(); // fire events here

One or many listeners should react to the events being fired, for example print their contents to the screen:

<?php
class EchoInventoryListener
{
    public function onInventoryItemCreated($event)
    {
        printf("New item created with name %s\n", $event->name);
    }

    public function onItemsCheckedIntoInventory($event)
    {
        printf("There were %d new items checked into inventory\n", $event->count);
    }
}

As the first building block, we need a Layer Supertype for our entities, called AggregateRoot adding the event raising capabilities to all entities that need it:

<?php
namespace MyProject\DomainSuperTypes;

abstract class AggregateRoot
{
    private $events = array();

    public function popEvents()
    {
        $events = $this->events;
        $this->events = array();

        return $events;
    }

    protected function raise($eventName, array $properties)
    {
        $this->events[] = new DomainEvent($eventName, $properties);
    }
}

This class allows us to add events to an entity using raise() as we have seen in the InventoryItem entity before.

Now we need Doctrine to process the events during a transaction (flush()). We do this by keeping all entities with domain events, and then triggering the domain events after the transaction has completed. This is a vital part of the domain events pattern, because it guarantees every listener that the state leading to the event is already in the database.

Technically we implement this with a Doctrine EventListener that listeners for the postFlush, postPersist, postUpdate and postRemove events:

<?php
namespace MyProject\DomainEvents;

use Doctrine\ORM\EntityManager;
use MyProject\DomainSuperTypes\AggregateRoot;

class DomainEventListener
{
    private $entities = array();

    public function postPersist($event)
    {
        $this->keepAggregateRoots($event);
    }

    public function postUpdate($event)
    {
        $this->keepAggregateRoots($event);
    }

    public function postRemove($event)
    {
        $this->keepAggregateRoots($event);
    }

    public function postFlush($event)
    {
        $entityManager = $event->getEntityManager();
        $evm = $entityManager->getEventManager();

        foreach ($this->entities as $entity) {
            $class = $entityManager->getClassMetadata(get_class($entity));

            foreach ($entity->popEvents() as $event) {
                $event->setAggregate($class->name, $class->getSingleIdReflectionProperty()->getValue($entity));
                $evm->dispatchEvent("on" . $event->getName(), $event);
            }
        }
        $this->entities = array();
    }

    private function keepAggregateRoots($event)
    {
        $entity = $event->getEntity();

        if (!($entity instanceof AggregateRoot)) {
            return;
        }

        $this->entities[] = $entity;
    }
}

Now we only need to put this code together with Doctrine to get it working:

<?php
$evm = new EventManager();
$evm->addEventListener(
    array('postInsert', 'postUpdate', 'postRemove', 'postFlush'),
    new DomainEventListener()
);
$evm->addEventListener(
    array('onInventoryItemCreated', 'onItemsCheckedIntoInventory'),
    new EchoInventoryListener
);
$conn = array(/* data */);
$config = new Configuration();
$entityManager = EntityManager::create($conn, $config, $evm);

Now the example from above, creating and checking in inventory items will trigger the EchoInventoryListener and print the data to the screen.

This example is very simple but shows how you can incorporate Domain Events into your Doctrine projects. The following improvements can be made if necessary:

  • Use a base EventSubscriber for the Domain Listeners, that automatically registers all the domain event listener methods. This way you don’t have to manually list them when calling $evm->addListener().
  • Implement one class for every domain event, allowing to typehint the events in listeners, with much more helpful information about the contained data.
  • If you prefer asynchroneous processing, serialize the events into a domain_events table instead of triggering the events directly. Add a daemon to your project that tails the table and triggers the events in the background.

I hope this blog post helps you when considering to use Domain Events Pattern with Doctrine. Again, see the code on this Gist for a working example.

]]>
Wed, 24 Jul 2013 00:00:00 +0200
https://beberlei.de/2013/06/27/extending_symfony2__controller_utilities.html https://beberlei.de/2013/06/27/extending_symfony2__controller_utilities.html <![CDATA[Extending Symfony2: Controller Utilities]]> Extending Symfony2: Controller Utilities

Controllers as a service are a heated topic in the Symfony world. Developers mainly choose to extend the base class, because its much simpler to use and less to write. But less to write is not necessarily true.

The problem: Injecting tons of services

The Symfony controller base class uses quite a lot of services, if you need them in your controller as a service, you have to inject them:

<?php
class UserController
{
    public function _construct(
        RouterInterface $router,
        EngineInterface $engine,
        HttpKernel $kernel,
        FormFactoryInterface $formFactory,
        SecurityContextInterface $security
    )
    {
        $this->router = $router;
        $this->engine = $engine;
        $this->kernel = $kernel;
        $this->formFactory = $formFactory;
        $this->security = $security;
    }
}

There are some problems here:

  • The services are not loaded lazily, we probably end up with a resource overuse here.
  • The constructor is MUCH too big, this doesn’t even include your own application, database or mailer services.

The quick and dirty solution

You could as a first solution, inject the container into your controller. But this is actually the same as just using the base controller from Symfony or just use the ContainerAware interface in a controller of your own.

The solution: Introduce a Utilities Service

To avoid the mentioned problems with controller as a service, introduce a controller utilities service and register it as controller_utils service:

<?php

class ControllerUtils
{
    private $container;

    public function __construct($container)
    {
        $this->container = $container;
    }

    public function createForm($type, $data, $options)
    {
        // ..
    }

    public function render($template, $parameters)
    {
        // ..
    }
    // ...
}

You can implement this by just copying the over from Symfony\Bundle\FrameworkBundle\Controller\Controller all the methods you need.

Then you can simplify the controller as a service:

<?php

class UserController
{
    private $utils;

    public function _construct(ControllerUtils $utils)
    {
        $this->utils = $utils;
    }
}

You don’t have to stop here. You can introduce lots of helper objects and inject them into your controller, depending on your use-cases. Generic handling of file uploads comes to mind for example.

From medium to large applications this can lead to much smaller controllers, because they can much more easily reuse code between each other than in the case of inheritance.

]]>
Thu, 27 Jun 2013 00:00:00 +0200
https://beberlei.de/2013/06/24/bounded_contexts.html https://beberlei.de/2013/06/24/bounded_contexts.html <![CDATA[Bounded Contexts]]> Bounded Contexts

I regularly fall into the same trap when building applications: Trying to find the one model that unifies all the use-cases and allows to solve every problem. From discussions with other developers I know not to be the only one making this mistake.

It is propably a human trait trying to find the one line that connects every dot. However in software design there is a case to be made for the separation of contexts to tackle complexity. Eric Evans described this in detail with the “Bounded Context” pattern in his “Domain-Driven Design” book. According to an interview with Evans I read somewhere, he considers it one of the most important patterns in the whole book.

The essence of this strategic pattern is to embrance the separation of different parts of an application and develop different models as different domain contexts. Consistency and constraints are enforced within one context, but don’t necessarily hold in another context.

The goal is simplication and freeing us from the burden of finding that one model that fits all use cases. It is much less awkward to have some parts of the model reimplemented for different purposes than creating God objects that try to unify everything, and failing at this task.

Evans goes even further and introduces a significant amount of patterns that describe the relationship between different bounded contexts.

An Experiment

At the SoCraTes conference in August last year, I participated in a DDD architecture game by Cyrille Martraire that focused on distilling bounded contexts. We were given a very simple domain concept “Customer” and assigned different roles in the business.

Everyone described their perfect customer very differently, imagining the Customer object that fits all these requirements was quite an eye opener. Even if there is no such single software that operates a business, going to the extreme and defining 10 different angles for a single concept made me clear, that sometimes different overlapping implementations are not such a bad thing.

Problems and Solutions

One problem when applying bounded contexts is data synchronization. Synchronization between contexts can be reached in many different ways, three of which are:

  1. The Domain Event pattern that integrates nicelly into Domain-Driven Design and simplifies the synchronization with other parts of an application.
  2. Correlation Ids of the same entity into the different contexts and a clean seperation of the data (no shared data necessary).
  3. Another way are immutable read models (value objects) of the data managed in another context (entity).

In any way relying on Eventual Consistency helps solving the synchronization problem.

Depending on how your bounded contexts seperate from each other, duplication of classes is necessary as well. This is something that has to be accounted for and you should make sure that only one context is responsible for changing the same data. Different contexts can have different behaviors on the same data. This is actually a point where Domain-Driven Design and Data, Context and Interaction (DCI) intersect.

Conclusion

Applying bounded contexts to an application or a cluster of applications helps developers and domain experts to focus on problems in their specific context. Sometimes each context even has its own domain-expert. The benefit for developers is a simplified abstraction of different parts of the modelled problem.

]]>
Mon, 24 Jun 2013 00:00:00 +0200
https://beberlei.de/2013/05/23/symfony2_on_windows_azure_websites.html https://beberlei.de/2013/05/23/symfony2_on_windows_azure_websites.html <![CDATA[Symfony2 on Windows Azure Websites]]> Symfony2 on Windows Azure Websites

With the Windows Azure Websites platform-as-a-service (PaaS), which was released as a preview to a general audiance in June last year, PHP applications can be deployed to Azure with as much as a Git push to a remote repository. At the same time Microsoft released a new and much improved PHP SDK, which integrates with all the Azure webservices.

I blogged about deployment with Composer on Azure Websites last November and since then I have been working with Microsoft to improve Symfony2 support on Windows Azure by developing and releasing the AzureDistributionBundle. This is a new version of the bundle and contains the following improvements:

  • Full support for the PHP SDK through the DependencyInjection container
  • Installation of project dependendencies using Composer during deployment. I will blog about the Composer support in much more detail in the next days, because this information is not only valuable for Symfony2 projects.
  • a full demo application (derived from symfony-standard) to show PHP, Symfony & Azure integration
  • documentation of the Windows Azure Websites deployment process and possiblities for PHP configuration
  • a Stream Wrapper for Azure Blob Storage, currently being reviewed for inclusion into the Azure PHP SDK itself.

You can install both the Bundle or the Demo application via Composer. For the Demo application call:

$ php composer.phar create-project beberlei/symfony-azure-edition

The README includes more information about how to deploy the demo to your Azure websites account.

The bundle can be installed using the following composer.json:

{
    "require": {
        "beberlei/azure-distribution-bundle": "*"
    },
    "repositories": [
        {
            "type": "pear",
            "url": "http://pear.php.net"
        }
    ],
}

Registering PEAR is necessary, because the Azure PHP SDK depends on some PEAR components.

]]>
Thu, 23 May 2013 00:00:00 +0200
https://beberlei.de/2013/04/12/traits_are_static_access.html https://beberlei.de/2013/04/12/traits_are_static_access.html <![CDATA[Traits are Static Access]]> Traits are Static Access

In a Twitter discussion yesterday I formulated my negative opinion about traits and Matthew asked me to clarify it:

https://beberlei.de/_static/traits_are_static_access.png

I used to look forward to traits as a feature in PHP 5.4, but after discussions with Kore I came to the conclusion that traits are nothing else than static access in disguise. They actually lead to the exact same code smells. Familiar with the outcome of too much static use, we should reject traits as just another way of statically coupling your code to other classes.

Let’s step back and take a look at the code smells that Static code produces:

  • Tight coupling, no way to exchange at runtime
  • Not mockable
  • Static code cannot be overwritten through inheritance
  • Global state (increases likelihood of unwanted side effects)

This blog post shows that Traits actually have the first three problems themselves and exhibit the same code smells. But they even have some additional problems on their own:

  • Not testable in isolation
  • Theoretically stateless, but not enforced in PHP
  • Very high impact on the code base (Code-Rank)

Take the following code, which tries to implement reusable controller code through traits:

<?php
class MyController
{
    use Redirector;

    protected $container;

    public function __construct($container)
    {
        $this->container = $container;
    }

    public function someAction()
    {
        return $this->redirect("route_xyz", array());
    }
}

trait Redirector
{
    public function redirect($routeName, $parameters)
    {
        return new RedirectResponse(
            $this->generateUrl($routeName, $parameters)
        );
    }

    public function generateUrl($routeName, $parameters)
    {
        return $this->container->get('router')->generateUrl(
            $routeName,
            $parameters
        );
    }
}

Lets spot the problems:

  1. Redirector is tightly coupled to MyController. There is no way to change this during runtime, by using a different type of redirector, it has to be exactly the trait used at compile time. This is exactly what static access enforces as well.

  2. The trait accesses $this->container without defining it and couples itself against the implementing classes. We can actually refactor this to include an abstract method getContainer() in the trait. But if we have multiple traits now, all having a getContainer() method then we run into method class problems that cannot be solved. We could pass the container as an argument to the method, but that actually defeats the purpose of the abstraction here.

    Traits using state of their “parents” usually create bidirectional coupling between classes, something which should be avoided for good software design.

  3. No way to overwrite subset of functionality. If I want to use only one method of a trait and a second one slighty different, then I cannot overwrite this function of Redirector for example in MyController.

  4. I cannot mock the traits functionality, therefore I cannot test MyController as a unit only in combination with a trait.

  5. I cannot test the trait as a unit, I always have to create a class that uses the trait to be able to write a test for it. This prevents me from testing traits in isolation.

  6. Once you start using Redirector in many controllers, its impact on your code base (see Code Rank) increases a lot. Traits are concrete implementations and therefore violate the Dependency Inversion SOLID principle: Changes in the trait require adoptions in all the implementing classes.

    With aggregation you could depend on an abstraction Redirector or turn it into an abstraction in the moment that you need different functionality.

The discovery of this properties of traits me to the conclusion that traits are just static access in disguise.

To see this argument a bit more drastically, you can “rewrite” a PHP 5.4 trait into “pseudo” static code:

<?php
class MyController
{
    public $container;

    public function __construct($container)
    {
        $this->container = $container;
    }

    public function someAction()
    {
        return Redirector::redirect("route_xyz", array());
    }
}

class Redirector
{
    public function redirect($routeName, $parameters)
    {
        return new RedirectResponse(
            self::generateUrl($routeName, $parameters)
        );
    }

    public function generateUrl($routeName, $parameters)
    {
        return $this->container->get('router')->generateUrl(
            $routeName,
            $parameters
        );
    }
}

Calling dynamic methods statically actually works right now (and access to $this of the parent class will luckily be removed in PHP 5.5). Let’s reformulate it into something that is actually using static methods and will work on 5.5, requires changes to the visibility of properties though:

<?php
class MyController
{
    public $container;

    public function __construct($container)
    {
        $this->container = $container;
    }

    public function someAction()
    {
        return Redirector::redirect($this, "route_xyz", array());
    }
}

class Redirector
{
    public static function redirect($thiz, $routeName, $parameters)
    {
        return new RedirectResponse(
            self::generateUrl($thiz, $routeName, $parameters)
        );
    }

    public static function generateUrl($thiz, $routeName, $parameters)
    {
        return $thiz->container->get('router')->generateUrl(
            $routeName,
            $parameters
        );
    }
}

Can you see the familiarity? If Traits can be rewritten as calls to static methods, how can they be any better than static methods? They exhibit the exact same problems and produce the same code smells.

Conclusion: Traits should be avoided at all costs, just like static methods.

Rule of Thumb: If you want to use a trait, try to think how to solve the problem with aggregation.

If you want to read more about problems with traits, Anthony wrote about them quite a while ago.

]]>
Fri, 12 Apr 2013 00:00:00 +0200
https://beberlei.de/2013/04/06/a_failed_side_project.html https://beberlei.de/2013/04/06/a_failed_side_project.html <![CDATA[Lessons learned: A failed side project]]> Lessons learned: A failed side project

Side projects have always been an important part of how I learn new skills and technologies. This usually ends with me dumping some prototype on Github, sometimes even with an open source project that I commit to maintain over a long time.

Two years ago, for the first time, I pursued a project idea which should not be open-source but a commercial SAAS product. It grew out of the Doctrine teams desire to synchronize Github Pull Requests with our Jira instance: A workflow engine for HTTP services. If this, then that (IFTTT) already existed at that point, but was far too limited for my use-case. I wanted something to allow:

  • Accept a Github Pull Request Event
  • Open a Jira Ticket
  • Comment back on the Pull Request with a link to the Pull Request
  • Optionally mention that the Pull Request should be made to master, instead of $BRANCH

I started developing this in my free time based on PHP, Symfony2, CouchDB, PostgreSQL and Backbone.js and got a prototype working early. However instead of reaching the state where I could release the project to others I hit some hurdles:

  1. The project had a UI where you could add/remove and reconnect tasks through a graphical workflow editor. The javascript became very messy fast, because I didn’t understand Backbone fully and also didn’t know patterns for decoupling and testing Javascript code.
  2. I had to restart with the core domain service side code, because I based it on the Zeta Components workflow library and it was too unflexible for my special requirements, messing the code up.
  3. I wanted to implement way too many features and had a huge backlog of issues to implement before “beta”.

Last year in July, when I finally had something remotely usable Zapier hit the market with a beautiful product and support for gazillions of services. At that point my service just had support for Github, Twitter and Jira and for generic HTTP POST requests and a UI that could not be operated by anyone else but me. I was quite demotivated that day I found out about Zapier.

Nevertheless I continued to work on the project and tried to make it even more powerful than Zapier, by introducing more features that neither IFTTT nor Zapier had. Adding this complexity ended up being the nail into the coffin of the project.

Each week I worked less and less on the project, because I couldn’t find the motivation. When Zapier got funded in October I was both sad and happy: Apparently my idea was a good one, however somebody else executed this idea much better than myself. I stopped working on the project that week.

Today I took the day to migrate the Doctrine Pull Request workflow to a simple hardcoded application and disabled my projects website entirely.

I want to use this moment to share my personal lessons learned:

  • Choosing a scope that allows you to finish a working prototype within 1-2 months is an important key to success.

    The longer it takes to get something working and useable for others, the less motivation you have. I know from my open-source experience that people using your project is a huge motivation boost.

    The side projects I started since last October are much smaller in scope.

  • You can actually get burned out by a side project: by designing its scope way too big.

  • You cannot compete feature-wise with startups that put their full attention into a project from morning to night. Either make something small working better than commercial products or quit your job and put your full time into this.

  • Side projects are either about learning new technologies or about trying to build something commercially successful. Don’t try to combine this or you might get frustrated by choosing the wrong technology for the job.

  • Never ever use an existing open-source library as the core of your complicated business domain. If your domain is something remotely interesting you will fail to achieve your goals with the restrictions of the library.

  • Starting a big project alone is not a good idea. I found out that discussing ideas with people is very valuable and at the point where I started sharing my idea with others I was already too far into the project to be able to take most of the advice into account.

  • Keeping a project idea secret is completely useless. Others will come up with the same idea regardless. People have ideas all the day, however nobody ever has time to implement them. When they do, execution is even more important than the idea itself.

What are your lessons learned from failed side projects? I would be happy to hear other stories.

]]>
Sat, 06 Apr 2013 00:00:00 +0200
https://beberlei.de/2013/03/12/code_coffee_bonn.html https://beberlei.de/2013/03/12/code_coffee_bonn.html <![CDATA[Code & Coffee Bonn]]> Code & Coffee Bonn

On last years SoCraTes Conference I had the pleasure to meet Sandro Mancuso from the London Software Craftsmanship Community (LSCC). Following his Twitter account I stumbled upon a meetup concept called “Code & Coffee” which the LSCC organizes very regularly. At a “Code & Coffee” developers would meet up early in the morning before work to discuss and hack at some coffee place.

I liked the idea very much and started such an event in Bonn. Our first test-event was a success, so that we now open up the concept to anyone who wants to join.

There is a Google+ Community Code & Coffee Bonn where you can join and get regular invites to the events.

Our next event is on March 26th. For more details see the Google+ page.

Looking forward to seeing you there!

]]>
Tue, 12 Mar 2013 00:00:00 +0100
https://beberlei.de/2013/03/04/doctrine_repositories.html https://beberlei.de/2013/03/04/doctrine_repositories.html <![CDATA[On Taming Repository Classes in Doctrine]]> On Taming Repository Classes in Doctrine

Over at the easybib/dev Blog Anne posted an entry about their usage of Doctrine Repositories with a growing amount of query responsibilities. I want to respond to this blog post with two alternative approaches, because I have seen the easybib approach multiple times in different projects by different teams and think it can be approved upon a lot.

The problems with the approach outlined are:

  • The Repository API does not hide implementation details of the ORM, the QueryBuilder API is returned to the client code. This might seen like nitpicking, however it leads to bloated client code doing the query builder work over and over again. For example the ->getQuery()->getSingleResult(AbstractQuery::HYDRATE_ARRAY) call.

  • Different parts of the QueryBuilder filtering cannot be composed together, because of the way the API is created. Assume we have the filterGroupsForApi() call, there is no way to combine it with another call filterGroupsForPermissions(). Instead reusing this code will lead to a third method filterGroupsForApiAndPermissions().

    This can lead to combinatorial explosion of methods that the developer using the repository needs to know. And wading through a list of 100 methods to find the right one is never fun, most importantly when the naming of methods is imprecise.

Generally introducing a new object such as a repository should pass the “Composite is simpler than the sum of its parts” rule. However the approach also clearly demonstrates a bad abstraction. In OOP the primary goal is avoiding changes to affect the whole system.

Introduce Criteria Objects

Instead of using the QueryBuilder outside of the Repository, lets start with an alternative refactoring. I will introduce a Criteria class for the User:

<?php
class UserCriteria
{
    public $groupId;
    public $hydrateMode = Query::HYDRATE_OBJECT;
}

It is important not to introduce a constructor here, because when we add more and more criterions, the constructor will get bloated. Static factory methods that create a criteria do make sense however.

Now we can introduce a match method on the UserRepository. Lets see that on an interface level first, to see how simple usage is for the client side of the repository:

<?php
interface UserRepository
{
    /**
     * @param UserCriteria $criteria
     * @return array<User>|array<array>
     ***/
    public function match(UserCriteria $criteria);
}

Put in a $criteria get back users or array data. Very nice and simple! The implementation would look like this:

<?php
/**
 * @param UserCriteria $criteria
 * @return array<User>
 ***/
public function match(UserCriteria $criteria)
{
    $qb = $this->createQueryBuilder('u');

    if ($criteria->groupId !== null) {
        $this->matchGroup($qb, $criteria);
    }

    return $qb->getQuery()->getResult($criteria->hydrateMode);
}

private function matchGroup($qb, $criteria)
{
    $qb->where('u.group = :group')->setParameter('group', $criteria->groupId);
}

The benefit here is, that we can add additional conditions and processing by only adding a new property on the UserCriteria and then handling this inside UserRepository#match(). Additionally you can save the UserCriteria in the session, or even in the database to that users can “save filter” or return to a search overview, with the previous criteria still known.

The client code now looks like:

<?php
$criteria = new UserCriteria();
$criteria->groupId = $groupId;
$criteria->hydrateMode = Query::HYDRATE_ARRAY;

$groups = $app['orm.ems']['api']
    ->getRepository('EasyBib\Api\Entity\User')
    ->match($criteria);

What we achieved in this step, is a simple API for the developer using the Repository and a simple way to compose conditions by setting new properties in the criteria.

If you complain that the solution has the same amount of lines, than the original EasyBib solution, then you are missing the point. We have factored away a violation of the Law Of Demeter and calls on an API (Doctrine) that should be implementation detail of the repository.

Lets try this by adding a new filter criteria, for example permissions I mentioned before:

<?php
class UserCriteria
{
    const PERMISSION_READ = 'read';
    const PERMISSION_WRITE = 'write';
    //...
    public $permissions;
}
class UserRepository
{
    public function match(UserCriteria $criteria)
    {
        // ...
        if ($criteria->permissions !== null) {
            $this->matchPermissions($criteria);
        }
        // ...
    }
}

Simple enough, now we can use it everywhere we want by adding for example $criteria->permissions = UserCriteria::PERMISSION_WRITE in our client code.

Specification Pattern

The Criteria object gets us very far in abstracting lots of query building behind a very simple API, but it fails short when:

  • Composing Conditions using combinations of Not/And/Or is not possible without a tree structure, however Criteria is just a single object.
  • Removing duplication of code between different repositories. If you have similar conditions, limit or ordering requirements then you can only solve this by having all repositories extend a base repository. But Inheritance is evil.

The Specification pattern solves this issue. There are several ways to implement it, in the spirit of refactoring I will approach it from our existing Criteria.

Lets move the QueryBuilder code from the repository, into the Criteria object and rename it UserSpecification. Its important here to change the query builder code to use expressions that can be composed.

<?php
class UserSpecification
{
    public $groupId;
    public $hydrateMode = Query::HYDRATE_OBJECT;
    public $permissions;

    public function match(QueryBuilder $qb, $dqlAlias)
    {
        $expr = "1=1";

        if ($this->groupId !== null) {
            $expr = $qb->expr()->and($expr, $this->matchGroup($qb));
        }

        if ($this->permissions !== null) {
            $expr = $qb->expr()->and($expr, $this->matchPermissions($qb));
        }

        return $expr;
    }

    public function modifyQuery(Query $query)
    {
        $query->setHydrationMode($this->hydrateMode);
    }

    private function matchGroup($qb)
    {
        $qb->setParameter('group', $this->groupId);

        return $qb->expr()->eq('u.group', ':group');
    }

    private function matchPermissions($qb)
    {
        // ...
    }
}

The repository is then delegating the expression generation and puts the result into the where() method of the builder

<?php
class UserRepository
{
    public function match(UserSpecification $specification)
    {
        $qb = $this->createQueryBuilder('u');
        $expr = $specification->match($qb, 'u');

        $query = $qb->where($expr)->getQuery();

        $specification->modifyQuery($query);

        return $query->getResult();
    }
}

Strictly speaking, the UserSpecification violates the single responsibility principle, which prevents the composability of specifications and reuse in different repositories. This is apparent by the $expr = "1=1"; line that is required to make the combination of conditions possible. Lets factor away the violation of the single responsibility principle by introducing three specifications:

<?php
interface Specification
{
    /**
     * @param \Doctrine\ORM\QueryBuilder $qb
     * @param string $dqlAlias
     *
     * @return \Doctrine\ORM\Query\Expr
     ***/
    public function match(QueryBuilder $qb, $dqlAlias);

    /**
     * @param \Doctrine\ORM\Query $query
     ***/
    public function modifyQuery(Query $query);
}

class AsArray implements Specification
{
    private $parent;

    public function __construct(Specification $parent)
    {
        $this->parent = $parent;
    }

    public function modifyQuery(Query $query)
    {
        $query->setHydrationMode(Query::HYDRATE_ARRAY);
    }

    public function match(QueryBuilder $qb, $dqlAlias)
    {
        return $this->parent->match($qb, $dqlAlias);
    }
}

class FilterGroup implements Specification
{
    private $group;

    public function __construct($group)
    {
        $this->group = $group;
    }

    public function match(QueryBuilder $qb, $dqlAlias)
    {
        $qb->setParameter('group', $this->group);

        return $qb->expr()->eq($dqlAlias . '.group', ':group');
    }

    public function modifyQuery(Query $query) { /* empty ***/ }
}

class FilterPermission implements Specification
{
    private $permissions;

    public function __construct($permissions)
    {
        $this->permissions = $permissions;
    }

    public function match(QueryBuilder $qb, $dqlAlias)
    {
        // ...
    }

    public function modifyQuery(Query $query) { /* empty ***/ }
}

Now we need a new And-Specification to combine this in our code. This looks rather abstract and complex on the inside, but for clients of this object, the usage is simple and obvious.

<?php
class AndX implements Specification
{
    private $children;

    public function __construct()
    {
        $this->children = func_get_args();
    }

    public function match(QueryBuilder $qb, $dqlAlias)
    {
        return call_user_func_array(
            array($qb->expr(), 'andX'),
            array_map(function ($specification) use ($qb, $dqlAlias) {
                return $specification->match($qb, $dqlAlias);
            }, $this->children
        ));
    }

    public function modifyQuery(Query $query)
    {
        foreach ($this->children as $child) {
            $child->modifyQuery($query);
        }
    }
}

Assuming we import all specifications from a common namespace Spec, our client code will look like this:

<?php
$specification = new Spec\AsArray(new Spec\AndX(
    new Spec\FilterGroup($groupId),
    new Spec\FilterPermission($permission)
));

$groups = $app['orm.ems']['api']
    ->getRepository('\EasyBib\Api\Entity\Group')
    ->match($specification);

In contrast to the criteria, we could now implement or and not specifications to enhance query capabilities.

Improving Specifications

You can now introduce reusability across different repositories by adding functionality to check if a specification supports a given entity.

<?php
interface Specification
{
    // ..
    /**
     * @param string $className
     * @return bool
     ***/
    public function supports($className);
}

Every composite can delegate this operation to its children, and every leaf of the tree can return true or false. The Repository can then check for a valid specification in its match method:

<?php

abstract class EntitySpecificationRepository
{
    public function match(Specification $specification)
    {
        if ( ! $specification->supports($this->getEntityName())) {
            throw new \InvalidArgumentException("Specification not supported by this repository.");
        }

        $qb = $this->createQueryBuilder('r');
        $expr = $specification->match($qb, 'r');

        $query = $qb->where($expr)->getQuery();

        $specification->modifyQuery($query);

        return $query->getResult();
    }
}

Now we can introduce very generic specifications, such as OnlyPage($page, Specification $spec)` for limit queries, or Equals($field, $value). For more readable code, you can then create a domain language for your specifications that is composed of more simple specifications:

<?php
class PowerUsers implements Specification
{
    private $spec;

    public function __construct()
    {
        $this->spec = new OnlyPage(1, new AndX(
            new UsersWithInteraction(),
            new EnabledUsers(),
        ));
    }

    public function match(QueryBuilder $qb, $dqlAlias)
    {
        return $this->spec->match($qb, $dqlAlias);
    }

    public function modifyQuery(Query $query)
    {
        $this->spec->modifyQuery($query);
    }

    public function supports($className)
    {
        return ($className === 'EasyBib\Api\Entity\User');
    }
}

$top20powerUsers = new Spec\PowerUsers();

Hiding this kind of composition inside another specification allows you to reuse query logic in different places in the application easily and in terms of the domain language.

Testability of Doctrine Repositories

One reasons outlined by Anne for this design is testability: Because the Repository returns the QueryBuilder you have access to the generated SQL. However testing Doctrine Repositories should never be verifying the generated SQL. I see a lot of people doing this and it is very fragile and dangerous. Doctrine is a third party library and as such a rather complex one. Possible changes that break the test are:

  • Doctrine adds/removes whitespaces to SQL in a next version
  • Doctrine performs SQL optimizations in certain cases, the result is the same though.
  • You add a field/column to any of the tables involved that does not affect the result.
  • You change something in the Doctrine mapping files, that leads to a reordering of SQL.

These are 4 changes that have absolutely nothing to do with the feature you are actually testing, making the test code very fragile. In terms of abstraction SQL generation is an implementation detail of the Doctrine ORM and you as developer are only interested in the public API, which the SQL generation is not part of.

The code should really be tested against Doctrine itself. Since you are using Doctrine to get rid of SQL query generation for some use-cases, why should you use them as measure of quality in your testing efforts.

Testing repositories with the Specification pattern is testing the different specifications in isolation against a real Doctrine database backend. This will not be super simple to setup, but the isolation of specifications and their reusability across repositories actually allows us to keep the number of tests very small. The pattern avoids the problem of combinatorial explosion of test-cases very neatly.

The real benefit of testability is achieved in tests of repository client code. Before we were not able to unit-test this code, because of the Doctrine EntityManager, Query + QueryBuilder dependencies. Now We can inject the repositories into our controllers and services and then use mock objects in the tests.

]]>
Mon, 04 Mar 2013 00:00:00 +0100
https://beberlei.de/2013/02/19/extending_symfony2__paramconverter.html https://beberlei.de/2013/02/19/extending_symfony2__paramconverter.html <![CDATA[Extending Symfony2: ParamConverter]]> Extending Symfony2: ParamConverter

Symfony2 is an extremely extendable framework, everything is extendable or overwritable through the Dependency Injection Container. The problem developers face is knowing about the extension points and when to use them. If you don’t know the extension points, your Symfony application will end up with code duplication, too much inheritance and very little unit-testable code.

This blog post will be the first in a series, describing Symfony2 extension points that help you achieve clean and duplicateless code. In my experience, using Symfony extension points to avoid code duplication helps you avoid writing thousands of lines of code in your controllers.

The problem: Duplicate finder logic

Inside controllers you can easily end with lots of duplication using the same general finder logic again in several actions. Take this following example:

<?php
class UserController extends Controller
{
    public function showAction($id)
    {
        $dql = "SELECT u, d, a
                  FROM MyBundle\Entity\User u
                  JOIN u.details d
                  JOIN u.addresses a
                  WHERE u.id = ?1";

        $user = $this->get('doctrine.orm.default_entity_manager')
            ->createQuery($dql)
            ->setParameter(1, $id)
            ->getSingleResult();

        if ( ! $user) {
            throw new NotFoundHttpException();
        }

        return array('user' => $user);
    }
}

If we need this block of code in several actions of different controllers, we will end up with duplication that has to be eliminated.

The Quick and Dirty Solution

One way of resolving the duplication appearing in this case, is moving the finder + not found logic into a common controller base class or into a trait. But this leaves us with a helper method buried in the code and a static dependency to a base class or a trait that we want to avoid.

<?php
class AbstractController extends Controller
{
    protected function findUser($id)
    {
        $dql = "SELECT u, d, a
                  FROM MyBundle\Entity\User u
                  JOIN u.details d
                  JOIN u.addresses a
                  WHERE u.id = ?1";

        $user = $this->get('doctrine.orm.default_entity_manager')
            ->createQuery($dql)
            ->setParameter(1, $id)
            ->getSingleResult();

        if ( ! $user) {
            throw new NotFoundHttpException();
        }

        return $user;
    }
}

There are two problems with this sort of refactoring:

  1. We are using inheritance for code-reuse.
  2. We hide the findUser behavior in an abstract class and make it hard to test.

The Preferred Solution

The SensioFrameworkExtraBundle offers an extension hook called Parameter Converters to transform Request attributes to objects directly for controller method arguments. They hook into the kernel.controller event that you can use yourself to achieve the same goal.

Lets see how the action will look like after our refactoring:

<?php
class UserController extends Controller
{
    public function showAction(User $user)
    {
        return array('user' => $user);
    }
}

Very concise and easy to read. The param converter doing the heavy lifting looks like this:

<?php
namespace MyProject\Request\ParamConverter;

use Sensio\Bundle\FrameworkExtraBundle\Configuration\ConfigurationInterface;
use Sensio\Bundle\FrameworkExtraBundle\Request\ParamConverter\ParamConverterInterface;
use Symfony\Component\HttpFoundation\Request;
use Symfony\Component\HttpKernel\Exception\NotFoundHttpException;
use Doctrine\ORM\EntityManager;

class UserParamConverter implements ParamConverterInterface
{
    private $entityManager;

    public function __construct(EntityManager $entityManager)
    {
        $this->entityManager = $entityManager;
    }

    public function apply(Request $request, ConfigurationInterface $configuration)
    {
        $id = $request->attributes->get('id');

        $dql = "SELECT u, d, a
                  FROM MyBundle\Entity\User u
                  JOIN u.details d
                  JOIN u.addresses a
                  WHERE u.id = ?1";

        $user = $this->get('doctrine.orm.default_entity_manager')
            ->createQuery($dql)
            ->setParameter(1, $id)
            ->getSingleResult();

        if ( ! $user) {
            throw new NotFoundHttpException();
        }

        $param = $configuration->getName();
        $request->attributes($param, $user);

        return true;
    }

    public function supports(ConfigurationInterface $configuration)
    {
        return "MyProject\Entity\User" === $configuration->getClass();
    }
}

Now we only need to register this class in the dependency injection container:

<service id="my_project.user_param_converter"
      class="MyProject\Request\ParamConverter\UserParamConverter">
    <argument type="service" id="doctrine.orm.default_entity_manager" />

    <tag name="request.param_converter" converter="user" priority="10" />
</service>

With the priority configuration the User entity is now always handled by our custom param converter and not by the default Doctrine converter.

In a next step, we should extract the query logic from the ParamConverter into a custom Doctrine entity repository. But that is a task for another blog post in this series.

]]>
Tue, 19 Feb 2013 00:00:00 +0100
https://beberlei.de/2013/02/04/doctrine_and_solid.html https://beberlei.de/2013/02/04/doctrine_and_solid.html <![CDATA[Doctrine and SOLID]]> Doctrine and SOLID

I often get asked how you can use Doctrine2 and implement the SOLID principles at the same time. It bugs me having to reply: It is not really possible. Because Doctrine2 does not support value-objects (or embedded-objects), it is very hard to pull the Single Responsibility Principle off.

These problems are related to the inability to share behavioral code through aggregation and the complexity of state transformations. Combining both, your average entity with 5-15 fields can end up with hundreds or thousands lines of code. The solutions to both problems boil down to minimizing duplication and maximizing clarity.

Extracting Value Objects: Minimize Duplication

Entity classes responsibility are the state transformations of their internal fields. This can simply be done by using setter methods or when avoiding setters, with use-case driven methods. These state transformations can be part of different responsibilities, specifically when properties belong to different groups of concepts.

Take a very simple entity that contains updated/created at logic:

<?php
use Doctrine\ORM\Mapping as ORM;

/**
 * @ORM\Entity
 * @ORM\HasLifecycleCallbacks
 **/
class Person
{
    /**
     * @ORM\Column(type="datetime")
     **/
    private $createdAt;
    /**
     * @ORM\Column(type="datetime")
     **/
    private $updatedAt;

    public function __construct()
    {
        $this->createdAt = new \DateTime("now");
        $this->updatedAt = new \DateTime("now");
    }

    /**
     * @ORM\PreUpdate
     **/
    public function updatedAt()
    {
        $this->updatedAt = new \DateTime("now");
    }
}

If you want to duplicate this logic to another second entity, then in Doctrine the solution for this is using traits. Kore will dismiss this solution, because traits create a hard dependency. Even if we accept the static dependency, traits are not perfect even for this very simple example, because its very likely that you cannot override the constructor of every entity.

The solution is to extract a value object Timestamped that contains all the logic:

<?php
class Timestamped
{
    private $createdAt;
    private $updatedAt;

    public function __construct()
    {
        $this->createdAt = new \DateTime("now");
        $this->updatedAt = new \DateTime("now");
    }

    public function updatedAt()
    {
        $this->updatedAt = new \DateTime("now");
    }
}

class Person
{
    private $timestamped;

    public function __construct()
    {
        $this->timestamped = new Timestamped();
    }
}

See how all the code could be moved into Timestamped and is now reusable in other entities.

Doctrine has no support for embedded objects, which is very sad. I am working very hard to get this feature into Doctrine as soon as possible. You can use the “object” type as a workaround and serialize() the value object into the database. However this is beyond ugly in my opinion.

Extract Method Objects: Maximizing clarity

Once you have identified groups of fields that are modified, then the complexity of the state transformations can attract lots of code.

Take an Order object that has a method for calculating the shipping costs, depending all the order items and products. To separate calculations from state transformations you can extract method objects instead of inlining the code into the Order object.

For this kind of extraction I create a folder Order and put all the extracted method objects in the Order subnamespace.

<?php
namespace MyProject\Entity {

    class Order
    {
        public function calculateShippingCosts()
        {
            $calculator = new ShippingCostCalculator();
            $this->shippingCosts = $calculator->calculate($this);
        }
    }
}

namespace MyProject\Entity\Order {

    class ShippingCostCalculator
    {
        public function calculate(Order $order)
        {
            return 0;
        }
    }
}

From this step its easy to make the code reusable by passing the shipping cost calculator:

<?php
class Order
{
    public function calculateShippingCosts(ShippingCostCalculator $calculator)
    {
        $this->shippingCosts = $calculator->calculate($this);
    }
}

Another benefit is that you can test the shipping cost calculator directly in a unit-test and avoid checking for the correctness indirectly through a getter method for the shipping costs.

Extracting every method of an entity into a method object is obviously overkill. You should exercise caution and common sense when performing this refactoring.

Conclusion

Not all the techniques to implement SOLID code can be exploited when using Doctrine for technical reasons. In the future I hope to support value objects in Doctrine to make this possible.

]]>
Mon, 04 Feb 2013 00:00:00 +0100
https://beberlei.de/2012/11/19/composer_and_azure_websites_git_deployment.html https://beberlei.de/2012/11/19/composer_and_azure_websites_git_deployment.html <![CDATA[Composer and Azure Websites Git Deployment]]> Composer and Azure Websites Git Deployment

Continuing my series on PHP PaaS Clouds (Fortrabbit), I turn to Microsoft Azure today. After some research I found out Azure supports post deployment hooks to run Composer and allows you to configure environment variables from the Management console.

Microsoft launched Azure Websites in June this year. It is a platform as a service solution where you can deploy your websites via FTP or Git. With Azure Websites you can avoid having to deal with the complex deployment automation that is necessary for Azure Hosted Cloud Services. As long as your website only requires an SQLServer, MySQL or external APIs you can actually achieve quite a lot already.

For a Symfony2, Silex or any other modern project however you want Composer support during the deployment as long as failures with Composer don’t break your site.

It turns out that Azure Websites - to support other platforms that require compiling - actually has an extremely robust deployment system (as far as I understood its internals). The Git repository is separated from the actual code that is served from the webserver and a number of old checkouts is kept to allow rollbacks through the Web interface. Once a Git push was recognized, Azure websites will execute a build step, and then copy all the files over to a directory with the name of the Git SHA hash. This directory is then symlinked to the document root.

If Composer fails during this step, the website will still be served from the currently running version and you don’t have to face unexpected downtime.

To actually run Composer as a post-deployment tool you have to do some manual work. Create a .deployment file in your project root folder:

[config]
command = "D:\Program Files (x86)\PHP\v5.3\php.exe" build_azure.php

If you are using PHP 5.4, then you probably have to change the command version name to “v5.4”, but I haven’t tested this.

Then you need to create the build_azure.php file that is referenced in the deployment command. The actual implementation is up to you, my version is the most simple one:

<?php
if (!file_exists("composer.phar")) {
    $url = 'https://getcomposer.org/composer.phar';
    file_put_contents("composer.phar", file_get_contents($url));
}

$_SERVER['argv'][1] = "update";
$_SERVER['argv'][2] = "--prefer-dist";
$_SERVER['argv'][3] = "-v";
require "composer.phar";

Note

This currently only works with a Composer version, where PR-1341 is applied. You can check out my Composer fork and run ./bin/compile to generate a composer.phar that works.

Instead of using this approach, you could ship composer.phar with your repository for example. You can of course execute additional steps in the build_azure.php, for example warmup caches.

]]>
Mon, 19 Nov 2012 00:00:00 +0100
https://beberlei.de/2012/10/27/babysteps_with_fortrabbit_php_cloud.html https://beberlei.de/2012/10/27/babysteps_with_fortrabbit_php_cloud.html <![CDATA[Babysteps with Fortrabbit PHP Cloud]]> Babysteps with Fortrabbit PHP Cloud

I wanted to host a simple application to play around with the Tent Protocol (see my last blogpost), so I checked through some of the PHP Cloud providers again for the fun of it. One requirement was to deploy from Git and update dependencies from Composer. I found a new provider that i haven’t heard of before, Fortrabbit that supports this so I had to check it out. It has some more features that I really dig:

Fortrabbit provides you with a Git url where you can push your repository. It is directly deployed during the push operation. You can trigger composer updates by having a special syntax in the commit message. The push output informs you about all the steps performed during deployment.

Additionally you can configure environment variables in the web-administration interface that are available in the application through $_SERVER. You can easily use them to configure your application, if its hosted in a public repository. Great to share sample applications on Github and host them from there.

You get SSH access to the machines, where you can take a look at the apache and php error log files. You have vim available. Quite cool and very helpful for any kind of debugging necessary. Deployments overwrite every change you make, so its still a save environment.

The composer support allows for post install scripts, which is cool to perform all sorts of cache warmup and other deployment tasks.

You can host one application for free, however they shutdown after 48 hours if you don’t regularly click on a reset button. Its definitely enough to get small prototypes up and running. From there you can upgrade in many small steps from small plans (10€/month) to bigger ones (80€/month).

]]>
Sat, 27 Oct 2012 00:00:00 +0200
https://beberlei.de/2012/10/22/a_tent_io_app__zelten_bookmarks.html https://beberlei.de/2012/10/22/a_tent_io_app__zelten_bookmarks.html <![CDATA[A tent.io app: Zelten Bookmarks]]> A tent.io app: Zelten Bookmarks

Over the weekend I built a very simple bookmarking application for tent.io. I wanted to try the tent functionality to create arbitrary entities of data using something as simple as a “Bookmark”.

The application retrieves and stores bookmarks on your own tent-server, nothing is kept on the applications server and as a user you keep full control over your content (bookmarks).

The code for the application is on Github, as well as the TentPHP library I have written to support the protocol. The application is using PHP, the Silex microframework and MySQL.

So what is Tent and why bother?

About two month ago a new distributed social networking protocol was launched called Tent.io. It works over HTTP using JSON and distributes data using webhooks. This happened in the shadow of app.net funding, just two weeks after they collected over 500k USD. Tent.io got lots of attention but not as much as app.net sadly. Its distributed nature however is much more suited to achieving true independence of large companies and true privacy for the users though. Compared to Diaspora it has the benefit of being a protocol, not an application first.

Now two month later, there are reference implementations for Server, Client and a Twitter like Status application. You can use Tent.is (Now defunct) to create a free or payed tent account including access to the status application or setup your own tentd server on Heroku.

As a user I can connect any application to the tent server I have registered with and control the visibility and sharing policies. Content can be private, public or visible to certain groups only. A Tent server then makes sure to distribute content to the appropriate subscribers.

In this early state of the protocol it makes sense to register on the central tent.is service, however when the server gets more stable or alternative server implementations popup its easy to move all your data off tent.is to your own server. One feature that will hit tent.is soonish is registration of domain records. That will be the first step to your own tent server, I am eagerly waiting to host all my content at https://tent.beberlei.de at some day.

Tent.io sounds awesome, how can I help?

Right at the moment the tent software stack is developed by the team that also hosts Tent.is. If all you can provide is money, then signing up for the 12 USD per month is the way to go.

If you are a developer, then take a look at the Wiki where related projects are listed. You will find a bunch of client libraries for major languages already, helping you to get started.

I suppose the major work lies in implementing Status-Post clients for all major smartphone platforms and desktops. As well as the host of applications that Facebook and other social networks provides:

  • Manage and share Photo albums
  • Plan and share events
  • Managing contact data/address book
  • Resume details (Linkedin/Xing)
  • Track User Events (Timeline)
  • Sharing Music and Videos (Myspace)
  • Importing all sorts of existing content (RSS,..)
]]>
Mon, 22 Oct 2012 00:00:00 +0200
https://beberlei.de/2012/09/04/using_assertions_for_validation.html https://beberlei.de/2012/09/04/using_assertions_for_validation.html <![CDATA[Using Assertions for validation]]> Using Assertions for validation

We all know PHP is weakly typed and has some weird type-conversion rules. This can often complicate our code checking for invalid or illegal data. Using if/else clauses we can do all sorts of validation, but with standard coding styles (PSR, Zend, PEAR) we end up with at least 3 lines per check, cluttering the whole code-base.

Take this example from the old deprecated Windows Azure SDK, a method for putting a binary file into Azures blob storage:

<?php
public function putBlob($containerName = '', $blobName = '', $localFileName = '', $metadata = array(), $leaseId = null, $additionalHeaders = array())
{
    if ($containerName === '') {
        throw new Microsoft_WindowsAzure_Exception('Container name is not specified.');
    }
    if (!self::isValidContainerName($containerName)) {
        throw new Microsoft_WindowsAzure_Exception('Container name does not adhere to container naming conventions. See http://msdn.microsoft.com/en-us/library/dd135715.aspx for more information.');
    }
    if ($blobName === '') {
        throw new Microsoft_WindowsAzure_Exception('Blob name is not specified.');
    }
    if ($localFileName === '') {
        throw new Microsoft_WindowsAzure_Exception('Local file name is not specified.');
    }
    if (!file_exists($localFileName)) {
        throw new Microsoft_WindowsAzure_Exception('Local file not found.');
    }
    if ($containerName === '$root' && strpos($blobName, '/') !== false) {
        throw new Microsoft_WindowsAzure_Exception('Blobs stored in the root container can not have a name containing a forward slash (/).');
    }
    // rest of the code here
}

The rest of this components public API methods look about the same. It is a very complete validation of the input data, however its not very readable. Instead you have 6 if branches, which increase the complexity of the method and almost take up half the screen without even getting to the actual code.

The assertion pattern is really helpful here to reduce the complexity of this code and hide the if/throw conditions into little helper methods. Have a look at a refactored method using my Assert micro-library.

<?php
public function putBlob($containerName = '', $blobName = '', $localFileName = '', $metadata = array(), $leaseId = null, $additionalHeaders = array())
{
    Assertion::notEmpty($containerName, 'Container name is not specified');
    self::assertValidContainerName($containerName);
    Assertion::notEmpty($blobName, 'Blob name is not specified.');
    Assertion::notEmpty($localFileName, 'Local file name is not specified.');
    Assertion::file($localFileName, 'Local file name is not specified.');
    self::assertValidRootContainerBlobName($containerName, $blobName);

    // rest of the code
}

This is much more readable, using two custom assertions and some general assertions.

There is one risk here though, you introduce additional coupling to another library/namespace. Especially when you start using this pattern on a large scale, you have to make sure this assertions are VERY stable and unit-tested.

That is why you should extend from Assertion and use your own class when using the Assert library. This returns some control in your hands and even allows you to overwrite the exception class:

<?php
namespace MyProject\Util;

use Assert\Assertion as BaseAssertion;

class Assertion extends BaseAssertion
{
    static protected $exceptionClass = 'MyProject\Util\AssertionFailedException';
}
]]>
Tue, 04 Sep 2012 00:00:00 +0200
https://beberlei.de/2012/08/26/symfony__monolog_and_different_log_types.html https://beberlei.de/2012/08/26/symfony__monolog_and_different_log_types.html <![CDATA[Symfony, Monolog and different log types]]> Symfony, Monolog and different log types

Symfony uses the excellent Monolog library for logging everything related to the symfony application stack. But sometimes you need a different logger that responds differently to the log error levels. Ideally you don’t want to reuse the application log mechanism to avoid messing with the fingers crossed listener of Monolog, that keeps the application log files small in case no errors occur.

This can be easily done with Symfonys MonologBundle with a feature called channel. Every log handler, such as file, syslog, mail or fingers crossed is attached to the default logger called “app”. This is referenced by the dependency injection service called monolog.logger.

If you want to create a different logger service that is responsible for a different type of your application stack, just define it using the channels option:

monolog:
    handlers:
        myfile:
            type: stream
            path: %kernel.logs_dir%/myfile.log
            channels: mychannel

To actually write to mychannel you have to tag all the services that use mychannel with the following tag:

<service id="my_service" class="MyService">
    <tag name="monolog.logger" channel="mychannel" />
    <argument type="service" id="logger" />
</service>

With this approach you are now as flexible as with the default channel app regarding to logging in development, testing and production environments for example. Once there is a channel created through the DIC tag, you can actually fetch it from the DIC with monolog.logger.mychannel in our example. I don’t like this approach too much, because it misuses tags for modifying arguments, where tags are actually for the services themselves. However it is usable and works.

This feature is included starting with Symfony 2.1. It is documented in a cookbook entry, however some insights of this blog post are still inside a pending Pull Request.

]]>
Sun, 26 Aug 2012 00:00:00 +0200
https://beberlei.de/2012/08/26/froscon_2012.html https://beberlei.de/2012/08/26/froscon_2012.html <![CDATA[FrOSCon 2012]]> FrOSCon 2012

This years Free and Open Source Conference (FrOSCon) took place this weekend in Bonn, St. Augustin. Because I live in Bonn, visiting the conference is a given for very many years now. This year I also did a talk on “Architecture Design Patterns” with Gordon so there was an incentive to show up besides meeting friends, all the Club Mate and good open source coffee.

The most interesting topic in this years conference for me has been the Graylog2 talk by Lennart Koopmann. Graylog is a logging server that uses Elastic Search for persistence and searching. It is sponsored by Xing, a pretty big german based Linkedin clone.

The talk The State of PHPUnit by Volker was interesting to see all the new stuff in PHPUnit 3.7 and that it fixes a bunch of annoyances that I stumbled on myself. I actually realized that they are so deeply rooted in my brain already, that they constitute features for me, not something that could actually be optimized. Well done to Volker and Sebastian to the new version.

Igors talk about building an HTTP server in PHP was interesting to see the stream/socket server APIs of PHP at work. Ever since I took part in the SoCraTes 2012 Conference Maexchen UDP Bot challenge I think network programming is an interesting topic for learning about languages and programming itself.

Volker did a second talk on sunday about Clean Code that was both entertaining and interesting. He was reminding the audience that shipping the code is what actually creates the value in our profession. Hence clean code is not about the beauty of code itself, but its ability to be shippable, changeable and adaptable to the business.

Again FrOSCon 2012 was awesome. Thank you very much especially to all the volunteers that organized this event. It really good event to bring together people from many different open-source communities and I already look forward to the next FrOSCon 2013.

]]>
Sun, 26 Aug 2012 00:00:00 +0200
https://beberlei.de/2012/08/25/decoupling_applications_with_domain_events.html https://beberlei.de/2012/08/25/decoupling_applications_with_domain_events.html <![CDATA[Decoupling applications with Domains Events]]> Decoupling applications with Domains Events

In the previous posts I have described 3 architectural patterns for business applications. Applying them to your software helps you decouple the business logic from the application/framework/UI. However given sufficient complexity, you will want to decouple different parts of your business model from each other as well.

As an example, lets think of a batch process in your application that updates orders from a CRM or logistics system:

  • You receive an XML with full users and order representations
  • You need to update certain user fields
  • You need to update certain order fields
  • Some orders require creating accounts in different remote systems
  • If the user has not confirmed his email yet he should receive an opt-in mail instead.
  • If the user confirms his opt-in mail, all outstanding remote accounts are created.

You can build all the steps into a single batch processing service. This will be a particularly huge service and in violation of the Single Responsibility principle.

the part with updating of users and fields has nothing to with the sending of mails and creation of remote accounts. We want to decouple them from each other.

The first obvious choice is to decouple them into distinct services:

  • Import Service
  • Authentication Service
  • Account Generation Service

All these services have dependencies on infrastructure objects, database, mailer and so on. We could inject the authentication and account generation services into the Import Service, but with rising complexity this will lead to a lasagna of services and the execution path will dig deep into this and this is not nearly as tasty as eating real lasagna.

ImportService
|_AuthenticationService
| |_Mailer
| |_Database
|_AccountGenerationService
| |_Database
| \_RemoteFacade
\_Database

This becomes more complicated when transaction semantics need to be taken care of. For example dependencies between mailer and database services. Also you have to take into account that the code in entities and value objects participating in this use-case.

What we want instead is a sequential execution of those nested services, but only if the parent service executed successfully:

ImportService
\_Database

AuthenticationService
|_Mailer
\_Database

AccountGenerationService
|_Database
\_RemoteFacade

We could use an event dispatcher in all of the services to notify each other, but the DomainEvent pattern does this more in a much cleaner way:

Every entity is an event provider and can emit events. Whenever a transaction is committed and an hence an operation is completed successfully, we take all the events emitted from all entities (looking at the identity map for example) and trigger observing event handlers. However if the operation fails, we do not trigger these event handlers.

<?php

class Order implements EventProviderInterface
{
    use EventProvider;

    public function importData(array $data)
    {
        // 1. do something with $data

        // 2. raise event
        $this->raise(new OrderImportCompleted(array(
            "id" => $this->id,
            "data" => $data
        )));
    }
}

interface EventProviderInterface
{
    public function dequeueEmittedEvents();
}

The Event provider trait aggregates all the events, and offers and API for external services to pull them from the entity:

<?php
trait EventProvider
{
    private $emittedEvents = array();

    protected function raise(DomainEvent Event)
    {
        $event->getMessageHeader()->setEntity($this);
        $this->emittedEvents[] = $event;
    }

    public function dequeueEmittedEvents()
    {
        $events = $this->emittedEvents;
        $this->emittedEvents = array();
        return $events;
    }
}

Our infrastructure must then trigger event handlers, based on the event names. It will use dequeueEmittedEvents and making sure the events are not emitted multiple times.

For reasons described below, we want the following command/event chain to happen in our system:

  • Command executes
  • Entities emit events
  • Command transaction succeeds
  • Events trigger event handlers
  • Event handlers execute more commands
  • Restart from 1.

With this approach we can decouple all services from each other and avoid deep nesting in each other. Yet we still have transactional dependencies, by dropping all events when the parent command fails. Transactions over multiple commands will not have ACID properties though, instead you will have to look into BASE transactions that are important in systems with eventual consistency. This is one downside that you need to take into account.

The Domain Event pattern is a prerequisite for full blown CQRS. My LiteCQRS library includes a simple implementation of DomainEvent and EventProvider classes and integration into Symfony and Doctrine ORM. Generally this pattern is very easy to implement though, so that you can just have a look at the implementation and take the best parts for your own.

]]>
Sat, 25 Aug 2012 00:00:00 +0200
https://beberlei.de/2012/08/22/building_an_object_model__no_setters_allowed.html https://beberlei.de/2012/08/22/building_an_object_model__no_setters_allowed.html <![CDATA[Building an Object Model: No setters allowed]]> Building an Object Model: No setters allowed

If you are using an object relational mapper or any other database abstraction technology that converts rows to objects, then you will probably use getter/setter methods or properties (C#) to encapsulate object properties.

Take a look at the default User entity from the Symfony2 FOSUserBundle plugin for example: A colossus of getter/setter methods with nearly no real business logic. This is how most of our persistence related objects look like in PHP. A Rails ActiveRecord such as Redmines “Issue” avoids the explicit getter/setters, however generates accessors magically for you. Nevertheless you have to add considerable amount of code to configure all the properties. And code using these active records becomes ambiguous as well.

Why do we use getters/setters so much?

  • Tools generate objects from a database, adding getters and setters automatically.
  • Frameworks make them automatically available for database records/rows.
  • IDEs can magically create getter/setter pairs for fields.
  • We want the flexibility to change every field whenever we want.
  • It became natural in OOP to have write access to every field.

However Getters/setters violate the open/closed principle, prevent information hiding and should be considered evil (Long version). Using getters and setters allows to decouples the business logic for setting values from the actual storage. Something that object-orientation was suppose to avoid.

Getting rid of setters

Avoiding setters is much simpler than getters, so lets start with them.

One way to avoid writing setters is a task based approach to the model. Think of every task that is performed in an application and add a method that changes all the affected fields at once, to perform this task. In the famous blog example the Post object may look like:

<?php
class Post
{
    public function compose($headline, $text, array $tags)
    {
        // check invariants, business logic, filters

        $this->headline = $headline;
        $this->text     = $text;
        $this->tags     = $tags;
    }

    public function publish(\DateTime $onDate = null)
    {
        // check invariants, business logic, filters

        $this->state       = "published";
        $this->publishDate = $onDate ?: new \DateTime("now");
    }
}

The Post class is now much more protected from the outside and is actually much better to read and understand. You can clearly see the behaviors that exist on this object. In the future you might even be able to change the state of this object without breaking client code.

You could even call this code domain driven, but actually its just applying the SOLID principles to entities.

Tackling getters

Avoiding getters is a bit more cumbersome and given no setters, maybe not worth the trouble anymore.

You still need getters to access the object state, either if you display models directly in the view or for testing purposes. There is another way to get rid of all the getters that you don’t need explicitly in a domain model, using the visitor pattern in combination with view models:

<?php
class Post
{
    public function render(PostView $view)
    {
        $view->id          = $this->id;
        $view->publishDate = $this->publishDate;
        $view->headline    = $this->headline;
        $view->text        = $this->text;
        $view->author      = $this->author->render(new AuthorView);

        return $view;
    }
}

class PostView
{
    public $id;
    //... more public properties
}

There are drawbacks with this approach though:

  • Why use the Post model in memory, when you are only passing PostView instances to the controllers and views only anyways? Its much more efficient to have the database map to the view objects directly. This is what CQRS postulates as separation of read- and write model.
  • You have to write additional classes for every entity (Data transfer objects) instead of passing the entities directly to the view. But if you want to cleanly separate the model from the application/framework, you don’t get around view model/data transfer objects anyways.
  • It looks awkward in tests at first, but you can write some custom assertions to get your sanity back for this task.

What about the automagic form mapping?

Some form frameworks like the Symfony2 or Zend Framework 2 ones map forms directly to objects and back. Without getters/setters this is obviously not possible anymore. However if you are decoupling the model from the framework, then using this kind of form framework on entities is a huge no go anyways.

Think back to the tasks we are performing on our Post entity:

  • Edit (title, body, tags)
  • Publish (publishDate)

Both tasks allow only a subset of the properties to be modified. For each of these tasks we need a custom form “model”. Think of these models as command objects:

<?php
class EditPostCommand
{
    public $id;
    public $headline;
    public $text;
    public $tags = array();
}

In our application we could attach these form models to our form framework and then pass these as commands into our “real model” through a service layer, message bus or something equivalent:

<?php
class PostController
{
    public function editAction(Request $request)
    {
        $post = $this->findPostViewModel($request->get('id'));

        // This could need some more automation/generic code
        $editPostCommand           = new EditPostCommand();
        $editPostCommand->id       = $request->get('id');
        $editPostCommand->headline = $post->headline;
        $editPostCommand->text     = $post->text;
        $editPostCommand->tags     = $post->tags;

        // here be the form framework handling...
        $form = $this->createForm(new EditPostType(), $editPostCommand);
        $form->bind($request);

        if (!$form->isValid()) {
            // invalid, show errors
        }

        // here we invoke the model, finally, through the service layer
        $this->postService->edit($editPostCommand);
    }
}

class PostService
{
    public function edit(EditPostCommand $command)
    {
        $post = $this->postRepository->find($command->id);
        $post->compose($command->headline, $command->text, $command->tags);
    }
}

This way we separated the business model from the application framework.

A word about RAD

Rapid-application development or rapid prototyping is a wide-spread approach in web development. My explicit approach seems to be completely against this kind of development and much slower as well. But I think you don’t loose much time in the long run:

  • Simple command objects can be code-generated or generated by IDEs in a matter of seconds. Or you could even extend ORMs code generation capabilities to generate these dummy objects for you. Since you don’t need ORM mapping information for these objects or think about performance implications in context with the ORM, you don’t need to spend much thinking about their creation.
  • Explicit models are much simpler to unit-test and those tests run much faster than tests through the UI that RAD prototypes need.
  • Generated Rapid prototypes can get hard to maintain quickly. That does not mean they are unmaintainable, but they seem to favour reimplementation instead of refactoring, something that leads to problems given the low code coverage that these prototypes normally have.

Conclusion

If we take a step back from all our tools suggesting to generate getter/setters we find that there is a simple way to avoid using setters when focusing on the tasks that objects perform. This actually makes our code much more readable and is one building block towards clean object oriented code and domain driven design.

I am very interested in your opinions on this topic and my attempt to avoid them, please leave comments when leaving this website :-)

]]>
Wed, 22 Aug 2012 00:00:00 +0200
https://beberlei.de/2012/08/18/oop_business_applications__command_query_responsibility_seggregation.html https://beberlei.de/2012/08/18/oop_business_applications__command_query_responsibility_seggregation.html <![CDATA[OOP Business Applications: Command-Query-Responsibility-Segregation (CQRS)]]> OOP Business Applications: Command-Query-Responsibility-Segregation (CQRS)

Other posts in this series about OOP Business Applications:

The last pattern for designing your business logic is called “Command-Query-Responsibility-Segregation” or short CQRS. It has its roots in the “Command Query Separation” principle that was put forward by Bertrand Meyer.

Wikipedia has a very good summary about what this principle is about:

Command-Query-Separation states that every method should either be a
command that performs an action, or a query that returns data to the
caller, but not both. In other words, asking a question should not change
the answer. More formally, methods should return a value only if they are
referentially transparent and hence possess no side effects.

CQRS is an OOP architecture design pattern building on that imperative principle.

Note

There are many different variations of the CQRS pattern that all have their place. This post describes a simple variant that does not use Messaging, DomainEvent or EventSourcing patterns, which are commonly mentioned with CQRS.

At its heart is the separation of Read and Write methods. This can be easily achieved by just having two different service layer classes instead of one. This insight alone is not really helpful, its just moving methods around. Coupled to this change however is the notion, that read and write models don’t necessarily have to be the same:

  • Getter/Setters: We don’t need setters in the write model, instead we use explicit methods for each write operation. Entities are also much simpler without bi-directional relationships that read/write models often need. Read-only models (ViewModel) don’t need getters/setters. They could just use public properties or even be arrays.
  • ORM Performance, Lazy Loading and Query Restrictions: In a write scenario we mostly need queries by primary key, changing only some objects. In read scenarios we need complex queries, joins and aggregations. A separation prevents both from affecting each other negatively.
  • Mapping Data-Transfer objects becomes simple. Instead of sending the whole model back and forth, for the write scenario you define use-case specific tasks and updates. This is much simpler than generic mapping of Data Transfer objects to complex object graphs.
  • Transforming SQL directly to a view model in a highly optimized way. Simple concept, but it has huge implications: A pure read-only model is simple to maintain and refactor for any performance requirements that come up.

Furthermore with CQRS our write service methods are not allowed to return data anymore (With the exceptions to the rule of course). This may increase the complexity at first, but it has benefits for scalability and decoupling. Following this principle allows us to turn every command from synchronous to asynchronous processing later without much problems. Services are allowed to throw exceptions and refuse the execution. Proper validation of input data should happen before executing write operations though.

One drawback of CQRS: A naive implementation doubles the amount of classes and services to write. But there are simple shortcuts to avoid having to write a full blown read-model if you don’t apply CQRS religiously:

  • Don’t use different data-stores for both models.
  • Mapping SQL to PHP arrays and stdClass objects through a simple gateway is very simple. You can easily generalize this to a simple object that gives efficient access to all your data.
  • Share parts of the write and read models if you don’t have to compromise much.

The important thing to take away from this separation is, that the read part of your application is seldom the part where business value lies in. Moving this code into a different part of your applications allows you to work with the underlying data-source much more directly, without need for much abstractions.

One thing that CQRS puts forward is the concept of Commands, Command Handlers and Command Bus and how they interact. Commands are simple objects that contain all the data necessary to execute an operation. Each command is processed by exactly one command handler. The Command Bus is responsible to route a command to its appropriate command handler.

Example

Here is the most simple implementation of the Command Bus, just holds a hash map mapping command class names to command handlers:

<?php
interface Handler
{
    public function handle($command);
}
class CommandBus
{
    private $handlers;
    public function register($commandClassName, Handler $handler)
    {
        $this->handlers[$commandClassName] = $handler;
    }

    public function handle($command)
    {
        $this->handlers[get_class($command)]->handle($command);
    }
}

In your code you would always pass Commands to the bus. That way the command bus is a central entry point to any write operation. With this we can rewrite the TransferMoney service:

<?php
class TransferMoneyCommand
{
    public $sourceId;
    public $destinationId;
    public $money;
}

class MoneyTransfer implements Handler
{
    private $accountDao; // ctor omitted

    public function handle($command)
    {
        $source      = $this->accountDao->find($command->sourceId);
        $destination = $this->accountDao->find($command->destinationId);
        $money       = new Money($command->amount);

        $source->withdraw($money);
        $destination->deposit($money);
    }
}

There is also a benefit from the mental model of commands compared to the “generic” term of model requests in EBI. Your write model gets task oriented.

Routing everything through the command bus has several benefits as well:

  • Nesting handlers that take care of transactions, logging, 2 phase commits
  • Order nested command calls sequentially instead of deep into each other.
  • Asynchronous processing with message queue become an option

The command bus acts as the application boundary as described in the Entity-Boundary-Interactor pattern, usage is simple:

<?php
class MoneyController
{
    public function transferAction(Request $request)
    {
        $commandBus = $this->container->get('command_bus');
        $commandBus->handle(new TransferMoneyCommand(
            $request->request->get('sourceId'),
            $request->request->get('destinationId'),
            new Money($request->request->get('amount')
        ));
    }
}

Pros and Cons

I really like CQRS for multiple reasons. It offers explicit guidance how solve tasks by making use of a range of different design patterns. This is very helpful, because the code-base is based on conventions that are simple to understand by everyone on the team. This structure liberates you from the curse of choice and gives you a cookbook of best-practices how to solve problems. You want this structure for all your applications, regardless of what architectural pattern you have chosen.

Embracing the difference between read and write models is another plus of CQRS. It is very helpful, not to try fitting both read and write into one model. You also don’t run into so many read-related problems with an ORM any more. You design the entities to be optimized for the write model only, loose the bidirectional associations and avoid all the optimizations here and there for read performance.

Compared to EBI, where we had to maintain a mapping between DTOs and entities, CQRS explicitly uses the command based approach to avoid the complexity of these mappings. You will still have to map between command and entity, however doing so in the context of a use-case simplifies the code considerably compared to a generic mapping solution.

One negative point: Its difficult to work with commands not returning any data in some cases. You need to find simple ways to return messages to the user. For that you also need to validate commands on the “client” or controller side, using the read model, so that you can prevent invalid/illegal commands from being sent as often as possible.

Without return values from your models, you are left to using mocks as means of testing. This might be more difficult for developers to use and understand. This problem goes away however, if you combine CQRS with Event Sourcing. A topic that I will discuss in the next blog post.

]]>
Sat, 18 Aug 2012 00:00:00 +0200
https://beberlei.de/2012/08/16/oop_business_applications__data__context__interaction.html https://beberlei.de/2012/08/16/oop_business_applications__data__context__interaction.html <![CDATA[OOP Business Applications: Data, Context, Interaction]]> OOP Business Applications: Data, Context, Interaction

Other posts in this series:

The next design pattern for technical system design is called “Data, Context, Interaction”. It’s inventors Trygve Reenskaug and James Coplien call it a new paradigm, in the context of PHPs language constraints it can be classified as architectural design pattern. I came across this pattern through Gordon Oheim. He has also invested quite some time going through the book on DCI and trying to understand this pattern with me.

As in the EBI pattern, DCI separates data objects (Data) from behavior implemented in use-cases (Context). Data objects are never directly involved in these use-cases, but are “casted” to their respective roles that they are taking in the use-case. This can be implemented with aggregation or traits in PHP.

One goal of DCI is to get object-oriented programming to a place, where you can look at a use-case and its roles and be very sure that the program is running correctly, because you can guarantee that there are no unexpected side effects. It also aims at keeping the amount of code required to keep in mind for a given use-case as small as possible, allowing developers to reason about the code with more confidence.

Terminology

In DCI, data are objects that encapsulate the state of the application. As in EBI, they only contain logic that is true for all the possible use-cases. These classes represent what the system is.

Roles and Context represent what the system does and are supposed to capture the End User’s Mental Model as close as possible.

Roles add behavior to the data objects by either wrapping them or acting as traits for the data objects.

Context is the use-case and fulfils its role by making roles interact with each other. These roles then modify the underlying data.

Example

Starting from the bank account example of the EBI post, we can introduce DCI quite easily by adding the missing concept of roles. In the use case of transferring money that would be:

  • TransferMoneySource
  • TransferMoneySink (Destination)

We can do this by either introducing two interfaces and two concrete implementations that accept the BankAccount as object to work on or by building two traits. I will go down the route with traits, to show one possible use-case for this new PHP 5.4 feature. The idea here is to implement all the behavior necessary on a trait and then have simple dummy objects using them in the tests for this use-case.

<?php
trait TransferMoneySource
{
    public function transferTo($destination, Money $money)
    {
        $destination->transferFrom($destination, $money);
    }
    abstract public function withdraw(Money $money);
}
trait TransferMoneySink
{
    public function transferFrom($source, Money $money)
    {
        $source->withdraw($money);
        $this->deposit($money);
    }
    abstract public function deposit(Money $money);
}

Any checks for balances, credit limits or other things would also happen here. The withdraw and deposit on the data objects should be as dumb as possible, so that their implementation does not cause any side-effects on the execution of this use-case.

The bank account would then be modified to look:

<?php

class BankAccount
{
    use TransferMoneySource, TransferMoneySink;

    private $balance;

    public function __construct()
    {
        $this->balance = new Money(0);
    }

    public function withdraw(Money $money)
    {
        $this->balance = $this->balance->subtract($money);
    }

    public function deposit(Money $money)
    {
        $this->balance = $this->balance->add($money);
    }
}

The use-case TransferMoney would then be modified to create Roles instead of Data objects from the DAO. This can be a bit tricky when you have multiple data objects implementing the same role and you have no way of knowing which underlying data object to pick. The binding of data objects to roles happens in the use-case. The use-case needs a means to retrieve objects with certain roles, which then access underlying data sources. To avoid that your use-cases have to know about how to bind roles to data, you could use GUIDs in your application and fetch all objects from one data store. Another way would be to implement data access objects for roles, that then know how to retrieve their corresponding data.

<?php
class MoneyTransfer
{
    private $source;
    private $destination;

    public function __construct($moneySource, $moneySink)
    {
        $this->source = $moneySource;
        $this->destination = $moneySink;
    }

    public function transferMoney(Money $money)
    {
        $this->source->transferTo($this->destination);
    }
}

The simplicity of this is appealing, however don’t forget that we have abstracted I/O completely here. There has to be code that deals with that part of the system somewhere. However this again is not at the heart of all the DCI examples out there, making it difficult to reason about the actual practical implications.

Note

One drawback with this example is, that PHP does not support typehinting for traits.

Here is an example of how the bank application service could look like:

<?php
class BankApplicationService
{
    public function transferMoney($sourceId, $destinationId, Money $amount)
    {
        $source      = $this->objectStorage->find($sourceId);
        $destination = $this->objectStorage->find($destinationId);

        $useCase = new MoneyTransfer($source, $destination);

        $conn = $this->conn->beginTransaction();

        try {
            $result = $useCase->transferMoney($amount);
            $conn->commit();

            return $result;
        } catch(\Exception $e) {
            $conn->rollback();
        }
    }
}

The ObjectStorage here is a service (repository) that can find any persistent data object by a global ID. This is necessary, because it doesn’t actually matter what data object uses the necessary traits for this use-case.

Again as in EBI, in a bigger system you would need to find some abstraction layer that does this in a more generic way.

Conclusion

When Gordon started showing me this pattern we were both puzzled as how to actually implement this in the real world. Especially the concept of binding roles to data objects still confuses us. Most notably why the use of traits or aggregates should actually constitute a new programming paradigm instead of just another way to do OOP.

In Scala casting data objects to roles is actually possible by binding traits to objects at runtime. This is not possible in PHP however and has to be done statically.

Compared to EBI, DCI focuses drastically on transaction script domain logic, by suggesting to implement roles for every use-case for the sake of avoiding side-effects. This is actually is very valuable lesson from this pattern. Finding means to decrease the complexity of software is always a good thing. And the explicit definition of this concept as roles is actually easy to explain to other programmers.

One thing that is lacking in DCI is that there is no concrete mechanism to deal with the boundary to other parts of the system. This is actually a step back from EBI and I suggest using EBI pattern in combination with DCI to solve this.

The largest benefit from DCI (and its self proclaimed goal) is the mapping from the users/customers mental model directly into the code by using Use-Cases and Roles. The communication with the customer about behavior can exclusively focus on the current context and its use-case. Mapping this behavior to actual data can then be done in a different step.

This simplification of use-cases and reduction of side-effects between different parts of the system has other benefits: It can lead to easier to test code and makes it much easier for developers to develop on small and isolated parts of the system.

]]>
Thu, 16 Aug 2012 00:00:00 +0200
https://beberlei.de/2012/08/13/oop_business_applications_entity_boundary_interactor.html https://beberlei.de/2012/08/13/oop_business_applications_entity_boundary_interactor.html <![CDATA[OOP Business Applications: Entity, Boundary, Interactor]]> OOP Business Applications: Entity, Boundary, Interactor

Other posts in this series:

Continuing the series, I will talk about Entity, Boundary and Interactor (EBI) an architectural design pattern. I first heard about it in a keynote video of Uncle Bob on Ruby Midwest called “The lost years of architecture”. This is also described as Hexagonal architecture or “Ports and Adapters” by Alistair Cockburn.

In this pattern any client of the system talks to the model using a model request object and receives data from the model in form of model responses objects. Additionally when your model talks to any kind of persistence or subsystem it should go through an adapter that is replaceable. For a model architecture designed this way, you can replace any part of the UI oder lower layers by writing new adapters and plug them in.

Terminology

To iterate the terminology:

Entities are objects that represent the system data of the application. They don’t contain much logic except rules that are set in stone and are never going to change (based on physic or mathematical laws or something).

Interactors are use-case objects, they contain the business logic that is valid in the current use-case and works with entities to fulfil their task.

The boundary is responsible for translating model request/response into a format that the UI or actor can understand. They also mediate between model and lower levels, for example to manage database transactions.

An example

Lets start with an example in PHP code. We will keep using a common example throughout all blog posts, the usual Bank Account/Money Transfer. The use case is the money transfer from one bank account (source) to another one (destination).

We start by implementing the entity BankAccount:

<?php
class BankAccount
{
    private $balance;

    public function __construct()
    {
        $this->balance = new Money(0);
    }

    public function withdraw(Money $money)
    {
        $this->balance = $this->balance->subtract($money);
    }

    public function deposit(Money $money)
    {
        $this->balance = $this->balance->add($money);
    }
}

It is straightforward to implement the withdraw and deposit functionality. The Money object implementation is omitted here.

The Interactor handles the use-case of transferring money from the source to the destination account:

<?php
class MoneyTransferRequest
{
    public $sourceId;
    public $destinationId;
    public $amount;
}
class MoneyTransferResponse
{
}

class MoneyTransfer
{
    private $accountDao; // ctor omitted

    public function transferMoney(MoneyTransferRequest $transfer)
    {
        $source      = $this->accountDao->find($transfer->sourceId);
        $destination = $this->accountDao->find($transfer->destinationId);
        $money       = new Money($transfer->amount);

        $source->withdraw($money);
        $destination->deposit($money);

        return new MoneyTransferResponse();
    }
}

The MoneyTransferRequest and MoneyTransferResponse objects are dumb php objects (so called data-transfer objects, DTO).

You can see in the example that we use a Data Access object to retrieve the source and destination account entities from some storage subsystem. To follow the EBI design pattern, we have to decouple this data access object from the model, by offering a port (Interface):

<?php
interface AccountDaoInterface
{
    public function find($accountId);
}

This way our business logic is storage independent.

An example for a boundary would be the requirement for a transaction in the bank account sample. We need to wrap the whole MoneyTransfer use-case in a transaction. Lets say the invocation of our Use-Case is controlled through some kind of application boundary object:

<?php
class BankApplicationBoundary
{
    private $applicationFactory;

    public function transferMoney(MoneyTransferRequest $request)
    {
        $connection = $this->applicationFactory->createConnection();
        $connection->beginTransaction();

        try {
            $useCase = new MoneyTransfer($factory->createAccountDao());
            $result = $useCase->transferMoney($request);
            $connection->commit();

            return $result;
        } catch(\Exception $e) {
            $connection->rollback();
            throw $e;
        }
    }
}

This is a very elaborate way to describe that calling the transfer money use-case is wrapped in a Transaction, another port for the storage system to manage transactions in this case. The code here is very explicit about the actual task. In a real application you would probably find a more generic approach to getting this job done.

Boundary Abstraction

Thinking about the boundaries I came up with a library several month ago called Context, which I deprecated already (Reason below). It allows you to wrap calls to the model by some sort of proxy that transforms the request and response and also handles transactions and such. Loosely spoken this was actually some kind of AOP library, using the limited ways that PHP provides to implement AOP (magic __call proxies).

With context you would do something like:

<?php
$context = $this->getContext();

// 1. direct invocation
$myService = new MyService();
$context->execute(array('service' => $myService, 'method' => 'doSomething', 'arguments' => $args));

// 2. proxy wrapping
$myService = $context->wrap(new MyService());
$myService->doSomething($args);

The second way is obviously way more readable, but its also rather magic.

I deprecated this library because in the end it wasn’t really helpful that much. Implementing an application specific proxy for services is done in almost no time and then it solves all your specific needs. My main problem with the library is that it tries to magically take away the need to design the boundary of your application yourself - in a way that is not really coherent to other developers.

In my own current greenfield applications I quickly went away from using it, since a custom application proxy as shown in this Gist is really much simpler to implement and use.

Using with Symfony2

As I am currently exclusively developing Symfony2/Silex applications, applying EBI to Symfony2 framework based applications is very important to me. The biggest difficulty here is the Form layer, especially the request data-mapping and validation concerns, which are normally part of the model. There are two approaches I came up with to solve this:

  • Build Forms for arrays or DTOs and send them through to the boundary to the model. You have to validate the data again on the model, which is annoying, but in this case the clean way. This is not so easy to do with complex forms though as you need to map the request objects to your entities.
  • Create a Model Request that wraps and hides the form behind a simple data mapping API. This way you can make it look as if you would map a DTO onto an object, but in this case you are using the Form API as the mapper.
<?php
class MyService
{
    public function edit(EditRequest $request)
    {
        $entity = $this->dao->find($request->id);
        $this->dataMapper->transform($request, $data);
    }
}

The problem with this approach is, that you cant really unit-test these methods anymore, because the complexity of the form layer mapping cannot be mocked with this API. Instead your tests always need the full form layer (with validation, depending services etc.) to allow for real-world testing. Additionally you have to make the DataMapper throw an exception that you can catch in the controller, rendering the appropriate response. Using exceptions for controlling executions paths is not a very good practice though.

Another thing that actually helped was the SensioFrameworkExtraBundle and ParamConverters. In my project I now have the framework building the Model Request objects by convention from the HTTP Request, so that I only need to pass them on and can skip the actual mapping of HTTP Request to Model Request.

Pros and Cons

This design pattern very closely resembles what Fowler calls Service Layer pattern in PoEAA. EBI is going a bit more into detail by naming individual parts of the pattern more explicit. Without more restrictions however using this pattern will drive you towards many of the problems described in my previous post.

Clean separation from frameworks is achieved, depending on the actual usage however only at a significant cost. Never forget stepping back and thinking about further abstractions, otherwise applying EBI is leading to lots of code being manually written.

This already shows one particular annoyance are the data-transfer objects. You need to invest quite some work to get a mapping working from entities to transfer objects and back. In the process you will loose the convenience of “Open Entity Manager in the View”, where you can lazy load any data you want to access in the view. This is quite a painful step, because you are loosing lots of flexibility. Much more annoying is the need to update entities from data-transfer objects, requiring sophisticated code for merging of partial object graphs.

What this design pattern improves is the testability of code and also the execution of tests is MUCH better, when you don’t have to go through the whole application stack to test something.

Implementing behavior into the use-cases also avoids lots of lasagna code compared to a messy domain driven design. You get a very good overview of what is actually happening just by looking at the Model Request and Interactor classes. However depending on the use-case the classes can get very big and might need lots of collaborators, which make the problem complex again.

It is important to note that aggregating the domain logic in the use-cases actually means going to some sort of transaction script processing, away from domain driven design. I am pretty sure that this is not necessarily the intention of this design pattern from a POV of Uncle Bob. However depending on the sophistication of the applications domain logic, transaction script is actually a very good pattern for simple to medium complex use-cases and I like to have this as a general rule for developers (“Put behavior on the use-case”).

In conclusion I can partially recommend using the EBI pattern. You have to be careful to find abstraction layers that keep your code DRY and SOLID however, something which does not come naturally with this pattern. If you are not careful you end up with all the “messy points” that I mentioned in my previous blog post.

You should be especially careful to avoid lots of DTO <-> Entity Mapping code by using some code-generation for example to do parts of this job for you. The worst outcome with this pattern is, when you manually code layers for HTTP Request/Form => DTO => Entity mapping and the other way around.

]]>
Mon, 13 Aug 2012 00:00:00 +0200
https://beberlei.de/2012/08/11/oop_business_applications__trying_to_escape_the_mess.html https://beberlei.de/2012/08/11/oop_business_applications__trying_to_escape_the_mess.html <![CDATA[OOP Business Applications: Trying to escape the mess]]> OOP Business Applications: Trying to escape the mess

Note

tl;dr Version: I present a list of problems with OOP business applications from my personal experience and then introduce 3 approaches that I was put in contact with over the last year, DCI, EBI and CQRS. This post is an introduction to a series about this approaches.

I mentally divide the world of programming into two large parts:

  1. Library and system programming
  2. business applications

The first produces code solving technical problems, often in a beautiful and dedicated way. The second regularly produces mess, hopefully still serving customers by optimizing their business. This differentiation is an exaggeration, but from my experience its far easier to run a greenfield business project into the ground than a new library.

There has to be a reason why so many programmers dedicate much free time to open source projects. It might be empirical prove, that all our business projects don’t make us happy at the end of the day and we have to show ourselves that we can do better.

I enjoy writing business applications more than libraries, but they are also much more difficult to get right in both social and technical dimensions. One motivational driver of my open source activities always was to simplify the technical dimension of OOP business applications.

My Personal List of Annoyances

This blog post is not about blaming customers (as the usual suspect), its about finding the problems in usual OOP code of business systems from my experience with PHP projects. To keep things short, I will list my personal list of technical annoyances in business projects in no particular order. These are highly subjective points.

  • Getter/Setter Madness of Objects that are used to store data into databases. Leading to huge objects that are essentially just meaningless stores of data.
  • Updating complex object graphs from user input.
  • Separation of model, controller and views, especially when dealing with forms. This is very complicated to achieve.
  • Lazy Loading and Query performance problems when using an ORM. Additionally replacing parts of the views with hand-crafted SQL is often difficult and/or time intensive, when already using an ORM.
  • Inability to decouple code that was written with CRUD (generators and utilities) in mind towards more domain driven code, when the requirements change.
  • Dependency mess leading to hard to test code, generally leading to untested applications. Too often frameworks create huge dependencies that make your domain model messy and complex to mock in tests.
  • Focus on synchronous request/response handling makes later usage of message queues very complicated and expensive to refactor towards.
  • Tight coupling of model against the framework code.
  • Junior developers have too much freedom, when there is not much structure in the code, but rather just a big ball of mud. Additionally with the tight coupling of so many components, junior developers get easily lost in the code and produce unexpected side-effects.
  • Use-cases easily span more than 5 classes. To grasp the interaction you have to keep lots of code in your head, especially complicated for junior developers again. As they cannot really work on a problem in isolation.
  • Constant violations of SOLID principles due to framework or persistence requirements.
  • Compared to library development, user-facing applications often have much higher coupling between classes and namespaces. In libraries its often easy to find different concerns, but in applications they seem hard to divide for developers.
  • In combination with a rich javascript client, requirements to update objects through their CRUD API produces concurrency and lost-update hell, that dynamic server-side applications could mostly avoid before.

Essentially if you work in a tight schedule, project based environment where the decision makers sell rapid application development and prototyping, then you often have only one, maybe one and a half attempts to get the big picture right. From that moment on you have to build on that decision and hope for the best.

We have tried not to go the RAD+CRUD tools ways in several projects, to escape the problems listed above. But without changes in your mindest you end up with hand written mess, compared to getting there with tools.

Specifically Domain Driven Design applied naively can make the problem even worse. It easily leads to lasagna code, where you have layers of layers that are very hard to understand. Personally I prefer spaghetti code over lasagna code, because its comparatively simpler to understand.

Finding new Approaches

Rather than to embrace the suck and dive deeper into CRUD architectures, I felt there has to be some solution to organize business models structurally to avoid all (or most) of these problems. In the PHP world with Symfony2 and Doctrine2 you have a powerful toolbox to avoid many of the problems above, but it is still not simple to write clean object oriented applications.

After years of participation in both open source projects I still feel there is a missing puzzle piece, to reach a clean separation of all the model concerns from framework and persistence.

At some point I would really like to close the gap between desired technical state of a project and the state it is actually in. I know there is no size fits all solution, but I fell a checklist of architectural patterns with pro and con arguments allows me to adjust the expectations about how a system looks in the end.

Thanks to Gordon, Stefan and Thomas I was introduced to several approaches to OO application design that I started to explore in the past last months:

They are not new and have all been around for several years already. I would describe all three of them as architectural design patterns, though some people might probably disagree with this classification. All three approaches make you think about application development beyond just “service layers” in radically new ways. All three have helped me rethink business applications in different ways.

In the next weeks I will talk about each of these approaches individually, show some examples and then wrap up my thoughts about all of them.

Note

Gordon and I will talk about this topic on the Free and Open Source Conference in Sankt Augustin Germany on 25th/26th August of 2012. Feel free to drop by and discuss this topic!

]]>
Sat, 11 Aug 2012 00:00:00 +0200
https://beberlei.de/2012/06/03/setting_up_development_machines_with_puppet.html https://beberlei.de/2012/06/03/setting_up_development_machines_with_puppet.html <![CDATA[Setting up Development Machines with Puppet]]> Setting up Development Machines with Puppet

I wrote about Vagrant last week and mentioned to talk about Puppet in the future. Before I will dive into Puppet in combination with Vagrant I want to share my general experiences with Puppet (beginners level, I just played with this today).

Puppet is a configuration management tool. It uses a declarative language to describe how a system should look like. Upon invocation of puppet it attempts to get the system from its current state into the desired state by installing packages, executing programs/scripts and creating files/directories.

My laptop was still on Ubuntu 10.04 this morning, but for Symfony Live I needed to upgrade to 12.04. This was necessary to run a Virtualbox VM on my laptop that I built on my 12.04 desktop. Additionally I really like the Unity desktop and wanted to get away from the Gnome that is in 10.04.

After a backup and fresh installation of Ubuntu I realized how many things I have to install to get this machine productive:

  • LAMP stack + a bunch of other databases and services
  • Git, Svn, Hg, ..
  • Virtualbox + Vagrant
  • Vim with my Dot-Files, including Command-T which requires compilation, and a bunch of development tools such as tree, ack-grep.
  • PHP Tools (PHPUnit, PHPCS, Xdebug, Xhprof)
  • Git with Author Information and global ignore file
  • A bunch of .bashrc initializations
  • Some open source projects I always have around (Doctrine, …)
  • Setup automatic backup with my cloud storage provider (Strato Hidrive - they support rsync)
  • and I probably forgot a bunch of tools just thinking about this today.

I realized not only virtual machines, but also their hosts (my development machines) could be automatically setup with Puppet.

So instead of saving my .bash_history to remember the steps I automated them using Puppet. In Puppet you define Resources of several types, for example Files, Packages or shell commands. You group these resources together in a .pp file and then invoke puppet apply <file>.pp. You can install Puppet via sudo apt-get install puppet on Ubuntu.

The idea is to define Puppet resources in a way that they are performed once and then the system has this resource. This means you can run puppet as often as you want and it will only activate the missing resources.

I separate the installation process into two puppet files, one that has to be run with root ~/.puppet/devmachine-root.pp and another one that has to be run with my own user ~/.puppet/devmachine.pp.

Because every dev-machine setup is unique, I will just show some examples of my puppet file to get the message across:

# /home/benny/.puppet/devmachine.pp

# VIM Dot Files
vcsrepo { 'vim-dotfiles':
    path     => "/home/$id/.vim",
    ensure   => present,
    provider => git,
    source   => 'https://...'
}

file { 'vim-dotfiles-symlink':
    path   => "/home/$id/.vimrc",
    ensure => link,
    target => '.vim/vimrc',
    require => Vcsrepo['vim-dotfiles']
}

exec { 'vim-make-command-t':
    command => 'rake make',
    cwd     => "/home/$id/.vim/bundle/Command-T",
    unless  => "ls -aFlh /home/$id/.vim/bundle/Command-T|grep 'command-t.recipe'",
    require => Vcsrepo['vim-dotfiles']
}

The three keywords vcsrepo, file and exec are called types. They define what functionality should be executed by puppet. Vcsrepo is a custom type that you can grab from Github.

  • The first block uses Git to make sure that in my home directory the ~/.vim directory is a checkout of a given remote git repository.
  • The second file block makes sure that ~/.vimrc points to ~/.vim/vimrc using a symlink.
  • The third block compiles the Command-T VIM plugin. Inside the exec block you can see the unless condition. This is necessary to prevent the command from being executed whenever devmachine.pp is executed with puppet. The require makes sure this resource is only activated after the dotfiles were installed.

Another nice example is the configuration of Git:

# Configure Git
exec { "git-author-name":
    command => 'git config --global user.name "Benjamin Eberlei"',
    unless => "git config --global --get user.name|grep 'Benjamin Eberlei'"
}

exec { "git-author-email":
    command => 'git config --global user.email "kontakt@beberlei.de"',
    unless => "git config --global --get user.email|grep 'kontakt@beberlei.de'"
}

exec { "git-global-ignore":
    command => "git config --global core.excludesfile /home/$id/.gitignore",
    unless  => 'git config -l --global|grep excludesfile'
}

file { "git-global-ignorefile":
    path => "/home/$id/.gitignore",
    ensure => present,
    content => "*.swo\n*.swp\n"
}

Here A bunch of commands is executed unless some option is already set. At last the global .gitingore is filled with the patterns of Vim temporary files.

I also automated all the other steps listed above, but will spare you the details. For the LAMP Stack + PHP I used the now defunct dietervds/puppet-symfony2 repository as an inspiration. It ships a set of Puppet Modules. You can just put them into your ~/.puppet/modules folder and they are then available in any of your puppet files.

I can now reuse this on the different machines: desktop at home and at work and my laptop. Whenever I install a new tool or change some configuration of my system I can directly put this into puppet and keep the setup synchronized on all the machines I work with.

You can find more detailed resources on the PuppetLabs website:

]]>
Sun, 03 Jun 2012 00:00:00 +0200
https://beberlei.de/2012/05/31/virtual_machines_with_vagrant__veewee_and_puppet.html https://beberlei.de/2012/05/31/virtual_machines_with_vagrant__veewee_and_puppet.html <![CDATA[Virtual Machines with Vagrant, Veewee and Puppet]]> Virtual Machines with Vagrant, Veewee and Puppet

I have been playing with Vagrant these last days, a tool to automate Virtualbox VM management very efficiently. Being always late to new technologies investigating Vagrant now felt about right. I am using Virtualbox for virtualization for about a year now but have build all my boxes manually. I have these machines setup:

  • Windows 7
  • Ubuntu with lots of crazy PHP builds
  • Ubuntu with Oracle, PostgreSQL for Doctrine Unit-Tests

The main benefit from this is that I don’t have to clutter my main machine with packages and dependencies that might potentially break.

To benefit from Virtualization on a larger scale I wanted to investigate automation of box-building and was pointed to Vagrant. This is a ruby tool that allows to copy and modify virtualboxes on the fly. You define a basebox and inherit from this box in your projects. Whenever you work on the project you start a copy of this basebox and install more system-dependencies using Chef or Puppet. That allows you to put the VM into a defined state just for this project. No matter how many different and weird dependencies you need, they will always be in this VM that is just created for the project and destroyed at the end of the day.

I am using Ubuntu (currently 12.04) and tried starting with the Vagrant example. It fails, because their example box uses a more current Virtualbox Guest Additions. To be able to use vagrant, you need a basebox with the virtualbox and guest additions versions matching correctly.

To build your own Virtualbox you can use the Vagrant plugin Veewee. Install it from Github to get all the latest VM templates:

$ git clone https://github.com/jedi4ever/veewee.git
$ cd veewee
$ sudo gem install bundler
$ sudo bundle install
$ alias vagrant="bundle exec vagrant"

You need Ruby and Gems installed for this to work. Veewee can be installed using Gems, but this not necessarily gives you the version with the most recent templates that match your own operating system version.

Now lets define our own basebox definition and copy from an existing template that Veewee provides:

$ vagrant basebox define 'beberlei-ubuntu-12.04-i386-php' 'ubuntu-server-12.04-i386'

This creates a copy from the default ubuntu server template into a folder definitions/beberlei-ubuntu-12.04-i386-php. You can start modifying the files in there to adjust the build process of the virtual machine image. For me its important to modify the definitions.rb and add the following, otherwise building the VMs fails:

:virtualbox => { : vm_options => ["pae" => "on"]},

You can then open up the postinstall.sh and install more packages that you need in your personal basebox. For example a LAMP stack. Do this right after the lines that do apt-get install -y:

export DEBIAN_FRONTEND=noninteractive
apt-get -y install php5 php5-cli php5 mysql-server-5.5 libapache2-mod-php5

If you are done you can start building a virtualbox image with:

$ vagrant basebox build

Make sure not to start typing in the console window that open ups. It is automatically controlled from a ruby script and every change to the keysequence breaks the building. This step takes a while. (Enough to write a blog post while watching a boring football game for example).

When this is done, you can verify and export the VM into virtualbox/vagrant.

$ veewee vbox validate 'beberlei-ubuntu-12.04-i386-php'
$ vagrant basebox export 'beberlei-ubuntu-12.04-i386-php'
$ vagrant box add 'beberlei-ubuntu-12.04-i386-php' 'beberlei-ubuntu-12.04-i386-php.box'

Now this box is also available as Vagrant base box. For example in a project that we want to use with vagrant, do:

$ cd myproject/
$ vagrant init 'beberlei-ubuntu-12.04-i386-php'
$ vagrant up
$ vagrant ssh

Although the blog-title suggests it, Puppet hasn’t been used much up to this point. The use of puppet with Vagrant will be part of a next blog post.

]]>
Thu, 31 May 2012 00:00:00 +0200
https://beberlei.de/2012/05/26/doctrine_goes_mit.html https://beberlei.de/2012/05/26/doctrine_goes_mit.html <![CDATA[Doctrine switches from LGPL to MIT]]> Doctrine switches from LGPL to MIT
tl;dr We moved all the Doctrine PHP projects from LGPL to MIT license successfully in roughly 4 weeks. For that we used a small web-based tool to ask the permission of 358 authors that contributed in the last 6 years.

The Doctrine project has been LGPL licensed since it was started in 2006. When I first started contributing in the project late in 2009 we were constantly asked about changing the license to a permissive license such as MIT or New-BSD. My subjective feeling is that there are much more libraries with permissive licenses than 10 years ago. And there are others that back my feeling up with facts. The fear of getting screwed when using a permissive license has probably declined in favor of the benefits. As Lukas puts it:

Maybe with enough experience you start to realize that it happens close to never that a proprietary fork of an open source project ends up outpacing the original project.

We wanted all the benefits of permissive licenses as well. So at the beginning of this year we as Doctrine project first started to investigate how the task of switching the license could be done. After a short email with the FSF it was clear that we had to get the approval of every single committer or remove their code.

VLC successfully attempted a license change last year from GPL to LGPL, so we we’re hopeful to get this done. After discussing with Lukas we came up with the idea of a tool that helps this process, so I wrote it. It features:

  • Import of all commits from one or many Github projects
  • Aggregation of commits to authors based on e-mail address from Git log
  • Status approved/not-approved for every author.
  • An e-mail engine to send a request for approval to every author using the awesome Mailgun service, who has not approved yet. We use Mailgun for fun and to have a way to act on bounces easily.
  • Admin functionality to mark commits as “trivial or deleted”.
  • An audit trail to account for all changes and who has done them.
  • Symfony2/Doctrine2 application with debian packaging using FPM.

This project is not yet open-sourced but will be when I have some time to clean up the hardcoded passwords and write a small README on how to adjust the templates.

After importing the commits of all the LGPL licensed Doctrine projects we realized that we have 358 unique committers and about 40-50 without an e-mail address (from the old SVN days). So we started googling for nicknames and eventually collected all e-mail addresses except maybe 6-8 missing ones.

Now after 4 weeks of bi-weekly e-mail reminders, everyone except 16 committers have accepted the license change. No single person refused the change. All commits of the 16 people left we’re luckily not part of the code base anymore when we completely rewrote Doctrine 2 or trivial one-line changes.

I am very happy to announce that Doctrine is now an MIT licensed project. Have fun with it.

]]>
Sat, 26 May 2012 00:00:00 +0200
https://beberlei.de/2012/02/25/symfony2_controller_testing.html https://beberlei.de/2012/02/25/symfony2_controller_testing.html <![CDATA[Symfony2 Controller Testing without Application]]> Symfony2 Controller Testing without Application

Controller testing using the WebTestCase requires a Symfony Kernel and with that a complete application. However you can just ship your own simple kernel with just the dependencies necessary to test your application. This way you can easily create functional tests for bundles without the bundle requiring an application.

Create a TestKernel

<?php
// Tests/Controller/App/AppKernel.php

use Symfony\Component\HttpKernel\Kernel;
use Symfony\Component\Config\Loader\LoaderInterface;

class AppKernel extends Kernel
{
    public function registerBundles()
    {
        $bundles = array(
            // Dependencies
            new Symfony\Bundle\FrameworkBundle\FrameworkBundle(),
            new Symfony\Bundle\SecurityBundle\SecurityBundle(),
            new Symfony\Bundle\MonologBundle\MonologBundle(),
            new Symfony\Bundle\TwigBundle\TwigBundle(),
            new Sensio\Bundle\FrameworkExtraBundle\SensioFrameworkExtraBundle(),
            new JMS\SerializerBundle\JMSSerializerBundle($this),
            new FOS\RestBundle\FOSRestBundle(),
            // My Bundle to test
            new Beberlei\WorkflowBundle\BeberleiWorkflowBundle(),
        );

        return $bundles;
    }

    public function registerContainerConfiguration(LoaderInterface $loader)
    {
        // We don't need that Environment stuff, just one config
        $loader->load(__DIR__.'/config.yml');
    }
}

Creating the config.yml

# Tests/Controller/App/config.yml
framework:
    secret:          secret
    charset:         UTF-8
    test: ~
    router:          { resource: "%kernel.root_dir%/routing.yml" }
    form:            true
    csrf_protection: true
    validation:      { enable_annotations: true }
    templating:      { engines: ['twig'] }
    session:
        auto_start:     false
        storage_id: session.storage.filesystem

monolog:
    handlers:
        main:
            type:         fingers_crossed
            action_level: error
            handler:      nested
        nested:
            type:  stream
            path:  %kernel.logs_dir%/%kernel.environment%.log
            level: debug

Creating the routing.yml

# Tests/Controller/App/routing.yml
BeberleiWorkflowBundle:
    resource: "@BeberleiWorkflowBundle/Controller/"
    type:     annotation
    prefix:   /

Adding a PHPUnit bootstrap

I assume the setup that Henrik described in his “Travis & Composer sitting in a Tree K-I-S-S-I-N-G” blog post.

His setup is missing the spl_autoload_register() call in the bootstrap file though.

<?php
// Tests/bootstrap.php
$loader = @include __DIR__ . '/../vendor/.composer/autoload.php';
if (!$loader) {
    die(<<<'EOT'
You must set up the project dependencies, run the following commands:
wget http://getcomposer.org/composer.phar
php composer.phar install
EOT
    );
}
\Doctrine\Common\Annotations\AnnotationRegistry::registerLoader(array($loader, 'loadClass'));

spl_autoload_register(function($class) {
    if (0 === strpos($class, 'Beberlei\\WorkflowBundle\\')) {
        $path = __DIR__.'/../'.implode('/', array_slice(explode('\\', $class), 2)).'.php';
        if (!stream_resolve_include_path($path)) {
            return false;
        }
        require_once $path;
        return true;
    }
});

That means your bundle should have a composer.json that loads all the dependencies.

Modifying the phpunit.xml.dist

We have to tell the WebTestCase base class where to find this kernel:

<!-- phpunit.xml.dist -->
<phpunit bootstrap="Tests/bootstrap.php">
    <php>
        <server name="KERNEL_DIR" value="Tests/Controller/App" />
    </php>
</phpunit>

Now just run your web-test cases.

If you want to debug the logging happening inside the Kernel just comment out the Monolog lines to get the log-messages printed to the screen.

You have to add the Tests/App/cache and Tests/App/logs to your version control ignore files.

]]>
Sat, 25 Feb 2012 00:00:00 +0100
https://beberlei.de/2010/02/07/resources-for-a-php-and-hudson-ci-integration.html https://beberlei.de/2010/02/07/resources-for-a-php-and-hudson-ci-integration.html <![CDATA[Resources for a PHP and Hudson CI Integration]]> Resources for a PHP and Hudson CI Integration

Yesterday I finally had the time to setup my first continuous integration environment. Possible solutions for CI are phpUnderControl, Hudson (now Jenkins CI) or Arbit. Although phpUnderControl is the most wide-spread, but from I heard complex to setup/maintain, solution [STRIKEOUT:supposedly a hack] and Arbit just in an early Alpha I decided to give Hudson a shoot. Another reason for this decision, I heard it has a simple plugin architecture and is easy to install and use. Additionally Hudson is easily integrated into Netbeans and Redmine, and I use both tools regularly in development.

My motivation to dive into CI is easily explained. I just never felt it was necessary to add a continuous integration environment to my projects, since I had one or two simple bash scripts that did the job. In general this is rather annoying, because they mostly only run PHPUnit and have to be done using a cronjob or manually, without any real process of notification. Additionally you have no way to navigate the test-results, code-coverage and no history of the last builds. For projects like Doctrine 2 we have the additional requirement to support 4 different database platforms, i.e. 4 different PHPUnit configurations. Currently that is solved by me using a Bash script that iterates over the configuration file names and invokes PHPUnit.

There are already some awesome sources how to get Hudson and PHP working. I’ll list them here:

All those guides are already awesome and I would recommend choosing one of those to install Hudson, I think i can’t add anything new to those. I have used Sebastians Howto, however i also like the third one. David Luhmans guide adds lots of details that are important to get the different parts of a build process to work.

Now what these tutorials all do is that they use a bash command to execute the build process or specify an Ant Build file. However there is also a Phing Build process plugin for Hudson that allows to specify the build.xml targets to execute in the process. From the “Available Plugins” list you can choose the “Phing plugin”.

After installation you have to configure the Phing install. The Phing Plugin Wiki Page shows how to do this. You have to go to “Manage Hudson” => “Configure System” and look for Phing. There you find the dialog to configure your phing installations.

In the context of choosing a build script for your project you can now choose “Phing” instead of Ant:

You can enter the targets to build, for example on my local Hudson instance I only execute “test” for Doctrine 2, since I am not interested in the building and deployment onto the PEAR channel at this development stage.

From inside Netbeans you can then start builds by pointing to the Hudson instance. See this tutorial by one of the Hudson + Netbeans Developers. You can then start all the builds from inside Netbeans and be notified of the success or failure.

]]>
Sun, 07 Feb 2010 00:00:00 +0100
https://beberlei.de/2010/05/02/testing-database-locks-with-phpunit-and-gearman.html https://beberlei.de/2010/05/02/testing-database-locks-with-phpunit-and-gearman.html <![CDATA[Testing Database Locks with PHPUnit and Gearman]]> Testing Database Locks with PHPUnit and Gearman

For the Beta 2 release of Doctrine 2 we plan to integrate pessimistic Database-level locks across all the supported vendors (MySQL, Oracle, PostgreSql, IBM DB2 so far). This means row-level locking as defined in the ANSI SQL Standard using “SELECT .. FOR UPDATE” will be available optionally in DQL Queries and Finder methods. The Implementation of this extension to SELECT statements is rather trivial, however functional testing of this feature is not.

A general approach would look like this:

  1. Run Query 1 and 2 with FOR UPDATE into the background
  2. Have both queries lock the row for a specified time x (using sleep)
  3. Verify that one of the two processes/threads runs approximately 2*x the lock time.

Since PHP does not support process forking or threads naturally you run into a serious problem. How do you execute two database queries in parallel and verify that indeed one query is locking read access for the second one?

Side note: There are some drawbacks to this testing approach. It could be that one background threads finishes the lock sleep already when the second just starts. The locking would work in these cases, however the lock time would not nearly be 2*x seconds, producing a test-failure. We are talking about a functional test though and I will accept a failure from time to time just to be 99% sure that locking works.

Solving this problem with Gearman provides a pretty nice “real-world” example for the Job-Server that I wanted to share. This blog post contains a stripped down code-example from the Doctrine 2 testsuite. If you are interested, you can see the complete Gearman Locking Tests in on GitHub.

Gearman allows to register worker processes with the job-server and offers clients to execute jobs on those workers in parallel. After installing the Gearman job-server and PHP pecl/gearman extension (Rasmus Lerdorf has a post on installation) we can go on writing our locking tests with Gearman.

The first bit is the worker, a PHP script that tries to acquire a database lock and then sleeps for a second. The return value of this script is the total time required for acquiring the lock and sleeping.

class LockAgentWorker
{
    public function findWithLock($job)
    {
        $fixture = $this->processWorkload($job); // setup doctrine in here

        $s = microtime(true);
        $this->em->beginTransaction();

        $entity = $this->em->find($fixture['entityName'], $fixture['entityId'], $fixture['lockMode']);

        sleep(1);
        $this->em->rollback(); // clean up doctrine

        return (microtime(true) - $s);
    }
}

The glue-code for the worker script contains of the registering of the worker method with the job-server and a simple infinite loop:

$lockAgent = new LockAgentWorker();

$worker = new \GearmanWorker();
$worker->addServer();
$worker->addFunction("findWithLock", array($lockAgent, "findWithLock"));

while($worker->work()) {
    if ($worker->returnCode() != GEARMAN_SUCCESS) {
        break;
    }
}

We need two running workers for this to work, since one worker only processes one task at a time. Just open up two terminals and launch the php scripts. They will wait for their first task to process.

Now we need to write our PHPUnit TestCase, which will contain a GearmanClient to execute two of the “findWithLock” in parallel. Our locking assertion will work like this:

  1. Register two tasks for the “findWithLock” method that access the same database row.
  2. Register a completed callback using “GearmanClient::setCompleteCallback()” that collects the run-time of the individual workers.
  3. Execute this tasks in parallel using “GearmanClient::runTasks()”.
  4. Assert that the maximum run-time is around 2 seconds (since each worker sleeps 1 second)

The code for this steps could look like:

class GearmanLockTest extends \Doctrine\Tests\OrmFunctionalTestCase
{
    private $gearman = null;
    private $maxRunTime = 0;
    private $articleId;

    public function testLockIsAcquired()
    {
        // .. write fixture data into the database

        $gearman = new \GearmanClient();
        $gearman->addServer();
        $gearman->setCompleteCallback(array($this, "gearmanTaskCompleted"));

        $workload = array(); // necessary workload data to configure workers
        $gearman->addTask("findWithLock", serialize($workload));
        $gearman->addTask("findWithLock", serialize($workload));

        $gearman->runTasks();

        $this->assertTrue($this->maxRunTime >= 2);
    }

    public function gearmanTaskCompleted($task)
    {
        $this->maxRunTime = max($this->maxRunTime, $task->data());
    }
}

Now if both workers are waiting for processing the task we can run this test and get a green bar for a working lock support.

]]>
Sun, 02 May 2010 00:00:00 +0200
https://beberlei.de/2010/01/08/immutable-datetime-objects.html https://beberlei.de/2010/01/08/immutable-datetime-objects.html <![CDATA[Immutable DateTime Objects]]> Immutable DateTime Objects

One serious drawback of PHPs DateTime extension is the mutability of its instances. It can lead to serious bugs if a DateTime instance is passed between different functions and is modified at unexpected places. Additionally this possibly rules out several optimizations for scripts that make very heavy use of dates and could share references to equal timestamps.

Warning: All the code assumes that you work with one timezone only!

The following code is an immutable version of PHP’s DateTime. All setter methods throw an exception and add(),sub() and modify() clone the current instance, apply the operation and return the new instance.

<?php
namespace Whitewashing\DateTime;

class DateTime extends \DateTime
{
    /**
     * To prevent infinite recursions
     *
     * @var bool
     */
    private static $_immutable = true;

    public function add($interval)
    {
        if (self::$_immutable) {
            self::$_immutable = false;
            $newDate = clone $this;
            $newDate->add($interval);
            self::$_immutable = true;
            return $newDate;
        } else {
            return parent::add($interval);
        }
    }

    public function modify($modify)
    {
        if (self::$_immutable) {
            self::$_immutable = false;
            $newDate = clone $this;
            $newDate->modify($modify);
            self::$_immutable = true;
            return $newDate;
        } else {
            return parent::modify($modify);
        }
    }

    public function sub($interval)
    {
        if (self::$_immutable) {
            self::$_immutable = false;
            $newDate = clone $this;
            $newDate->sub($interval);
            self::$_immutable = true;
            return $newDate;
        } else {
            return parent::sub($interval);
        }
    }

    public function setDate($year, $month, $day) {
        throw new ImmutableException();
    }
    public function setISODate($year, $week, $day=null) {
        throw new ImmutableException();
    }
    public function setTime($hour, $minute, $second=null) {
        throw new ImmutableException();
    }
    public function setTimestamp($timestamp) {
        throw new ImmutableException();
    }
    public function setTimezone($timezone) {
        throw new ImmutableException();
    }
}
class ImmutableException extends \Exception
{
    public function __construct()
    {
        parent::__construct("Cannot modify Whitewashing\DateTime\DateTime instance, its immutable!");
    }
}

Its not the prettiest code, but it works.

A next optimization would be a DateFactory that manages DateTime instances by returning already existing instances for specific dates. This is not a perfect solution, since you won’t be able to enforce a single instance when you are using the relative descriptive dates or when calculating with dates using add(), sub() and modify(), however for lots of dates created from a database or other external source it might be quite a powerful optimization depending on your use-case:

namespace Whitewashing\DateTime;

class DateFactory
{
    static private $_dates = array();

    static public function create($hour, $minute, $second, $month, $day, $year)
    {
        $ts = mktime($hour, $minute, $second, $month, $day, $year);
        if (!isset(self::$_dates[$ts])) {
            self::$_dates[$ts] = new DateTime('@'.$ts);
        }
        return self::$_dates[$ts];
    }

    static public function createFromMysqlDate($mysqlDate)
    {
        list($date, $time) = explode(" ", $mysqlDate);
        if($time == null) {
            $hour = $minute = $second = 0;
        } else {
            list($hour, $minute, $second) = explode(":", $time);
        }
        list($year, $month, $day) = explode("-", $mysqlDate);
        return self::create($hour, $minute, $second, $month, $day, $year);
    }
}

This includes some date time calculations and date creation with mktime() and DateTimes unix timestamp capabilities to be able to work. Otherwise the sharing of instances could not be implemented. If you need to create shareable instances from other formats you could just create another creation method for it and convert the format for create() to be used.

]]>
Fri, 08 Jan 2010 00:00:00 +0100
https://beberlei.de/2010/01/10/application-lifecycle-management-and-deployment-with-pear-and-phar-revisited-update.html https://beberlei.de/2010/01/10/application-lifecycle-management-and-deployment-with-pear-and-phar-revisited-update.html <![CDATA[Application Lifecycle Management and Deployment with PEAR and PHAR (revisited) UPDATE]]> Application Lifecycle Management and Deployment with PEAR and PHAR (revisited) UPDATE

Some weeks ago I posted an article on using PEAR and PHAR for application lifecycle management and deployment. Since then I have gotten some feedback through comments, but also discussed the topic with colleagues. I have also optimized the approach quite considerably and even made an open-source project out of parts of it and I want to share all that is new with you. First of all, yes the presented solution was somewhat complex, partly because it is still a proposed idea and definitely up for optimizations. However I am still very convinced of my approach, something I should discuss in more detail.

The only other two languages I have ever programmed something in with more than 50 lines of code are Java and C#. In both languages you can explicitly import different dependencies like libraries and frameworks by adding them to your application, i.e. in java you would add for example Spring as an MVC Web layer, Hibernate as an ORM and several other things to your project and directly bundle them in your executable. This is very easy to configure and maintain in IDEs like Netbeans or Eclipse (C# has the same with allowing you to attach DDLs of libraries to your project). It also makes for a much more straightforward deployment.

In the PHP world this situation was quite different (up to PHP 5.3) for several reasons:

  • You could not package a library into a single distributable file.
  • The PEAR installer as the only tool for updating and managing dependencies of your application by default installs into a system/global directory. This means dependencies your application uses are located in a completely different location than your application code.
  • You can’t manage multiple versions of the same package with PEAR in this system directory, making it very hard to control servers with different applications.
  • The global directory with your application dependencies is most often not under version control, which makes deployment of applications with PEAR dependencies somewhat difficult.

There are some solutions to this problems:

  • Don’t use PEAR, but put all dependencies in your version control system.
  • Don’t use PEAR, and bundle dependencies to your code in the build/deployment process.
  • Use PEAR like in the article described, on a per project basis.

The solutions that don’t use PEAR suffer from the disadvantage that you need to keep track of all the library and framework dependencies yourself and upgrade them yourself. This might not be such a huge problem from a first glance, but in my opinion many PHP applications and projects suffer from using either no framework/library or just exactly one. There is no real cherry-picking going on the PHP world, for example I would really like to use Zend Framework for the general application layout, but still use Doctrine for the Modelling and HTML Purifier for the Output Escaping. Certain tasks might then only be solvable with the help of eZ Components, all of which are then to some extend dependencies of my application. With PEARHUB (defunct) and PEARFARM (defunct) on the horizon Read Padraic on this topic even more PHP code will be distributed using PEAR channels in the near future. My immutable DateTime code for example makes for a great little open source library that could be distributed via PEAR, as well as Yadif - a dependency injection container I am using extensively.

Question: Are you really going to manage all these dependencies correctly manually? Is everything up to date all the time, and upgradeable with ease?

The PEAR driven solution then begins to look very desirable in this light, however it has a considerable disadvantage: The PEAR installer itself works on a system-wide/user-centric basis, making it impossible to manage dependencies of several applications using only one linux user. My little Pearanha to the rescue: I have taken the PEAR installer code (which is distributed with all PHP installations across all systems) and put a new CLI tool on top of it. Its a very simple code-generator that allows to generate a re-configured PEAR installer script which only manages a single application in a given directory. This approach is also used by the symfony plugin mechanism, which internally uses the PEAR installer (did you know?).

Lets revisit my blog application example from my previous PEAR post, first install from Github and make the “pearanha” file executable and put it in your PATH (A PEAR Server Channel will follow any time soon).

Now we need to have an existing application directory somewhere, for example /var/www/blog and then we can put Pearanha on top of it with:

benny@desktop: pearanha generate-project

You then need to specify the project base dir and then the project style (for example Zend Framework or Manual) which prompts your for the directory that should be used for as the vendor/library directory that PEAR will install all the code in. You will also be prompted for a binary/scripts directory which will then hold a new PHP file for you, the file my_phpiranha.

Pro Argument: Switching to Pearanha can be done at any point in your application lifecycle. Just define an additional vendor directory for all the dependencies to go in and generate the applications pear installer and you are good to go.

The generated script is your new application specific PEAR installer and you can begin to install all the required dependencies of your application:

benny@desktop:~$ cd /var/www/blog
benny@desktop:/var/www/blog$ ./vendor/pear/my_pearanha channel-discover pear.zfcampus.org
benny@desktop:/var/www/blog$ ./vendor/pear/my_pearanha install zfcampus/zf-alpha
benny@desktop:/var/www/blog$ ./vendor/pear/my_pearanha channel-discover htmlpurifier.org
benny@desktop:/var/www/blog$ ./vendor/pear/my_pearanha install hp/HTMLPurifier
benny@desktop:/var/www/blog$ ./vendor/pear/my_pearanha channel-discover pear.phpdoctrine.org
benny@desktop:/var/www/blog$ ./vendor/pear/my_pearanha install pear.phpdoctrine.org/DoctrineORM-2.0.0
benny@desktop:/var/www/blog$ ./vendor/pear/my_pearanha channel-discover components.ez.no
benny@desktop:/var/www/blog$ ./vendor/pear/my_pearanha install ezc/ezcGraph

All this stuff is now located in /var/www/blog/vendor. Again you can use PEARs complete upgrade, remove and install functionality for your application, now without the hazzle of having to create a linux user for each project you want to manage this way, which in my opinion is a considerable simplification. The complete application (including its dependencies) can then be put under version control and be readily packaged as a single executable PHAR file by your build process.

As a side node, I did try Pyrus instead of PEAR for the same discussed purpose, however most of the current PEAR channels don’t validate against Pyrus strict standards for the package.xml file. In the future this might change and a Pyrus based application installer will then be integrated into Pearanha.

UPDATE: I renamed PHPiranha to Pearanha as its more appropriate. Also after apinsteins comment on “pear config-create” I rewrote the generate-project parts to use the config-create functionality internally, which allowed me to throw away half of the self-written code. Thanks!

]]>
Sun, 10 Jan 2010 00:00:00 +0100
https://beberlei.de/2010/04/27/doctrine-2-beta-1-released.html https://beberlei.de/2010/04/27/doctrine-2-beta-1-released.html <![CDATA[Doctrine 2 Beta 1 released]]> Doctrine 2 Beta 1 released

Today we are happy to announce that the first beta of Doctrine 2 has been released and we fixed 165 issues kindly reported by several early adopters.

You can grab the code from the Github project:

http://github.com/doctrine/doctrine2/tree/2.0.0-BETA1
git clone git://github.com/doctrine/doctrine2.git

Or download it from our website:

It is our believe that Doctrine 2 brings PHP ORMs to a new level. We are leaving behind the Active Record pattern because we think it hurts testability, project maintainability and is not a suitable abstraction (80/20) for models that exceed the complexity of a blog or otherwise simple web application. Instead we implemented a pure Data Mapper approach with help of the new Reflection functionalities of PHP 5.3, so your Domain Objects neither have to extend a base class nor implement an interface.

We also dropped most of the magical features of Doctrine 1 in favour of a simple and standardized API that is loosely based on the Java Persistence API, a technical standard for Object Relational Mappers. However we try not to blindly follow the “Program PHP like Java” approach and and deviated from JPA where applicable to make the concepts fit better into the PHP environment, such as alternatively hydrating all results into nested array structures for very high read performance.

The cornerstone of Doctrine 2 is the query language DQL. It allows to execute queries on the object level defined by your metadata in a similar fashion to SQL. You can even do Joins, Subselects and Aggregates and Group Clauses in DQL, eliminating the need to circumvent ORMs for more advanced SQL features. Nevertheless it is also possible to write plain SQL and let Doctrine 2 hydrate the results into an object graph.

In the last three month since alpha 4 we have done considerable changes and integrated lots of feedback from our users. The most notable changes are:

  • Allowing Constructors of your Domain Objects to have non-optional parameters.
  • Allow to define a natural ordering of to Many Collections that is automatically enforced trough an SQL ORDER BY statement when retrieved from the database.
  • Shipping the Symfony Console Component to replace our own Console Implementation
  • New DQL syntax to load objects partially, omitting potentially expensive fields from retrieval for the current request.
  • Changes to how bi-directional have to be defined in the mapping files.
  • Several changes to the Events API inside the ORM, to make sure many possible extension scenarios work smoothly.
  • Enhancements to our Console Tools
  • Surpassed the 1000 unit-tests each running against Postgres, Mysql, Sqlite and Oci8 drivers.
  • Moved from SVN to Git: http://github.com/doctrine/doctrine2

We also did several painful backwards incompatible changes that seemed necessary to clean up and optimize the API or allow the ORM to be even faster than before. The beta phase beginning today will not contain any larger BC breaks anymore, opening up this release for a broader testing audience.

For the next iteration several enhancements are planned:

  • Support for PDO IBM, IBM DB2, SqlSrv and PDO SqlSrv und MsSql drivers
  • Pessimistic Lock Support (FOR UPDATE and SHARED)
  • Support for Custom Hydration Modes
  • Support for Custom Persister Implementations
  • Support for handling very large collections of related objects without needing an in-memory representation of the collection
  • Separation of Doctrine\Common, Doctrine\DBAL and Doctrine\ORM into three different projects
  • Extend the documentation even further, adding quickstart tutorials, cookbook recipes and enhancing the existing chapters.

We also plan to support several extensions for the 2.0 release such as:

  • Migrations
  • PHPUnit Database Testing Integration
  • NestedSet Support
  • Symfony 2 Support
  • Zend Framework 2 Support

Please try out the new Beta release If you find the time and leave your feedback in our Issue Tracker, the Mailing-List or come discuss with us on Doctrine 2 on Freenode #doctrine-dev.

]]>
Tue, 27 Apr 2010 00:00:00 +0200
https://beberlei.de/2010/04/18/new-netbeans-php-codesniffer-plugin-version.html https://beberlei.de/2010/04/18/new-netbeans-php-codesniffer-plugin-version.html <![CDATA[New Netbeans PHP CodeSniffer Plugin Version]]> New Netbeans PHP CodeSniffer Plugin Version

This morning I took the time to merge several changes by Manuel Piechler into my Github branch of the CodeSniffer Plugin and released a new NBM file for you to download. Here are the changes done by Manuel:

  • Ability to specify the path to CodeSniffer Binary
  • Automatic Detection of all the installed Coding Standards
  • Fixed Automatic Binary Detection on Windows

The only open TODO for this project is the possibility to specify different standards on a per project basis, currently you can only choose a global standard.

]]>
Sun, 18 Apr 2010 00:00:00 +0200
https://beberlei.de/2010/12/18/generate-proxy-code-using-a-stream-wrapper.html https://beberlei.de/2010/12/18/generate-proxy-code-using-a-stream-wrapper.html <![CDATA[Generate Proxy code using a stream wrapper]]> Generate Proxy code using a stream wrapper

This blog-post is about an experiment and I am curios what others have to say about it. For Doctrine 2 we need to generate Proxy classes for the mapped objects to implement the Lazy Loading pattern. This is currently done with a console command or during runtime (for development). For Doctrine 2.1 we were thinking about using eval() as a third option for shared hosting or development environments. One annoyance of having to generate proxy code is that you have to regenerate it every time you change the original file. Additionally this makes deployment a little more complicated.

Today I got the idea to use stream wrappers to solve this problem. You can use stream-wrappers in combination with include/require and if you are using APC the generated php code can even be cached. That means you only have to generate the code once and after that everything is served from APCs opcode cache. Additionally by using the return values of stat() for the original file you can automatically regenerate your proxy code in APC when the original file changes.

I haven’t found a good way to pass state into a stream wrapper, that is why I put the data into $GLOBALS before calling include(). The client code looks like this:

<?php
stream_wrapper_register("dc2proxy", "Doctrine\ORM\Proxy\ProxyStreamWrapper");

$proxyDir = "dc2proxy:///proxies";
$GLOBALS['DOCTRINE2_PSW_EM'] = $em;
$GLOBALS['DOCTRINE2_PSW_ENTITYNAME'] = $className;
$GLOBALS['DOCTRINE2_PSW_PROXYCLASS'] = $proxyClass;

require $proxyDir. "/".str_replace("\\", "", $proxyClass) . ".php";

unset($GLOBALS['DOCTRINE2_PSW_EM'], $GLOBALS['DOCTRINE2_PSW_ENTITYNAME'], $GLOBALS['DOCTRINE2_PSW_PROXYCLASS']);

Not the nicest code, but it works. I can generate PHP code and have APC cache it for me until the original code changes. The stream wrapper to make this work looks like this:

<?php
namespace Doctrine\ORM\Proxy;

class ProxyStreamWrapper
{
    private $proxyCode;

    function stream_open($path, $mode, $options, &$opened_path)
    {
        $this->position = 0;
        $this->proxyCode = $GLOBALS['DOCTRINE2_PSW_EM']
                ->getProxyFactory()
                ->getProxyClassCode($GLOBALS['DOCTRINE2_PSW_ENTITYNAME'], $GLOBALS['DOCTRINE2_PSW_PROXYCLASS']);

        return true;
    }

    public function stream_stat()
    {
        $reflClass = new \ReflectionClass($GLOBALS['DOCTRINE2_PSW_ENTITYNAME']);
        $stat = stat($reflClass->getFileName());
        $stat[0] *= -1;
        $stat["ino"] *= -1;
        return $stat;
    }

    public function url_stat()
    {
        $reflClass = new \ReflectionClass($GLOBALS['DOCTRINE2_PSW_ENTITYNAME']);
        $stat = stat($reflClass->getFileName());
        $stat[0] *= -1;
        $stat["ino"] *= -1;
        return $stat;
    }

    function stream_read($count)
    {
        $ret = substr($this->proxyCode, $this->position, $count);
        $this->position += strlen($ret);
        return $ret;
    }

    function stream_write($data)
    {
        $left = substr($this->proxyCode, 0, $this->position);
        $right = substr($this->proxyCode, $this->position + strlen($data));
        $this->proxyCode = $left . $data . $right;
        $this->position += strlen($data);
        return strlen($data);
    }

    function stream_tell()
    {
        return $this->position;
    }

    function stream_eof()
    {
        return $this->position >= strlen($this->proxyCode);
    }

    function stream_seek($offset, $whence)
    {
        switch ($whence) {
            case SEEK_SET:
                if ($offset < strlen($this->proxyCode) && $offset >= 0) {
                     $this->position = $offset;
                     return true;
                } else {
                     return false;
                }
                break;

            case SEEK_CUR:
                if ($offset >= 0) {
                     $this->position += $offset;
                     return true;
                } else {
                     return false;
                }
                break;

            case SEEK_END:
                if (strlen($this->proxyCode) + $offset >= 0) {
                     $this->position = strlen($this->proxyCode) + $offset;
                     return true;
                } else {
                     return false;
                }
                break;

            default:
                return false;
        }
    }
}

What do you think about this approach? Are there any potential problems I am not seeing?

]]>
Sat, 18 Dec 2010 00:00:00 +0100
https://beberlei.de/2010/12/07/blog-refactorings-with-symfony2-doctrine-2-and-zeta-components.html https://beberlei.de/2010/12/07/blog-refactorings-with-symfony2-doctrine-2-and-zeta-components.html <![CDATA[Blog Refactorings with Symfony2, Doctrine 2 and Zeta Components]]> Blog Refactorings with Symfony2, Doctrine 2 and Zeta Components

I have been playing around with Symfony 2 quite a lot lately and have rewritten this blog using Symfony 2, Doctrine 2 and Zeta Components. You can go over to Github and find its project page.

Since I have found Sphinx I am pretty impressed with Restructured Text and have now integrated that as writing language into my backend. Using Zeta Components excellent Document component I transform the written ReST to XHTML. I hooked into the Document rendering and it now supports using the Sphinx directive “.. code-block:: <language>” and runs the subsequent code through Geshi for highlighting. For example:

<?php
echo "hello world with ReST and Zeta Components!";

Using the jQuery Plugin Tabby and MarkitUp with a ReST extension I can also write more conveniently now. Also to battle spam I have moved the comment system to Disqus.

My next plans for the blog bundle are:

  • WebDav support for authoring. All articles will be named “slug.inputformat”, for example “blog-refactorings-with-symfony2-doctrine2-zetacomponents.rst”. For this I plan to write a custom backend for ezcWebdav.
  • Making the Bundle simpler to re-use by others.
  • More jQuery love to the backend.
  • Trigger some events in the blog post cycle and hook a twitter + facebook notification for new posts in there.

Btw: This blog post is really just a bad excuse for me to test the ReST Editor in the backend. I hope you still enjoyed it ;)

]]>
Tue, 07 Dec 2010 00:00:00 +0100
https://beberlei.de/2009/02/25/git-is-great-helping-out-with-mutateme.html https://beberlei.de/2009/02/25/git-is-great-helping-out-with-mutateme.html <![CDATA[Git is great: Helping out with MutateMe]]> Git is great: Helping out with MutateMe

These last days I had fun with Padraic Bradys MutateMe project. It dynamically changes your code (using PECL Runkit extension) and checks weather your test cases fetch the errors occurring from changing the source code. This is a great addition to having a test suite, because it finds all the problematic test cases, that try to do too much and fail to cover the code correctly.

I have also contributed to the MutateMe project already and this was only possible due to Git and GitHub, which makes contributing to any project easy like writing a hello world example in a random programing language. As such I have to agree with Padraic: I love Git, too! He already integrated my and Sebastians changes back into the master and we didn’t have to focus on merging, communication and integration a lot.

My PUMA project is also hosted on GitHub (although nobody has looked into that yet), but I am beginning to write a complete Runner now that calls all three components PHPUnit, PDepend and CodeSniffer at once and directly processes the results. When this is done, I will pack it all up as a PEAR package for anyone to test out.

]]>
Wed, 25 Feb 2009 00:00:00 +0100
https://beberlei.de/2009/02/08/unittest-and-metrics-aggregator-tool-for-php.html https://beberlei.de/2009/02/08/unittest-and-metrics-aggregator-tool-for-php.html <![CDATA[Unittest and Metrics Aggregator Tool for PHP]]> Unittest and Metrics Aggregator Tool for PHP

Both PHPUnit and PDepend offer export functionality in XML or in a Test-database that is not quite readable for any user from the start. Over the last month I have gradually written a nice webbased tool, that aggregates this data and (todo in the future) relates them to help me with my open source projects.

PHPUnit can be used with a –test-db-dsn command, which saves all information about tests into a Database and PDepend has a strong Package centric source parser for all sorts of project metrics.

What I needed for my Zend Framework related work (and other projects) was a tool that does a run of the complete test-suite for me and saves the information, so that I can see where problems occur (and if they are due to my changes or other peoples). Since PHPUnit will stopped calculating additional metrics, I have also integrated the fabulous PDepend Tool into this aggregator. It shows me, what classes and functions need refactoring due to complexity issues and draws some nice graphs that summarize all sorts of project related information on a per package basis.

You can download or clone the an alpha version at Github. Its written with ezcMvcTools, so you need this to work too. A list of some screenshots is at the Github Wiki page.

]]>
Sun, 08 Feb 2009 00:00:00 +0100
https://beberlei.de/2009/02/13/update-on-unittest-and-metrics-tool.html https://beberlei.de/2009/02/13/update-on-unittest-and-metrics-tool.html <![CDATA[Update on Unittest and Metrics Tool]]> Update on Unittest and Metrics Tool

Some days ago i posted about the PHP Unittest and Metrics Aggregator tool that i have written on (and which I have dubbed PUMA). Discussing it with people I came to the conclusion that the approach using ezcMvcTools is quite problematic that it forces to use this application, although the reporting and the application are quite separate. This is not against ezcMvcTools: I love it, its just the wrong type of support.

I began to split up the aggregator into some sort of importing-exporting tool. You can specify library-, test- and output-directory of your application and it will use the tools at hand (PHPUnit, PDepend and CodeSniffer currently) to generate their XML formatted reports. It then parses those XML outputted files and combines their result to give a consistent view on your application.

The Import-To-Output generator uses a three-step approach. An importer emits signals to report observers, which will then hand over their collected data to specific html pages that are then generated to the disc. This is a very flexible approach that allows anyone to extend and re-use the tool to generate project metrics, unittest overview and other interesting details on your application.

Have a look at the tool on Github and play with it. I would really like to hear your thoughts.

]]>
Fri, 13 Feb 2009 00:00:00 +0100
https://beberlei.de/2009/05/01/explicit-code-requires-no-comments-only-bad-code-does.html https://beberlei.de/2009/05/01/explicit-code-requires-no-comments-only-bad-code-does.html <![CDATA[Explicit Code requires no comments - Only bad code does]]> Explicit Code requires no comments - Only bad code does

Following the recent discussion on commenting source code by Brandon Savage and Marco Tabini, I wanted to add upon Marcos code example to show that it not even needs any inline comments and can be greatly enhanced in readability with taking 2-5 minutes of time and refactoring the code. The original code sample was:

function __construct(array $pathInfo) {
    $section = $pathInfo[0];
    $file = $pathInfo[1];

    // Assume that the location of this file is inside the main trunk, and that pathInfo has already been filtered.

    $path = dirname(__FILE__) . '/../../' . $section . '/xml/static/' . $file . '.xml';

    if (!file_exists($path)) {
        Controller::singleton()->doError(404);
    }

    parent::__construct('main/xsl/template.xsl', $path);
}

This sample is somehow related to the response and view rendering of an application and its pretty understandable. For me there are some squirks though that would make me have to scroll to other classes to see how they interact. For example the path information falls from heaven. Why is it an array of 2 values and are these sections set the same all the time? And what exactly happens when the front controller does the 404 error? Is there a die() or exit in the doError()

The Agile movement postulates the phase of refactoring for a finished piece of code. Rebuilding the hacked solutions that did it to make them more understandable. I applied this to Marcos code snippet. The refactoring produces about double the code, but its done entirely by using NetBeans nice Rename variable and copy paste on extracting methods, which takes not more than 5 minutes.

class StaticXmlXsltRendererImpl extends AbstractXsltView
{
    function render(array $filteredPathInfo) {
        $templateValuesXmlPath = $this->getTemplateValuesXmlFile($filteredPathInfo);

        if(!$this->templateExists($templateValuesXmlPath)) {
            Controller::singleton()->doError(404);
        } else {
            parent::render('main/xsl/template.xsl', $templateValuesXmlPath);
        }
    }

    private function getTemplateValuesXmlFile($filteredPathInfo) {
        $section = $this->getSection($filteredPathInfo);
        $file = $this->getFile($filteredPathInfo);

        // Assume that the location of this file is inside the main trunk
        return dirname(__FILE__) . '/../../' . $section . '/xml/static/' . $file . '.xml';
    }


    private function getSection($pathInfo) {
        return $pathInfo[0];
    }

    private function getFile($pathInfo) {
        return $pathInfo[1];
    }

    private function templateExists($path) {
        return file_exists($path);
    }
}

In this code snippet the class name, variable names and methods clearly communicate what is done, which wasn’t the case in the previous example. The only change for the clients is the change of using the constructor to using the render() method, which communicates the intend more. Also the render method now clearly communicates that either the error or the parent rendering is executed and leaves no doubt about it.

The getTemplateValuesXmlFile() method still uses the comment to show the assumption about relative paths, but this is only the case because application configuration is made implicit into the code. This has to be extracted to be an explicit configuration constant and the comment can go.

private function getTemplateValuesXmlFile($filteredPathInfo) {
    $section = $this->getSection($filteredPathInfo);
    $file = $this->getFile($filteredPathInfo);

    return APPLICATION_ROOT . '/' . $section . '/xml/static/' . $file . '.xml';
}

In my opinion commenting code is necessary only for non-refactored code that has been hacked into existence and is hard to understand. Either the programmer has to get it done and has no chance to clearly communicate the intend. Or what is even worse the to be changed legacy code is hard to understand but you can’t take the chances to refactor it because its also already in production and has no tests. Now from the second point it is obvious that upon ugly code you put only more and more ugly code to fix the problems. This is what leads to the legacy maintenance problems that pretty much every programmer faces. And then commenting comes into play: You have to add a new feature X and it has to be done fast. You don’t really understand the code or how it works together but you know you can put the new code for the feature into existence but its really unintuitive. So you begin to comment it excessively, because its the only way to clearly show its intend.

There are two mindestting factors that help to write code that is understandable without having to excessively comment it:

  • From the beginning, do not write code for the computer but for developers. Changing this attitude really helps to write understandable code like the one above.
  • Only leave code better than it was before, never worse.

There are five technical practices that - when followed - allow to write clearly communicating code from the beginning:

  1. Giving classes, variables and methods good names. This is a no-brainer but few people seem to follow it anyways.
  2. Following the object-oriented Single Responsibility Principle by never giving a class more than one responsibility. Macros example seems to follow this one.
  3. Methods should never switch in the level of detail. Micro-work at the datastructure level should never be mixed with macro level delegation to executing large chunks of code. Macros code violates this by mixing the path building micro-level work with the macro-level work of rendering the XSLT template. The path building code can be hidden behind a method to communicate intend more clearly.
  4. Exchange if conditions with private methods that explain the condition being checked for. In Marcos example this is not really necessary, because file_exists already is quite a good description to the condition. But in cases of logical combinations of conditions the method extracting is a superior way to explain the conditions intend without having to write a comment.
  5. Seperate Query and Command: A method never should do a query which returns the state of an object and a command which executes a set of rules on the state.

These practices sum up to one guideline: Make code explicit. This obviously requires less commenting since a comment of explicit code would be duplication and duplication is bad. What if you have a project that does not follow this guidelines? Then of course comments should be used to explain code, but in the long run this should be refactored to self-explaining code. Additionally every new feature should be programmed explicitly to follow the “leave code better than before” principle.

In my opinion two refactoring tools are missing that would greatly help PHP programmers write nice to read code: Extract method and Replace magic value with constant. Can someone integrate them into NetBeans please?

Update: Fixed a creepy copy-paste code bug, thanks to azeroth for pointing out. Moved methods around a bit to be more reading friendly.

]]>
Fri, 01 May 2009 00:00:00 +0200
https://beberlei.de/2009/06/16/using-a-dependency-injection-container-with-zend-application.html https://beberlei.de/2009/06/16/using-a-dependency-injection-container-with-zend-application.html <![CDATA[Using a Dependency Injection Container with Zend_Application]]> Using a Dependency Injection Container with Zend_Application

Much has been written on Dependency Injection lately, mostly by Padraic Brady (1, 2) and Fabien Potencier (1, 2, 3, 4, 5). My subjective feeling tells me there are now more PHP DI containers out there than CMS or ORMs implemented in PHP, including two written by myself (an overengineered and a useful one). Its an awesome pattern if used on a larger scale and can (re-) wire a complex business application according to a clients needs without having to change much of the domain code. It aims at a complete separation of object instantiation and dependency tracking from the business logic.

Beginning with version 1.8 Zend Framework is able to integrate any of these DI containers into its Zend_Application component easily. The Application component initializes a set of common resources and pushes them into the MVC stack as additional Front Controller parameters. Technically a Zend_Registry instance holds all these resources with their respective resource names as keys. The resources are accessible inside the Front Controller and the Action Controller classes. Assume the resource ‘Db’ is initialized in your application, you can access it with:

$front = Zend_Controller_Front::getInstance();
$container = $front->getParam('bootstrap')->getContainer();
$db = $container->db;

// or:
class FooController extends Zend_Controller_Action {
    public function barAction()
    {
        $container = $this->getInvokeArg('bootstrap')->getContainer();
        $db = $container->db;
    }
}

This is pretty much dependency injection already, but the default registry approach suffers from two serious problem: New instances can only be added to the Container by implementing new Zend_Application resources and these instances cannot be lazy loaded. All resources used by Zend_Application are always loaded on every request. But the application container implementation was developed with Dependency Injection in mind and is not tied to the use of Zend_Registry. Only three magic methods are required by any container that wants to be Zend_Application compliant: __get(), __set() and __isset(). Each instantiated resource is pushed via __set() into the container. If required by another resource, __isset() is used to check whether a resource with the given name exists inside the container and __get() is used to retrieve the instances from the container.

Some month ago I extended Yadif, a lightweight PicoContainer-like DI container for PHP written by Thomas McKelvey, with several features that make it a very powerful DI container in my opinion. It has a detailed documentation and examples on the GitHub Page if you want to check it out. I extended Yadif to be Zend_Application compliant in a way that the application-wide Zend_Application resources can be used as dependencies for objects that are lazy-loaded from the container. Skipping the tech-talk, here is an example. First we have to replace the default Container with Yadif:

$objects = new Zend_Config_Xml(APPLICATION_PATH."/config/objects.xml");
$container = new Yadif_Container($objects);

$application = new Zend_Application(
    APPLICATION_ENV,
    APPLICATION_PATH . '/config/application.xml'
);
// Set Yadif as Container
$application->getBootstrap()->setContainer($container);
$application->bootstrap()
            ->run();

Assume you run Zend_Application with a “Db” and a “Cache” resource. These resources are loaded on every request. The objects configured in Yadif however are not instantiated until they are requested for the first time from the container. We can merge these two worlds and make use of the Application resources inside the Yadif Configuration “objects.xml”, which looks like:

<?xml version="1.0" ?>
<objects>
    <myModel>
        <class>MyModel</class>
        <arguments arg1="db" arg2="cache" />
    </myModel>
</objects>

Instantiating a myModel class inside the Action Controller uses the the Database and Cache resources as constructor arguments:

class FooController extends Zend_Controller_Action {
    public function barAction()
    {
        $container = $this->getInvokeArg('bootstrap')->getContainer();
        $myModel = $container->myModel;
    }
}

Given the possibilites by the Yadif_Container you are now empowered to use Dependency Injection for all your application objects and make use of Zend_Applications resource system. Furthermore any other dependency injection container, simple or complex, can also be integrated easily.

]]>
Tue, 16 Jun 2009 00:00:00 +0200
https://beberlei.de/2009/11/14/netbeans-php-codesniffer-plugin-now-with-options-dialog.html https://beberlei.de/2009/11/14/netbeans-php-codesniffer-plugin-now-with-options-dialog.html <![CDATA[Netbeans PHP CodeSniffer Plugin: Now with Options Dialog]]> Netbeans PHP CodeSniffer Plugin: Now with Options Dialog

Just two days before the International PHP Conference 09 will start I got an email from alexandrehaguiar that he forked my Netbeans PHP CodeSniffer plugin and added the long awaited Options screen. This is just awesome! This way I don’t have to ask the Netbeans guys at IPC all the questions about how to implement it. I merged the changes back into my branch and released a new version which also fixed another small bug. For me it works with both Netbeans 6.7.1 and 6.8 Beta. I want to thank alexandre very much for this awesome contribution.

What you can do now is chose the Coding Standard you want to use in an Options Dialog under the “Miscellaneous” button. You can choose from a list of pre-defined standards, or even enter your own if you use one. Additionally you can disable that warning are shown, so that with any given coding standard you only see the errors.

|image0|

You can download the new version from the Github Project Downloads page.

]]>
Sat, 14 Nov 2009 00:00:00 +0100
https://beberlei.de/2009/08/31/enums-in-php.html https://beberlei.de/2009/08/31/enums-in-php.html <![CDATA[Enums in PHP]]> Enums in PHP

A colleague of mine complained about the missing support for Enums in PHP and how one often wants to have a variable that is constant and has one of several specific values. We came up with an elegant solution that we want to share, maybe its even helpful to you. If you want to implement Enum behaviour with a simple string or int value you end up with having to validate the values at several different locations in the code rather than being able to strictly enforce the Enum structure by using typehints.

We discussed several approaches to this problem that all seemed a bit ugly in one or another way. For example the SplEnum class is a step in the right direction, however you can only use a type hint for “SplEnum” or add another inheritance layer. Also you have to implement different classes for each enum type, which requires you to implement a factory method for your enum to help with creation of the different types.

We came up with a very simple Enum concept. It uses reflection in the constructor to check if the given value is a valid Enum value by checking it against all the defined constants of the implementing class and throws an Exception if it is not. The __toString() magic method is implemented to allow for simple checks of the enums value. Strict type-checks are not possible with this construct, however in our opinion it is a very elegant solution to enforce a limited set of specific values throughout your code-base.

Here is the code plus a small example:

abstract class MyEnum
{
    final public function __construct($value)
    {
        $c = new ReflectionClass($this);
        if(!in_array($value, $c->getConstants())) {
            throw IllegalArgumentException();
        }
        $this->value = $value;
    }

    final public function __toString()
    {
        return $this->value;
    }
}

class Foo extends MyEnum
{
    const FOO = "foo";
    const BAR = "bar";
}

$a = new Foo(Foo::FOO);
$b = new Foo(Foo::BAR);
$c = new Foo(Foo::BAR);

if($a == Foo::FOO) {
    echo "My value is Foo::FOO\n";
} else {
    echo "I dont match!\n";
}

if($a == $b) {
    echo "a value equals b value!\n";
}
if($b == $c) {
    echo "b value equals c value!\n";
}

Now you could nice things such as:

function doStuff(Foo $foo) {
    switch($foo) {
        case Foo::FOO:
            echo "do more here!\n";
            break;
        case Foo::BAR;
            echo "evil stop!\n";
            break;
    }
}

doStuff($a);

What are your thoughts?

]]>
Mon, 31 Aug 2009 00:00:00 +0200
https://beberlei.de/2009/01/23/zend-form-and-the-model-yet-another-perspective-using-a-mediator.html https://beberlei.de/2009/01/23/zend-form-and-the-model-yet-another-perspective-using-a-mediator.html <![CDATA[Zend_Form and the Model: Yet another perspective using a Mediator]]> Zend_Form and the Model: Yet another perspective using a Mediator

Matthew Weier O’Phinney of the Zend Framework Devteam wrote a controversial post on integrating Zend_Form and the Model last month. He separated concerns of view and model that communicate via a Form, by calling just thee validation functions on the Form inside the mode. On request you could retrieve the model to the controller and view layers. I already wrote into his comments that I didn’t like the solution because it relies on implicit rules for the developers to use the Form component correctly in all layers. Additionally the building of the form using this approach would be performed inside the model, although strictly speaking this is responsibility of the View Layer. Another negative point is duplication of input filtering code that has to be performed to use certain variables inside the controller or when different forms talk with the same model.

Jani took it up and proposed writing validators for forms and attaching them to the Form as sort of a mediator. I am not a fan of this approach either, because the validator would have to include domain logic but is not really a part of the domain logic anymore but just a validator. Developers might forget using the validator inside the model for all their actions or there would be duplication of code in some places. In a perfect world, only functions of the models public interface should be called for validation.

My personal favorite for Form and Model integration is a mediator object between the two layers. Your model will have to include an additional interface with one function acceptFormRequest($values); which accepts an array of validated Zend Form field values. It then tries to apply the validated data into a record. Additional required validations of the model can take place in this function, which separates the concerns of Form validation and Model data validation. Still the mediator merges those differences together: You can throw an Exception and it will be attached as a custom error message to the Form. The following very short code will show the required interface and the mediator code. This code is very simple and might produce maintenance overhead fast, but I propose some refactoring enhancements later in the discussion.

interface WW_Model_AcceptFormRequest
{
    /**
     * Acceept a form request
     * @param array $values
     * @return WW_Record_Interface
     */
    public function acceptFormRequest($values);
}
class WW_Model_FormMediator
{
    /**
     * Try to push the form request to the model
     *
     * @param Zend_Form $form
     * @param WW_Model_AcceptFormRequest $model
     * @return WW_Record_Interface
     */
    public function pushFormToModel(Zend_Form $form, WW_Model_AcceptFormRequest $model)
    {
        if(!$form->isValid()) {
            throw new Exception("Form not valid!");
        } else {
            $values = $form->getValues();
            try {
                $record = $model->acceptFormRequest($values);
            } catch(Exception $e) {
                // This exception message comes from the model, because validation failed
                $form->addErrorMessage($e->getMessage());
                throw new Exception("Form request not accepted by model!");
            }
        }
        return $record;
    }
}

You can see the mediator has two different stages where errors can occur: When the form is not valid or the model is not valid. Both exits can be catched inside the controller and are the indicator that the form has to be displayed again for further input corrections. When successful the model returns a valid record that applies to the form and model requirements and can be displayed. If this record should be persistent this would have been done inside the acceptFormRequest function already. An example using a very simple Model using the a BankAccount example. We have a form that validates all the incoming request data for a withdrawal of money, though does not validate it against the models internal state. Our BankAccountModel implements the WW_Model_AcceptFormRequest interface and returns a valid BankAccount. If found the given amount is withdrawn.

class BankAccountModel implements WW_Model_AcceptFormRequest {
    public function acceptFormRequest($values)
    {
        $bankAccount = $this->getBankAccountBy($values['bankAccountNumber'], $values['pin']);
        if($values['action'] == "withdraw") {
            $bankAccount->withdraw($values['amount']);
            $this->save($bankAccount);
        } else {
            // unknown action...
        }
    }
    public function getBankAccountBy($key, $password) {
        // Find by Primary Key returning 'BankAccount' instance or exception if not found.
    }
    public function save(BankAccount $ba) {
        // Sql for saving the Bank Account
    }
}

class BankAccount
{
    public function withdraw($amount)
    {
        if( ($this->getBalance()-$amount) < 0 ) {
            throw new Exception("You cannot withdraw more money than your bank account holds!");
        }
        $this->balance -= $amount;
    }
}

Two exceptions might be thrown in this case: The Bank Account number does not exist or the password is wrong. Or you are not allowed to withdraw the given amount of money. If any of those exceptions is thrown the Model does not accept the form data and the form will have to be displayed again for the client showing the new error message that was returned from the model. The controller handling this process would look like this:

class BankAccountController extends Zend_Controller_Action {
    public function performWithdrawlAction() {
        $form = new BankAccountWithdrawlForm(); // extends Zend_Form and builds the form

        if($this->getRequest()->isPost()) {
            $mediator         = new WW_Model_FormMediator();
            $bankAccountModel = new BankAccountModel();
            try {
                $bankAccount = $mediator->pushFormToModel($form, $bankAccountModel);

                $this->view->assign('bankAccount', $bankAccount); // Show new balance in view!
            } catch(Exception $e) {
                $this->view->assign('withdrawlForm', $form);
                $this->_redirect('showWithdrawl');
            }
        } else {
            $this->view->assign('withdrawlForm', $form);
            $this->_redirect('showWithdrawl');
        }
    }
}

You can see the mediator tightly integrates Form and Model without both components knowing too much of each other. Still you can add error messages received from the model into the Form and redisplay it. One negative point of this approach is the fact that you only have one method for accepting form data, which could result in variable checking and redispatching in the case of many different operations that can be performed on the same model. For this case you might want to either:

  1. Rewrite the mediator to accept a specific model class (not the interface) and call the required custom method that matches the forms request. (Best approach for separation concerns)
  2. Rewrite the mediator to also pass the get_class($form); value to the model for decision making (Faster approach)

There is still some overhead on using the mediator. Since its generic you could build an Action Helper for it and use the direct call mechanism to save some lines of code.

]]>
Fri, 23 Jan 2009 00:00:00 +0100
https://beberlei.de/2009/01/14/finally-zend-mail-charset-and-or-long-lines-header-encoding-bug-fixed.html https://beberlei.de/2009/01/14/finally-zend-mail-charset-and-or-long-lines-header-encoding-bug-fixed.html <![CDATA[Finally: Zend_Mail charset and/or long lines header encoding bug fixed]]> Finally: Zend_Mail charset and/or long lines header encoding bug fixed

There was this lurking bug in Zend_Mail which destroyed every Mail-Header (and corresponding Mail) with non US-ASCII Chars and more than an encoded length of 74 chars. This is quite a huge subset of mails, but it seems a nice solution was not so easy, at least nobody tried to fixed it for quite some time.

Where many hackish solutions we’re offered, Ota Mares aka Littlex spent incredible time to hunt the original problem down and with his help I tag-teamed the bug to death today. Saturo Yoshida of Zend Fame added some further spice regarding an alternative solution with Base64 Encoding instead of Quoted Printable Mime Header encoding.

In the end the solution we chose was, not to re-use the Mime encoding function that is specific to MIME bodies according to RFC2045, but to write a completely new algorithm for Mime Headers, whose rules are specified in RFC2047. This is now done and unit-tests prove its working according to standard.

What is missing now is people trying that fix on as many Mail platforms as possible and giving feedback in the issue if a lengthy subject with non-ASCII chars is displayed correctly.

]]>
Wed, 14 Jan 2009 00:00:00 +0100
https://beberlei.de/2009/01/11/howto-file-a-good-bug-report-suggestions-for-framework-users.html https://beberlei.de/2009/01/11/howto-file-a-good-bug-report-suggestions-for-framework-users.html <![CDATA[Howto file a good bug report: Suggestions for framework users]]> Howto file a good bug report: Suggestions for framework users

I have fixed quite a number of bugs for the ZF lately, which lead me to this post about how to file a good bug report. There are many annoying bug reports out there, where the reporter of the bug withholds important information to the bugfixer unintended. This advice applies bug reports in general of course.

What are the benefits of a good bug report? The bug generally gets fixed faster, when the developer has more information at hand. Additionally other developers might come to rescue since they can understand the issue faster. These benefits are good for both parties. If you take no time for a good bug report, your issue might risk to end up getting old or closed unfixed.

  • Post the whole Exception Stack Trace: If the library throws an Exception into your application that is unexpected and may indicate an bug: Do not post the Message or Exception name only. The exception may be thrown in many different places or due to different reasons. The PHP exception class offers the method getTraceAsString(), which offers many information to the developer what the cause of the exception might be. Please use it!
  • Post codefixes in a patch format: When you find a bug in the framework, it is quite possible that you can offer a fix directly. Writing “Replace x in line y with z” does not help very often. The component might be in flux and the line positions change more often than you think. Please create a diff file of this changes that indicate the precise position of the change. This diff also includes 2 lines above and below the patched code for direction of the developer. SVN Diffs are even more useful since they include the revision where you fixed the bug in.
  • Post reproducible cases as PHPUnit Test: If you find a bug and can show how to reproduce it: Write a unittest to prove it. It is ZF policy to create a unittest for each bugfix showing that the bug was indeed fixed and previous functionality remains the same, so this unit-test has to be written anyways. Many show-offs rely on massive echo statements or var_dump, which render them almost useless for the developer.
  • Attach a unit-test to a submitted patch: This is related to the previous point. If you add a unit test your patch will get more attention. It will prove that you have thought about the patch, its consequences and that you might have checked it does not break backwards compatibility. This is worth a lot.
  • Run the test suite with your patch: If you want to provide a patch. Run the testsuite of the Zend Framework. It might break expected behaviour. When you post a patch that will break BC, it will be recognized. Your bug report might be closed, which helps nobody.

When you find a bug you have probably thought about it and how to fix it. This is valuable information. Disregarding one of this points will lead to missing information on part of the developer that he has to “learn” again. This takes time, which may make your bug last longer than it should.

]]>
Sun, 11 Jan 2009 00:00:00 +0100
https://beberlei.de/2009/01/11/what-will-be-new-in-zf-1-8.html https://beberlei.de/2009/01/11/what-will-be-new-in-zf-1-8.html <![CDATA[What will be new in ZF 1.8]]> What will be new in ZF 1.8

In February or March 2009 the 1.8 version of the Zend Framework is scheduled to be released. I have contributed some stuff already regarding ZendX_JQuery and Zend_Soap.

Both components have seen numerous bugfixes and I managed to get the JQuery helper down to zero open issues. I have also taken over the Zend_Json_Expr proposal, which will be a huge benefit to everything JSON that can be done with ZF. Foremost it is an integral part for the jQuery component which heavily relies on javascript callbacks.

The Soap Autodiscover and WSDL classes compatibility with Java and .NET has been optimized due to great user feedback, as well as some bugfixes to the newly added WSDL type detection strategies.

Additionally I went on another bug killing spree and fixed around 20-30 old bugs in a wide range of different components.

]]>
Sun, 11 Jan 2009 00:00:00 +0100
https://beberlei.de/2009/01/12/on-publishing-webservices-within-mvc-frameworks.html https://beberlei.de/2009/01/12/on-publishing-webservices-within-mvc-frameworks.html <![CDATA[On Publishing Webservices within MVC Frameworks]]> On Publishing Webservices within MVC Frameworks

Webservices are a very important part in today’s enterprise applications. They tie applications of different age or programming languages together or allow applications of different subcontracters to speak to each other. Because they use HTTP, a stateless network protocol, considerable overhead floods the pipes when you use them, which should be minimized.

Martin Fowler writes in his PEAA book, that if you have the option not to use distributed objects (which are implemented via webservice) you should not distribute them. Considerable effort has to be brought into keeping complex webservices performant.

Still people make mistakes about webservices all the time (me included for example proposing a dispatcher for the ZF that could be used for webservices).

When people report problems with the Zend Soap component they often post a stripped down example that involes their webservice being instantiated within a controller. This is a very bad decision based on different arguments:

  • Dispatching overhead: Dispatching, Routing, Pre- and Postfiltering is costly in all frameworks. You give up the performance of having numerous PHP scripts that act as controller on their own. You get centralized filtering, authentication and other benefits. But those benefits generally do not aply to XML, JSON or SOAP requests, because you cannot parse them or access their properties. You give up the performance of a page controller for webservices to gain mostly nothing.
  • HTTP Request uselessness: Web frameworks work with HTTP request objects. The request of webservices facilitates HTTP to act as a far more complex request. No framework I know off, allows to work with the webservice requests outside the Webservice handler. What a SOAP or XML-RPC request does in your MVC is only get passed through numerous costly stages that offer no benefit, before it hits the target. Only the parsing of HTTP-Headers might offer additional benefit, but the gain is low, since they are available to PHP scripts at no cost.
  • Webservices already seperate concerns: Take the PHP SOAPServer as an example. It is an MVC application on its own, the controlling aspect of the SOAPServer parses the SOAP Request and sends it to the model, a class given by the user, which in turn works and returns the result as an SOAP Response View. You have to decouple model and view for a webservice handler otherwise it would generate invalid responses. Why nest a perfectly separated operation into another one? You gain no more of this additional separation, except performance decrease.

So what are good practices to implement webservices?

  • Use a page controller that generates no MVC overhead. In context of the Zend Framework: Add a new php script to your web root and add a new route into your .htaccess file that redirectes the desired location of the webservice to the script that overwrites the standard catch-all incoming requests to the front controller script.
  • Use the proxy pattern and the invaluable __call() method to implement wrapper objects for authentication and session management of the webservice. These classes can easily be reused by all webservice page controllers of your site. If you do your homework you can even share parts of these objects inside your Web-MVC application to keep the code DRY. Those proxies keep authentication logic out of your service class.
  • Use the remote facade pattern to implement a few, powerful methods that delegate the service request to underlying domain objects. Never ever publish direct access to domain objects with your webservices. As a rule of thumb, talking to a webservice during a logical operation should never involve more than one or two calls. The first call is for data fetching, the second for data saving. Authentication should be handled via HTTP Authentication to save an additional call.

If you follow these simple rules, you should get around the performance issues that generally come with webservices, without loosing flexibility at all.

]]>
Mon, 12 Jan 2009 00:00:00 +0100
https://beberlei.de/2009/01/07/seven-things.html https://beberlei.de/2009/01/07/seven-things.html <![CDATA[Seven Things]]> Seven Things

I have been tagged by Freeaqingme and the blog letter machinery. So read on for seven things you probably may not know about me.

  • I will get my degree in some weeks, currently intensively writting on my thesis. My major is economics though. I probably won’t get a job with that sooner or later. I actually don’t want to, I have a job as programmer already.
  • One christmas many years ago (Pentium PCs to 200Mhz were the coolest thing around) i wished and wished for a computer for playing Command and Conquer and stuff. What i got was a 3/86 i could only run crap with. So i had to start programming.
  • In 11th grade (7 years ago) I took part in the german computer science olympics (exercise still online) using PHP and writing thousands of lines of nested aray and for loop code and print the solution to the browser. No need to say I utterly failed, because I didn’t know a bit about algorithms, pointers and objects at all.
  • I have only learned about what good programming practices are (OO, functional, whatever) really means in the last two years, spending almost every free minute of those many i had on projects to test around with patterns and programming styles.
  • I went to my first conference last year september, the IPC in Mainz, getting to know lots of great people in the community.
  • I don’t have a license at all (Reference to Sara). I am afraid of driving cars, although I drive my bike through the biggest crossroads in town like nuts.
  • I am a timeline oriented reader, having made me masterplans to read both all the Discworld and Ian Rankin novels in the correct order to not miss a single reference.

Its sad that everyone I would want to tag was already tagged by someone else, still for the head count:

  • Matthew Weierophinney, because he helped me understand Zend Framework alot.
  • Ben Scholzen, because he is a fun guy to be around #zftalk.dev.
  • Thomas Weidner, because I like his dedication towards his ZF components.
  • Kore Nordmann, Bastian Feder and Thomas Weinert, because I had great discussions with him on IPC.
  • zomg aka Jani because of his skills to be referenced by PHPDeveloper.org (joke) and the good IRC discussions.
  • Petr Pisl because of the awesome NetBeans PHP support.
]]>
Wed, 07 Jan 2009 00:00:00 +0100
https://beberlei.de/2009/03/11/using-zend-soap-server-and-autodiscover-in-a-controller.html https://beberlei.de/2009/03/11/using-zend-soap-server-and-autodiscover-in-a-controller.html <![CDATA[Using Zend_Soap Server and Autodiscover in a Controller]]> Using Zend_Soap Server and Autodiscover in a Controller

I am in a dilemma. I have condemned the usage of Zend_Soap_Server (or any other webservice server handler) inside a Model-View-Controller application before. Still I get questions about my old Zend_Soap_Server and Zend_Soap_AutoDiscover example not working in the MVC scenario. The following example provides you with a working Soap server inside a Zend_Controller_Action, although I discourage the use of it and would suggest using a dedicated script outside the dispatching process to gain multitudes of performance, which webservices often require.

require_once "/path/to/HelloWorldService.php";

class MyDiscouragedSoapServerController extends Zend_Controller_Action
{
    public function serverAction()
    {
        $server = new Zend_Soap_Server("http://example.com/pathto/wsdl");
        $server->setClass('HelloWorldService');
        $server->handle();
    }

    public function wsdlAction()
    {
        $wsdl = new Zend_Soap_AutoDiscover();
        $wsdl->setUri('http://example.com/pathto/server');
        $wsdl->setClass('HelloWorldService');
        $wsdl->handle();
    }
}

Now all you have to do is create two routes, one that makes http://example.com/pathto/server point to MyDiscouragedSoapServerController::serverAction and the other route that makes http://example.com/pathto/wsdl point to MyDiscouragedSoapServerController::wsdlAction. The wrong version (Version Mismatch) error comes from sending a request for the WSDL file to the actual Soap Server, which he doesn’t like.

]]>
Wed, 11 Mar 2009 00:00:00 +0100
https://beberlei.de/2009/03/27/test-your-legacy-php-application-with-function-mocks.html https://beberlei.de/2009/03/27/test-your-legacy-php-application-with-function-mocks.html <![CDATA[Test your Legacy PHP Application with Function Mocks!]]> Test your Legacy PHP Application with Function Mocks!

Much talking is going on about Unittesting, Mocks and TDD in the PHP world. For the most this discussions surround object-oriented PHP code, frameworks and applications.

Yet I would assert that the reality for PHP developers (me included) is dealing with PHP 4, PHP 5 migrated, or non-object oriented legacy applications which are near to impossible to bring under test. Still this code works, is in production and needs maintenance and possibly extension for new features.

For example many applications may still use mysql_* functions at length inside their model or have multiple nested function levels (Example: Wordpress). Runkit to the rescue: A PECL extension that can hack your running PHP code, such that method or functions are repointed to execute new implementations. Using this extension you can actually mock out internal PHP functions, which is great to bring legacy code under test.

Consider this following proof of concept, using the Runkit extension build by Padraic Brady (the one from pecl.php.net does not compile on PHP 5.2), which replaces the functionality of mysql_query(). You have to set the following option in your php.ini: runkit.internal_override = On for this to work. By default only user-defined functions may be overwritten by Runkit.

class FunctionMocker
{
    protected $_mockedFuncBehaviourMap = array();

    public function mock($funcName, $return=null)
    {
        $newFuncCode = 'return "'.$return.'";';

        $renamedName = "__".$funcName."_mockOriginalCopy";
        runkit_function_copy($funcName, $renamedName);
        runkit_function_redefine($funcName, '', $newFuncCode);

        $this->_mockedFuncBehaviourMap[$funcName] = $renamedName;
    }

    public function reset()
    {
        foreach($this->_mockedFuncBehaviourMap AS $funcName => $renamedName) {
            runkit_function_remove($funcName);
            runkit_function_copy($renamedName, $funcName);
            runkit_function_remove($renamedName);
        }
        $this->_mockedFuncBehaviourMap = array();
    }
}

$mocker = new FunctionMocker();
$mocker->mock('mysql_query', 'hello world!');

echo mysql_query(); // hello world

$mocker->reset();

mysql_query(); // error

This example, only allows for string return values of the mock and has no support for replacing the arguments of the mocked function. Also chaining of different return values based on input or call number might be interesting. Some kind of Code Generator Tool would have to be implemented to support this functionality. Additionally Assertions and Verifying should be implemented for the function arguments. All in all this would allow to mock functions as you would mock interfaces/classes, which would be a great addition for all those legacy applications that use procedural PHP.

Additionally what the real killer for runkit would be: The possibility to insert PHP callbacks instead of real PHP code into the runkit_function_redefine.

]]>
Fri, 27 Mar 2009 00:00:00 +0100
https://beberlei.de/2009/03/10/speaking-about-framework-quality-on-the-ipc-spring-09.html https://beberlei.de/2009/03/10/speaking-about-framework-quality-on-the-ipc-spring-09.html <![CDATA[Speaking about Framework Quality on the IPC Spring 09]]> Speaking about Framework Quality on the IPC Spring 09

Just a short notice, my thesis is taking its last breath, I will be speaking on the IPC Spring 09 conference in Berlin end of may this year on Framework Quality and Unittests:

Framework Quality: Unit-Tests as silent witness
Unit-Tests are often overlooked in framework discussions that focus

on features, performance, tools or database abstraction. However they are a silent witness on how poor or well frameworks are designed. In this presentation we will look at the testsuites of the big players in the PHP framework business - Zend Framework, ezComponents, Symfony and CakePHP - and reveal their hidden secrets.

This talk will make heavy use of the PHP Unittest and Metrics Aggregator tool, which i haven written on several times. For both CakePHP and Symfony extensions for their testtools may have to be written.

I am really excited for my first time being speaker on a conference and looking forward to see you at IPC.

]]>
Tue, 10 Mar 2009 00:00:00 +0100
https://beberlei.de/2009/10/05/php-codesniffer-for-netbeans-v0-2.html https://beberlei.de/2009/10/05/php-codesniffer-for-netbeans-v0-2.html <![CDATA[PHP CodeSniffer for Netbeans v0.2]]> PHP CodeSniffer for Netbeans v0.2

I finally found some time to spend some time on the PHP CodeSniffer for Netbeans plugin. Previously the plugin used an unnecessary API which restricted the use to Netbeans 6.7.0 only. This API was removed so that the plugin should now work with all Netbeans Versions >= 6.7.0.

Additionally when working the previous version would scan every PHP script on a file per file basis when the all projects or main projects filters were activated. This rendered made the plugin almost useless. In the current version scans of filters that contain more than one file are blocked. That means you will only see Coding Violations when you enable the “Current File” filter. Using any other filter will just do nothing and won’t put your Netbeans in permanent hibernation mode (aka useless mode).

There is also some preference “phpcs.codingStandard” using the `NbPreferences API <http://bits.netbeans.org/dev/javadoc/org-openide-util/org/openide/util/NbPreferences.html>`_ which allows to configure the coding standard. [STRIKEOUT:However this feature was contributed by Manuel Pichler and I don’t yet understand how I can manipulate it. Maybe someone knows how (*Looking in Mapis general direction*)?] By default the Zend Coding Standard is used.

Update: Coding Standard can be changed by hacking into the config file .netbeans/6.7/config/Preferences.properties setting “phpcs.CodingStandard=”. Additionally I fixed several bugs with the inline highlighting that did not refresh when lines in the file changed.

]]>
Mon, 05 Oct 2009 00:00:00 +0200
https://beberlei.de/2009/12/19/trying-a-two-step-pear-phar-approach-to-develop-and-deploy.html https://beberlei.de/2009/12/19/trying-a-two-step-pear-phar-approach-to-develop-and-deploy.html <![CDATA[Trying a Two Step PEAR/PHAR approach to develop and deploy]]> Trying a Two Step PEAR/PHAR approach to develop and deploy

With PHP 5.3 the PHAR extension is a pretty powerful concept for all your deployment needs, however it does not tell the complete story. Frameworks, Libraries and many different of them are used throughout applications and in recent times people even began to chery-pick the best components from each of the frameworks and package them together. With Pirum being a simple PEAR channel server there is also momentum for projects to distribute their code via PEAR.

However PEAR is mostly used in the server-wide configuration use-case, which is not very useful if you plan to distribute your complete application in one PHAR file. I just recently had the idea for this scenario, so please bear with me and add all the feedback and comments you can come up with. I tested this with the ongoing rewrite of my blog software.

First we’ll add a new user that we develop our application with:

sudo useradd -m -g www-data whitewashing
sudo passwd whitewashing
su - whitewashing

Now there is the possibility that with this user PHP and PEAR is not in your $PATH environment, so you might have to add it. In my case on Ubuntu i also had to switch the console from /bin/sh to /bin/bash for this user. Then we need to setup our application, I am going to use the Zend Framework project style here but with a little twist. We will add a distinction between vendor and project libraries by adding a vendor directory into the main folder.

But first we create a folder for our application, and create a Zend Framework project in the subfolder “trunk”, which will be the focus of our development.

whitewashing@desktop:~$ mkdir whitewashing
whitewashing@desktop:~$ zf create project whitewashing/trunk
Creating project at /home/whitewashing/whitewashing/trunk
whitewashing@desktop:~$ mkdir whitewashing/trunk/vendor

Now we can configure an Apache virtual host to point to our /home/whitewashing/whitewashing/public directory, i call this “whitewashing-dev” add it to my /etc/hosts and can visit the dummy project page.

We then configure our PEAR installation for the specific application user and re-configure the bin and php library paths:

whitewashing@desktop:~$ pear create-config /home/whitewashing/ .pearrc
whitewashing@desktop:~$ pear config-set php_dir /home/whitewashing/whitewashing/trunk/vendor
whitewashing@desktop:~$ pear config-set bin_dir /home/whitewashing/whitewashing/trunk/bin

This configuration assumes that we will install all our stuff into our development trunk. From there also the PEAR installed libraries might be copied into branches or tags. PEAR Project tests, configuration and web-files will still be put by default under $HOME/pear/*. We don’t need them for our applications.

Now we install all the dependencies our project needs, in this case Zend Framework, Doctrine 2, HTML Purifier:

whitewashing@desktop:~$ pear channel-discover pear.zfcampus.org
whitewashing@desktop:~$ pear install zfcampus/zf-alpha
whitewashing@desktop:~$ pear channel-discover htmlpurifier.org
whitewashing@desktop:~$ pear install hp/HTMLPurifier
whitewashing@desktop:~$ pear channel-discover pear.phpdoctrine.org
whitewashing@desktop:~$ pear install pear.phpdoctrine.org/DoctrineORM-2.0.0

Now we have all three of the packages installed in our project folder whitewashing/trunk/vendor, see:

whitewashing@desktop:~$ ls -aFl whitewashing/trunk/vendor/
total 680
drwxr-xr-x  7 whitewashing www-data   4096 2009-12-13 14:45 ./
drwxr-xr-x  8 whitewashing www-data   4096 2009-12-13 14:36 ../
drwxr-xr-x  3 whitewashing www-data   4096 2009-12-13 14:43 .channels/
-rw-r--r--  1 whitewashing www-data     57 2009-12-13 14:45 .depdb
-rw-r--r--  1 whitewashing www-data      0 2009-12-13 14:45 .depdblock
drwxr-xr-x  5 whitewashing www-data   4096 2009-12-13 14:45 Doctrine/
-rw-r--r--  1 whitewashing www-data 582208 2009-12-13 14:45 .filemap
drwxr-xr-x 20 whitewashing www-data   4096 2009-12-13 14:39 HTMLPurifier/
-rw-r--r--  1 whitewashing www-data    629 2009-12-13 14:39 HTMLPurifier.autoload.php
-rw-r--r--  1 whitewashing www-data    274 2009-12-13 14:39 HTMLPurifier.auto.php
-rw-r--r--  1 whitewashing www-data    545 2009-12-13 14:39 HTMLPurifier.func.php
-rw-r--r--  1 whitewashing www-data   9299 2009-12-13 14:39 HTMLPurifier.includes.php
-rw-r--r--  1 whitewashing www-data    955 2009-12-13 14:39 HTMLPurifier.kses.php
-rw-r--r--  1 whitewashing www-data   8831 2009-12-13 14:39 HTMLPurifier.php
-rw-r--r--  1 whitewashing www-data  11901 2009-12-13 14:39 HTMLPurifier.safe-includes.php
-rw-r--r--  1 whitewashing www-data      0 2009-12-13 14:45 .lock
drwxr-xr-x  8 whitewashing www-data   4096 2009-12-13 14:43 .registry/
drwxr-xr-x 59 whitewashing www-data   4096 2009-12-13 14:36 Zend/
-rw-r--r--  1 whitewashing www-data  19537 2009-12-13 14:36 zf.php

And both Doctrine and ZF registered their binary CLi tools inside the whitewashing/trunk/bin/ folder:

whitewashing@desktop:~$ ls -aFl bin/
total 20
drwxr-xr-x 2 whitewashing www-data 4096 2009-12-13 14:45 ./
drwxr-xr-x 8 whitewashing www-data 4096 2009-12-13 14:36 ../
-rwxr-xr-x 1 whitewashing www-data   50 2009-12-13 14:45 doctrine*
-rwxr-xr-x 1 whitewashing www-data  169 2009-12-13 14:45 doctrine.php*
-rwxr-xr-x 1 whitewashing www-data 1511 2009-12-13 14:36 zf*

We now have the full control over the versions of our dependencies, we can call “pear upgrade ” whenever we want to update one of the ZF, Doctrine or HtmlPurifier libraries inside our application.

Now some magic is gonna happen, we start to develop our application and such which is all not really interesting for this topic. At some point we want to package it all up into a PHAR file and distribute it. We want to package our application in one big phar file. We also want to make sure that the configuration files in whitewashing/trunk/application/configs/ are not distributed, but have to be created on the server and are kept that way. We could write an installer script for this configuration management.

The reference for PHAR files is the PHP Manual for the Basics and Cal Evans’ two posts (1, 2) on this topic, aswell as a post on Geekmonkey. Contrary to most other PHP extensions, PHAR has an extensive documentation, however its not organized terribly well. Also there are no real use-cases and scenarios discussed, methods are only looked at in isolation. Cals posts are very good on understanding how to package up different libraries, but there is no word on distributing web applications. That is where the Geekmonkey post comes in to wire it all together.

For a Zend Framework application that should have both a web and a cli (cronjobs) entry point into the application we need a specific stub file for the PHAR bootstrapping. A stub is a little PHP script that is executed whenever your PHAR file is included into your php script. It is essentially a front-controller for your PHAR application. It also has mount capabilities that allow to import files or directories from outside into the PHAR context. This is a powerful feature that is required to distribute configurable applications like our blog.

This screenshot shows how the application is currently structured in development mode. In production its structure should look like:

whitewashing
|--application
|  |--configs
|     |-- my application config files are all here...
|--bin
|  |--whitewashing.php
|--public
|  |--index.php
|  |--.htaccess
|--whitewashing.phar

The whitewashing.php and index.php files are the application entry points that only include the phar file and trigger the application bootstrapping that will be included in the Stub file. They both look like:

<?php
define('EXTERNAL_APPLICATION_ROOT', __DIR__."/../");
include EXTERNAL_APPLICATION_ROOT."/whitewashing.phar";

Including a PHAR file essentially has two consequences:

  • The PHAR path will be added to your include path.
  • The stub file will be executed.

Our application stub looks like this:

<?php

if(defined('EXTERNAL_APPLICATION_ROOT')) {
    // Mount the external application/configs directory as config if it exists.
    if (file_exists(EXTERNAL_APPLICATION_ROOT."/application/configs")) {
        Phar::mount("application/configs", EXTERNAL_APPLICATION_ROOT."/application/configs");
    }
}

/** Zend_Loader_Autoloader */
require_once 'Zend/Loader/Autoloader.php';
$autoloader = Zend_Loader_Autoloader::getInstance();

if (php_sapi_name() == "cli") {
    require_once 'bin/whitewashing.php';
} else {
    require_once 'public/index.php';
}

__HALT_COMPILER();

The first bit of the stub mounts the external application configs directory into the stub and hides possible directories that are present at this location in the PHAR file. This allows us to distribute our application with a default configuration, but allows any user to replace the configuration files to fit the application to his need.

The second bit loads Zend Framework Autoloader that is required by the bootstrapping mechanism. The third bit decides whether this request is executed from the CLI- or the Web-Entry point of the application. The fourth bit, __HALT_COMPILER(); is a technically required call inside your stub-file.

Now that we have a stub-file for our application, we can package it and distribute it. I am using a modified version of Cal Evans example for this. I have extracted his directory traversal to find all the relevant into a re-usable FilterIterator implementation. I pasted my package.php a Gist on Github. Now this should probably be put into the build context of your application, possibly as a phing or ant task or something alike.

Now what this build process does not manage is the creation of the application entry point php and .htaccess files, but since they won’t ever change its easy to add them to the build directory for now. An even more sophisticated version of the build script would lead to the creation of an additional tar.gz of the complete application folder. Our deployment process would then be as easy as:

  • If the application is not installed yet, unpack the tarball into its location.
  • If the application should be updated, just replace the PHAR file.

If you need the ability to go back to any version of your application you could make use of symlinks.

]]>
Sat, 19 Dec 2009 00:00:00 +0100
https://beberlei.de/2009/07/30/php-codesniffer-support-for-netbeans.html https://beberlei.de/2009/07/30/php-codesniffer-support-for-netbeans.html <![CDATA[PHP CodeSniffer Support for Netbeans]]> PHP CodeSniffer Support for Netbeans

I dived into the code of my new favorite IDE Netbeans these last days and came up with an extension module, which adds PHP CodeSniffer Support on a per file basis to make my life much easier. It shows warnings and errors as annotations to the Editor and marks the affected lines in yellow and red.

|image0|

[STRIKEOUT:My Java skills being very bad, it will only work on Linux currently, since the PHP Code Sniffer “binary” is hardcoded into the Java Source code. You have to create a “/usr/bin/phpcs2” executable, which is a wrapper that looks like:]

With Manuals extensions (see the comments) the module now works without the wrapper script. It might even work under Windows now. Yet now the Zend Coding standard is enforced though. I am working on making that one configurable next.

You can install the NBM module install file from the GitHub repository into Netbeans and it “should” work then.

I hope to get more familiar with Netbeans in the future to add some more PHP tools and enhance Code Sniffer support.

]]>
Thu, 30 Jul 2009 00:00:00 +0200
https://beberlei.de/2009/07/04/using-symfony-dependency-injection-with-zend-application.html https://beberlei.de/2009/07/04/using-symfony-dependency-injection-with-zend-application.html <![CDATA[Using Symfony Dependency Injection with Zend_Application]]> Using Symfony Dependency Injection with Zend_Application

As a follow-up to my recent post on Dependency Injection containers and Zend_Application I was eager to find out if its possible to integrate the new Symfony Dependency Injection Container into the Zend Framework. To my surprise it’s possible without having to make any changes to one of the two components. An example use-case would look like:

$container = new sfServiceContainerBuilder();

$loader = new sfServiceContainerLoaderFileXml($container);
$loader->load(APPLICATION_PATH.'/config/objects.xml');

$application = new Zend_Application(
    APPLICATION_ENV,
    APPLICATION_PATH . '/config/application.xml'
);
$application->getBootstrap()->setContainer($container);
$application->bootstrap()
            ->run();

Resources instantiated by Zend_Application are then injected into the container by the name of the resource and are given the resource instance. Any object from the Symfony container can then use these dependencies in their object setup. The only drawback of the integration is the fact that the Symfony Container is case-sensitive in regards to the service names but Zend_Application lower-cases all service before injecting them into the container. The following code is a restatement of my previous example of a MyModel class which requires a Zend_Db and Zend_Cache constructor argument.

$container->register('myModel', 'MyModel')
          ->addArgument(new sfServiceReference('db'))
          ->addArgument(new sfServiceReference('cache'));

Access to a MyModel instance with its dependencies is then granted through the call $container->myModel throughout the application. Make sure to call this after running Zend_Application::bootstrap, so that the Resource dependencies are injected first.

]]>
Sat, 04 Jul 2009 00:00:00 +0200
https://beberlei.de/2008/02/12/pdt-back-to-easyeclipse-with-php-extension.html https://beberlei.de/2008/02/12/pdt-back-to-easyeclipse-with-php-extension.html <![CDATA[PDT: Back to easyeclipse with PHP extension]]> PDT: Back to easyeclipse with PHP extension

I am disappointed by PDT for Eclipse. The current version strikes my rather old machine with “just” 512mb to death. Building projects is not possible even when assigning eclipse all my available memory, because of an Java Heap Error (See this blog post). This effectively hinders you to use all the completion and hint features, because PDT does not know about any functions, classes and their phpdoc descriptions.

PDT is also missing some very important features. Marking all occurrences of a function, class, constant or variable on click for example. PHP Errors and Warnings are display in a way so that you won’t find them and if you do, you won’t understand what they mean.

So I am going back to my easyeclipse with php combination, which always worked well, the only problem being that it is no longer under active development. At least it has all the features I need for now.

]]>
Tue, 12 Feb 2008 00:00:00 +0100
https://beberlei.de/2008/05/25/introducing-zend-controller-scaffolding.html https://beberlei.de/2008/05/25/introducing-zend-controller-scaffolding.html <![CDATA[Introducing: Zend Controller Scaffolding]]> Introducing: Zend Controller Scaffolding

In the last couple of weeks I came across the idea of building a little component for scaffolding in Zend Framework. After playing around with this idea a little I came up with an Zend_Controller_Scaffolding object, which extends the Zend_Controller_Action object.

It takes any Zend_Db_Table_Abstract object and generates create and update forms using Zend_Form and displays them in the Controller. The component is very easy to use, as this example shows:

class SomeController extends WW_Controller_Scaffolding {
    public function init() {
        $this->setScaffolding(new ZendDbTableModel(), $options);
    }
}

You now have access to the following actions of the controller: some/create some/edit some/delete and some/index (some/list). They handle all your model editing dreams.

All you have to own is the WW_Controller_Scaffolding Library Class and a folder of scaffolding view templates both of which are bundled in an archive you can download.

Download

Changes

  • 0.5.5: Put Component into own Namespace (WW = Whitewashing). Make it possible to hide fields in the list via the options. Allow to specify scaffolding view scripts folder via options.

Install & Usage

Untar the two folders include/ and views/ into your
Zend Framework project application directory.

Make sure the include folder is in your Zend_Loader
include path or move the Scaffolding.php so that it
is placed in your library include path.

Define each Controller that should be a Scaffolding Interface
as:

class SomeController extends WW_Controller_Scaffolding {
    public function init() {
        $this->setScaffolding(new ZendDbTableModel(), $options);
    }
}

Where $options is an array or Zend_Config object with any of the following keys:

  'allow_edit_primary_key' True/false - Whether the form allows you to set the
                          Primary Key fields or not

 'field_names' an Array of the Database Tables field names mapped to Label Names

 'hide_list_fields' Array of database table field names that should not be displayed in the list overview.

 'checkbox' an Array of Database Table field names that should be represented
            as Checkbox. Useful for TINYINT(1) fields that represent boolean decisions.

 'view_folder' Sometimes you want to use different views for scaffolding in one project. Use
               this variable and copy the scaffolding folder for each component you want to
               change the view basics for.

Todos & Problems:

  • Many To Many Relationships are not implemented yet (Using MultiSelec).
  • Compound Keys are probably not working correctly
  • Relationships on non-primary key fields probably don’t work as expected
  • Relationships with lots of data are not scaled down for easy administration.

Please report any bugs and feature requests or recommendations to kontakt at beberlei dot de.

]]>
Sun, 25 May 2008 00:00:00 +0200
https://beberlei.de/2008/05/18/my-2-cents-on-zend-search-lucene-update-problems.html https://beberlei.de/2008/05/18/my-2-cents-on-zend-search-lucene-update-problems.html <![CDATA[My 2 cents on Zend_Search_Lucene Update Problems]]> My 2 cents on Zend_Search_Lucene Update Problems

Zend_Search_Lucene (or Lucene in general) does not support an update statement, therefore we must check via find() if a specific article that should be updated already exists.

Numerous solutions exist for this problem (for example this PDF Tutorial by a Zend Programmer). Mine looks as follows. The first thing is to Set an Analyzer other from the default one. The Default Analyzer only looks at letters, not at numbers.

Zend_Search_Lucene_Analysis_Analyzer::setDefault(new
    Zend_Search_Lucene_Analysis_Analyzer_Common_Utf8Num_CaseInsensitive()
);
$index = Zend_Search_Lucene::open('/path/to/index/directory');

We can now save an identifier containing letters and numbers, for example the md5 string of the Article Database ID.

$doc = new Zend_Search_Lucene_Document();
$field = Zend_Search_Lucene_Field::Keyword('id', md5($data['id']));
$doc->addField($field);
$index->addDocument($doc);

Before updating an entry you have to delete any entry that has been indexed before:

$hits = $index->find('id:'.md5($articleID));
foreach($hits AS $hit) {
    $index->delete($hit->id);
}

The Tutorial Talk on Search Lucene (given above) uses an extended version of this delete routine snippet that also checks if an article is up to date and does not have to be reindexed and deletes multiple entries of one article.

]]>
Sun, 18 May 2008 00:00:00 +0200
https://beberlei.de/2008/05/17/integrated-zend-layout-into-this-blog.html https://beberlei.de/2008/05/17/integrated-zend-layout-into-this-blog.html <![CDATA[Integrated Zend_Layout into this blog]]> Integrated Zend_Layout into this blog

I finished with integrating Zend_Layout into this blog software today. At first I thought it is quite hard to use especially if you want to integrate dynamic navigational contents depending on the current module/controller/action setup. But I came up with a combination of actionStack and Named Response Sections as described in the ZF Documentation of the Layout component, which is quite easy to understand, use and extend.

At each of the major Controllers I put different actions from a specific Navigation Controller on the ActionStack (mostly global per Controller using the init() class method).

A typical controller looks like this:

class BlogController extends Zend_Controller_Action
{
    public function init()
    {
        $this->_helper->actionStack('tagcloud', 'Navigation');
        $this->_helper->actionStack('userinfo', 'Navigation');
    }
}

The NavigationController looks like this:

class NavigationController extends Zend_Controller_Action
{
    public function tagcloudAction()
    {
        // Do not render the content of this action to the default output.
        $this->getHelper("ViewRenderer")->setNoRender();

        [...]

        // Append content to secondaryNavigation named response section
        $this->getResponse()->append('secondaryNavigation', $this->view->render('navigation/tagcloud.phtml'));
    }
}

The layout.pthml then just calls certain named sections, for example <=? $this->layout()->secondaryNavigation; ?>

]]>
Sat, 17 May 2008 00:00:00 +0200
https://beberlei.de/2008/05/17/using-zend-http-client-to-ping-technorati.html https://beberlei.de/2008/05/17/using-zend-http-client-to-ping-technorati.html <![CDATA[Using Zend_Http_Client to Ping Technorati]]> Using Zend_Http_Client to Ping Technorati

Send a ping to Technorati manually is a pain in the ass. Writing a little Method that handles this functionality for your own blog software is quite easy. Call the following method after you created or updated a post of yours:

private function _pingTechnorati()
{
    $xml = $this->view->render('ping_technorati.phtml');

    $httpclient = new Zend_Http_Client('http://rpc.technorati.com/rpc/ping');
            $httpclient->setHeaders('User-Agent', 'PHP/Zend Framework/Zend_Http');

    $httpclient->setRawData($xml, 'text/xml')->request('POST');

    $response = $httpclient->request();

    if($response->isSuccessful() === true) {
        return true;
    } else {
        return false;
    }
}

You also need a little view in XML format like it is described in the Technorati Ping Configuration Guide and ready to go you are!

]]>
Sat, 17 May 2008 00:00:00 +0200
https://beberlei.de/2008/05/07/zend-form-rapid-development-plugin-for-eclipse.html https://beberlei.de/2008/05/07/zend-form-rapid-development-plugin-for-eclipse.html <![CDATA[Zend_Form - Rapid Development Plugin for Eclipse?]]> Zend_Form - Rapid Development Plugin for Eclipse?

I found some time in the last weeks to reconsider Zend Framework, this time in the 1.5 version, for this blog software and rewrote all Forms using the new Zend_Form component. I have to say, i love it. Its very easy using the already existing Zend_Validate and Zend_Filter components as input for the generated form fields. Another plus is the handling of errors in input and re-entering/displaying text.

There is also a neat feature of the Zend_Form component, which allows to generate complete forms by passing an appropriately formatted Zend_Config_Ini object to the Zend_Form Constructor. So my idea was, why not program an Eclipse Plugin allowing to rapidly generate and edit forms using a point-and-click plugin window approach, which would ultimately generate your Form INI code.

This might have been a great Google Summer of Code idea for the Eclipse Foundation (since Zend is not taking part).

]]>
Wed, 07 May 2008 00:00:00 +0200
https://beberlei.de/2008/06/30/implementing-zend-auth-acl-and-caching.html https://beberlei.de/2008/06/30/implementing-zend-auth-acl-and-caching.html <![CDATA[Implementing Zend Auth, Acl and Caching]]> Implementing Zend Auth, Acl and Caching

I managed to work a bit on the blog, enabling me to have new instance in the series, How to refactor and extend your own blog software using Zend Framework. This time: Implementing meaningful Auth and ACL mechanisms and fixing view caching, which did not work in its original implementation due to different users groups sharing the same cached views.

In the last days I came across two in depth tutorials on Zend_Acl and Zend_Auth integration to MVC. Most people probably saw the DevZone article “Zend_Acl and MVC Integration (part 1)” by Aldemar Bernal on the Frameworks Frontpage. Another good article was written by Frank Ruske in the latest german PHP Magazin (Zipped Source Code of the Example). I took the best ideas of both articles and merged them into the existing components of my blog.

This now enables me to cache the site depending on the Auth status of the page user agent. The blog is now caching all views that are generated for guest users. Since there is no “registered” or “member” account yet, this means the blogs content is cached and served from cache for everyone except me when I am logged in. To get this work I added some simple additional check in Matthews Cache View Controller Plugin:

public function dispatchLoopStartup(Zend_Controller_Request_Abstract $request)
{
    $auth = Zend_Auth::getInstance();
    if($auth->hasIdentity()) {
        self::$doNotCache = true;
        return;
    }
    [..]
}

This prevents caching for registered identities.

]]>
Mon, 30 Jun 2008 00:00:00 +0200
https://beberlei.de/2008/06/14/exim4-and-virtual-users-forward-files.html https://beberlei.de/2008/06/14/exim4-and-virtual-users-forward-files.html <![CDATA[Exim4 and Virtual Users .forward Files]]> Exim4 and Virtual Users .forward Files

Some days ago I began to wonder why some mails in mail account were filtered by Exims .forward file why some are not. It took until today that I found out what was the problem and how to fix it.

I implemented a virtual users/domains system for Exim using several different documentations and tutorials. Right now some parts of my system include the documentation on

http://www.tty1.net/virtual_domains_en.html

Now the problem is that the userforward router is only working for accounts whose $local_part is an existing system user. I came up with the following additional router that solves the problem. Its written for the MySQL table layout that is specified in the Tutorial above.

virtualuserforward:
  debug_print = "R: virtualuserforward for $local_part@$domain"
  driver = redirect
  domains = +local_domains
  file = ${lookup mysql{SELECT CONCAT(home,'.forward') AS forward FROM users \
        WHERE id='${quote_mysql:$local_part}@${quote_mysql:$domain}'}}
  no_verify
  no_expn
  check_ancestor
  directory_transport = address_directory
  file_transport = address_file
  pipe_transport = address_pipe
  reply_transport = address_reply
  allow_filter

  user = ${lookup mysql{SELECT uid FROM users \
        WHERE id='${quote_mysql:$local_part}@${quote_mysql:$domain}'}}
  group = ${lookup mysql{SELECT gid FROM users \
        WHERE id = '${quote_mysql:$local_part}@${quote_mysql:$domain}'}}

Now each virtual user is using the .forward file in its actual system user account home directory. The next problem was: $home is not defined in this case in the .forward file syntax: Its empty. So replacing $home with its obvious path leads to the final solution.

]]>
Sat, 14 Jun 2008 00:00:00 +0200
https://beberlei.de/2008/06/25/multimediatreff-on-frameworks.html https://beberlei.de/2008/06/25/multimediatreff-on-frameworks.html <![CDATA[Multimediatreff on Frameworks]]> Multimediatreff on Frameworks

My neighbour gladly called my attention to the Multimediatreff, a relaxed conference that is held about three times a year, this last saturday on the topic: Frameworks and Content-Management-Systems.

It was held in a very nice conference center in cologne, and from what I heard about 200 people attended. The first two topics were on Joomla! and barrier-free Typo3. Since I am more the framework guy, I did not enjoy them too much. At least I got some input on how a barrier-free website has to function. The Joomla! presentation supported my prejudice, that programming in and for Joomla! produces ugly code.

I did like the presentations on frameworks, the most interesting being the CakePHP presentation by Timo Derstappen, since I haven’t looked into Cake for about one and a half years. I have seen that with the new to come version 2, the framework became even better, now supporting easy code generation and flexible site generation. But its still somehow a Ruby clone and forces me too much in my programming style.

The Zend Framework presentation was done by Carsten Möhrke, who also wrote the first german ZF Book. He showed the creation of a blog using Zend Framework, being the Hello World example for frameworks. He specifically showed, that you can use anything you want as a model, a Zend_Db_Table object or any other object. He also showed something like a Data-Mapper pattern in his application. I got some ideas from this I probably will write about some later time.

I also enjoyed the Rails presentation, because I never got the chance to look the Ruby guys at work of the shoulder. But this is even worse than CakePHP was my first observation. I am no fan of ActiveRecord being forced upon you as Model implementation.

The pizza battle, mysteriously announced by the host, was another highlight, as well as the drinks and snacks in between. All in all I can highly recommend the Multimediatreff for the next time.

]]>
Wed, 25 Jun 2008 00:00:00 +0200
https://beberlei.de/2008/06/22/zf-caching-pages-via-front-controller-plugin.html https://beberlei.de/2008/06/22/zf-caching-pages-via-front-controller-plugin.html <![CDATA[ZF: Caching Pages via Front Controller Plugin]]> ZF: Caching Pages via Front Controller Plugin

Today I implemented caching using a front controller plugin (see Matthews post on Zend Devzone) into the Whitewashing blog software via Zend_Cache and the Filesystem Backend. I ran some superficial Apache Benchmark tests to have a look at the gain change in performance.

First of all i used the complete Plugin from Matthews article on Zend Devzone and a missing function getCache() in the code fragment:

public function getCache()
{
    if( ($response = $this->cache->load($this->key)) != false) {
        return $response;
    }
    return false;
}

I loaded the Plugin into the front controller and initialized it with a Zend_Config_Ini object. This all works very smoothly and caching is ready to begin.

Testing came to the result that with 1000 requests, 10 of them concurrent (ab -n 1000 -c 10), which I know is a rather unrealistic assumption for this blog, the request time dropped by half, from six to three seconds (still lot of time, but this isn’t the best webserver).

Results without caching

Requests per second:    1.49 [#/sec] (mean)
Time per request:       6695.213 [ms] (mean)

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       26   30   4.0     30      43
Processing:  1556 6452 3372.1   5511   14768
Waiting:     1436 6129 3174.1   5145   14595
Total:       1583 6483 3373.2   5538   14810

Results with caching

Requests per second:    3.03 [#/sec] (mean)
Time per request:       3304.534 [ms] (mean)

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       26   31   6.3     30      73
Processing:   781 3214 1757.7   3103    7445
Waiting:      650 3055 1731.4   2928    7190
Total:        809 3245 1757.8   3135    7475

As said before this testing is superficial, but its gives a broad sense of what performance gain is possible with just some lines of code and a temporary directory. Using memcache will probably speed up the process another good amount.

Later I realized that complete page caching (rather than block element caching) sucks when you inject admin area links into the navigation, which would allow non-logged in users to see parts of the admin area so I had to disable caching totally. You never know in the first place (you should though). I have to rework my admin area or extend my authentication or the caching plugin somehow. This will probably be the topic another time.

]]>
Sun, 22 Jun 2008 00:00:00 +0200
https://beberlei.de/2008/06/21/on-frameworks-and-javascript-coupling.html https://beberlei.de/2008/06/21/on-frameworks-and-javascript-coupling.html <![CDATA[On Frameworks and Javascript Coupling]]> On Frameworks and Javascript Coupling

Zend announced they work together with the Dojo Team to integrate Javascript support into the Zend Framework. What this actually means is that you can write your Javascript Code in PHP if you’re javascript library of choice is Dojo.

Since Dojo is not my library of choice I would have to write a JQuery View Helper on my own or wait until somebody else releases one.

But why not write javascript? Its very easy with all those libraries and addresses most cross browser incompatibilities: One Django, one Zend Framework developer argue framework and javascript coupling is not the way to go based on the arguments that its breaks the MVC pattern, leads to function calls equal to those in javascript in its framework respective language, and that javascript is actually something any good webdeveloper should be able to program, even when its just using library components.

Using JQuery without any help from tools for about a year now I can say there are still those days where javascript drives me mad, but most of the time I its working perfect. Since Ajax related calls like GET, POST, and FORM stuff are oneliners I don’t see why one would need tighter integration of JS and frameworks.

Larger Javascript components like Popup Calendars, Autocomplete and the like are harder to integrate than in Rails or cakePHP, but thinking about the problem a little longer leads to powerful reusable solutions, that save time on integration after the first initial setup.

]]>
Sat, 21 Jun 2008 00:00:00 +0200
https://beberlei.de/2008/06/21/simplifying-zf-scope-of-variables-in-view-and-controller.html https://beberlei.de/2008/06/21/simplifying-zf-scope-of-variables-in-view-and-controller.html <![CDATA[Simplifying ZF: Scope of Variables in View and Controller]]> Simplifying ZF: Scope of Variables in View and Controller

As a follow up on the Zend Framework, “Web 2.0 Framework” My Ass! article referenced earlier I came up with some simplifications of the Zend View and Controller variable coupling.

Using some code in the article I created a new Zend_View_Extended class which offers the possibility to circumvent the access of variables via $this and directly registers each variable key of the Zend_View object as an own variable in the script template:

class Zend_View_Extended extends Zend_View_Abstract
{
    protected function _run()
    {
        while( list( $k, $v ) = each( $this ) ) ${$k} = $v;

        include func_get_arg(0);
    }
}

Now rather than calling $this->data you can call $data in a script template. Of course this comes with additional overhead of each variable being registered twice. I don’t know how this handles performance wise, but maybe unsetting $this variables after copying solves this. Also you have to overwrite the initView() method of your base Controller Action class.

Another simplification would be to allow for the following direct variable settings in any Controller Action, which would shorten the $this->view->variable call, to derive the same functionality via $this->variable. I haven’t tested this though.

class Zend_Controller_SimpleAction extends Zend_Controller_Action
{
    function __set($name, $value)
    {
        $this->view->{$name} = $value;
    }
}
]]>
Sat, 21 Jun 2008 00:00:00 +0200
https://beberlei.de/2008/06/24/teaching-zend-form-some-mvc.html https://beberlei.de/2008/06/24/teaching-zend-form-some-mvc.html <![CDATA[Teaching Zend_Form some MVC]]> Teaching Zend_Form some MVC

Lots of people complain that Zend Form runs counter to the MVC spirit, because it handles validation, business logic and view elements all in one acting as Model, Controller and View.

Extending the Zend Form component to handle all this different aspects in different layers of the application is rather easy though. What we want of a MVC compatible Form object is the following:

  • The model escapes and validates all the data that is put into the form.
  • The view decides on how the form is displayed.
  • The controller moves data from the model to the view and back, handling the stages of the form request.

The first step is, allowing any Model that implements Zend_Validator_Interface to hook into the Zend_Form Validation process. We generate a new class, WW_Form_Mvc and allow a function setModel() to insert any Model object that implements Zend_Validator_Interface into the form. We extend isValid(), getMessages() and getErrors() to not only check all the form elements validators, but also the models validators. Please note that the array_merge() solution is not the correct way of how this snippet should work. Any merge operation of the messages and errors has to be on a field key level, which is not currently done.

class WW_Form_Mvc extends Zend_Form
{
    protected $model = null;

    /**
     * Extends isValid() method of Zend Form to check for validity of specified model
     * @param Array $data
     * @return Boolean
     */
    public function isValid($data)
    {
        $valid = parent::isValid($data);

        if($valid == true && !is_null($model)) {
            $valid = $this->model->isValid($data) && $valid;
        }
        return $valid;
    }

    /**
     * Extends getMessages() Validator Interface implementation of Zend Form to also
     * return the messages of the Model validation.
     * @return Array
     */
    public function getMessages($name = null, $suppressArrayNotation = false)
    {
        $messages = parent::getMessages($name, $suppressArrayNotation);

        if(!is_null($model)) {
             $form_messages = $this->model->getMessages();

             $messages = array_merge($messages, $form_messages);
        }

        return $messages;
    }

    /**
     * Extends getErrors() Validator Interface implementation of Zend Form to also
     * return the errors of the Model validation.
     * @return Array
     */
    public function getErrors($name = null)
    {
         $messages = parent::getErrors($name);

        if(!is_null($model)) {
             $form_messages = $this->model->getErrors();

             $messages = array_merge($messages, $form_messages);
        }

        return $messages;
    }

    /**
     * Set a Model object, which has to implement Zend_Validate_Interface
     * @throws Zend_Exception
     */
    public function setModel($model)
    {
        if($model instanceof Zend_Validate_Interface) {
            $this->model = $model;
        } else {
             throw new Zend_Exception('WW_Form_Mvc expects a model of type Zend_Validate_Interface');
        }
    }
}

We then extend the WW_Form_Mvc class to disable the automatic loading of decorators in the Constructor and additionally allow to pass a Model to the constructor as second argument:

class WW_Form_Mvc
{
    protected $model = null;

    /**
     * this overrides the original Zend_Form Constructor and skips
     * the decorator initialisation, because this is now being handled
     * by View Helpers
     */
    public function __construct($options=null, $model=null)
    {
        if (is_array($options)) {
            $this->setOptions($options);
        } elseif ($options instanceof Zend_Config) {
            $this->setConfig($options);
        }

        // Extensions...
        $this->init();

        if(!is_null($model)) {
             $this->setModel($model);
        }
    }

    // All the other stuff here
}

In our views we want to use helper methods to manage the displaying of the form. For each different style of form displaying, we can generate different helpers. For example a helper that would only apply the default decorators would look like this:

class WW_View_Helper_FormDefault
{
    /**
     * Load only default decorators on this Zend_Form object
     *
     * @param Zend_Form $form
     */
    public function formDefault(Zend_Form $form)
    {
        if($form instanceof Zend_Form) {
            $form->loadDefaultDecorators();
            return $form;
        }
    }
}

We can now use this helper and in any template say: <?= $this->formDefault($this->someForm); ?> We can now look at our controller action that implements this form and we will see that it does not look different from what we would have done before:

function formAction()
{
    $model = new SomeModel();

    $form = new WW_Form_Mvc();
    $form->setModel($model);

    // generate form here, adding elements and stuff

    if($form->isValid($_POST)) {
        $model->insert($form->getValues());
        $this->view->form = "Form was submitted!";
    } else {
        $this->view->form = $form;
    }
}

Isnt that nice? Now each part of the equation is doing what its supposed to do.

]]>
Tue, 24 Jun 2008 00:00:00 +0200
https://beberlei.de/2008/11/05/overwrite-ezcmvccontroller-a-bit-more-rapid.html https://beberlei.de/2008/11/05/overwrite-ezcmvccontroller-a-bit-more-rapid.html <![CDATA[Overwrite ezcMvcController - A bit more rapid]]> Overwrite ezcMvcController - A bit more rapid

ezComponents gets Mvc in its 2008.2 version. I have played around a bit with the alpha version, since I am currently searching for a good framework for a high performance application. My first benchmarks on ezcMvc just say: wh000pie! A lot faster as compared to the Zend Framework. Still ezcMvc is VERY loosely coupled and you have to write lots of lines to get where ZF gets you with less (more magic involved). As you can see from the tutorial on the ezcMvcTools component it is currently a bit unwieldily to work with ezcMvcController since you have to create and return result objects everywhere. This is very nice since it abstracts from the actual response type (could be www, mail, cli, anything). I have created a very little extension of the ezcMvcController class that hopefully serves you quite some time. You can append variables to the ezcMvcResult object by calling the magic __get and __set on the controller. Plus it offers a method to use the ezcMvcInternalRedirect instead of the result. See for yourself:

class myController extends ezcMvcController
{
    protected $result;

    public function createResult()
    {
        $actionMethod = $this->createActionMethodName();

        if ( method_exists( $this, $actionMethod ) ) {
            $status = $this->$actionMethod();
            if($status != 0) {
                $this->getResult()->status = $status;
            }
            return $this->getResult();
        } else {
            throw new ezcMvcActionNotFoundException( $this->action );
        }
    }

    protected function _redirect($uri)
    {
        $request = clone $this->request;
        $request->uri = $uri;
        $this->result = new ezcMvcInternalRedirect($request);
    }

    public function __get($name)
    {
        if(isset($this->getResult()->variables[$name])) {
            return $this->getResult()->variables[$name];
        }
        return null;
    }

    public function __set($name, $value)
    {
        $this->getResult()->variables[$name] = $value;
    }

    public function __isset($name)
    {
        return isset($this->getResult()->variables[$name]);
    }

    protected function getResult()
    {
        if($this->result === null) {
            $this->result = new ezcMvcResult();
        }
        return $this->result;
    }
}

You can now use a controller in the following way:

class dashboardController extends myController
{
    public function doIndex()
    {
        $this->cookie = "Cookie!"; // Proxy to $ezcMvcResult->variables['cookie']
    }

    public function doRedirect()
    {
        $this->_redirect("/");
    }
}

Very nice! The next thing I have to extend in ezcMvc is automagical matching of controller and action names to view output names with a special View Handler that takes care of this. This saves another bunch of work you have to cope with in the current standard setup.

]]>
Wed, 05 Nov 2008 00:00:00 +0100
https://beberlei.de/2008/11/25/next-zf-mini-release-1-7-1.html https://beberlei.de/2008/11/25/next-zf-mini-release-1-7-1.html <![CDATA[Next ZF Mini Release: 1.7.1]]> Next ZF Mini Release: 1.7.1

In the next days the next mini release of the Zend Framework will be released and I want to keep you up to date of my progress on it. I got lots of feedback on the jQuery component and also had the possibility to fix some bugs and documentation issues. I also fixed some minor bugs that have been in the Zend_Soap_Wsdl stuff i added for the 1.7 release. And a question for the 1.8 release that is due around February 2009: What do you think is missing in the jQuery support to ZF? Any widgets or stuff that you want to have included?

]]>
Tue, 25 Nov 2008 00:00:00 +0100
https://beberlei.de/2008/11/01/zf-1-7-jquery-is-in-zend-soap-with-lots-of-bugfixes.html https://beberlei.de/2008/11/01/zf-1-7-jquery-is-in-zend-soap-with-lots-of-bugfixes.html <![CDATA[ZF 1.7: jQuery is in! Zend_Soap with lots of Bugfixes.]]> ZF 1.7: jQuery is in! Zend_Soap with lots of Bugfixes.

The next version of Zend Framework will appear quite soon (16th November) and there will be lots of buzz about Zend Amf, the new component sponsored by Adobe to support Adobe Flash (or other tools) to PHP communications.

Besides the buzz there is some stuff from me getting into the Zend Framework. jQuery View and Form Helpers will allow you to throw the Dojo stuff away and use the great jQuery. This is of course inherently subjective, but still different opportunities are always great.

Additionally I have been fixing quite some bugs on the Zend Soap component and will manage to fix further stuff until the 1.7 release I hope. This will make especially Wsdl and AutoDiscover produce valid WSDL XML plus adds setObject() support to Zend_Soap_Server.

You can now choose between several strategies to evaluate complex types in WSDL Autodiscovering. Array of Datatypes (simple or complex) via type[] syntax and complex objects can be parsed differently now based on settings.

]]>
Sat, 01 Nov 2008 00:00:00 +0100
https://beberlei.de/2008/11/01/performance-in-calypso-and-bugs.html https://beberlei.de/2008/11/01/performance-in-calypso-and-bugs.html <![CDATA[Performance in Calypso (and Bugs)]]> Performance in Calypso (and Bugs)

I have tested Calypso performance some weeks ago and have not been able to write on it before. Don’t use it please its fucking slow and cannot be optimized in the current architecture, because the template will NOT be parsed into intermediate PHP code.

Additionally there are some bugs:

  1. You cannot currently use UTF8 encoding since htmlentities deep in the escape mechanism cannot be given the utf-8 specification.
  2. Triple or higher inheritance may not work under some circumstances. Which sucks.

I sadly have no time currently to fix any of the issues poping up with Calypso, so please don’t use it for anything production.

]]>
Sat, 01 Nov 2008 00:00:00 +0100
https://beberlei.de/2008/11/29/encouraging-forks-for-calypso-dtl.html https://beberlei.de/2008/11/29/encouraging-forks-for-calypso-dtl.html <![CDATA[Encouraging forks for Calypso DTL]]> Encouraging forks for Calypso DTL

I still get a bunch of emails regarding the Calypso DTL template engine that I have written some month ago and which i can currently not maintain any further (see my posting on that). Still I encourage anyone to fork the project and develop it further. Personally what I think is missing are some bugfixes and enhancements regarding the compiling and parsing. I created a GITHub Project for Calypso DTL, so collaborative development is very easy and added general TODOS to the README.txt that is on the front page of github project page.

]]>
Sat, 29 Nov 2008 00:00:00 +0100
https://beberlei.de/2008/11/12/updating-from-kde3-5-to-kde4-with-mac-style-menubars.html https://beberlei.de/2008/11/12/updating-from-kde3-5-to-kde4-with-mac-style-menubars.html <![CDATA[Updating from KDE3.5 to KDE4 with Mac-Style Menubars]]> Updating from KDE3.5 to KDE4 with Mac-Style Menubars

It seems KDE4 has no Mac-Style menubars and when you upgrade from 3.5 to 4.1 within the same config folder (say you use the kubuntu distribution upgrade installer like I did) you end up with no menubars at all. To solve this little issue find a variable “macStyle = true” in the /home/username/.kde/share/config/kdeglobals file and switch it to macStyle = false and you have your menubars back.

]]>
Wed, 12 Nov 2008 00:00:00 +0100
https://beberlei.de/2008/08/02/dojox-django-template-language-example-with-data-from-php-json.html https://beberlei.de/2008/08/02/dojox-django-template-language-example-with-data-from-php-json.html <![CDATA[DojoX Django Template Language Example with Data from PHP/JSON]]> DojoX Django Template Language Example with Data from PHP/JSON

This is a simple example how you would use DojoX Django Template Language component filled with data from a JSON GET Response:

<script type="text/javascript" src="/development/dojo-release-1.1.1/dojo/dojo.js" djConfig="parseOnLoad:true, isDebug:true"></script>

<script language="javascript" type="text/javascript"><!--
dojo.require("dojox.dtl");
dojo.require("dojox.dtl.Context");
dojo.require("dojox.dtl.ext-dojo.NodeList");
dojo.addOnLoad(
    dojo.xhrGet({url: "dojotest.php", handleAs: "json", handle: function(data,ioArgs){
        if(typeof data == "error"){
            console.log("error?",data);
        } else {
            dojo.query("#test").dtl("I am eating {{food.fruit}} and {{food.meat}}", data);
        }
    }
}));
--></script>

::

    <?php
    header('Content-type: text/json');
    $food = array('food' => array('fruit' => 'apple', 'meat' => 'chicken'));
    echo json_encode($food);
    ?>
]]>
Sat, 02 Aug 2008 00:00:00 +0200
https://beberlei.de/2008/08/02/wrestling-with-dojo.html https://beberlei.de/2008/08/02/wrestling-with-dojo.html <![CDATA[Wrestling with Dojo]]> Wrestling with Dojo

I have to complain about the Dojo Toolkit (Considering the project I am using this for). It has the worst documentation and API guides I have ever seen. The documentation (called the Dojo Book) has an unnecessary complex structure to click through and its contents are wrapped around examples that are so overly complex that you click away the pages when they get smashed at your head.

The API too, has no clear structure and hides its contents behind cryptic names such as “dnd”, “fx”, “rpc”. Descriptions of the functions sum to very small sentences and additional options are listed without any description at all other than a link to where they are defined in the project. There are no short examples given to any function like jQuery does.

In general I think the Dojos Site is too complex and hard to navigate. The layout seems to change massively on each click and you it is hard recognize common patterns. Dear Dojo Team, please make your project more accessible!

]]>
Sat, 02 Aug 2008 00:00:00 +0200
https://beberlei.de/2008/08/08/first-impressions-on-zend-soap-and-a-basic-implementation.html https://beberlei.de/2008/08/08/first-impressions-on-zend-soap-and-a-basic-implementation.html <![CDATA[First impressions on Zend_Soap and a basic implementation]]> First impressions on Zend_Soap and a basic implementation

The Zend Framework release candidate for version 1.6 includes a new component for SOAP operations. Zend_Soap_Server/Client extend the PHP functionality of the SOAPClient and SOAPServer objects, which by itself is trivial. The more important functionality it ads to the package is the AutoDiscovery Component.

Generally you can use SOAP in the so called “non-wsdl” mode, that is if you specify the correct options like the soap service location and uri, you don’t need any description of the service for the clients. This is only useful if you’re using the SOAP Service internally since you then know about all the available functions and methods. If you want to offer the Service for external users you want to use the WSDL-mode: Generate a WSDL that describes the services available methods, their parameters and return types is an important task.

Using Zend_Soap_AutoDiscover you can generate your WSDL file automatically by reflecting on the given Service Class methods. This works as follows. We setup a simple service:

class HelloWorldService
{
    /**
     * @return string
     */
    public function helloWorld()
    {
        return "Hello World!";
    }

    /**
     * @return array
     */
    public function getFruits()
    {
        return array('apple', 'orange', 'banana');
    }
}

It is important that you specify the Doc Comments @param and @return otherwise the AutoDiscovery of the correct parameter and return types cannot be resolved. We will now setup a simple SOAP Server access point, that will also generate our WSDL file for description of this HelloWorld service:

require_once "HelloWorldService.php";
require_once "Zend/Soap/Server.php";
require_once "Zend/Soap/AutoDiscover.php";

if(isset($_GET['wsdl'])) {
    $autodiscover = new Zend_Soap_AutoDiscover();
    $autodiscover->setClass('HelloWorldService');
    $autodiscover->handle();
} else {

    $soap = new Zend_Soap_Server("http://localhost/soapserver.php?wsdl"); // this current file here
    $soap->setClass('HelloWorldService');
    $soap->handle();
}

Now we have our SOAP Service up and running and any client can access the HelloWorldService class from a remote or local location with just this simple lines:

require_once "Zend/Exception.php";
require_once "Zend/Soap/Client.php";

try {
    $client = new Zend_Soap_Client("http://localhost/soapserver.php?wsdl"); // Servers WSDL Location
    $string =  $client->helloWorld();
    $fruits = $client->getFruits();

    var_dump($string);
    var_dump($fruits);
} catch(Zend_Exception $e) {
    echo $e->getMessage();
}

This is easy. Additionally Zend_Soap offers a WSDL class to generate your own WSDL file based on your special preferences, which is a nice feature. I guess only some people can write a correct WSDL XML specification file on their own from scratch, so using a powerful helper is reasonable.

The usage of Zend_Soap is quite simple and straightforward as is PHP5’s internal SOAP Service and might therefore gain widespread use. I have never tested PEAR’s WSDL Autodiscovery, so i cannot draw comparisons.

]]>
Fri, 08 Aug 2008 00:00:00 +0200
https://beberlei.de/2008/08/22/zend-helper-for-jquery-ui-widgets-demo.html https://beberlei.de/2008/08/22/zend-helper-for-jquery-ui-widgets-demo.html <![CDATA[Zend Helper for jQuery UI Widgets Demo]]> Zend Helper for jQuery UI Widgets Demo

I finished a pretty demo for all the jQuery UI widgets view helpers for my Zend Framework proposal. It utilizes the jQuery View helper which is further down the proposal road. You can have a look at how nicely the Javascript is generated for the different jQuery UI Widgets. A further demo showing functionality of the form decorators and elements will follow when I finished these components.

Update: I also added a demo showing the form elements usage I have just finished in their first versions. I committed the classes to the SVN so you can already check them if you want.

]]>
Fri, 22 Aug 2008 00:00:00 +0200
https://beberlei.de/2008/08/28/benchmark-and-optimize-zend-framework-performance.html https://beberlei.de/2008/08/28/benchmark-and-optimize-zend-framework-performance.html <![CDATA[Benchmark and Optimize Zend Framework Performance]]> Benchmark and Optimize Zend Framework Performance

In the fall of Rasmus presentation (Simple is Hard) on FrOSCon I tried to optimize Zend Frameworks performance a little bit. There has also been a little discussion on the Zend Framework Mailing List on this topic.

I haven’t changed great parts of the code or anything, I just benchmarked how different global include strategies affect the overall performance of my blog software written with help of the ZF. The following include strategies were tested (I’ve used the Zend Framework 1.6 RC2 package for all of them):

  1. Zend_Loader Autoload, Default PHP Include Path
  2. Zend_Loader Autoload, Swapped Include Path
  3. ZF 1.6 RC2 stripped from all “require_once” dependencies, Zend_Loader Autoload, Swapped Include Path
  4. ZF 1.6 RC2 stripped from all “require_once” dependencies, no autoload, used inclued to find file dependencies and require (not _once) them all on startup.

To strip all the require_once from the Zend Framework source code, i built a little script to do that for me. For the last test I wrote a little script that used the inclued_get_data() function to built a correct dependency tree for all includes. I have put each configuration of my Zend Framework install 30 seconds under siege with 5 concurrent requests. I have rerun all tests with APC and without APC.

Results without APC

Include Strategy Response time Trans/sec Performance %
Autoload, default include path 0.20 24.65 100%
Autoload, swapped include path 0.19 26.83 95%
Autoload, ZF w/o require_once 0.17 29.27 85%
No-Autoload, require up front 0.16 31.82 80%

You can see that each step makes a ZF application run faster and that the full optimization with requiring all scripts up front is about 20% faster than the default configuration.

Results with APC

Include Strategy Response time Trans/sec Performance %
Autoload, default include path 0.11 45.76 100%
Autoload, swapped include path 0.09 53.05 81,81%
Autoload, ZF w/o require_once 0.08 60.90 72,72%
No-Autoload, require up front 0.07 73.99 63,63%

Turning APC on gives a boost of about 50% to your application no matter what include strategy you are following (so there is excuse not using APC). But switching between different include strategies still makes a huge difference in performance. Percentage-wise this is a larger difference than without APC. Requiring all dependent scripts up front takes only about 63% of the default configuration time, which can make a major difference on any production server.

In an application knowing which includes of the Zend Framework will be needed on each request is difficult, since different helpers and classes might be needed based on which url is requested. It maybe a good practice to just include all the classes that might be needed. If this is a job to hard to do you can still get lots of performance gain out of your application by fixing the include path, using Zend Loaders Autoload and stripping all require_once calls from all of the Zend Framework files.

]]>
Thu, 28 Aug 2008 00:00:00 +0200
https://beberlei.de/2008/08/16/zendx-jquery-is-ready-for-recommendation.html https://beberlei.de/2008/08/16/zendx-jquery-is-ready-for-recommendation.html <![CDATA[ZendX_JQuery is Ready for Recommendation]]> ZendX_JQuery is Ready for Recommendation

Yesterday night I finished the proposal for ZendX_JQuery, a View Helper and an Ajax Helper hopefully to be included in the Zend Framework Extras Library in the next release after 1.6. I moved the proposal to the “Ready for Recommendation” section now, so it can be reviewed a last time by the community and by the Zend people.

For those people already using an earlier version, i have squashed about 4-5 bugs I found writing the Unit-Tests for this component. The test coverage for both helpers is 100% although the jQLink Helper might need some additional Use-Case scenarios to ensure proper working. You can find the current version at my SVN repository.

]]>
Sat, 16 Aug 2008 00:00:00 +0200
https://beberlei.de/2008/08/10/import-and-pythonpath-problems-with-pydev-and-linux.html https://beberlei.de/2008/08/10/import-and-pythonpath-problems-with-pydev-and-linux.html <![CDATA[Import and PYTHONPATH Problems with PyDev and Linux]]> Import and PYTHONPATH Problems with PyDev and Linux

Today is began to configure my PyDev Eclipse environment since I want to learn Python and need it for a programming aspect of my forthcoming diploma thesis. I ran into several difficulties getting the plugins run dialog to work. It seems that configuring the python interpreter under linux and adding all the correct include paths is not enough, even when following the PyDev FAQ.

Nearly giving up I came up with sort of a hack to get it running. Eclipse can execute arbitrary “External Tools” (Find it at: “Run -> External Tools -> Open External Tools Dioalog”). I used this facility to integrate an external tool “Python” with path “/usr/bin/python”. Now to automatically execute the currently active python script, you can specify the line: “${resource_loc}” in the arguments field of the dialog and you’re good to go.

You now circumvented the PyDev internal execution scheme and work with the pure command line interpreter. This way all the problems of PyDev with the PythonPath, Symlinks and whatever are gone.

Update: Another note maybe, you can also add the action “running the last used external tool again” as keyboard shortcut in the Eclipse preferences. This simplifies the execution even more.

]]>
Sun, 10 Aug 2008 00:00:00 +0200
https://beberlei.de/2008/08/04/complete-django-template-engine-implementation-for-php5-downloadable-now.html https://beberlei.de/2008/08/04/complete-django-template-engine-implementation-for-php5-downloadable-now.html <![CDATA[Complete Django Template Engine Implementation for PHP5 - Downloadable Now]]> Complete Django Template Engine Implementation for PHP5 - Downloadable Now

I prepared a downloadable version of my Django Template Language Engine Clone for PHP5 and the Zend Framework (although it can easily be used Standalone). It also implements two helpers that aid in rapid ajax/Dojox deployment. You can easily specify template blocks that should be rendered by Javascript via DojoX DTL, which are then marked, compiled and shown by the client using JSON data from the server. I included a demo to show this feature.

You can grab the Tarball and there is also a README with some information. I would appreciate if anybody would test the lib and gave some feedback.

I will soon start to push this component in the ZF proposal process. I still have to finish the class structure before I can move to the Ready for Review stage. The library will max end up in Zend Extras. Therefore I will have to say again: This View Renderer shall be no replacement for the current Zend View. It is only an alternative which allows for some neat functionality and stricter View / Controller logic separation.

UPDATE: I finished a little page for what I call the Calypso DTL. See it here: Calypso Site. I have put up a little tutorial how to get the Template Engine running with Zend Framework and as a standalone component.

]]>
Mon, 04 Aug 2008 00:00:00 +0200
https://beberlei.de/2008/08/31/greaat-python-can-control-your-wiimote.html https://beberlei.de/2008/08/31/greaat-python-can-control-your-wiimote.html <![CDATA[Greaat, Python can control your Wiimote]]> Greaat, Python can control your Wiimote

In the last days I had lots of time to play with my Wiimote and its bluetooth capabilities. Sadly there is actively supported driver in non-C (and I am not the biggest C programmer). I found this little prototype driver written in Python by Will Woods which is using pyBluez to communicate with my wiimote and have extended it quite a bit. WiiLinux is a huge resource that offers information on reverse engineering of the Wiimote protocol.

The driver is still far from finished or usable outside some demo applications, but that might still be included in the future. You can download this little goodie and have a look at it.

]]>
Sun, 31 Aug 2008 00:00:00 +0200
https://beberlei.de/2008/01/23/constructions.html https://beberlei.de/2008/01/23/constructions.html <![CDATA[Constructions…]]> Constructions…

Some people complained: Yes comments do not work YET, and also the links behind categories and Tags show nothing. Everything else follows shortly. I am programming this whole blog using the Zend Framework to have a bit of learning experience. It will take some more time than if I would do it in “the old ways”.

]]>
Wed, 23 Jan 2008 00:00:00 +0100
https://beberlei.de/2008/01/14/helpers-in-zend-view-class-prefix.html https://beberlei.de/2008/01/14/helpers-in-zend-view-class-prefix.html <![CDATA[Helpers in Zend_View: Class Prefix]]> Helpers in Zend_View: Class Prefix

I have to say, the Zend Framework documentation is a bit imprecise sometimes. Regarding custom Helpers in Zend_View it states:

The class name must, at the very minimum, end with the helper name itself, using CamelCaps. E.g., if you were writing a helper called “specialPurpose”, the class name would minimally need to be “SpecialPurpose”. You may, and should, give the class name a prefix, and it is recommended that you use ‘View_Helper’ as part of that prefix: “My_View_Helper_SpecialPurpose”. (You will need to pass in the prefix, with or without the trailing underscore, to addHelperPath() or setHelperPath()).

What does that mean regarding how your Helper has to be called? The default prefix for a Helper function that should be called foobar() is actually:

class Zend_View_Helper_Foobar()
{
    function foobar() { }
}

It took me some time to realize that. Why doesn’t Zend just give an example like that? To change the prefix in a context of the Zend_Controller_Action you have to do something like this:

class SomeController extends Zend_Controller_Action
{
    function init() {
        $this->initView();
        $this->view->addHelperPath("pathtohelpers", "ClassPrefix");
    }
}
]]>
Mon, 14 Jan 2008 00:00:00 +0100
https://beberlei.de/2008/01/22/a-knight-s-tour-solver-in-pascal.html https://beberlei.de/2008/01/22/a-knight-s-tour-solver-in-pascal.html <![CDATA[A knight’s tour solver in pascal]]> A knight’s tour solver in pascal

My professor has a task on the current problem set of my pascal programming course to program, demanding a recursive program finding a solution to the knights tour problem on a chess board of size n>=3 given a starting position. I used a backtracking algorithm (at least that is what wikipedia tells me it is), by trying all possible combinations and setting steps back, when they do not lead to a solution.

Beginning with n = 7 my algorithms gets very slow, caused by the backtracking algorithm which is not very efficient. I found some interesting articles regarding this problem using graph theory to solve it. Some pretty big minds have thought about the problem, I guess its not for me to find such an solution for a little task of the weekly problem set.

]]>
Tue, 22 Jan 2008 00:00:00 +0100
https://beberlei.de/2008/01/22/knights-tour-using-warndorff-s-rule.html https://beberlei.de/2008/01/22/knights-tour-using-warndorff-s-rule.html <![CDATA[Knights Tour using Warndorff’s rule]]> Knights Tour using Warndorff’s rule

When using Warndorff’s rule for solving the Knights Tour problem, the solution is found within a second. It took some time for me to realize it in PASCAL though using a RECORD type to match the jumps to the number of free fields and then sort them. You can view my source code here.

]]>
Tue, 22 Jan 2008 00:00:00 +0100
https://beberlei.de/2008/01/22/a-knight-s-tour-solver-in-pascal-supplement.html https://beberlei.de/2008/01/22/a-knight-s-tour-solver-in-pascal-supplement.html <![CDATA[A knight’s tour solver in pascal - supplement]]> A knight’s tour solver in pascal - supplement

Following my earlier post on the Knights tour: It seems it is crucial for a backtracking algorithm what the knights starting position is, and even more important in what order the jumps are tried. Because there are 8 possible moves for a knight, you have to decide in which order they should be performed. If there is not a deviation when backtracking the algorithm gets stuck in almost the same positions very very often.

I rewrote the order of jumps for my 7x7 board and found a solution in seconds. Using the same order for an 8x8 board gets me stuck in endless tries again. I see its important to implement the Warnsdorff’s algorithm, perhaps I will do so.

]]>
Tue, 22 Jan 2008 00:00:00 +0100
https://beberlei.de/2008/01/28/linux-style-mysql-shell-prompt.html https://beberlei.de/2008/01/28/linux-style-mysql-shell-prompt.html <![CDATA[Linux style MySQL Shell prompt]]> Linux style MySQL Shell prompt
Just add the following to the [mysql] section in your .my.cnf file:
prompt=\u@\h [\d]>\
]]>
Mon, 28 Jan 2008 00:00:00 +0100
https://beberlei.de/2008/01/27/writing-a-game-for-kde4.html https://beberlei.de/2008/01/27/writing-a-game-for-kde4.html <![CDATA[Writing a Game for KDE4]]> Writing a Game for KDE4

I am writing PHP / Web Applications for about 8 years now, but I am not so much the pro in writing desktop applications. I know little or no C/C++/Java, but want to change that soon. Since I use KDE for both my desktop and laptop it seems reasonable to write a KDE 4 application.

What about ideas? I wanted to write a game for so long and in the last years my girlfriend and I tried lots of 2 person player games. One of them I wanted to re-create as a computer game.

The possibilities are:

All 3 games are time intensive and very strategic. Moruba can be played for lots of hours (My gf and I played one game for about 5-6 hours). Its probably more easy to write algorithms for the computer player of Oware or Moruba than for Amazons, but that should not affect my decision. I will decide in the next weeks, but I should probably concentrate on my semesters final exams in the first place.

Update: Moruba seems to have different rules than the game we are playing. I will elaborate on that soon.

]]>
Sun, 27 Jan 2008 00:00:00 +0100
https://beberlei.de/2008/01/20/zend-view-hack-for-implementation-of-rss-feed.html https://beberlei.de/2008/01/20/zend-view-hack-for-implementation-of-rss-feed.html <![CDATA[Zend_View Hack for implementation of RSS Feed]]> Zend_View Hack for implementation of RSS Feed

Using Zend_View for your site and want to implement an RSS Feed? You might run into the problem that the XML header definition is interpreted as PHP code:

<?xml version="1.0" encoding="ISO-8859-1"?>

Before you start cursing the world try this (rather obvious) workaround:

<?php echo '<?xml version="1.0" encoding="ISO-8859-1"?>'; ?>
]]>
Sun, 20 Jan 2008 00:00:00 +0100
https://beberlei.de/2008/01/20/using-a-self-defined-zend-loader.html https://beberlei.de/2008/01/20/using-a-self-defined-zend-loader.html <![CDATA[Using a self defined Zend_Loader]]> Using a self defined Zend_Loader

When you start writing with a Framework like Zend like I did in the last weeks, you probably are still haunted by all the bad habits of pseudo object orientated PHP design, using lots of require and include’s. Zend_Loader doesn’t seem to help at first, its just another command (Zend_Loader::loadClass or ::loadFile). It sucks to register lots of autoload paths too.

I wrote my own Zend_Loader child class. It grabs all include paths from a config File using the Zend_Config class, and completely frees you of thinking about includes.

class WWLoader extends Zend_Loader
{
    private static $dirs = NULL;

    public static function loadClass($class)
    {
        if(self::$dirs===NULL) {
            //
            // Include directories of this application are saved in a configuration file. If it has not been
            // loaded before do so now and safe everything to the private static variable $dirs which will
            // then be used in further loadings
            //
            $dirs = array();
            $conf_app_path = new Zend_Config_Ini(sprintf('%s%s', constant('ZEND_CONFIG_PATH'), 'application.ini'), 'appincludepath');
            self::extractPaths($conf_app_path->toArray(), $dirs);
            self::$dirs = $dirs;
        }

        parent::loadClass($class, self::$dirs, true);
    }

    public static function autoload($class)
    {
        try {
            self::loadClass($class);
            return $class;
        } catch (Exception $e) {
            return false;
        }
    }

    /**
     * Given an array with subkeys of include paths this function unifies this array to
     * a single one and returns the result in the second argument $dirs which is given
     * by reference.
     *
     * @param Array $array
     * @param Array $dirs
     */
    public static function extractPaths($array, &$dirs)
    {
        foreach($array AS $k => $v) {
            if(is_array($v)) {
                self::extractPaths($v, $dirs);
            } else {
                $dirs[] = $v;
            }
        }
    }
}

ZEND_CONFIG_PATH is the only constant I use in my application. I initialize the following at the beginning of my bootstrap file:

define('ZEND_CONFIG_PATH', dirname(__FILE__)."/../application/config/");

require_once 'Zend/Loader.php';
require_once dirname(__FILE__).'/../application/include/WWLoader.php';
Zend_Loader::registerAutoload();
Zend_Loader::registerAutoload('WWLoader');

After that, its just Objects.

]]>
Sun, 20 Jan 2008 00:00:00 +0100
https://beberlei.de/2008/01/17/zf-managing-404-errors-with-version-1-0-3.html https://beberlei.de/2008/01/17/zf-managing-404-errors-with-version-1-0-3.html <![CDATA[ZF: Managing 404 Errors with Version 1.0.3]]> ZF: Managing 404 Errors with Version 1.0.3

Ok, the description of the Link referring to the version 0.9 solution for Managing 404 errors due to missing modules or actions is wrong. It is not enough to replace Zend::loadClass with Zend_Loader::loadClass. Other functions have been renamed, my current snippet is:

/**
 * Original Snippet from: http://www.bigroom.co.uk/blog/managing-404-errors-in-the-zend-framework
 */
    class NoroutePlugin extends Zend_Controller_Plugin_Abstract
    {
        public function preDispatch(Zend_Controller_Request_Abstract $request )
        {
            $dispatcher = Zend_Controller_Front::getInstance()->getDispatcher();

            $controllerName = $request->getControllerName();
            if (empty($controllerName)) {
                $controllerName = $dispatcher->getDefaultControllerClass($request);
            }
            $className = $dispatcher->formatControllerName($controllerName);
            if ($className)
            {
                try
                {
                    // if this fails, an exception will be thrown and
                    // caught below, indicating that the class can't
                    // be loaded.
                    Zend_Loader::loadClass($className, $dispatcher->getControllerDirectory());
                    $actionName = $request->getActionName();
                    if (empty($actionName)) {
                        $actionName = $dispatcher->getDefaultAction();
                    }
                    $methodName = $dispatcher->formatActionName($actionName);

                    $class = new ReflectionClass( $className );
                    if( $class->hasMethod( $methodName ) )
                    {
                        // all is well - exit now
                        return;
                    }
                }
                catch (Zend_Exception $e)
                {
                    // Couldn't load the class. No need to act yet,
                    // just catch the exception and fall out of the
                    // if
                }
            }

            // we only arrive here if can't find controller or action
            $request->setControllerName( 'blog' );
            $request->setActionName( 'noroute' );
            $request->setDispatched( false );
        }
    }

I hope someone needs this as much as me.

]]>
Thu, 17 Jan 2008 00:00:00 +0100
https://beberlei.de/2008/01/24/complaining-about-zend-filter-input.html https://beberlei.de/2008/01/24/complaining-about-zend-filter-input.html <![CDATA[Complaining about Zend_Filter_Input]]> Complaining about Zend_Filter_Input

Zend_Filter_Input is really nice to be sure form data is filtered correctly, but i have some serious complaints: Why the hell are the error messages so user-unfriendly? I really would like to use the output of Zend_Filter_Input::getMessages(), but I could never trust to show them to a user of my website: They are obviously written for developers. Some examples:

‘’ does not appear to be an integer ‘’ is an empty string

I use a ton of validators to filter the comments form and all possible error messages that can be thrown are not “user-save”. Because all these messages are handled in each Validator class its almost impossible to change them without going nuts:

$validators = array(
    'article_id' => array(
        'Int',
        new Zend_Validate_Int(),
        array('GreaterThan', 0)
        ),
    'username' => 'Alnum',
    'userEmail' => new Zend_Validate_EmailAddress(Zend_Validate_Hostname::ALLOW_DNS | Zend_Validate_Hostname::ALLOW_LOCAL, true),
    'cp'   => array(
        'Digits',                // string
        new Zend_Validate_Int(), // object instance
        array('Between', 1138, 1138)  // string with constructor arguments
    ),
    'comment' => 'NotEmpty',
);

This simple $validators requirement creates templates for error messages in each validator object, that are: Zend_Validate_Int, Zend_Validate_Alnum, Zend_Validate_EmailAddress, Zend_Validate_Hostname, and the not empty message. To allow for user friendly messages one had to make child objects of each of the Validators and edit the error messages accordingly. I avoid this by printing my own errors from outside the Zend_Filter_Input, rather than using the object for what it is probably supposed to do.

]]>
Thu, 24 Jan 2008 00:00:00 +0100
https://beberlei.de/2008/01/24/how-my-zend-db-table-models-look-like.html https://beberlei.de/2008/01/24/how-my-zend-db-table-models-look-like.html <![CDATA[How my Zend_Db_Table models look like]]> How my Zend_Db_Table models look like

I am not too happy with the ZF Frameworks Table access solution. Not that I have anything against the Table Gateway pattern, but in most web applications you almost always have to join data some way or another. Even this rather simple application, a blog, needs joins for displaying articles so that category, comment count and tags can be displayed. Therefore I use an instance of Zend_Db_Table_Abstract only for the simplest purposes and pimp it by using lots of public methods and Zend_Db_Select. An example:

class SomeModel extends Zend_Db_Table_Abstract
{
  $_name = "someTable";
  $_primary = "id";

  public function getSomethingByJoin($param1, $param2)
  {
     $db = $this->getAdapter();
     $select = $db->select();
     ...[build select]
     $result = $db->query($select);

     return $result;
  }
}

That way you still follow the MVC pattern and don’t have to take one compromise after another for getting the Table Gateway to reproduce a result that looks like a join.

]]>
Thu, 24 Jan 2008 00:00:00 +0100
https://beberlei.de/2008/10/30/php-conference-recap-netbeans-ezcomponents-mapreduce.html https://beberlei.de/2008/10/30/php-conference-recap-netbeans-ezcomponents-mapreduce.html <![CDATA[PHP Conference Recap: NetBeans, ezComponents, MapReduce..]]> PHP Conference Recap: NetBeans, ezComponents, MapReduce..

This years october edition of IPC in Mainz, Germany was the first php conference I have been to. It ended some hours ago and i am quite happy to have been there. I had several great discussions which pointed me to new stuff (to follow) and also met great people.

First thing that completely fascinated me is NetBeans 6.5 with PHP support. It feels much more easy to use than Eclipse PDT. Its faster (from the little tests i could make) and does not strike when you add several large libraries into the include path. It offers much more help on refactoring and organizing than Eclipse PDT does, for example it offers to generate getter/setter, constructor, inheritance methods. Much thanks to Petr who explained to me in detail how NetBeans works.

Ez had lots of people at the conference (to Kore and Tobias i spoke in more detail) and I had some talks with them about ezComponents. This was especially interesting after the great session on object persistence in ez, as well when I heard that ezcMvc, a model-view-controller component, wil be included in the next release of ezc in December 2008. ezComponents has a Database layer, that does not seem to enforce too much overhead on you but still offers great extensibility through a persistence layer and a library that offers to build Database Diffs. The latter feature is really great and I will definitely have a look at this.

Sebastian Bergmann had a great talk on MapReduce which is an old LISP programming paradigm to work with huge datasets. This style was recently revived by Google because tasks can easily be split and distributed for computing on different nodes with hundreds of threads. Implementing a working example in PHP is really easy, just finding a good way to process your large data seems to be a topic that needs lots of thinking.

Additional great things:

  • Meet Thomas, the guy that wrote Weaverslave (I use it for years to build simple php applications).
  • He is working on a cool open source project Carica Cachegrind with Bastian Feder that will process xdebug cachegrind files via a webinterface (They told me webgrind is actually calculating the numbers wrong).
  • Jan Lehnhart and Kore Nordmann had some great things to say about CouchDB.
  • Brian Acker talked about Drizzle, which is a forked MySQL that throws out lots of legacy stuff from MySQL out, to build a micro-kernel database server that suites the web.
  • Ulf Wendel talked about Mysqli + Mysqlnd. Mysqlnd will be a new internal driver for PHP to access MySQL through Mysqli and optimizes processing and memory consumption. Additionally Mysqli will be extended to offer asynchronous queries that can be fired at the database and polled latter. To be included in PHP 5.3

Not so good things: There were absolutely to few power lines. People using their notebooks were packed on hotspots where power was available. Please more power plug possibilities next time!

]]>
Thu, 30 Oct 2008 00:00:00 +0100
https://beberlei.de/2008/10/01/my-recent-zf-ongoings-jquery-action-controller-couchdb.html https://beberlei.de/2008/10/01/my-recent-zf-ongoings-jquery-action-controller-couchdb.html <![CDATA[My recent ZF ongoings: JQuery, Action Controller, CouchDb]]> My recent ZF ongoings: JQuery, Action Controller, CouchDb

In the last weeks lots of stuff came up for me regarding the Zend Framework. My jQuery component is finished, i have even committed the documentation into the SVN already. Do read it you have to be either an XML fetishist or compile it using a docbook compiler. Lots of questions about ZF and jQuery come up regularly on the mailing lists so I really look forward for the first ZF 1.7 release candidate which will include the helpers for a broader audience. This component comes at a good point, since a few days ago the jQuery team announced a deal with Microsoft and Nokia. Microsoft will include jQuery as the framework to go into its Webbased ASP.Net applications and Nokia will include the library in their mobile phones.

Additionally i reported an important issue (ZF-4385) that offers a patch to create an interface of the Action Controller and therefore allows everyone to implement their own action controller classes. May it be lightweight implementations or like my current use-case webservice only controllers that route all their actions through soap, xml-rpc or a rest server. I would really appreciate if more people would vote +1 on this issue to get it included in 1.7

In the last weeks i also played around with CouchDb, a document based database. Its the perfect storage medium for blogs, wikis or other knowledge-based applications and comes with a lightweight and easy REST Api for ClientSide access. Within the Zym project i managed to implement a prototype that is based on Jurrien’s prototype for a CouchDb client. Matthew of Zend Fame published a proposal for a Zend Couch component yesterday and we teamed up to implement two prototypes (mine is based on Matthews and integrates lots of stuff of the Zym component).

]]>
Wed, 01 Oct 2008 00:00:00 +0200
https://beberlei.de/2008/04/27/ox-cross-correlation-coefficients-with-lags.html https://beberlei.de/2008/04/27/ox-cross-correlation-coefficients-with-lags.html <![CDATA[Ox: Cross-correlation coefficients with lags]]> Ox: Cross-correlation coefficients with lags

Working on the homework for my Monetary Economics class I realized that Ox has different mechanisms to calculate correlation coefficients like Mathlab has.

Our assignment was to replicate the Dynamic Correlations graph in Walshs’ Monetary Theory and Policy for European Data, which shows the cross-correlation of different monetary aggregates (M0,M1,M2) with GDP over lagging periods of -8 to +8 quarters.

What was missing for me to do that assignment in Ox was a function to calculate cross-correlation coefficients for two vectors for a specified lag length:

#include <oxfloat.h>

/**
 * Computes the cross-correlation coefficient for two vector series allowing
 * to optionally specify the number of positive and negative lags the ccc
 * should be calculated for.
 *
 * @author Benjamin Eberlei (kontakt at beberlei dot de)
 * @param vVarY Tx1 vector of variable related to
 * @param vVarX Tx1 vector of variable which is tested in different lags
 * @param lags Integer indicating the number of negative and positive lags.
 * @returns (1+lags*2)*1 vector of correlations from -lags to lags
 **/
ccf(const vVarY, const vVarX, const lags)
{
    // Generate positive and negative lags of given length and fill with NaN, so
    // that rows with not available numbers can be dropped from calculating the CCF later on.
    decl mXLag = lag0(vVarX, range(lags, -lags), M_NAN);

    decl mCorr, sCov;

    // initialize result vector holding one crosscorrelation per lag
    decl vCorrLags = zeros(1+lags*2, 1);

    // sadly there are no matrix operations to ease this computation, loop over all lags
    for(decl i = 0; i < 1+lags*2; i++) {
        mCorr = deleter(vVarY ~ mXLag[][i]); // stick y and current lagged x together and delete NaN rows

        // calculate covariance of both time series
        sCov = 1/(rows(mCorr)-1) * sumc( (mCorr[][0]-meanc(mCorr[][0]))' * (mCorr[][1]-meanc(mCorr[][1])) );

        // calculate correlation coefficient
        vCorrLags[i][0] = sCov / ( sqrt(varc(mCorr[][0])) * sqrt(varc(mCorr[][1])) );
    }

    return vCorrLags;
}

It may prove useful to someone.

]]>
Sun, 27 Apr 2008 00:00:00 +0200
https://beberlei.de/2008/09/05/jquery-component-approved-for-extras-library.html https://beberlei.de/2008/09/05/jquery-component-approved-for-extras-library.html <![CDATA[jQuery Component approved for extras library]]> jQuery Component approved for extras library

Yesterday both my jQuery Helper proposals (Core Helper, UI Widgets) for the Zend Framework have been accepted for development. I have already checked my working prototypes into the Zend Framework SVN. In my opinion the usage is quite stable already. There is likely to be no argument flip, function renaming or whatsoever. I have renamed the jqLink helper to ajaxLink though, because other libraries are offering the same functionality to make Ajax related calls and this way the learning curve for people using different libraries in different projects may be more easy.

What is still missing? I have to add captureStart()/captureEnd() functions to the Container Widgets Accordion and Tabs, and will create additional pane helpers for both of them. This will likely be finished this weekend and then there is only the documentation missing. I am very confident this component will be finished for the Zend Framework 1.7 release later this year.

Since this component also includes helpers for the jQuery UI Widgets 1.6 release (which is not released yet), a little waiting time before release is still good. I hope this all works out and the jQuery UI 1.6 release is in the Google CDN, when ZF 1.7 comes out.

]]>
Fri, 05 Sep 2008 00:00:00 +0200
https://beberlei.de/2008/09/11/new-calypso-dtl-version-0-2.html https://beberlei.de/2008/09/11/new-calypso-dtl-version-0-2.html <![CDATA[New Calypso DTL Version 0.2]]> New Calypso DTL Version 0.2

I have got some hints about bugs and optimized the performance of loops a little bit, so that you can now download version 0.2 of the Calypso DTL template engine. Have fun.

]]>
Thu, 11 Sep 2008 00:00:00 +0200
https://beberlei.de/2008/09/20/dependency-injection-via-interface-in-php-an-example.html https://beberlei.de/2008/09/20/dependency-injection-via-interface-in-php-an-example.html <![CDATA[Dependency Injection via Interface in PHP: An example]]> Dependency Injection via Interface in PHP: An example

Personal projects and work currently both force me to think about application design and I came across dependency injection the other day. Its quite a complex Design Pattern and hard to grasp, even with the number of examples you can find in the internet. Basically Dependency Injection means you register components that might be a dependency for other components and when loading new classes via the dependency injection framework, it knows of this relationships and assembles the object in need and all its dependencies and returns them fully initialized. This pattern leads to completely encapsulated objects where any class in the object graph can be exchanged for a new implementation without breaking all its dependencies. It also makes testing complex components and object relationships very easy.

As far as I could find out, three different types of Dependency Injection implementations exist: Via Constructor, via Setter and via Interfaces. There exist implementations of Dependency Injection for Java (Spring) that are configured via XML config files and seem very complex. They are also hard to read in code I presume, since you always have to be aware of the current application configuration. There are also some more lightweight implementations (PicoContainer) that assemble their relationships in the application and work with config methods that act as configuration of the dependencies. Martin Folwer also discusses an implementation via interfaces, which seems most accessible for me.

All PHP clones of any dependency injection framework seem to go the complex way via configuration though. With traits and multiple class inheritance on the horizon, some kind of interface injection seems mighty powerful though. So I implemented a really lightweight implementation of Interface Dependency Injection for PHP. The Container works with static method only, is therefore in the global scope, such that configuration is reduced to a minimum. Generally you have to write interfaces for object injection, for example:

interface InjectDbAdapter
{
    public function injectDbAdapter(Zend_Db_Adapter_Abstract $db);
}

Other potential examples include “InjectLogger”, “InjectSoapClient”, or “InjectAppConfig”. You then have to register a component for usage with this interface:

Whitewashing_Container::registerComponent('InjectDbAdapter', Zend_Db::factory($dbConfig));

Now any class that implements the InjectDbAdapter interface can be instantiated via:

$obj = Whitewashing_Container::load('Class');

and the loader takes care of calling the ‘Class’ implementation of injectDbAdapter with the given Database Adapter. A negative consequence of this approach is that you have to implement the inject methods for all interfaces in your concrete class implementations. With Traits (multiple class inheritance) being a new feature in PHP soon, injection via parent classes seems to become a very powerful approach though, which can handle the concrete implementations.

The Container takes care of all the dependency building, so when testing your components you can register lots of mock objects. You can also exchange dependencies for only a subset of objects very easily. I implemented a method “Whitewashing_Container::registerClassComponent”, which registers a dependency component that is used with higher priority in construction of the given class. You can also specify a third parameter $localInterfaceOverride for the highest priority:

$obj = Whitewashing_Container::load('Class', null, array('InjectDbAdapter' => $newDbAdapter));

Speedwise the usage of the Container reduces class instantiation by about 50%, from 0.5 sec for 10000 classes with setting dependencies to 1 sec on my machine. But naturally only classes with dependencies and lots of them should be using this mechanism and with good application design, this shouldn’t be to many. You can download Container and Example sourcecode, to take a look. Currently this Container is not really creating new dependencies for each new class generation but rather inverts the usage of a registry. It should be an easy task to extend the registering method to decide between new class generation and using the globally registered class instance.

]]>
Sat, 20 Sep 2008 00:00:00 +0200
https://beberlei.de/2008/12/30/dependency-injection-with-php-introducing-sphicy.html https://beberlei.de/2008/12/30/dependency-injection-with-php-introducing-sphicy.html <![CDATA[Dependency Injection with PHP: Introducing Sphicy]]> Dependency Injection with PHP: Introducing Sphicy

I have written on dependency injection before christmas and how Guice, googles DI framework, offers a simple solution. I copied the functionality for a PHP clone of Guice and named it Sphicy. You saw an early prototype of it in the blogpost mentioned above.

Sphicy configures object dependencies with modules: You explicitly state which implementation should be bound to which interface. An injector then creates instances of these objects via reflection: All constructor dependencies are resolved by looking at the given type hints and initializes those according to the specified bindings.

Two examples included in the source code of Sphicy are Zend Framework and ezComponents MVC bootstrapping modules. Sadly both frameworks default front controllers are engineered in such a way that useful dependency injection needs some workarounds.

As an example I will now discuss the Sphicy Module for Zend Framework MVC applications. To circumvent the singleton and protected Constructor of Zend_Controller_Front, we have to build a new front controller that wraps around it and requires all the dependencies:

class Sphicy_Controller_Front
{
    protected $front;

    /**
     * Create Front Controller for Zend Framework using explicitly dependencies created by Sphicy.
     *
     * @param Zend_Controller_Request_Abstract      $request
     * @param Zend_Controller_Response_Abstract     $response
     * @param Zend_Controller_Router_Interface      $router
     * @param Zend_Controller_Dispatcher_Interface  $dispatcher
     */
    public function __construct(
        Zend_Controller_Request_Abstract $request,
        Zend_Controller_Response_Abstract $response,
        Zend_Controller_Router_Interface $router,
        Zend_Controller_Dispatcher_Interface $dispatcher=null
    )
    {
        $front = Zend_Controller_Front::getInstance();
        $front->setRequest($request);
        $front->setResponse($response);
        $front->setRouter($router);

        if($dispatcher === null) {
            $dispatcher = $front->getDispatcher();
        }
        $front->setDispatcher($dispatcher);
    }

    public function dispatch()
    {
        $this->front->dispatch();
    }
}

You can see that the Sphicy_Controller_Front class requires dependencies in its constructor that are then forward injected into Zend_Controller_Front. You can now create a module that binds all the required dependencies to concrete implementations, for example a module for a Zend MVC Http Application might look like:

class Sphicy_ZendMvc_ExampleModule implements spModule {
     public function configure(spBinder $binder) {
         // Sphicy_Controller_Front does not extend Zend_Controller_Front, because of Singletonitis
         // It offers the dispatch method to proxy against Zend_Controller_Front::dispatch.
         $binder->bind("Sphicy_Controller_Front")->to("Sphicy_Controller_Front");
         $binder->bind("Zend_Controller_Request_Abstract")->to("Zend_Controller_Request_Http");
         $binder->bind("Zend_Controller_Response_Abstract")->to("Zend_Controller_Response_Http");
         // loads all routes
         $binder->bind("Zend_Controller_Router_Interface")->to("MyApplication_Router");
     }
 }

The class MyApplication_Router might look up all the routing information of the application via a hardcoded configuration mechanism. You may say this is a hard dependency, but actually you can just switch modules or implementations at this position to replace the router with another implementation. You can also see that no implementation for the dispatcher is bound. But this dependency is optional and will be created automatically as can be seen in the Sphicy_Controller_Front class.

The front controllers is now created by calling:

$injector = new spDefaultInjector(new Sphicy_ZendMvc_ExampleModule());
$front = $injector->getInstance("Sphicy_Controller_Front");
$front->dispatch();

What happens in the $injector->getInstance() line? Sphicy looks at Sphicy_Controller_Front’s constructor and finds that four dependencies are needed: Request, Response, Router and Dispatcher implementations. It looks up the bindings and searches for them, creating Zend_Controller_Request_Http, Zend_Controller_Response_Http and MyApplication_Router objects. A dispatcher implementation is not found, but Sphicy recognizes that null is a valid paramater and injects it. The 3 concrete implementations and one null are instantiated and used to construct a valid Sphicy_Controller_Front instance.

You have now stated the dependencies of the Zend Controller Front explicitly and can switch them instantaneously by switching the bindings of interface to implementations in the configuration module.

Have a look at the Sphicy Website and FAQ to see more examples and information about the possibilites of this dependency injection framework.

]]>
Tue, 30 Dec 2008 00:00:00 +0100
https://beberlei.de/2008/12/01/zend-jquery-enhancements-ajax-forms-tooltip-and-autofill.html https://beberlei.de/2008/12/01/zend-jquery-enhancements-ajax-forms-tooltip-and-autofill.html <![CDATA[Zend + jQuery Enhancements: Ajax Forms, Tooltip and Autofill]]> Zend + jQuery Enhancements: Ajax Forms, Tooltip and Autofill

Some weeks ago Zend Framework 1.7 was released with the jQuery support I contributed. What is the essential advantage of using jQuery support in ZF? You can develop forms with ajax support either by using View Helpers or directly by the integrated Zend_Form support. The implementation of a DatePicker or AutoComplete functionality becomes as easy as using 2-3 lines of php code.

Currently only support for the jQuery UI library is shipped, but you can easily extend the jQuery support on your own and this blog post will show you how using three very popular jQuery plugins: AjaxForm, Tooltip and AutoFill. This will be a series of installments, beginning with the first one: AjaxForms

AjaxForm allows you to enhance any form of yours to submit the data with ajax to the server, so no additional overhead of loading a new page is necessary. Combining the power of Zend_Form and jQuery’s ajaxForm you can even go so far as differentiating between successful and non-validated form submits. We will build a Form Decorator that integrates the AjaxForm plugin in any of your Zend_Form’s. On submit it will send the form data to the server via ajax and clears the form afterwards. Clients that have javascript disabled will work too, the form is submitted to the server in a standard pre-ajax fashion and processed that way.

First what we need is obviously the new decorator, we will call it “My_JQuery_Form_Decorator_AjaxForm” and it will inherit from Zend_Form_Decorator_Form. What we then realize is, that this is just using a view helper to render, so what we need additionally is a “My_JQuery_View_Helper_AjaxForm” that extends from “Zend_View_Helper_Form”. The code of the view helper will have to look as follows to fulfil our needs:

require_once "Zend/View/Helper/Form.php";class ZendX_JQuery_View_Helper_AjaxForm extends Zend_View_Helper_Form{  /**   * Contains reference to the jQuery view helper   *   * @var ZendX_JQuery_View_Helper_JQuery_Container   */  protected $jquery;  /**   * Set view and enable jQuery Core and UI libraries   *   * @param Zend_View_Interface $view   * @return ZendX_JQuery_View_Helper_Widget   */  public function setView(Zend_View_Interface $view)  {    parent::setView($view);    $this->jquery = $this->view->jQuery();    $this->jquery->enable()           ->uiEnable();    return $this;  }  public function ajaxForm($name, $attribs = null, $content = false, array $options=array())  {    $id = $name;    if(isset($attribs['id'])) {      $id = $attribs['id'];    }    if(!isset($options['clearForm'])) {      $options['clearForm'] = true;    }    if(count($options) > 0) {      require_once "Zend/Json.php";      $jsonOptions = Zend_Json::encode($options);      // Fix Callbacks if present      if(isset($options['beforeSubmit'])) {        $jsonOptions = str_replace('"beforeSubmit":"'.$options['beforeSubmit'].'"', '"beforeSubmit":'.$options['beforeSubmit'], $jsonOptions);      }      if(isset($options['success'])) {        $jsonOptions = str_replace('"success":"'.$options['success'].'"', '"success":'.$options['success'], $jsonOptions);      }    } else {      $jsonOptions = "{}";    }    $this->jquery->addOnLoad(sprintf(      '$("#%s").ajaxForm(%s)', $id, $jsonOptions    ));    return parent::form($name, $attribs, $content);  }}

It takes all the form-tag building of the inherited view helper for granted and just appends the necessary jQuery code to the jQuery onLoadActions stack. They will be outputted to the clients browser when calling <?php $this->jQuery(); ?> in your layout or view script. Make sure that you include the jQuery Form plugin in your code, for example with <?php $view->jQuery()->addJavascriptFile(..); >

Programming the decorator becomes a simple trick now:

require_once "Zend/Form/Decorator/Form.php";class My_JQuery_Form_Decorator_AjaxForm extends Zend_Form_Decorator_Form{  protected $_helper = "ajaxForm";  protected $_jQueryParams = array();  public function getOptions()  {    $options = parent::getOptions();    if(isset($options['jQueryParams'])) {      $this->_jQueryParams = $options['jQueryParams'];      unset($options['jQueryParams']);      unset($this->_options['jQueryParams']);    }    return $options;  }  /**   * Render a form   *   * Replaces $content entirely from currently set element.   *   * @param string $content   * @return string   */  public function render($content)  {    $form  = $this->getElement();    $view  = $form->getView();    if (null === $view) {      return $content;    }    $helper    = $this->getHelper();    $attribs    = $this->getOptions();    $name     = $form->getFullyQualifiedName();    $attribs['id'] = $form->getId();    return $view->$helper($name, $attribs, $content, $this->_jQueryParams);  }}

Now to use either the decorator for your form, or just the view helper to print your form tag with jQuery code you can invoke:

$form->addPrefixPath('My_JQuery_Form_Decorator', 'My/JQuery/Form/Decorator', 'decorator');$form->removeDecorator('Form')->addDecorator(array('AjaxForm', array(  'jQueryParams' => array(),)));$view->addHelperPath("My/JQuery/View/Helper", "My_JQuery_View_Helper");$view->ajaxForm("formId1", $attribs, $content, $options);

Now we finished up the view side of our script. Assuming that we use the Form Decorator instead of the View Helper, we can additionally add some fancy logic and error handling ajax fun to the action controller that is handling the Zend_Form instance.

class IndexController extends Zend_Controller_Action{  public function indexAction()  {    $foo = new MyAjaxTestForm();    try {      if(!$foo->isValid($_POST)) {        throw new Exception("Form is not valid!");      } else {        // do much saving and stuff here        if($this->getRequest()->isXmlHttpRequest()) {          $this->_helper->json(array("success" => "SUCCESSMESSAGEHERE"));        }      }    } catch(Exception $e) {      if($this->getRequest()->isXmlHttpRequest()) {        $jsonErrors = array();        foreach( ( new RecursiveIteratorIterator(new RecursiveArrayIterator($form->getMessages())) ) AS $error) {          $jsonErrors[] = $error;        }        $this->_helper->json->sendJson($jsonErrors);      }    }  }}

This has to be processed by a callback function of the AjaxForm and which may for example look like the following which uses a predefined div box (#formMessages, don’t forget to implement it) to render either the success or the error messages.

$form->addDecorator(array('AjaxForm', array(  'jQueryParams' => array(    'success' => "formCallback1",   ),)));$view->jQuery()->addJavascript('function formCallback1(data) {  if(data.errors) {    $("#formMessages").append("<ul>");    for each(var item in data.errors) {      $("#formMessages").append("<li>"+item+"</li>");    }    $("#formMessages").append("</ul>");  } else {    $("#formMessages").html(data.success);  }}');

This seems very complex, but you could include that javascript code into the AjaxForm decorator and implement an Action Helper to do the action controller side of the stuff. This will be an exercise for a future post.

AutoFill and Tooltip extensions will be topic of the next installments of this series, so be aware of new content soonish.

]]>
Mon, 01 Dec 2008 00:00:00 +0100
https://beberlei.de/2008/12/18/dependecy-injection-the-juicy-way.html https://beberlei.de/2008/12/18/dependecy-injection-the-juicy-way.html <![CDATA[Dependency Injection the juicy way]]> Dependency Injection the juicy way

I have written on dependency injection before and came up with a solution for PHP via interface injection. Thinking about it twice I didn’t like it very much. Its too much overkill that you have to implement all setter methods again and again.

Still dependency injection is the way to write good, testable and easily exchanged and re-wired object graphs. I tried to do lots of dependency injection via constructor lately and realized that it pollutes my constructors when my object graph is too deep.

An example: When I setup my database connection in the bootstrap file and encapsulate it in my model creation factory object, i have to insert the model factory through the configuration into the dispatcher into the different controllers and views to be accessible in the MVC pattern. The model factory has to walk 3 nodes in the object graph without being used at all in the “higher” steps. This creates very unnecessary dependencies.

I came across Misko Hevery’s Blog, which rocks. There are also some great Google Tech Talks by him, where he argues in favour of dependency injection and debunks singletons as being evil (he does that on his blog too). From there I learnt about Guice, a dependency injection framework for Java by Google.

What I like about Guice: Its easy to use and its immediately obvious to someone without experience, why it works so good and you don’t have to hand down objects deep down the object graph. I cloned the basic functionality for PHP and an example would work as follows.

We first have to implement a module, which defines which concrete implementation should be injected as a placeholder for which interface.

class ServiceModule implements Module{ public function configure(Binder $b) {  $b->bind("Service")->to("ConcreteService");  $b->bind("Add")->to("ConcreteAdd");  $b->bind("Sub")->to("ConcreteSub"); }}

We can now use this module to instantiate an injector object:

$injector = new Injector( new ServiceModule() );$service = $injector->getInstance("Service");

Given that the constructor of ConcreteService would expect an Add and a Sub object, the Injector would realize this and instantiate the concrete implementations ConcreteAdd and ConcreteSub and inject them into the constructor.

What makes this dependency injection so simple and great to use? You can instantiate an injector everywhere in your code and just have to configure it using the additional module implementation. This way you don’t have to make sure that you pass down the dependency injection container from the bootstrap into all nodes of the application. You can also easily work with many frameworks and still be able to use dependency injection without having to hack the whole core of the framework.

My guice clone does more. It allows to pass down additional non-object arguments into constructors, even for object dependencies. It offers a Provider interface to be able to wrap adapters around your already existing ServiceLocator or Registry objects. But its reflection capabilities have to be extended to docblock comments, so that better dependency detection is possible.

Because using dependency injection with only this little example I will refrain from releasing the source code yet. I have to provide some useful documentation for it to be of use to anyone.

]]>
Thu, 18 Dec 2008 00:00:00 +0100
https://beberlei.de/2008/12/03/ezcomponents-view-handler-for-zend-pdf.html https://beberlei.de/2008/12/03/ezcomponents-view-handler-for-zend-pdf.html <![CDATA[ezComponents View Handler for Zend_Pdf]]> ezComponents View Handler for Zend_Pdf

My previous posting discussed different view handlers based on routing information. One example was the PDF View which was implemented rather hackish through overwriting the createResponseBody() function of ezcMvcView. Derick told me the way to go would be writing my own PDF view handler. Since this is a rather lengthy topic I created this new post that only discusses using Zend_Pdf as a View Handler in the new ezComponents MvcTools.

The code for the View Handler would look like the following:

abstract class myPdfViewHandler implements ezcMvcViewHandler
{
    /**
     * Contains the zone name
     *
     * @var string
     */
    protected $zoneName;

    /**
     * Contains the variables that will be available in the template.
     *
     * @var array(mixed)
     */
    protected $variables = array();

    /**
     * Pdf object to be rendered.
     *
     * @var Zend_Pdf
     */
    protected $pdf;

    /**
     * Creates a new view handler, where $zoneName is the name of the block and
     * $templateLocation the location of a view template.
     *
     * @var string $zoneName
     * @var string $templateLocation
     */
    public function __construct( $zoneName, $templateLocation = null )
    {
        $this->zoneName = $zoneName;
    }

    /**
     * Adds a variable to the template, which can then be used for rendering
     * the view.
     *
     * @param string $name
     * @param mixed $value
     */
    public function send( $name, $value )
    {
        $this->variables[$name] = $value;
    }

    /**
     * Processes the template with the variables added by the send() method.
     * The result of this action should be retrievable through the getResult() method.
     */
    public function process( $last )
    {
        // template method
    }

    /**
     * Returns the value of the property $name.
     *
     * @throws ezcBasePropertyNotFoundException if the property does not exist.
     * @param string $name
     * @ignore
     */
    public function __get( $name )
    {
        return $this->variables[$name];
    }

    /**
     * Returns true if the property $name is set, otherwise false.
     *
     * @param string $name
     * @return bool
     * @ignore
     */
    public function __isset( $name )
    {
        return array_key_exists( $name, $this->variables );
    }

    /**
     * Returns the name of the template, as set in the constructor.
     *
     * @return string
     */
    public function getName()
    {
        return $this->zoneName;
    }

    /**
     * Returns the result of the process() method.
     *
     * @return mixed
     */
    public function getResult()
    {
        if($this->pdf instanceof Zend_Pdf) {
            return $this->pdf->render();
        } else {
            throw new Exception("Could not render PDF.");
        }
    }
}

Now you would implement a concrete PDF view handler by extending myPdfViewHandler.

class myConcretePdfViewHandler extends myPdfViewHandler {
    public function process( $last )
    {
        $pdf = new Zend_Pdf();
        // do concrete PDF drawing stuff here

        // save PDF here, will be rendered in getResult()
        $this->pdf = $pdf;
    }
}

And your ezcMvcView implementation will make of createZones() and look like the following:

class myPdfView extends ezcMvcView {
    function createZones( $layout )
    {
        $zones = array();
        // A decision which concrete Pdf Handler should be used would be decided on here.
        $zones[] = new myConcretePdfViewHandler( 'concreteA' );
        return $zones;
    }
}

There you go!

]]>
Wed, 03 Dec 2008 00:00:00 +0100
https://beberlei.de/2008/12/03/ezcomponents-2008-2-beta-mvc-separation-for-win.html https://beberlei.de/2008/12/03/ezcomponents-2008-2-beta-mvc-separation-for-win.html <![CDATA[ezComponents 2008.2 Beta - Mvc separation for win]]> ezComponents 2008.2 Beta - Mvc separation for win

I have written on the new ezComponents MvcTools component before already. Just yesterday the beta of this component was released with the general beta of the 2008.2 version of ezComponents. Several bugfixes and enhancements were included into the MvcTools which make it a perfect component for any Mvc based application.

I reviewed lots of the code myself and can only say i love the beauty of the code. Its very simple but by default the most powerful mvc solution in the PHP market. People working with unittests will like it very much, since all the parts are perfectly separated from each other allowing to test controllers, views, routers and filters in complete separation.

To show one very simple example howto benefit of the separation of view and controllers. If we need a view in both html and pdf, this should generally make no difference for the controller. We add two routes, one for the pdf one for the html view that execute the same controller and action:

class myRouter extends ezcMvcRouter{ public function createRoutes() {  return array(   new ezcMvcRailsRoute( '/pdf', 'SomeController', 'index'),   new ezcMvcRailsRoute( '/', 'SomeController', index' ),  ); }}class SomeController extends ezcMvcController{ public function doIndex() {  $result = new ezcMvcResult();  $result->variables['items'] = Model::retrieveLotsOfItems();  return $result; }}

Now howto decide between PDF and Html view? We use the createView method of our dispatcher configuration, but we still only need the http response writer, nothing more.

class myMvcConfiguration implements ezcMvcDispatcherConfiguration { [...] function createView( ezcMvcRoutingInformation $routeInfo, ezcMvcRequest $request, ezcMvcResult $result ) {  if(strstr($routeInfo->matchedRoute, "/pdf")) {   return new myHtmlView( $request, $result );  } else {   return new myPdfView( $request, $result );  } } function createResponseWriter( ezcMvcRoutingInformation $routeInfo, ezcMvcRequest $request, ezcMvcResult $result, ezcMvcResponse $response ) {  return new ezcMvcHttpResponseWriter( $response ); } [...]}

Now both myHtmlView and myPdfView can create their ezcMvcResponse objects that fill the response body depending on their type. Please note that overwriting createResponseBody() in myPdfView is a shortcut that circumvents me having to write a new PDF View Handler (which would be the way to go).

class myHtmlView extends ezcMvcView { function createZones( $layout ) {  $zones = array();   $zones[] = new ezcMvcPhpViewHandler( 'content', '../templates/index.phtml' );  $zones[] = new ezcMvcPhpViewHandler( 'page_layout', '../templates/layout.phtml' );  return $zones; }}class myPdfView extends ezcMvcView { function createZones() {  // empty, abstract method that has to be defined. } function createResponseBody() {  // Set PDF Content-Type Response Header  $this->result->content->type = "application/pdf";   $pdf = new Zend_Pdf();  // do pdf stuff  return $pdf->render(); }}

Now all the the logic that is potentially in the controller is completely separated from the view handling that may depend on the routing information not on the controller. And views can be tested separately from the controller result. Testability is very high.

]]>
Wed, 03 Dec 2008 00:00:00 +0100
https://beberlei.de/2008/12/04/rest-and-ajax-aware-controllers-in-ezcmvctools.html https://beberlei.de/2008/12/04/rest-and-ajax-aware-controllers-in-ezcmvctools.html <![CDATA[REST and Ajax Aware controllers in ezcMvcTools]]> REST and Ajax Aware controllers in ezcMvcTools

There are essentially two major different ways to implement a restful application using a web framework. You either implement a router that routes to different controller actions based on the HTTP method used on a requested resource. This is the fancy way, which sadly is not always so practical because many browsers do not support sending more than GET and POST requests. The other way would be to define suburls of a resource such as /user/1/delete for the resource /user/1 and take GET a a request for deleting and POST for the confirmation of the delete.

ezcMvcTools HTTP Request Parser and routing mechanisms currently offer no real help to decide on this issues, but its easy to extend this missing functionality. What we first add are simple checks of the current http request method. We extend ezcMvcHttpRequestParser which will return a derived ezcMvcRequest object that implements 7 new methods: isPost(), isGet(), isDelete(), isPut(), isXmlHttpRequest(), isMethod() and getMethod(). These methods can now be easily used on the request object to determine which action will be undertaken:

class myMvcRequest extends ezcMvcRequest{  public function isPost()  {    return $this->isMethod("POST");  }  public function isGet()  {    return $this->isMethod("GET");  }  public function isPut()  {    return $this->isMethod("PUT");  }  public function isDelete()  {    return $this->isMethod("DELETE");  }  public function isXmlHttpRequest()  {    if(isset($this->raw['HTTP_X_REQUESTED_WITH'])      && strtolower($this->raw['HTTP_X_REQUESTED_WITH']) == "xmlhttprequest") {      return true;    }    return false;  }    public function getMethod()  {    if(isset($this->raw['REQUEST_METHOD']))      return strtolower($this->raw['REQUEST_METHOD']);    }    return false;  }  public function isMethod($method)  {    if(isset($this->raw['REQUEST_METHOD']) &&       $this->getMethod() == strtolower($method)) {      return true;    }    return false;  }}class myMvcHttpRequestParser extends ezcMvcHttpRequestParser{  /**   * Uses the data from the superglobals.   *   * @return ezcMvcRequest   */  public function createRequest()  {    $this->request = new myMvcRequest;    $this->processStandardHeaders();    $this->processAcceptHeaders();    $this->processUserAgentHeaders();    $this->processFiles();    $this->processAuthVars();    $this->request->raw = &$_SERVER;    return $this->request;  }}

This helps us to implement simple decision mechanisms in a single controller action that takes both POST and GET requests. From the point separation of concerns this is not a good design decision. We need routes that can point to different controller actions based on their request method, so that two requests GET /user/1/delete and POST /user/1/delete lead to different methods, for example userController::doDeleteDialog and userController::doDelete. We will simply extend the ezcMvcRailsRoute to support decision based on http request methods:

class myMvcRestRoute extends ezcMvcRailsRoute{  protected $method;  public function __construct( $method, $pattern, $controllerClassName, $action = null, array $defaultValues = array() )  {    $this->method = $method;    parent::__construct($pattern, $controllerClassName, $action, $defaultValues);  }  public function matches( ezcMvcRequest $request )  {    if(strtolower($this->method) == strtolower($request->raw['REQUEST_METHOD'])) {      return parent::matches($request);    }    return null;  }}

This very simple extension is independent of the advanced request parser given above, so you could use it separately. In the light of our delete user example, you would use the new router in the following way:

class myRouter extends ezcMvcRouter{  public function createRoutes()  {    return array(      new myMvcRestRoute( 'GET', '/users/:id/delete', 'UserController', 'deleteDialog' ),      new myMvcRestRoute( 'POST', '/users/:id/delete', 'UserController', 'delete' ),    );  }}

We have now simple rest route support in our ezcMvcTools application.

]]>
Thu, 04 Dec 2008 00:00:00 +0100
https://beberlei.de/2008/12/12/bad-news-jquery-ui-1-6-ships-without-spinner-and-autocomplete.html https://beberlei.de/2008/12/12/bad-news-jquery-ui-1-6-ships-without-spinner-and-autocomplete.html <![CDATA[Bad news: jQuery UI 1.6 ships without Spinner and AutoComplete]]> Bad news: jQuery UI 1.6 ships without Spinner and AutoComplete

The jQuery UI team announced yesterday on its blog that jQuery UI 1.6 will not be shipped with AutoComplete and Spinner Plugin support. No further delay, for the original august 2008 estimated release, is wanted.

This brings about a problem on ZendX_JQuery. All view helpers and decorators that depend on AutoComplete and Spinner Plugins can only be run with SVN trunk or 1.6 release candidates, which have known bugs and problems.

I am adding a compatibility table to the ZendX jQuery manual today that will make its way to the manual of the 1.7.2 release. This will hopefully help and guide everyone to the correct dependencies.

]]>
Fri, 12 Dec 2008 00:00:00 +0100
https://beberlei.de/2008/07/14/jquery-helper-what-is-to-include.html https://beberlei.de/2008/07/14/jquery-helper-what-is-to-include.html <![CDATA[jQuery Helper - What is to include?]]> jQuery Helper - What is to include?

I took some time in the last days to develop the concept for a possible jQuery View Helper that could pass the integration into the ZF Core. Since jQuery is organized somewhat different than Dojo is, its possible implementation for ZF will differ.

The jQuery Helper itself will manage inclusion of jQuery javascript files and the base library, much like the Dojo component works (Using the Google CDN). On top of this simple Helpers should mimic $.get, $.post, $.load and $.getJSON in a simple way, so that with a simple function call you can generate an XmlHttpRequest that updates a specified part of the DOM (via injection). (See: CakePHP Ajax)

Problematic are the next ideas: Additional Helpers will allow to specify the jQuery UI Library components (DatePicker, Sortables, Draggable, Dropable..), jQuery Autocomplete or jQuery Form (AjaxForm and AjaxSubmit). These libraries need additional javascript content to be downloaded and implemented in the Zend Framework project. There currently exists no way to implement them using a CDN. Therefore any documentation must clearly specify which additional content has to be installed and how. The helpers will have to be general enough to support this.

My proposal is currently in the workings and will be on the ZF Wiki in the next couple of hours.

]]>
Mon, 14 Jul 2008 00:00:00 +0200
https://beberlei.de/2008/07/22/multidimensional-array-via-spl.html https://beberlei.de/2008/07/22/multidimensional-array-via-spl.html <![CDATA[Multidimensional Array via SPL]]> Multidimensional Array via SPL

Per default any implementation of ArrayObject does not allow to use the object as multidimensional array. This snippet here implements the support:

class ArrayMultiObject extends ArrayObject
{
    function __construct($array, $flags = 0, $iterator_class = "ArrayIterator")
    {
        $objects = array();
        foreach($array AS $key => $value) {
            if(is_array($value)) {
                $objects[$key] = new ArrayMultiObject($value, $flags, $iterator_class);
            } else {
                $objects[$key] = $value;
            }
        }

        parent::__construct($objects, $flags, $iterator_class);
    }

    public function offsetSet($name, $value)
    {
        if(is_array($value)) {
            $value = new ArrayMultiObject($value);
        }

        return parent::offsetSet($name, $value);
    }
}
]]>
Tue, 22 Jul 2008 00:00:00 +0200
https://beberlei.de/2008/07/17/zf-jquery-helper-first-prototype-ready.html https://beberlei.de/2008/07/17/zf-jquery-helper-first-prototype-ready.html <![CDATA[ZF jQuery Helper - First Prototype Ready]]> ZF jQuery Helper - First Prototype Ready

I checked in the first implementation of my jQuery Helper into my dev svn. It follows my suggested proposal in the Zend Framework Dev Wiki and implements all the advertised features. I also implemented 2 demos that nicely present the functionality.

Please test and comment!

]]>
Thu, 17 Jul 2008 00:00:00 +0200
https://beberlei.de/2008/07/21/zend-dojo-and-the-django-template-language.html https://beberlei.de/2008/07/21/zend-dojo-and-the-django-template-language.html <![CDATA[Zend, Dojo and the Django Template Language]]> Zend, Dojo and the Django Template Language

With the Zend Framework nearing its 1.6 release and full Dojo Toolkit support I took some time to look up what Dojo is actually capable of. I found the DojoX Django Template Language extension and remembered my neighbor talking about why Python + Django is so much better than PHP with any other framework. So I digged into the Django template language and found that it is quite awesome.

Variable filtering and evaluation looks almost like Smarty the syntax being {{var.key|filter1|filter2:”arg1”:”arg2”}}. The logical syntax is quite different though, taking an somehow object oriented view on templates you can extend an existing template and override specific parts with your more special implementation. Have a look at the following two snippets:

This is an example of Django template inheritance:

{% block helloworld %}Hello World!{% endblock %}
{% extends "base.html" %}

{%block helloworld %}Hello World for Object Oriented Views!{% endblock %}

What does Django do with this second template when rendering? It realizes it inherits logic from a parent template and substitutes all special blocks for the parent ones. With a little object oriented background you can guess the result looks like this:

This is an example of Django template inheritance:

Hello World for Object Oriented Views!

So what was all the talking about Dojo being able to parse this kind of templates? If you envision a helper component that would function like this: 1.) make an ajax request to $url 2.) retrieve json object from the controller 3.) render json object with $template into container $container. The helper would know that the template is needed in this HTML response and appends it to the Dojo Helper output. A link would be generated performing steps 1 and 2, handing over the json data to the template and render the output. What do you get? Templates that can be used Client and Server side.

For example on rendering the view of your blog you can send all the comments using a specific comment building Django template script. Additionally you can also use the same template to render any new comment to the comment list using via an ajax form submit, returning the (model or form) filtered data via JSON and appending it to the comment list using the DojoX DTL parser. For non-JS browsers you can always use a <noscript> variant to render the templates completely server-side.

This generally being a cool idea since it makes developing applications with AJAX technology very easy I began to port the DTL to the Zend Framework and the whole weekend later I got a working implementation that supports at least the “extends”, “for”, “include”, and “comment” tags. As a next task I will implement the helper for DojoX DTL and will report back on my efforts.

]]>
Mon, 21 Jul 2008 00:00:00 +0200
https://beberlei.de/2008/07/12/discussing-a-jquery-helper-for-zend-framework.html https://beberlei.de/2008/07/12/discussing-a-jquery-helper-for-zend-framework.html <![CDATA[Discussing a jQuery Helper for Zend Framework]]> Discussing a jQuery Helper for Zend Framework

Yesterday I had some time to test Matthews Dojo View Helper and the Zend_Dojo_Form component. At a first glance the implementation looks great and can almost instantly generate you a very nice form with all the possible Dijit form extensions that Dojo has to offer.

If one were to refactor the Dojo Component to allow for jQuery elements one bigger problem appears. Dojo has a CDN (Content Distribution Network) for all its components, that is, the Zend Dojo components can load all javascript and css files from a distant server. JQuery can only offer its main library to be loaded from a CDN. All additional components, for example the jQuery DatePicker, have to be installed locally. This significantly reduces the possibility for rapid development of Javascript/Ajax/Form components with jQuery and Zend Framework.

One could offer a complete dependency downloadable archive with all the CSS, Javascript and Images inside, but this would be very complex to maintain. Looking at the future Zend Tool capabilities one possibility would be to offer a download client for all the relevant jQuery plugins, but there would have to be a man-middle-server that maintains the most up to date locations of all the plugins, which also has to be maintained. Does anybody have a better solution to solve this problem? Perhaps the jQuery Team needs to be made aware that they need a CDN for their most stable plugins.

Abstracting from the CDN problem, I implemented a simple jQuery View Helper (mostly copy paste and simple rewrites from Matthews Dojo component) and a HtmlElement Form Helper which constructs a Date-Picker from within the template. This is very easy to use and looks great. In the next days I will add further helpers for the jQuery plugins I use in my day to day work live and hope to present a demo. I might even opting for a jQuery proposal aiming at inclusion in the Zend Extras Library.

Update: I got a reply on the jQuery mailing list stating that a CDN is planned for the jQuery UI library. Sadly this does not include plugins for which one might desperately need a View Helper except for maybe the Date Picker. I will post further comments on this issue in the near future.

]]>
Sat, 12 Jul 2008 00:00:00 +0200
https://beberlei.de/2008/07/24/finished-first-django-template-language-port-to-php-zf.html https://beberlei.de/2008/07/24/finished-first-django-template-language-port-to-php-zf.html <![CDATA[Finished first Django Template Language Port to PHP + ZF]]> Finished first Django Template Language Port to PHP + ZF

Today I finished the first almost complete Django Template Language Clone for PHP implementing the Zend Framework View Interface. My proposal is up and running on the ZF Wiki. The source code can be downloaded from my svn repository.

]]>
Thu, 24 Jul 2008 00:00:00 +0200