Drupal8.x and Symfony Developing Minding BC on Upgrades – Ajax Core Component: Part III Series

Screenshot 2014-05-24 05.02.33

So today I grabbed the Ajax core component. At first it seemed daunting because I did not know what I was going to find in so many flat directory structure. But after a bit of analysis I discovered that the many XCommand classes were really very simple implementations of a commond XCommandInterface. So the first thing I did was to move all those classes into a Command directory. These commands have nothing to do with the Command classes as we know them from Symfony.

Let me explain. It seems like in Drupal the requirement to communicate via JSON responses and in particular Ajax responses needed to be standardized. Drupal needs this because on this bus for ajax communications all other moving parts that are ajax related on the front end will revolve their actions. These actions are called commands, and it is pretty much information useful on the front side, possibly used by the javascript that runs on the front end.

So the ajax system, as we call it, comprises of many JavaScript libraries that are served and then the hub or bus for communications that is server side and therefore have to have a type of container or standard place where to put information and where to read information when doing JSON communications, that is request and responses that pertain to dynamic calls from the client.

In this ajax system, the part in PHP, server side, and that pertain to the Ajax core component, there are only 4 moving pieces namely: the AjaxResponse, the AjaxResponseRenderer, an AjaxSubscriber, and the Commands.

AjaxResponse is an extension of the Symfony’s JsonResponse where the commands are computed, rendered with the Drupal inner renderer functions that deal with arrays and the part that is still not OO, but that is wrapped, so that we can end up with a JsonResponse object that has a drupalized-array-of-commands payload on the JsonResponse.

The extension of the JsonResponse breaks the BC promise from Symfony in that it overrides a public method on a non api class. I did the PR to drupal here https://drupal.org/node/2273717 and filed an issue in Symfony asking for workarounds here https://github.com/symfony/symfony/issues/10985.

// AjaxResponse extends JsonResponse
<?php
  /**
   * {@inheritdoc}
   *
   * Sets the response's data to be the array of AJAX commands.
   */
  public function prepare(Request $request) {
    $this->prepareResponse($request);
    return $this;
  }
 
  /**
   * Sets the rendered AJAX right before the response is prepared.
   *
   * @param \Symfony\Component\HttpFoundation\Request $request
   *   The request object.
   */
  public function prepareResponse(Request $request) {
    if ($this->data == '{}') {
      $this->setData($this->ajaxRender($request));
    }
  }

We will see what happens. For certain at least there may have been a need to replug that call to the parent::prepare($request) that was missing. If not well the refactor of the tests seemed to me reasonable and also the folder organization. Drupal still have some ugly organization, think of flat folder structure and you get a pretty good picture. Another nuance regarding this is that digging inside let’s you see that there are 2 layers, one where it is all OO and the other where the drupal_x calls are still creeping in.

I have found some tickets that were opened and discussed but then closed and nothing was done. There were good chances to refactor and remove those calls but there seems to be a lack of interest in getting rid completely of these hooks and array manipulation. The array manipulations obscure the parts of the class where they are cornered. Even in the tests, you have to see these &getCommand ampersands just to, and I don’t know the real reason, to have access to an internal array inside the class. This is poor OO and it is a crutch on the side of the Drupal developers that they probably need to deal with. This causes impacts on every side though, tests that are not optimal, thereby violating the principles of closed classes or unit testing.

The next element is the AjaxResponseRenderer, it basically builds an AjaxResponse by calling conditionally and adding some specific commands, and then calling the monster of drupal_render to convert these commands further into the payload to send inside the AjaxResponse. Nothing to see here, just the render command and well what I have explained above.

The AjaxSubscriber is a bit interesting, it lacks a test, so I don’t know what is its use case, however it seems to set the format of an incoming Request. This is weird as it sets drupal fields that relate to application/vnd.drupal-x, maybe to ensure by forcing these into every incoming request for a specific call pattern. Smells like a workaround and not a robust solution.

Then comes the Commands, there is a diverse selection of commands and use cases. From adding some css into the page, adding more markup, setting dialogs, modals, invoking some js, etc. There are commands for that, you name it!

So that is it for this component. I hope you have had fun reading this, I stayed a bit late just to get this out.

Please think about supporting me. Thank you and mind I am doing this all with a lot of excitement and passion for developing.

undeservedly doing this with joy, your friend @cordoval

Drupal8.x and Symfony Deeper into Routing: Part II Series

Screenshot 2014-05-23 00.40.22

So today i went on reading the Drupal core element Access. I believe Access and the other elements, I call them like that, are the equivalent of components in Symfony. The difference is that they are coupled from mildly to badly. Let me explain what i mean. Drupal has also components but i prefer not to call them components because they are really all very similar in that they are `utils`. The component libraries are quaint and resolve one problem and even then they are a bit tiny coupled, however they are not the major components or players in Drupal. Drupal core is build of elements. Symfony is build on components with a layer of glue between bridges and framework bundles that do the coupling. In the case of Drupal, they are the core libraries or elements from now on.

So looking at the first element in Drupal I found an interesting element called Access. Access solves the problem of determining access to a route, the first class resource citizen in Drupal, naturally of a content management system. The agents that intervene on this problem are mainly and namely: An account, the request, the route name or route object itself. In practical terms is all determined by a given request and the state of the system.

Let me explain further. A request comes in, we pull the account from the session, that is the current user or the user that is being actuated as, we also match the request to a route and thereby have a route to do further processing.

The processing consist of extracting (even overriding) parameters from the route and setting into the route itself an array of checkers labels. These checkers bear the name of checks, but in reality they are checker labels, corresponding to checker services, similar to voters in Symfony lingo. Notice we don’t put back the stuff into the request but rather into the route. Since the route represents a resource in Drupal, then the resource is passed around to hold this important information. The voters or checkers are services relatively flexible. They only need to implement some Access related interface. The idea is that these checker services can be passed parameters and method information to invoke so that they can determine whether the access is granted or denied.

The problem is simple, given certain checkers and the route which will come with parameters on it, let’s resolve whether the account associated have access to this route.

This is how you can access from the admin dashboard of Drupal to the entities to roles and what not.

One thing to notice is the voter system is made flexible via checkers/voters that can vote for or against, but that also can override all other checkers. This is kind of like vital checkers, you must have this checker pass else you will suffer a lock down. The voter system Access also have capabilities for checking that all checkers comply or just any of them.

The AccessManager is the king in that component/element. It is a service that gets the important pieces injected via its constructor, that is: the url generator, a custom parameter converter, an argument resolver, and the router provider. It does its job checking the route name for a given account and route and request.

The AccessArgumentResolver is the obscure component there, and it is still buggy. The fetching of the parameters from the request and resolving which parameters should be passed from the request to do the check and the structure and the use cases make this class the worst organized from the element. It definitely needs some refactoring and testing.

In comparison to Symfony voters I guess is a nice exercise from Drupal to invent their own voter system. The system is a bit stiff but is a good example of what you can do with the Route, Request, and voters.

The AccessManager would be easy to test were it not because it was injected the ContainerAwareTrait. The first time I saw this I was bracing myself for the worst. Thankfully it only injects it for lazy loading the services that are functioning as checkers. That thing is required. The class seems to be divided almost in two, the lazy loading of services and then the more clear logic about the voting system.

The Route class from Symfony is very powerful and was used as key component here. Unfortunately Drupal has multiplied certain interfaces unnecessarily into other element/core components. I think that it should keep each core component with strong interfaces and then just use them across to avoid coupling it all over the place. Thankfully there is no Route wrapper and this is the way it should be, Symfony does not even have an interface for Route and we bet for a good reason.

These are the methods that were used from the Route in Drupal Access core component:

$route->getRequirements()
$route->getDefaults()
$route->getOption('_access_mode')
$route->setOption('_access_checks')

You can see that the Route uses its requirements, defaults, and options. The options is the strongest feature that gets used and this stores the access checker labels, checks that will be made out of the checker services and also the type of voting aka access mode “all” or “any”.

Let’s take a closer look on the tests for Route and see the behavior of the class Route:

  public function testOptions()
    {
        $route = new Route('/{foo}');
        $route->setOptions(array('foo' => 'bar'));
        $this->assertEquals(array_merge(array(
        'compiler_class'     => 'Symfony\\Component\\Routing\\RouteCompiler',
        ), array('foo' => 'bar')), $route->getOptions(), '->setOptions() sets the options');
        $this->assertEquals($route, $route->setOptions(array()), '->setOptions() implements a fluent interface');
 
        $route->setOptions(array('foo' => 'foo'));
        $route->addOptions(array('bar' => 'bar'));
        $this->assertEquals($route, $route->addOptions(array()), '->addOptions() implements a fluent interface');
        $this->assertEquals(array('foo' => 'foo', 'bar' => 'bar', 'compiler_class' => 'Symfony\\Component\\Routing\\RouteCompiler'), $route->getOptions(), '->addDefaults() keep previous defaults');
    }
 
    public function testOption()
    {
        $route = new Route('/{foo}');
        $this->assertFalse($route->hasOption('foo'), '->hasOption() return false if option is not set');
        $this->assertEquals($route, $route->setOption('foo', 'bar'), '->setOption() implements a fluent interface');
        $this->assertEquals('bar', $route->getOption('foo'), '->setOption() sets the option');
        $this->assertTrue($route->hasOption('foo'), '->hasOption() return true if option is set');
    }
 
    public function testDefaults()
    {
        $route = new Route('/{foo}');
        $route->setDefaults(array('foo' => 'bar'));
        $this->assertEquals(array('foo' => 'bar'), $route->getDefaults(), '->setDefaults() sets the defaults');
        $this->assertEquals($route, $route->setDefaults(array()), '->setDefaults() implements a fluent interface');
 
        $route->setDefault('foo', 'bar');
        $this->assertEquals('bar', $route->getDefault('foo'), '->setDefault() sets a default value');
 
        $route->setDefault('foo2', 'bar2');
        $this->assertEquals('bar2', $route->getDefault('foo2'), '->getDefault() return the default value');
        $this->assertNull($route->getDefault('not_defined'), '->getDefault() return null if default value is not set');
 
        $route->setDefault('_controller', $closure = function () { return 'Hello'; });
        $this->assertEquals($closure, $route->getDefault('_controller'), '->setDefault() sets a default value');
 
        $route->setDefaults(array('foo' => 'foo'));
        $route->addDefaults(array('bar' => 'bar'));
        $this->assertEquals($route, $route->addDefaults(array()), '->addDefaults() implements a fluent interface');
        $this->assertEquals(array('foo' => 'foo', 'bar' => 'bar'), $route->getDefaults(), '->addDefaults() keep previous defaults');
    }

Symfony tests for Route are well maintained and therefore read nicely. A quick read explain us the simplicity and minimum constraints required to make the Route a versatile class. Now the route is not alone, and that is where everything else will come to play, however we are just treating this for simplicity’s sake.

So far as it is concerned, our Access core component in Drupal only has that responsibility of deciding on access for account or profiles associated to a user interacting via requests with the system.

The Access core component also comes with an example, that is an implementation of a common checker. If you want to create your own checkers for your use cases, you can imitate how the CsrfAccessCheck implements the interface AccessInterface and you can go from there. I believe in Drupal you should be even able to replace the whole AccessManager implementation or any of its lower creatures, however, I may be going to far or too advanced for the generic use case.

I don’t know what awaits me in the other core components. If i have made many mistakes explaining this I ask for comprehension, this is what I read and deduce from the code which is the better documentation 🙂 and my Symfonic intuition.

Until then your friend @cordoval with undeserved joy doing this drupliconic publication.

If you like this series, I got accepted to DrupalCon2014 in Austin, I need to cover for the expenses of my air ticket. Please i kindly ask you for your support as a friend and I hope I will keep working on these series and provide more secrets and details. Please paypal me to my email cordoval at gmail dot com to say thanks. With your effort I will be able to spread it more! Thanks!

Drupal8.x and Symfony To the Aid Experiences: Part I Series

Screenshot 2014-05-21 00.45.27

Ok, so let’s do this. I decided to take a stab go into one of the tweets from jmolivas and picked a link and bam it was this module on drupal8x.

So i had already cloned drupal into a folder and had ran “`php -S localhost:8000“` and had gone through the installation on the web, they call it installation script which makes no sense :).

Then I had my drupal blue screen. The i was kindly oriented by yesCT and others in the IRC to plug some more modules, these are drag and drop. I know it sucks but once you download one and move a folder under modules you start to think is not too bad. We don’t do that in Symfony, is almost sinful, but bearing up with drupal legacy we just do this on this part of planet earth. So i moved the folder for the webprofile, that was neat and discovered they had a similar WDT like in Symfony. The same happen with a dependency and how they enable modules. Rather than running composer install or update package, you basically move the folder under “`modules“` and enable it in under “`Extend“` drop down on the navigation bar.

So this is the link of the repo i found in jmolivas tweet https://github.com/dmouse/vimeo_field.
What i found to my surprise was a test which was mocking Drupal core classes. The first thing that it striked me was, why would you mock all classes that you extend from your actual class under test. Well it turns out that the tests did not have a proper autoloader. I had two options, load as a require-dev drupal/drupal, or sourcing the autoload from drupal root directory and merge it with the current autoloader in a boostrap.php file. I opted for the first so that travis can work of course. At first i was looked as a rare bird by some drupalers but I went my own way knowing my theory.

    protected function setUp()
    {
        $this->formmaterItemListInterface = $this->getMockBuilder('Drupal\Core\Field\FieldItemListInterface')
            ->disableOriginalConstructor()
            ->getMock();
        $this->formmaterBase = $this->getMockBuilder('Drupal\Core\Field\FormatterBase')
            ->setMethods([])
            ->disableOriginalConstructor()
            ->getMock();
 
        $this->formatter = $this->getMockBuilder('Drupal\vimeo_field\Plugin\Field\FieldFormatter\VimeoDefaultFormatter')
            ->disableOriginalConstructor()
            ->setMethods(['t', 'getSetting'])
            ->getMock();
 
        $this->formatter->expects($this->any())
            ->method('t')
            ->with($this->isType('string'))
            ->will($this->returnCallback(function () {
                $args = func_get_args();
                return $args[0];
            }));

Notice the first mocks which have nothing to do in this test. The class variables were not even defined. But if you removed those lines you would get an error because they were there because phpunit would not know who the heck were those classes because the autoloader was not finding them. This is obviously a bad practice. Picture building a module very complex that depends on half of the classes on the core, poor you.

     "require-dev": {
 -        "phpunit/phpunit": "~4.1"
 +        "phpunit/phpunit": "~3.7",
 +        "drupal/drupal": "8.x-dev@dev"
      },

I have noticed also the test needs more integration tests, but for that more thinking is needed. For now is enough to include drupal/drupal as a dependency and make the testing of the module standalone.

After following the instructions and sending some PRs and comments I got the desired result 🙂

I also wanted to require-dev drush just in case but i rather enabled the bundle ehem module according to the instructions in the readme.

Drupal community needs our help as good PHP developers, especially if you know some Symfony it would help us a lot! now i can say i belong to two communities and is so cool!

Doing this work out of pure joy and very thankful for such undeserved favor, please enjoy too!

your friend @cordoval

En Route to DrupalCon Austin 2014

I am trying to see if I can take a stab at Drupal project. I have cloned it and checked out the core a bit, mostly the components and the core. One first impression is the familiarity with Symfony and well PHP in general. I am talking about past and present. There is code down there that is not very pleasant at all; however, there is good code too.

Regarding components, I believe it is currently a mix of byproduct drop copy/paste spaguetti code that was before all together and now we have put it in classes and removed the requires. This feels exactly like the book from Paul M. Jones. The process is not at all finished though, so there is this feeling of disconnection and of facade :), something fake. The components are really a move of pieces of code that treat a topic but they are somewhat still to specific to Drupal. The components are defined as code that could even be pulled out of Drupal, however i think they are not mature enough for that. They are good within Drupal for now, there is some more work to be done on them to make them really more generic oriented and more helpful for solving single responsibility problems. Some of them are code ideas split between the core and the components so really they are a refactor of some core classes.

Regarding contribution, I believe the current state of Drupal contribution is too adhoc and prone to errors and time spend on things that elongate the development process. I feel like going 0.00009X of the speed I go on Github. There is no excuse really. Drupal has issues, so Github, Drupal has patches, so Github PRs, Drupal has interdiffs, Github has Diffs and PRs to allow more than one contributor and has forks and much much more. Drupal has a search, so does Github have labels, Drupal has dreditor, Github has diffs and a world class web interface, Drupal has a test system, Github has travis, Drupal has millions of patches, so Github handles much much more across all languages and repos combined, Drupal.org (d.o) is slower than Github, Drupal has a broken API for posting tickets and such, Github has a brand working API interconnected with many libraries and support for integration into other systems like Jira, waffle.io, and what not, Drupal has threads, so does Github can sustain threads with drag and drop graphics, code markdown and realtime facebook-like updates, Drupal has follow feature, so Github has follow and subscribe features for PRs, comments, repos, etc, Drupal has many branches, so Github allows projects to handle branches and tagging and releasing, Drupal has projects so Github does and you have authentication, private and public sides, and you have complete freedom. The only reason I think Drupal does not move to Github perhaps is fear of change. That said Drupal has a great community and thrive, so much that we need to invite it to the PHP arena on Github. We cannot afford to not share with those on the other side of the wall.

Regarding help, there is yesCT which will help you if you approach Drupal channels on IRC. She is such an asset! Thanks for helping me so far figuring things out.

Regarding some good quality code, I saw a plugin system that is very powerful, maybe not fully tested, but good nonetheless. I dislike the CS because it not only is particular amongst good PHP projects but it is just non enforced well. It kind of conveys sloppiness imo, but yes as I take more informations I can change my mind on some things, many things maybe.

I am trying to send patch after patch as I advance reading the core code. I figured if I read the core then it would be helpful when I get to use the release and I can learn things from its source code too. I see very interesting line ideas, the problem is that I see a reminiscence of convoluted practices, but I see an emergent OO feel in Drupal, situation is changing, and there are great minds there. So don’t be shy and start looking at Drupal, after all is not all spaghetti code, it has very good parts, and in its beginnings its architecture is promising.

Often I hear the main argument for Drupal is size, but it is time to change that and let the core quality decide. Not just that is using Symfony but that its other components and core classes can well develop an architecture that is parallel or beyond Symfony.

Encouragements and thanks for reading this. All are my thoughts and never to be taken offensively of any work. These are just appreciations that are changing as I dig but is a first impression.