Composer The Latest of Our Alpha – Symfony2 Tricks

Ok folks, time to get up to speed with the latest developments in composer. Composer is trying hard to get out of alpha version. Let’s help spread the word to summon support and contribute. Here is the run down of new/updated features you may want to have under your arsenal belt:

logo-composer-transparent
[logo made by @wizardcat, btw i met this guy in Warsaw :)]

Suppose we were to start with a composer.json something like this:

{
    "type": "project",
    "license": "proprietary2",
    "require": {
        "lib-ICU": ">=53.1",
        "php-64bit": ">=5.5",
        "doctrine/orm" : "*",
        "symfony/event-dispatcher": ">=2.1"
    },
    "scripts": {
        "post-update-cmd": [
            "phpunit"
        ]
    }
}

Notice the new type `project`, and also the first couple of requires. Let’s validate it:

~ composer validate
./composer.json is valid for simple usage with composer but has
strict errors that make it unable to be published as a package:
See http://getcomposer.org/doc/04-schema.md for details on the schema
name : is missing and it is required
description : is missing and it is required
License "proprietary2" is not a valid SPDX license identifier, see http://www.spdx.org/licenses/ if you use an open license.
If the software is closed-source, you may use "proprietary" as license.

It even offers us the right suggestion. Now suppose we have network problems, and we want to report it or reproduce the problem. No worries, diagnose command to the rescue:

~ composer diagnose
Checking platform settings: OK
Checking http connectivity: OK
Checking composer.json: OK
Checking github.com oauth access: OK
Checking disk free space: OK
Checking composer version: FAIL
Your are not running the latest version

Suppose you get concerned by a rumor some of your bundles have GPL or such forbidding licenses, then just list them with `composer licenses | grep ‘GPL’` or even fail a test if this is different than empty :):

~ composer licenses
Name: __root__
Version: dev-master
Licenses: proprietary
Dependencies:
 
  doctrine/annotations       v1.1.2   MIT
  doctrine/cache             v1.3.0   MIT
  doctrine/collections       v1.1     MIT
  doctrine/common            v2.4.1   MIT
  doctrine/dbal              v2.4.1   MIT
  doctrine/inflector         v1.0     MIT
  doctrine/lexer             v1.0     MIT
  doctrine/orm               v2.4.1   MIT
  symfony/console            v2.4.0   MIT
  symfony/event-dispatcher   v2.4.0   MIT

Try running `composer install` now with a mac with updated brew formulas on icu. Currently we are on 52.1 so we will need to downgrade the 53.1 to 52.1 to be able to install the dependencies of our composer.json above or we get this:

    - The requested linked library lib-icu >=53.1 has the wrong version installed, try upgrading the intl extension.

Now because you contribute a lot you need to have a way to see if some of your dependencies have been modified and you need to launch a couple of PRs to your peeps. No worries, got you covered with `status` command:

~ composer status -v
You have changes in the following dependencies:
/Users/cordoval/Sites/libs/experiment/vendor/symfony/event-dispatcher/Symfony/Component/EventDispatcher:
    M CHANGELOG.md
composer archive symfony/event-dispatcher
symfony-event-dispatcher-11d844997699682f23b6fbb50c00d343c645654f-zip-00fc5c.tar

One more time you gotta run the test suite on your project, then use `run-script` (warning: currently only event named commands are allowed):

composer run-script post-update-cmd
PHPUnit 3.7.28 by Sebastian Bergmann.
 
// ...

Then imagine you are under pressure and composer did not work because of a low timeout number. You can display the merged configuration with the `config` command:

~ composer config --list
[repositories.0.type] composer
[repositories.0.url] http://satis.l
[repositories.packagist.type] composer
[repositories.packagist.url] https?://packagist.org
[repositories.packagist.allow_ssl_downgrade] true
[process-timeout] 300
[use-include-path] false
[preferred-install] auto
[notify-on-install] true
[github-protocols] [git, https]
[vendor-dir] vendor
[bin-dir] {$vendor-dir}/bin (vendor/bin)
[cache-dir] /Users/cordoval/.composer/cache
[cache-files-dir] {$cache-dir}/files (/Users/cordoval/.composer/cache/files)
[cache-repo-dir] {$cache-dir}/repo (/Users/cordoval/.composer/cache/repo)
[cache-vcs-dir] {$cache-dir}/vcs (/Users/cordoval/.composer/cache/vcs)
[cache-ttl] 15552000
[cache-files-ttl] 15552000
[cache-files-maxsize] 300MiB (314572800)
[discard-changes] false
[prepend-autoloader] true
[github-domains] [github.com]
[home] /Users/cordoval/.composer
[github-oauth.github.com] KEYHERE...

Then run the `config` with `–global key value` to up your network setting for composer to work:

composer config --global process-timeout 400

Inside your `~/.composer/config.json` will look like this:

~ cat ~/.composer/config.json
{
    "config": {
        "github-oauth": {
            "github.com": "KEYHERE"
        },
        "process-timeout": 400
    },
    "repositories": [
        {
            "type": "composer",
            "url": "http://satis.l"
        }
    ]
}

Finally, you are just tired of copying or typing your `composer.json` by hand. You then will like the `init` command to do like a boss:

~ composer init
 
 
  Welcome to the Composer config generator
 
 
 
This command will guide you through creating your composer.json config.
 
Package name (<vendor>/<name>) [cordoval@gmail.com/experiment]: cordoval/experiment
Description []: some description
Author [Luis Cordova <cordoval@gmail.com>]:
Minimum Stability []: dev
License []: MIT
 
Define your dependencies.
 
Would you like to define your dependencies (require) interactively [yes]?
Search for a package []: monolog
 
Found 15 packages matching monolog
 
   [0] monolog/monolog
   [1] symfony/monolog-bundle
   [2] symfony/monolog-bridge
   [3] kamisama/monolog-init
   [4] flynsarmy/slim-monolog
   [5] logentries/logentries-monolog-handler
   [6] ddtraceweb/monolog-parser
   [7] enlitepro/enlite-monolog
   [8] fancyguy/wordpress-monolog
   [9] swestcott/monolog-extension
  [10] lexik/monolog-browser-bundle
  [11] bazo/nette-monolog-extension
  [12] nodepub/monolog-json-service-provider
  [13] 0x20h/monoconf
  [14] beberlei/loggly-bundle
 
Enter package # to add, or the complete package name if it is not listed []: 0
Enter the version constraint to require []: dev-master
Search for a package []:
Would you like to define your dev dependencies (require-dev) interactively [yes]? n
 
{
    "name": "cordoval/experiment",
    "description": "some description",
    "require": {
        "monolog/monolog": "dev-master"
    },
    "license": "MIT",
    "authors": [
        {
            "name": "Luis Cordova",
            "email": "cordoval@gmail.com"
        }
    ],
    "minimum-stability": "dev"
}
 
Do you confirm generation [yes]?

You forgot what you were doing and wanted to see what it was installed?

~ composer show --installed
doctrine/annotations     v1.1.2 Docblock Annotations Parser
doctrine/cache           v1.3.0 Caching library offering an object-oriented API for many cache backends
doctrine/collections     v1.1   Collections Abstraction library
doctrine/common          v2.4.1 Common Library for Doctrine projects
doctrine/dbal            v2.4.1 Database Abstraction Layer
doctrine/inflector       v1.0   Common String Manipulations with regard to casing and singular/plural rules.
doctrine/lexer           v1.0   Base library for a lexer that can be used in Top-Down, Recursive Descent Parsers.
doctrine/orm             v2.4.1 Object-Relational-Mapper for PHP
symfony/console          v2.4.0 Symfony Console Component
symfony/event-dispatcher v2.4.0 Symfony EventDispatcher Component

Encouragements in all good, totally undeserving it and enjoying grace,

your friend @cordoval

PHPUnit Symfony Test Suite

Recently I was looking at tests at the symfony/symfony repository and found some interesting things. Because there were so many skipped tests I wanted to know the reason why so many tests on symfony were skipped. So I ran `phpunit –tap` on under `vendor/symfony/symfony` folder. I then realized there were some skipped tests mainly due to the mismatching intl and icu extension and versions, some other reasons and among them that we were not plugging the memcached, memcache, and mongodb php extensions. The option `tap` was very useful in doing this and I submitted a PR here.

Screenshot 2013-12-27 08.36.11

Besides this I wondered how can I run phpunit so that it will give me a note on which was the bottleneck in terms of tests performance. Which test is taking too long on the symfony suite?

I needed to do a custom logger or listener for phpunit. Here is the code output:

...
14.584405 = ok 2985 14.584405 - Symfony\Component\Finder\Tests\FinderTest :: testNonSeekableStream
 
Time: 2.54 minutes, Memory: 749.50Mb

So now we know which is the slowest test and the reason is because of the network dependency:

public function testNonSeekableStream()
{
    try {
        $i = Finder::create()->in('ftp://ftp.mozilla.org/')->depth(0)->getIterator();
    } catch (\UnexpectedValueException $e) {
        $this->markTestSkipped(sprintf('Unsupported stream "%s".', 'ftp'));
    }
 
    $contains = array(
        'ftp://ftp.mozilla.org'.DIRECTORY_SEPARATOR.'README',
        'ftp://ftp.mozilla.org'.DIRECTORY_SEPARATOR.'index.html',
        'ftp://ftp.mozilla.org'.DIRECTORY_SEPARATOR.'pub',
    );
 
    $this->assertIteratorInForeach($contains, $i);
}

You have to add it to the phpunit.xml.dist like this:

<listeners>
    <listener class="CustomLogger" file="./CustomLogger.php" />
</listeners>

And finally what makes it possible:

<?php
 
class CustomLogger extends PHPUnit_Util_Printer implements PHPUnit_Framework_TestListener
{
    protected $testNumber = 0;
    protected $testSuiteLevel = 0;
    protected $testSuccessful = TRUE;
    protected $largestTime;
    protected $largestTest;
 
    public function __construct($out = NULL)
    {
        parent::__construct($out);
        $this->largestTest = '';
        $this->largestTime = 0.0;
    }
 
    public function addError(PHPUnit_Framework_Test $test, Exception $e, $time)
    {
    }
 
    public function addFailure(PHPUnit_Framework_Test $test, PHPUnit_Framework_AssertionFailedError $e, $time)
    {
    }
 
    public function addIncompleteTest(PHPUnit_Framework_Test $test, Exception $e, $time)
    {
    }
 
    public function addSkippedTest(PHPUnit_Framework_Test $test, Exception $e, $time)
    {
        $this->testSuccessful = FALSE;
    }
 
    public function startTestSuite(PHPUnit_Framework_TestSuite $suite)
    {
        $this->testSuiteLevel++;
    }
 
    public function endTestSuite(PHPUnit_Framework_TestSuite $suite)
    {
        $this->testSuiteLevel--;
 
        if ($this->testSuiteLevel == 0) {
            $this->write(sprintf(
                "\n\n%f = %s\n",
                $this->largestTime,
                $this->largestTest
            ));
        }
    }
 
    public function startTest(PHPUnit_Framework_Test $test)
    {
        $this->testNumber++;
        $this->testSuccessful = TRUE;
    }
 
    public function endTest(PHPUnit_Framework_Test $test, $time)
    {
        if ($time > $this->largestTime) {
            $this->largestTime = $time;
            $this->largestTest = sprintf(
                "ok %d %f - %s\n",
                $this->testNumber,
                $time,
                get_class($test).' :: '.$test->getName()
            );
        }
        if ($this->testSuccessful === TRUE) {
            $this->write(
                sprintf(
                    "ok %d %f - %s\n",
                    $this->testNumber,
                    $time,
                    get_class($test).' :: '.$test->getName()
                )
            );
        }
    }
 
    protected function writeNotOk(PHPUnit_Framework_Test $test, $prefix = '', $directive = '')
    {
        $this->testSuccessful = FALSE;
    }
}

Thanks for reading and hope you find this very useful at work!

Encouragements in all good,

Luis

Hacking Day Warsaw Symfony Series: Part VII

giullio

Numbers do need to be taken care as they have no mathematical precision nor are they updated to the last minute since some things are hard to get noted on the same spreadsheet. But they are worst case, so the reality is much better.

Screenshot 2013-12-15 13.51.26

Screenshot 2013-12-15 13.38.09 source @liuggio

I think the numbers speak of that we tried hard forms, that even though @romainneutron did not have a large group that he got most things done than anybody else around, also that FOSRestBundle got some good love from the community, probably due to the fact that there was a talk about it that was clearly delivered. Overall we pound the symfony locs with some good arsenal. Docs was the second most PRed repo and probably the largest and more organized group that was around. The polish power was spread out throughout the tables bringing up the largest hacking day ever held within the symfony world. More can be said of many things, but let this bit encourage you to grow as a contributor and be part of our welcoming community.

Encouragements in all good,

totally undeserving of everything and standing in grace

your friend @cordoval

Hacking Day at Warsaw Series: Part VI

Screenshot 2013-12-13 03.51.40

I think the success of events are clear. We have a SymfonyCon with two interesting presentations, “How to pitch Symfony to your client” and “Community Building with Mentoring: What makes people crazy happy to work on an open source project?”. By these two at least you may know two intentions from @fabpot. First one is to help people like you and me to really get Symfony2 out there and nail clients/contracts thereby helping you in your work and to succeed. You succeed, Symfony succeeds. Smart move. The second one is looking forward to the community and caring about breeding the right type of community efforts.

Drupal is an example to study as they have many years of organizing and experimenting with what works and how it works on the FOSS field. Cathy They’s slides https://github.com/YesCT/warsaw set us in a good direction to be concerned about a community growth path a human roadmap. So far Symfony has been a community of top experts working on cool stuff, but with popularity and growth we have to take care of quality and a contributor culture. Symfony without a contributing community cannot thrive as Drupal does. Drupal has success because is indeed a well networked community. Symfony is getting there, but there is the danger of breeding things we don’t necessarily want as a community, namely bad quality and uncontrolled growth.

Mentoring is one of the answers but it is also hard on some people. I myself was mentored by several people along the road, it is not easy and a continuous thing, and it is hard. I think is good we learn in a challenging environment but it costs a lot of time and will. Mentoring helps a lot to easy that path. This saturday, that is tomorrow, we are going to see if we put things into practice. If you are not clear on what a hacking day looks likes please follow the guidance that we will give and have been giving along.

I think @fabpot did good in bringing this talks on board for the conference. But definitely it is not going to be an easy path, and it will take work not just from him but from the community. We have to raise the bar ultimately. How do we raise the bar and at the same time teach people and at the same time fit this into people business mindset and schedules?

My practical advice, get into code within your team, be ready, read issues, read and try. Meet people now at Warsaw that can help, or whom you can ask for help while you code. Don’t wait until the last minute, read tickets and then bring up questions from your findings. A larger community is rising.

I hope you will be with us this saturday.

Encouragements in all good, your friend

@cordoval

Symfony Hacking Day D Day Series: Part V

Time is up guys, time to start focusing on the hacking day.

My last notes about this go about the hanging out and getting to know symfony/symfony core better. So I have spent quite a bit amount of time meddling with the issues on github for the symfony/symfony repo. Here are some of the things I have noted:

Do your work, nevermind the rest: Whenever one submits a PR you will attract reviewers to your code. If changes are simple to do after some preliminary reviews then just do the work. If you receive a very strong criticism or is the wrong approach, just let it there and move onto something else, let it sit until it settles. Or it also can get settled by the major figures, repo maintainers, etc.

Surprises on level of support: Sometimes there will be issues on specific support of certain features. Remember to try to remain core. The core domain, the thing people will use the most. If it is a too edge case that only one person will ever use then you may not be investing your time wisely. And remember nobody is paying you for helping. Some components or pieces of software may not support a full specification for instance Yaml component. In this case in digging the issues you will find the reasons in a comment by the maintainer. This is very important as you are unveiling history and reasons why the project was lead to certain directions.

Be the change Try to participate and ask questions. Be careful you understand to the best of your ability the issue. But ask, is important to get clarifications to be able to work on failing tests or reproduction of the bug. Sometimes people will be lazy, they will just take forever to reply, or not be interested at all. Be different! Take things into your hands and submit PRs. By being the change you want there you will prompt other people to ask and constructively criticize certain decisions.

Never stop learning: Some things are hard, and that is a good thing, there is room to explore and feel a bit lost. Never stop learning details and open new doors. Yaml does not have a full spec implementation then check ruby implementation of yaml. Can’t it be ported? yes or no? why not? Learn to read and read, get familiar with the issues within the scope of a year or so or even more. Learn from the code of others. Don’t be afraid to ask.

Wait patiently and humbly: After all your PRs are submitted, keep calm, they will not necessarily be merged. It does not matter if they do. You are learning. If they are rejected is good because there is room to learn more and there is room to explore, why it is not merged. Also give time, the maintainer usually is very busy. Being a maintainer is a lifestyle, you know when and what you have to do, it is like being a ring bearer, is heavy. So just wait and relax, dive into other territories while waiting.

When a feature is requested please do not do thumbs up and leave: worst thing you can do is to spam others with just your desires. Please don’t do it, makes no sense. Same when you report a problem and do not even provide details on how to reproduce. Best is if you fork a Symfony Standard Edition exhibiting the problem. Never never please just do thumbs up and leave without pushing some code or details. You are not supporting, you are spamming. It is very different when the feature is complete and you do a thumbs up because you have reviewed it and yo have found no fault. So please stop spamming, we already receive a ton of email. No need for stupid thumbs up without review or code work. There is an exception for thumbs up which I think is valid, and is when you mount support to overcome a decision from the maintainer that is or seem to be somewhat political or unreasonable. Then it is about finding out how many people is really interested. That is different and needs to be pondered.

Please do review-oriented thumbs up with comments: PRs need your comments, thoughtful comments, and questions if you are contributing. You should contribute that way doing peer reviews. Thumbs up if you have the right reputation make a PR more trustworthy. The maintainer will be prone to merge it quicker. If you happen to be familiar with another related ticket that has not been tagged please link it in your comment. Is important to keep the issues interconnected so when they are closed there is little to search and easy to close related tickets operation is done seamlessly.

Make friends and pair program: While working on an issue please find opportunity to ask and befriend people. You need the right rubber ducks and helpers. I just pushed some hours ago an issue which i troubleshoot and was helped by another guy in UK. I wrote some code but ended up removing it and he found the fix after I have helped make setup for it to be found. It was a team effort, the change was small but significative. I couldn’t have done it alone and he couldn’t have found it out were it not for my testing code that put it to the surface concretely. I submitted the PR and I set the commit to his authorship. We are a team. So be friends and enjoy what you do.

All in all a hacking day is to put some greasy and start moving the pieces of the machinery. The faster you get the more excited you get the quicker reactions to problem the better. Come to the hacking day to learn the flow and to resolve problems.

Encouragements, your friend @cordoval

See you on the 14th! and i will be working from Warsaw, Lord willing, all this week on issues or my personal projects. Also I am looking for a decent job. /o/