Kiss Your Travis-CI Good-Bye with Symfony2 Vagrant

I have problems with vagrant right from the get go. The idea is to setup a travis like environment to pre-test tests to be send to travis.
The problem I am trying to overcome appears when we have too elaborated settings in travis and experience too long wait time that make the whole testing experience very tedious.

Right from the get go i ran into a wrong assumption. I assume that i had installed vagrant just because rvm and ruby 1.9.2 was present on my system. The vagrant site tells you to download the package and run it with standard procedures. However ubuntu has the right version on its channels if you are using precise. So just use that and don’t try to either download the package or try to install it on your rvm or gems. Yes sorry ruby got me frustrated this morning. I don’t say it sucks, I say it is a mess!

Then next step is to do:

~ git clone git://github.com/cordoval/vagrant-php-dev-box.git
~ cd vagrant-php-dev-box
~ vagrant box add precise64 http://files.vagrantup.com/precise64.box
~ #vagrant init precise64 (this will tell you there is an already vagrant file created so just move on)
~ vagrant up (first time failed me because of port collisions)

So if port collisions check it with (thanks @boonkerz):

~ netstat -lnptu
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 0.0.0.0:27017           0.0.0.0:*               LISTEN      -               
tcp        0      0 127.0.0.1:3306          0.0.0.0:*               LISTEN      -

Then enter the Vagrantfile on the root vagrant dev box project directory and change the numbers (27017 before):

  config.vm.forward_port 80, 1234

Then rerun `vagrant up` and there you go.

If you forget or need some special things to have either composer or your project dependencies to work well then you can proceed to do `vagrant ssh` and install inside the needed dependencies, for example:

~ sudo apt-get install php5-curl

If after you run `vagrant up` you get something like the following:

~ vagrant up
[default] Importing base box 'precise64'...
[default] The guest additions on this VM do not match the install version of
VirtualBox! This may cause things such as forwarded ports, shared
folders, and more to not work properly. If any of those things fail on
this machine, please update the guest additions and repackage the
box.
 
Guest Additions Version: 4.1.16
VirtualBox Version: 4.1.18
[default] Matching MAC address for NAT networking...
[default] Clearing any previously set forwarded ports...
[default] Forwarding ports...
[default] -- 22 => 2222 (adapter 1)
[default] -- 80 => 1234 (adapter 1)
[default] -- 27017 => 1235 (adapter 1)
[default] Creating shared folders metadata...
[default] Clearing any previously set network interfaces...
[default] Running any VM customizations...
[default] Booting VM...
[default] Waiting for VM to boot. This can take a few minutes.
[default] VM booted and ready for use!
[default] Mounting shared folders...
[default] -- v-root: /vagrant
[default] -- manifests: /tmp/vagrant-puppet/manifests
[default] Running provisioner: Vagrant::Provisioners::Puppet...
[default] Running Puppet with /tmp/vagrant-puppet/manifests/default.pp...
stdin: is not a tty
No LSB modules are available.
warning: Could not retrieve fact fqdn
notice: /Stage[main]/Development/Exec[apt-get update]/returns: executed successfully
notice: /Stage[main]/Php5/Package[php5-mysql]/ensure: ensure changed 'purged' to 'present'
notice: /Stage[main]/Development/Package[git]/ensure: created
notice: /Stage[main]/Php5/Package[php-apc]/ensure: ensure changed 'purged' to 'present'
notice: /Stage[main]/Php5/File[/etc/php5/conf.d/custom.ini]/ensure: defined content as '{md5}d918da65f01e420a256bd5721848f8a6'
notice: /Stage[main]/Php5/Package[php5-xdebug]/ensure: ensure changed 'purged' to 'present'
notice: /Stage[main]/Php5/Package[php5-sqlite]/ensure: ensure changed 'purged' to 'present'
notice: /Stage[main]/Development/Package[php-pear]/ensure: ensure changed 'purged' to 'present'
notice: /Stage[main]/Development/Package[curl]/ensure: ensure changed 'purged' to 'present'
notice: /Stage[main]/Php5/Package[php5-cgi]/ensure: ensure changed 'purged' to 'present'
notice: /Stage[main]/Development/Exec[set pear autodiscover]/returns: executed successfully
notice: /Stage[main]/Development/Exec[install phpunit]/returns: executed successfully
notice: /Stage[main]/Symfony-standard/Exec[run composer for symfony when composer is used]/returns: executed successfully
notice: /Stage[main]/Symfony-standard/Exec[install composer for symfony when needed]/returns: executed successfully
notice: /Stage[main]/Php5/Package[php5-intl]/ensure: ensure changed 'purged' to 'present'
notice: Finished catalog run in 494.53 seconds

Then even the virtual box was created you need to update your virtualbox guest additions with this script (thanks @debo).

# Installing the virtualbox guest additions
sudo apt-get -y install linux-headers-$(uname -r) build-essential
sudo apt-get -y install dkms
VBOX_VERSION=4.1.18 # this is the latest version at the time of the blog post
cd /tmp
wget http://download.virtualbox.org/virtualbox/$VBOX_VERSION/VBoxGuestAdditions_$VBOX_VERSION.iso
sudo mount -o loop VBoxGuestAdditions_$VBOX_VERSION.iso /mnt
sudo sh /mnt/VBoxLinuxAdditions.run
sudo umount /mnt

After this step you want to exit the virtualbox and then do a `virtual reload`. Be mindful you may need to reinstall some things but your virtualbox is certain to have Guest Additions already upgraded to 4.1.18. The Guest Additions step is a MUST because of this 3 day nightmare. Here is another link that can serve the explanation and also be useful. But the script is enough to make it work.

At this point I ran into this blog post: http://www.adayinthelifeof.nl/2012/06/29/using-vagrant-and-puppet-to-setup-your-symfony2-environment/ so the idea now is to add the user-group www-data to vagrant as in the blog post.

Another note I can make is regarding the time outs from composer within a vagrant slow connection box. Here is the link and the command:

vagrant@precise64:/vagrant/www/KnpBundles$ COMPOSER_PROCESS_TIMEOUT=4000 php composer.phar install --dev -v

http://www.ubuntuka.com/add-user-to-existing-group-ubuntu/
http://serverfault.com/questions/398414/vagrant-set-default-share-permissions

Composer’s New Colophon Friend: The Post Install Command Tail

Lately I got interested in composer’s feature of post install commands.

I created a simple library and its composer file as follows:

{
    "name": "cordoval/colophon",
    ...
    "require": {
        "php": ">=5.3.2",
        "symfony/console": "dev-master"
    },
    "autoload": {
        "psr-0": {
            "Colophon": "lib/"
        }
    },
    "bin": ["bin/colophon"]
}

The important thing is the `bin` and the `symfony/console` dependency. The `bin` will tell composer to place our library script bin/colophon to be taken into consideration so that it can be called from the your project’s top level directory `bin/colophon`. The `symfony/console` part is because the `bin/colophon` calls an application part of the console to run our lib command which is based on symfony command class inside `symfony/console`.

Our `cordoval/colophon` lib contains this class:

<?php
namespace Colophon;
 
class Colophon
{
    public static function postUpdate($event)
    {
        self::postInstall($event);
    }
 
    public static function postInstall($event)
    {
        //$composer = $event->getComposer();
 
        // 1st pull
        system('bin/colophon print');
    }
}

Colphon class shown above has static methods called by your project’s post install directives inside your project’s composer.json:

// ...
"require": {
        "php": ">=5.3.2",
        "cordoval/colophon": "*",
        // ...
}
// ...
"scripts": {
        "post-install-cmd": [
            "Colophon\\Colophon::postInstall",
        ],
        "post-update-cmd": [
            "Colophon\\Colophon::postUpdate",
        ]
    },
// ...
    "config": {
        "bin-dir": "bin"
    },
// ...

Your project’s composer.json also sets the require to get `cordoval/colophon` package and also tells your project to place all the scripts into the `bin` top level folder.

Notice `cordoval/colophon` defines a command `bin/colophon print` which basically outputs text into the cli. This would be used by `cordoval/colophon` to output some nice ascii text like the one image on top.

Thanks for your support. If you want to see me on San Francisco SymfonyLive please donate i need $1700 to go out of the initial $1800, thanks so much for all those who donated, it is so encouraging to see the community standing up on this!

BC Detection And Locking With Composer!


Background: It is often the case that our project has dependencies under development. If a new version of a dependency is released on packagist and introduces a BC break that you are not aware of then things can happen. You don’t want to become aware of this problems after you deploy!

Breaks of your application due to these dependency BC breaks can be detected writing good integration tests. In addition, you want to find out when these BC breaks were introduced and lock your good versions right before.

Here we show you an example on how to tackle this problem. We will use a silex app called Fazer with with a package dependency found in packagist, namely TodoList (cordoval/todolist).

First instantiate the project:

composer create-project cordoval/fazer /path/to/project/fazer
cd /path/to/project/fazer
~/path/to/project/fazer$

Second copy bisect.sh script to a location outside of the project. The reason why copying the latest version is that you may have versioned this script and earlier versions could be different. To measure whether a version is good or not we need a fix standard rather than a moving target.

~/path/to/project/fazer$ cp bisect.sh ../bisect.sh

Next, start the BC detection:

~/path/to/project/fazer$ git bisect start
~/path/to/project/fazer$

Next mark the current commit as bad (project shows already broken == tests fail):

~/path/to/project/fazer$ git bisect bad

Next mark the last known good commit (using good criteria and memory):

~/path/to/project/fazer$ git bisect good ee55ec2

Next run the script to bisect and find the last good commit right before the BC dependency break:

~/path/to/project/fazer$ git bisect run ../bisect.sh

The script bisect.sh will install cleanly the dependency versions at the time the commit was current in order to find when was the last time that everything worked ok. It will use composer for this and will run after every install the tests provided as shown here:

rm -rf vendor
rm composer.lock
composer install
phpunit -c phpunit.xml.dist

The process will start:

running ../bisect.sh
Installing dependencies
  - Installing pimple/pimple (dev-master)
    Cloning 86cae5c9e85cd02f1703c774ddf12f2448a5938c
...
PHPUnit 3.6.4 by Sebastian Bergmann.
 
Configuration read from /home/cordoval/sites-2/fazer/phpunit.xml.dist
 
.
 
Time: 0 seconds, Memory: 5.25Mb
 
OK (1 test, 2 assertions)
Bisecting: 0 revisions left to test after this (roughly 0 steps)

And will finish finding the last good commit:

running ../bisect.sh
Installing dependencies
  - Installing pimple/pimple (dev-master)
    Cloning 86cae5c9e85cd02f1703c774ddf12f2448a5938c
...
PHPUnit 3.6.4 by Sebastian Bergmann.
 
Configuration read from /home/cordoval/sites-2/fazer/phpunit.xml.dist
 
E
 
Time: 0 seconds, Memory: 4.75Mb
 
There was 1 error:
 
1) Fazer\Tests\FazerTest::testBisect
Missing argument 2 for Fazer\TodoList::__construct(), called in /home/cordoval/sites-2/fazer/src/controllers.php on line 36 and defined
...
FAILURES!
Tests: 1, Assertions: 0, Errors: 1.
9235d39bdeb0317d4f098bb4f8300f0a7f4e2920 is the first bad commit
commit 9235d39bdeb0317d4f098bb4f8300f0a7f4e2920
Author: Luis Cordova <cordoval@gmail.com>
Date:   Sat Jun 30 16:52:21 2012 -0500
 
    update vendors
 
:100644 100644 42534fd95ab08a782f1df71af131a00f515f2427 c8aed6e4d61aae84b8e8693f348fbb6519f85941 M	composer.json
bisect run success

Finally issue a `bisect reset` to go back to your former branch and continue work to do the corresponding fixes.

If you have reached this far, congratulations! I hope it was useful.
I thank the Lord Jesus for His undeserved grace given to me so far for enjoying leaning these things. Thanks Raul for hanging out.
Again if you would like to support me to attend symfonyLive event on San Francisco and this blog post has helped you please do consider donating towards the fund raising of $1800 for the air ticket.
I would really appreciate your help!

More blog posts coming! Please tweet to your friends and spread the word!

Kiss Your git clone’s Good-bye!: Part I

You heard me right, get ready to kiss your git clone’s good-bye with composer. If before you were accustomed to:

git clone git://github.com/username/repoName.git .

Then you would usually run more commands to get actually setup. And guess what, we don’t have time and we don’t have enough info for that. As PHP devs we are to get things done fast and in the best elegant way.
No more. Composer is here!

Sometimes we ignore that composer is all we need to start in the world of PHP. Today we are going to talk about the create-project command found in composer. I will assume you have a composer dev version installed on another folder and symlinked to /usr/bin/composer. The ability to bootstrap with dependencies ready a complete skeleton application lies into one command now thanks to this composer command. As long as there is a package registered in packagist.org with the ability to setup a project with dependencies of any kind you can just tell composer to ready it for you. The example could be a silex application:

composer create-project fabpot/silex-skeleton

Thankfully @fabpot took the job to create a sample project skeleton for silex. Let’s remember that one thing is the framework or microframework and another different is the folders, organization, bootstrap files, that carry along with a proper application using those frameworks. For the case of symfony, don’t worry guys, we have know it all along just we did not look hard enough:

composer create-project symfony/framework-standard-edition

You really name it, as long as there was someone careful enough to prepare a bootstrapped skeleton for that particular lib or framework you love, there will be an easy 1 punch setup.
Furthermore, if you were looking for a particular version you can do:

composer create-project symfony/framework-standard-edition sf21beta1Dir v2.1.0-BETA1

If you want to go straight to the source you could do right on the same directory bootstrap in a php-flow subdirectory the project Bergie’s nice project PHPFlow like this:

composer create-project PhpFlo/PhpFlo php-flow --dev --prefer-source

Note: If create-project uses a source install, you generally want to delete the .git folder and init a clean git repo for your project. (thanks @stof)

Finally, it is in the responsibility of the package maintainer to work on composer post install hooks. This is the information that will make the development faster and easier. Using those post install scripts is not exactly easy but it can be done. This will benefit anyone coming and just doing work on a completely setup environment!

If you enjoy this blog post please consider donating. I am fund raising 1,800$ for my ticket to SymfonyLive 2012 in San Francisco.
If you want to help me go there, or would like to meet me, I would love to meet you there and hangout with the community. Please help me get there! Thanks!

I thank the Lord Jesus for His grace so far on learning symfony2 and for the opportunity that is opening to attend my first SymfonyLive event. I am also thankful to KnpLabs, happy developers! for helping me with the conference ticket!
You guys rock!

SFCON’2012 Conference Report

The conference was a well attended ~60+ conference on symfony2. More people tried to register however there was a limitation already fixed due to lack of bigger facilities. The first conference in Latin America on Symfony2 was held this past weekend on Bello Horizonte, Brasil. Bello Horizonte had the best weather I have ever seen with green surroundings, my idea of it is a spread out city with one main center and 12-14 districts. Downtown is busy, people all over like to party and drink beer and coffee and go to the market to eat and eat out at restaurants too. I rather like the more simple view of the districts which resemble the life in province back in Peru. People is simple and kind, they have problems with education and opportunities for the poorest. Food is good, I liked beans, rice, and mixtures of vegetables, not so much pork, or the grease in the meat, but they have an excellent meat and sort of juices. I thank God for such a great weather I enjoyed for few days, it felt like Napa, California but with the district’s people touch from province. Could go back at anytime, the only impediment is the price of the tickets.

The conference started early on Saturday:

Desenvolvimento de aplicações PHP utilizando o Symfony 2
This one was given by Hugo Magalhães. It was an introduction to symfony2 which tried to be a warming up for those who were not using symfony2 already.
There was a mention of the cmf but not much detail. The good thing mentioned was a slight argument symfony2 vs drupal from a developer pov.

Doctrine 2 – Camada de persistência para PHP
This one was given by Fabio Silva. This was my first live introduction to doctrine. I learned some specifics about configuring cache mechanism for the ORM layer. Also was explained annotations and there were more things to review. There was not much details on more annotation arguments beyond the basic one but the explanations were good and simple. I did not know doctrine was actually based on Active Record. Also I answered a question on the origins of doctrine’s specification: JPA and won a doctrine t-shirt. One interesting thing was the concept of concrete table inheritance, single table inheritance, and class table inheritance. Also there was a quick mention on doctrine’s DQL and performance. The more one hears details on Doctrine the more one learns what one is already using unknowingly, it is so neat.

Silex – Talk is cheap, show me the code!
Jean was good and clear explaining the reason for silex. He showed code on most of his slides which made it very insightful and clear. I am looking forward to put here a link to the silex code on github or to his slides. The talk introduced details on silex I did not know like the after and before methods and also some mention on Session Provider. He was the first one to show something of composer as silex works well with it. Also there was methods to validate and a word on the importance of priorities for the method calls.

Conceitos de engenharia reversa aplicados na migração de sistemas legados Symfony 1.x para Symfony 2.x.
After lunch there came this talk. Guilherme Veras knows a lot more than what he presented but he was limited to talk more details on symfony2 by time and scope. I become sleepy after lunch and when the talk is about symfony1.4, however it was good to unwillingly learn some details about it because that gives me more of a proper view of what has been changed and improved. The reversing engineering did not mention the bundle to transfer 1.4 to 2 apps, however it was insightful in that there is a market for migrating old app from clients to latest. I think that recipes are good however they probably represent the 3% of the project advance. Sorry I was not very much brought in because lack of strength.

Inversão de Controle e Injeção de Dependência no Symfony 2 – E faça-se a luz!
I liked this talk for the reason that I am interested on this topic. Welington is a good friend and he went ahead and showered all DI information I can hardly find on the documentation. The example of the fish in a bowl and chest made it all very easy to follow. He had a fast accent but thanks to his “Belleza!” at the end I was able to catch up on the explanations. Welington seems very rounded and even though I cannot find anything on his github account I can at least see he has its eyes in good stuff. About the DI the explanation even touched on advance things such as scopes. Even though he went that far to explain DI that far, I felt like the Extensions concept and operation on containers was left out. The talk was on DI standlone so it was very good. Maybe there is a place to do a talk on DI + Symfony2. The thing that was very well laid out was the purpose of DI and when it should be used. I hope we get in the following days links to the code on the talk and his slides which well serve as a documentation grade intro to DI. Belleza!

TDD com Symfony 2
Augusto is a nice gentleman, very generous and real. He is one of the top devs in Brasil and very keen to explain newbies about his findings and experiences (at least the ones he can tell). Btw he is looking for a job and I just hope he finds a very core dev job with the best pay, he is a total asset for the real companies out there on PHP. I would hire him if I were you :D. About the lecture it was exactly what I expected besides one thing. In the talk he conveyed a full coded explanation on how to do TDD the right way, the basic concepts and why one does what. The code was a class Task which had injected PDO-like layer with some API methods. The meat was seeing the thought process live and the phpunit dotted screen everytime it failed or passed. The catch phrase was TDD is a process in which we better get accustomed to. Another take on his talk is the way about mocking one service/class one by one on the chain until we truly finish. This is probably the hardest step for when writing tests. Remember: “Tests ain’t gonna write themselves!”. He also confirmed this to me in that even when we end up testing a simple if statement after mocking the last service, it is all worth it and we should do it. Another idea that was bounced between us it was that in order to mock well one has to know the boundaries of surrounding services/classes well enough or the nature of the messages being passed at least.
The critique comes after I got off the plane and looked at the title again. I thought it was symfony2 TDD :D. Well, Augusto is very knowledgeable and have used symfony2 and knows it ok even to use his components on Respect project, the problem is that he did not feel perhaps very confident on doing TDD with symfony2. It is like someone that knows a lot and he only talks about something when he is totally twice the expert on. Anyway, it is cool, I am sure he will rock it in the next lecture with Symfony2 and become the jakuza for Latin America. I made a great friend in Augusto!

Aplicações Comportadas com Behat
Alexandre gave a great disclosure on his experience with BDD. It was a great joint between Augusto and Alexandre’s talk that make it even better to follow up. I have never seen the evolution of BDD’eing things so clearly explained, even though the talk was in Portuguese Alexandre was very direct and clear and showed his mastery on the topics and his experience. Alexandre and Augusto work together on Respect and have a great baggage to share. He made a like facebook type of example that carried the message to the four winds throughout the presentation. If these two things are not the most important tips, then I don’t know what else to tell you. First 1) show simple code on your presentation but show the evolution of that simple one code, 2) work things around that code and stick to that code to explain things. These guys know a lot from their experience with yahoo and personal experiences, I wish there could be more times in the future to learn from them and also ended up very interested in respect and working with them on bringing a great community movement to the LA region.

Three questions I would have wanted to be answered are:

1. The way one BDD with behat off-site authentication and if this can be done with the behat tool and how (this is for knpbundles.com).
2. DI how to generate from reading the Configurator.php class a yaml dump on the default configuration for a bundle repo clone (this is also for knpbundles.com).
3. Show us more experience and strategy on symfony2 TDD

Congratulations to organizers and community. I cannot emphasize enough the hospitality of the people of Bello Horizonte. Belleza!
Personal acknowledgements go to Moises, Renato, Augusto, Daniel, Diego, Alexandre, and many more I am forgetting their names sorry. My mind was all inmersed in portuguese and I now need to go back to English mindset. You guys keep the good work!

Special thanks to those who supported the @php_peru participation in the event:

– I thank God to give me the undeserved opportunity to travel to Brasil and accomplish much learning!
– ServerGrove: best hosting for php symfony2 projects!
– Diego Oliveira: thanks man, you rock!
– Daniel Oliveira: best friend in Brasil!
– Raul Rodriguez: He printed my eticket so that i can travel without problems!
– @php_peru: thanks my friends!
– and more! you know who you are!