Building a REST client with asynchronous calls using PHP and curl

One month ago I posted an article called Building a simple HTTP client with PHP. A REST client. In this post I tried to create a simple fluid interface to call REST webservices in PHP. Some days ago I read an article http://www.ibuildings.nl/blog/archives/811-Multithreading-in-PHP-with-CURL.html and one light switch on on my mind. I can use curl’s “multi” functions to improve my library and perform simultaneous calls very easily.

I’ve got a project that needs to call to different webservices. Those webservices sometimes are slow (2-3 seconds). If I need to call to, for example, three webservices my script will use the add of every single call’s time. With this improve to the library I will use only the time of the slowest webservice. 2 seconds instead of 2+2+2 seconds. Great.

For the example I’ve created a really complex php script that sleeps x seconds depend on an input param:

sleep((integer) $_REQUEST['sleep']);
echo $_REQUEST['sleep'];

With synchronous calls:

echo Http::connect('localhost', 8082)
    ->doGet('/tests/gam_http/sleep.php', array('sleep' => 3));
echo Http::connect('localhost', 8082)
    ->doPost('/tests/gam_http/sleep.php', array('sleep' => 2));
echo Http::connect('localhost', 8082)
    ->doGet('/tests/gam_http/sleep.php', array('sleep' => 1));

This script takes more or less 6 seconds (3+2+1)

But If I switch it to:

$out = Http::connect('localhost', 8082)
    ->get('/tests/gam_http/sleep.php', array('sleep' => 3))
    ->post('/tests/gam_http/sleep.php', array('sleep' => 2))
    ->get('/tests/gam_http/sleep.php', array('sleep' => 1))
    ->run();
print_r($out);

The script only uses 3 seconds (the slowest process)

I’ve got a project that uses it. But I have a problem. I have webservices in different hosts so I’ve done a bit change to the library:

$out = Http::multiConnect()
    ->add(Http::connect('localhost', 8082)->get('/tests/gam_http/sleep.php', array('sleep' => 3)))
    ->add(Http::connect('localhost', 8082)->post('/tests/gam_http/sleep.php', array('sleep' => 2)))
    ->add(Http::connect('localhost', 8082)->get('/tests/gam_http/sleep.php', array('sleep' => 1)))
    ->run();

With a single connection, the exceptions are easy to implement. If curl_getinfo() returns an error message I throw an exception, but now with a multiple interface how I can do it? I throw an exception if one call fail, or not? I have decided not to use exceptions in multiple interface. I always return an array with all the output of every webservice’s call and if something wrong happens instead of the output I will return an instance of Http_Multiple_Error class. Why I use a class instead of a error message? The answer is easy. If I want to check all the answers I can check if any of them is an instanceof Http_Multiple_Error. Also I don’t want to check anything I put a silentMode() function to switch off all error messages.

$out = Http::multiConnect()
    ->silentMode()
    ->add(Http::connect('localhost', 8082)->get('/tests/gam_http/sleep.php', array('sleep' => 3)))
    ->add(Http::connect('localhost', 8082)->post('/tests/gam_http/sleep.php', array('sleep' => 2)))
    ->add(Http::connect('localhost', 8082)->get('/tests/gam_http/sleep.php', array('sleep' => 1)))
    ->run();

The full code is available on google code but the main function is the following one:

    ...
    private function _run()
    {
        $headers = $this->_headers;
        $curly = $result = array();

        $mh = curl_multi_init();
        foreach ($this->_requests as $id => $reg) {
            $curly[$id] = curl_init();

            $type     = $reg[0];
            $url       = $reg[1];
            $params = $reg[2];

            if(!is_null($this->_user)){
               curl_setopt($curly[$id], CURLOPT_USERPWD, $this->_user.':'.$this->_pass);
            }

            switch ($type) {
                case self::DELETE:
                    curl_setopt($curly[$id], CURLOPT_URL, $url . '?' . http_build_query($params));
                    curl_setopt($curly[$id], CURLOPT_CUSTOMREQUEST, self::DELETE);
                    break;
                case self::POST:
                    curl_setopt($curly[$id], CURLOPT_URL, $url);
                    curl_setopt($curly[$id], CURLOPT_POST, true);
                    curl_setopt($curly[$id], CURLOPT_POSTFIELDS, $params);
                    break;
                case self::GET:
                    curl_setopt($curly[$id], CURLOPT_URL, $url . '?' . http_build_query($params));
                    break;
            }
            curl_setopt($curly[$id], CURLOPT_RETURNTRANSFER, true);
            curl_setopt($curly[$id], CURLOPT_HTTPHEADER, $headers);

            curl_multi_add_handle($mh, $curly[$id]);
        }

        $running = null;
        do {
            curl_multi_exec($mh, $running);
            sleep(0.2);
        } while($running > 0);

        foreach($curly as $id => $c) {
            $status = curl_getinfo($c, CURLINFO_HTTP_CODE);
            switch ($status) {
                case self::HTTP_OK:
                case self::HTTP_CREATED:
                case self::HTTP_ACEPTED:
                    $result[$id] = curl_multi_getcontent($c);
                    break;
                default:
                    if (!$this->_silentMode) {
                        $result[$id] = new Http_Multiple_Error($status, $type, $url, $params);
                    }
            }
            curl_multi_remove_handle($mh, $c);
        }

        curl_multi_close($mh);
        return $result;

Pivot tables in PHP

In my work as developer I normally need to transform data from one format to another. Typically our data input are a database and we need to show database data into a report, datagrid or something similar. It’s very typical to use pivot tables. It’s not very dificult to handle pivot tables by hand but the work is always the same: groups by, totals, subtotals, totals per row …. Now I want to write a class to pivot tables in PHP with the most common requirements (at least for me). I know we can create pivot tables with SQL. Group by and some other database specific commands like oracle’s GROUP BY ROLLUP/CUBE can do the work, but I like clean SQL querys. The business logic must be in PHP and SQL must be as clean as we can.

Let’s show you an example. We have the following recordset from a SQL:

$recordset = array(
    array('host' => 1, 'country' => 'fr', 'year' => 2010,
        'month' => 1, 'clicks' => 123, 'users' => 4),
    array('host' => 1, 'country' => 'fr', 'year' => 2010,
        'month' => 2, 'clicks' => 134, 'users' => 5),
    array('host' => 1, 'country' => 'fr', 'year' => 2010,
        'month' => 3, 'clicks' => 341, 'users' => 2),
    array('host' => 1, 'country' => 'es', 'year' => 2010,
        'month' => 1, 'clicks' => 113, 'users' => 4),
    array('host' => 1, 'country' => 'es', 'year' => 2010,
        'month' => 2, 'clicks' => 234, 'users' => 5),
    array('host' => 1, 'country' => 'es', 'year' => 2010,
        'month' => 3, 'clicks' => 421, 'users' => 2),
    array('host' => 1, 'country' => 'es', 'year' => 2010,
        'month' => 4, 'clicks' => 22,  'users' => 3),
    array('host' => 2, 'country' => 'es', 'year' => 2010,
        'month' => 1, 'clicks' => 111, 'users' => 2),
    array('host' => 2, 'country' => 'es', 'year' => 2010,
        'month' => 2, 'clicks' => 2,   'users' => 4),
    array('host' => 3, 'country' => 'es', 'year' => 2010,
        'month' => 3, 'clicks' => 34,  'users' => 2),
    array('host' => 3, 'country' => 'es', 'year' => 2010,
        'month' => 4, 'clicks' => 1,   'users' => 1),
);

host	country	year	month	clicks	users
1	fr	2010	1	123	4
1	fr	2010	2	134	5
1	fr	2010	3	341	2
1	es	2010	1	113	4
1	es	2010	2	234	5
1	es	2010	3	421	2
1	es	2010	4	22	3
2	es	2010	1	111	2
2	es	2010	2	2	4
3	es	2010	3	34	2
3	es	2010	4	1	1

And I need the information grouped by host and show it in a table with: each row one host and each column the sum of clicks and users per month

The interface I have created is the following one:

$data = Pivot::factory($recordset)
    ->pivotOn(array('host'))
    ->addColumn(array('year', 'month'), array('users', 'clicks',))
    ->fetch();

Let’s explain the interface:

Loads the original recorset into the class:

Pivot::factory($recordset)

I want to pivot the recordset on the field ‘host’

->pivotOn(array('host'))

The columns are grouped by year and month and each group will show the sum of users and clicks

->addColumn(array('year', 'month'), array('users', 'clicks',))

And finally I make the transformation:

->fetch();

Basically that’s it. I also has made some other helpers to:
Add totals per line
Add total per pivoted colum
Add total for all table

->fullTotal()
->pivotTotal()
->lineTotal()

In this class I allow to pivot tables with one, two or three fields. And also I allow to use calculated columns.
For example imagine you need to show users, clicks and the average of clicks per user. It’s easy clicks/users but you must take care in the calculated column because you can’t sum it. You must calculate the calculated column over the sum of clicks and user. Because of that I define calculated functions. This feature only works with PHP5.3 because it uses closures.

$averageCbk = function($reg)
{
    return round($reg['clicks']/$reg['users'],2);
};

$data = Pivot::factory($recordset)
    ->pivotOn(array('host', 'country'))
    ->addColumn(array('year', 'month'), array('users', 'clicks', Pivot::callback('average', $averageCbk)))
    ->lineTotal()
    ->pivotTotal()
    ->fullTotal()
    ->typeMark()
    ->fetch();

And now different usage examples:

pivot on ‘host’

$data = Pivot::factory($recordset)
    ->pivotOn(array('host'))
    ->addColumn(array('year', 'month'), array('users', 'clicks',))
    ->fetch();

_id	host	2010_1_users	2010_1_clicks	2010_2_users	2010_2_clicks	2010_3_users	2010_3_clicks	2010_4_users	2010_4_clicks
1	1	8	236	10	368	4	762	3	22
2	2	2	111	4	2
3	3					2	34	1	1

pivot on ‘host’ with totals

$data = Pivot::factory($recordset)
    ->pivotOn(array('host'))
    ->addColumn(array('year', 'month'), array('users', 'clicks',))
    ->fullTotal()
    ->lineTotal()
    ->fetch();

_id	host	2010_1_users	2010_1_clicks	2010_2_users	2010_2_clicks	2010_3_users	2010_3_clicks	2010_4_users	2010_4_clicks	TOT_users	TOT_clicks
1	1	8	236	10	368	4	762	3	22	25	1388
2	2	2	111	4	2					6	113
3	3					2	34	1	1	3	35
4	TOT	10	347	14	370	6	796	4	23	34	1536

pivot on ‘host’ and ‘country’

$data = Pivot::factory($recordset)
    ->pivotOn(array('host', 'country'))
    ->addColumn(array('year', 'month'), array('users', 'clicks',))
    ->fullTotal()
    ->pivotTotal()
    ->lineTotal()
    ->fetch();

_id	host	country	2010_1_users	2010_1_clicks	2010_2_users	2010_2_clicks	2010_3_users	2010_3_clicks	2010_4_users	2010_4_clicks	TOT_users	TOT_clicks
1	1	fr	4	123	5	134	2	341			11	598
2	1	es	4	113	5	234	2	421	3	22	14	790
3	TOT(host)		8	236	10	368	4	762	3	22	25	1388
4	2	es	2	111	4	2					6	113
5	TOT(host)		2	111	4	2	0	0	0	0	6	113
6	3	es					2	34	1	1	3	35
7	TOT(host)		0	0	0	0	2	34	1	1	3	35
8	TOT		10	347	14	370	6	796	4	23	34	1536

pivot on ‘host’ and ‘country’ with calculated columms

$averageCbk = function($reg)
{
    return round($reg['clicks']/$reg['users'],2);
};

$data = Pivot::factory($recordset)
    ->pivotOn(array('host', 'country'))
    ->addColumn(array('year', 'month'), array('users', 'clicks', Pivot::callback('average', $averageCbk)))
    ->lineTotal()
    ->pivotTotal()
    ->fullTotal()
    ->typeMark()
    ->fetch();

_id	type	host	country	2010_1_users	2010_1_clicks	2010_1_average	2010_2_users	2010_2_clicks	2010_2_average	2010_3_users	2010_3_clicks	2010_3_average	2010_4_users	2010_4_clicks	2010_4_average	TOT_users	TOT_clicks	TOT_average
1	0	1	fr	4	123	30.75	5	134	26.8	2	341	170.5			0	11	598	54.36
2	0	1	es	4	113	28.25	5	234	46.8	2	421	210.5	3	22	7.33	14	790	56.43
3	1	TOT(host)		8	236	29.5	10	368	36.8	4	762	190.5	3	22	7.33	25	1388	264.13
4	0	2	es	2	111	55.5	4	2	0.5			0			0	6	113	18.83
5	1	TOT(host)		2	111	55.5	4	2	0.5	0	0	0	0	0	0	6	113	56
6	0	3	es			0			0	2	34	17	1	1	1	3	35	11.67
7	1	TOT(host)		0	0	0	0	0	0	2	34	17	1	1	1	3	35	18
8	3	TOT		10	347	34.7	14	370	26.43	6	796	132.67	4	23	5.75	34	1536	45.18

The source code of the class is available in google code

Building a simple HTTP client with PHP. A REST client

REST webservices are cool. Nowadays there are a lot of APIS with REST and in combination with JSONP is really easy to build applications. Actually I am playing with CouchDB. It has a nice RESTfull API and I want to use it with PHP. There are several PHP libraries, even a nice PHP extension to work with CouchDB. I like Zend Framework’s REST client but as exercise I will develop a HTTP client. My idea is to create a simple class that allows me to perform GET, POST and DELETE requests to a remote server.

That’s the way I want to use my class:

echo Http::connect('localhost', 8082)
    ->doGet('tests/httpclient/dummy.php', array('a' => "a a a"));

First I want to do a lazy connection to server. That’s because I want to create the real connection only when I create the request. So I create a factory function that only saves the configuration of the client:

    private $_host = null;
    private $_port = null;
    private $_user = null;
    private $_pass = null;

    /**
     * Factory of the class. Lazy connect
     *
     * @param string $host
     * @param integer $port
     * @param string $user
     * @param string $pass
     * @return Http
     */
    static public function connect($host, $port, $user=null, $pass=null)
    {
        return new self($host, $port, $user, $pass);
    }

And now one public function for each action:

    const POST   = 'POST';
    const GET    = 'GET';
    const DELETE = 'DELETE';

    /**
     * POST request
     *
     * @param string $url
     * @param array $params
     * @return string
     */
    public function doPost($url, $params=array())
    {
        return $this->_exec(self::POST, $this->_url($url), $params);
    }

    /**
     * GET Request
     *
     * @param string $url
     * @param array $params
     * @return string
     */
    public function doGet($url, $params=array())
    {
        return $this->_exec(self::GET, $this->_url($url), $params);
    }

    /**
     * DELETE Request
     *
     * @param string $url
     * @param array $params
     * @return string
     */
    public function doDelete($url, $params=array())
    {
        return $this->_exec(self::DELETE, $this->_url($url), $params);
    }

And one extra function to set the headers of the request

    private $_headers = array();
    /**
     * setHeaders
     *
     * @param array $headers
     * @return Http
     */
    public function setHeaders($headers)
    {
        $this->_headers = $headers;
        return $this;
    }

I use a settler function and chainable (returns the instance of the class) because if I need to use some custom headers I want to use the following interface:

    echo Http::connect('localhost', 8082)
        ->setHeaders($myCustomHeaders)
        ->doGet('tests/httpclient/dummy.php', array('a' => "a a a"));

And finally the main function. The function that perform the real request. There are several ways to do that in PHP. For this example I will use CURL’s functions. You must take care about it because CURL extension is not always available in PHP installations. So please check your phpinfo(). If CURL is not available and if you are not allowed to install it you must chose another alternatives.

    const HTTP_OK = 200;
    const HTTP_CREATED = 201;
    const HTTP_ACEPTED = 202;

    /**
     * Performing the real request
     *
     * @param string $type
     * @param string $url
     * @param array $params
     * @return string
     */
    private function _exec($type, $url, $params = array())
    {
        $headers = $this->_headers;
        $s = curl_init();

        if(!is_null($this->_user)){
           curl_setopt($s, CURLOPT_USERPWD, $this->_user.':'.$this->_pass);
        }

        switch ($type) {
            case self::DELETE:
                curl_setopt($s, CURLOPT_URL, $url . '?' . http_build_query($params));
                curl_setopt($s, CURLOPT_CUSTOMREQUEST, self::DELETE);
                break;
            case self::POST:
                curl_setopt($s, CURLOPT_URL, $url);
                curl_setopt($s, CURLOPT_POST, true);
                curl_setopt($s, CURLOPT_POSTFIELDS, $params);
                break;
            case self::GET:
                curl_setopt($s, CURLOPT_URL, $url . '?' . http_build_query($params));
                break;
        }

        curl_setopt($s, CURLOPT_RETURNTRANSFER, true);
        curl_setopt($s, CURLOPT_HTTPHEADER, $headers);
        $_out = curl_exec($s);
        $status = curl_getinfo($s, CURLINFO_HTTP_CODE);
        curl_close($s);
        switch ($status) {
            case self::HTTP_OK:
            case self::HTTP_CREATED:
            case self::HTTP_ACEPTED:
                $out = $_out;
                break;
            default:
                throw new Http_Exception("http error: {$status}", $status);
        }
        return $out;
    }

I’ve have also created a Http_Exception Exception class:

class Http_Exception extends Exception{
    const NOT_MODIFIED = 304;
    const BAD_REQUEST = 400;
    const NOT_FOUND = 404;
    const NOT_ALOWED = 405;
    const CONFLICT = 409;
    const PRECONDITION_FAILED = 412;
    const INTERNAL_ERROR = 500;
}

Http throws Http_Exception when something wrong happens so I can play with it to build my application logic.
For example:

try {
   echo Http::connect('localhost', 8082)
        ->doGet('tests/couchdb/a.php', array('a' => "a a a"));
} catch (Http_Exception $e) {
    switch ($e) {
        case Http_Exception::INTERNAL_ERROR:
            // do something
            break;
    }
}

And that’s it. There is the full code available on bitbucket.

Clean way to call to multiple xmlrpc remote servers

The problem:

I’ve got a class library in PHP. This class library is distributed in a several servers and I want to call them synchronously. The class library has a XMLRPC interface with Zend Framework XMLRPC server.
There is an example class

class Gam_Dummy
{
    /**
     * foo
     *
     * @param integer $arg1
     * @param integer $arg2
     * @return integer
     */
    function foo($arg1, $arg2)
    {
        return $arg1 + $arg2;
    }
}

and the xmlrpc server:

$class = (string) $_GET['class'];
$server = new Zend_XmlRpc_Server();
$server->setClass($class);
echo $server->handle();

First solution

An easy a fast solution for calling remote interfaces is:

$class = "Gam_Dummy";
$client = new Zend_XmlRpc_Client("http://location/of/xmlrpc/server?class={class}");
echo $client->call('foo', array($arg1, $arg2));

and if we have several remote servers:

$servers = array(
    'server1' => 'http://location/of/xmlrpc/server1',
    'server2' => 'http://location/of/xmlrpc/server2',
    'server3' => 'http://location/of/xmlrpc/server3'
    );

$class = "Gam_Dummy";
foreach (array_values($servers) as $_server) {
    $server = "{_server}?class={$class}";
    $client = new Zend_XmlRpc_Client($server);
    echo $client->call('foo', array($arg1, $arg2));
}

Second solution (one remote server):

I want to use the following interface to call my remote class

$class = "Gam_Dummy";
Gam_Dummy::remote("Gam_Dummy", 'server1')->foo($arg1, $arg2);

Why? The answer is because I like to coding with the help of the IDE. If I use the first solution I must remember Gam_Dummy class has a foo function with two parameters. With the second solution if I place the PHPDoc code correctly my IDE will help me showing me the function list of the Gam_Dummy class and even when I type Gam_ IDE will show me all the classes of my repository starting with Gam_. That issue could sound irrelevant for a lot of people but for me is really useful

To get this interface I will change my Gam_Dummy class to:

class Gam_Dummy
{
    /**
     * foo
     *
     * @param integer $arg1
     * @param integer $arg2
     * @return integer
     */
    function foo($arg1, $arg2)
    {
        return $arg1 + $arg2;
    }

    /**
     * Remote interface
     *
     * @param string|array $server
     * @return Gam_Dummy
     */
    static function remote($server)
    {
        return new Remote(get_called_class(), $server);
    }
}

And of course Remote class:

class Remote
{
    private $_class  = null;
    private $_server = null;

    function __construct($class, $server)
    {
        $this->_class  = $class;
        $this->_server = $server;
    }

    function __call($method, $arguments)
    {
        if (class_exists($this->_class)) {
            $server = "{$this->_server}?class={$this->_class}";
            $client = new Zend_XmlRpc_Client($server);
            return $client->call($method, array($arg1, $arg2));
        }
    }
}

Cool. Isn’t it?. But there is a problem if I want to work with two or more remote servers I must write one line of code for each server:

Gam_Dummy::remote('http://location/of/xmlrpc/server1')->foo($arg1, $arg2);
Gam_Dummy::remote('http://location/of/xmlrpc/server2')->foo($arg1, $arg2);
Gam_Dummy::remote('http://location/of/xmlrpc/server3')->foo($arg1, $arg2);

or may better with the array $servers

foreach (array_values($servers) as $server) {
    Gam_Dummy::remote($server)->foo($arg1, $arg2);
}

Third solution for multiple remote servers:

I would like to use this interface instead of solution two with a foreach for multiple servers:

$servers = array(
    'server1' => 'http://location/of/xmlrpc/server1',
    'server2' => 'http://location/of/xmlrpc/server2',
    'server3' => 'http://location/of/xmlrpc/server3'
    );

Gam_Dummy::remote($servers)->foo($arg1, $arg2);

so I change Remote class to:

class Remote
{
    private $_class  = null;
    private $_server = null;

    function __construct($class, $server)
    {
        $this->_class  = $class;
        $this->_server = $server;
    }

    function __call($method, $arguments)
    {
        $out = array();
        if (is_array($this->_server)) {
            foreach ($this->_server as $key => $_server) {
                $server = "{$_server}?class={$this->_class}";
                $client = new Zend_XmlRpc_Client($server);
                $out[$key] = $client->call($method, $arguments);
            }
        } else {
            $server = "{$this->_server}?class={$this->_class}";
            $client = new Zend_XmlRpc_Client($server);
            $out = $client->call($method, $arguments);
        }
        return $out;
    }
}

Jugando con FriendFeed y Appengine: tv-sms.appspot.com

INTRODUCCIÓN

Estas navidades me he propuesto publicar un aplicación con google app engine.
La idea de la aplicación es simple. Hoy en día es muy común disponer de un portátil o netbook y estar en la sala viendo la tele con nuestro portátil. Los canales de televisión suelen permitir que la gente mande sms a unos números de teléfono y estos mensajes aparezcan en la parte inferior de nuestras pantallas. Personalmente no lo he usado nunca pero parece ser que el servicio tiene mucho éxito. Estos mensajes tienen un coste bastante elevado y son una fuente de ingresos para los canales y para las operadoras. La idea de la aplicación es hacer esto por Internet agarrándonos los sms aprovechando que tenemos un portátil en la sala junto a nosotros.

Los requisitos que me he impuesto son los siguientes:

• No dispongo de tiempo ilimitado. Tengo que publicarla en un par de días.
• Usar la API de FriendFeed.
• La aplicación tiene que limitar en lo posible el uso de recursos en el servidor.
• Publicar el código en google code.
• Subir la aplicación a appengine: http://tv-sms.appspot.com/
• Escribir una entrada en mi blog explicando como esta hecha.

Pues bien ya estoy en el ultimo punto así que paso a explicar como funciona.

FUNCIONAMIENTO DE LA APLICACIÓN

Creación de los grupos

La idea es la siguiente me creo con mi usuario un grupo público en FriendFeed por cada canal de televisión y después me creo un archivo javascript con la configuración de los canales:

var chanels = {
    'tvsmstve1' : 'TVE 1',
    'tvsmstve2' : 'TVE 2',
    'tvsms-antena3' : 'Antena 3',
    'tvsmscuatro' : 'Cuatro',
    'tvsms-tele5' : 'Tele 5',
    'tvsmslasexta' : 'La Sexta'
}

Entiendo que esta información la debería guardar en algún tipo de Base de datos pero por ahora se queda así. Al arrancar la aplicación leo esto y lo muestro en pantalla

function populateChanels() {
    $.each(chanels, function(i, chanel) {
        $('#chanels').append("<li class='status'><a class='chanelsLink' href='#' onclick='start(\"" + i + "\")'>" + chanel + "</a></li>");
    });
}

Listado de las últimas modificaciones de los grupos

Friendfeed tiene una API muy sencilla de usar usando REST y JSONP. Para obtener una la lista de actualizaciones de un canal simplemente tenemos que hacer una petición GET a la siguiente url:

http://friendfeed-api.com/v2/feed/grupoid

Por lo tanto y teniendo en cuenta que voy a utilizar jQuery para mi aplicación creo la siguiente función:

var users = {};
var frmInput = "<div class='share'><form method='post' action='?' onsubmit='post();return false;'><input type='text' id='body' name='body' style='width:300px'/><input id='submit' type='submit' value='Post'/></form></div>";

function _getLi(body, user, date) {
    var fDade = formatFriendFeedDate(date);
    return "<li class='status'><span class='thumb vcard author'><img height='50' width='50' src='http://friendfeed-api.com/v2/picture/" + user + "?size=medium' alt='" + user + "'/></span><span class='status-body'>" + user + " : " + body + "<span class='meta entry-meta'>" + fDade + "</span></span></li>";
}

function list(id) {
    var txt = '';
    $.getJSON("http://friendfeed-api.com/v2/feed/" + id + "?callback=?",
        function (data) {
            $('#ff').html('');
            $.each(data.entries, function(i, entry) {
                users[entry.from.id] = entry.from.id;
                txt+= _getLi(entry.body, entry.from.id, entry.date);
            });
            $("#ff").html(txt);
            if (logged) {
                $('#inputForm').html(frmInput);
            }
            $('#txtCanal').html(chanels[id]);

            cometClient(id);
        });
}

En esta función lo que estoy haciendo es obtener la lista en formato JSON y añadirla a un ol. Además de la lista le añado un formulario para que el usuario pueda introducir comentarios y arranco el cliente comet pero esto lo explicaré más adelante.

Actualizaciones en tiempo real

Lo que mas me gusta de Friendfeed y lo que me ha hecho decantarme por el y no por Twitter es que Friendfeed me permite lo que llama Real-Time updates. Esta caracteristica, que hasta donde yo se Twitter no nos la da nos permite hacer de forma muy sencilla clientes Comet. Aquí va el mio:

function cometClient(id, cursor) {
    var url = "http://friendfeed-api.com/v2/updates/feed/" + id + "?callback=?&timeout=5";
    if (cursor) {
        url += "&cursor==" + cursor;
    }
    $.getJSON(url, function (data) {
        $.each(data.entries, function(i, entry) {
            $('#ff').prepend(_getLi(entry.body, entry.from.id, entry.date));
        });
        t = setTimeout(function() {cometClient(id, data.realtime.cursor)}, 2000);
    });
}

Pues nada gracias a esto tengo actualizaciones a tiempo real de mi aplicación y sin usar ningún recurso de servidor.

Nuevo comentario

Para crear un nuevo comentario la API de FriendFeed no me deja hacerlo directamente desde javascript. Esto es una pequeña faena ya que si no tuviera esta restricción toda la aplicación correría sobre javascript y no necesitaría nada de código en servidor. Pero al no ser posible me tengo que autentificar con OAuth desde google appengine. Para hacer esto me baso en la aplicación de ejemplo que nos proporciona FriendFeed para jugar con su API.

El citado ejemplo hace todo lo que necesito y lo hace todo en el servidor. Ya que yo solo voy a usar la parte que FriendFeed denomina creating an entry, me limito a leer el código y eliminar la parte que no necesito. Como una de mis premisas es que no dispongo de tiempo ilimitado para esta aplicación no me voy a dedicar a entender cada linea de código. Me dedico a entender mas o menos como funciona.

En mi aplicación la opción de nuevo comentario va a ser llamada desde javascript y espero un JSON por lo que a parte de eliminar las partes del código que no me interesan, hago unas pequeñas modificaciones para que la respuesta sea JSON

class EntryHandler(webapp.RequestHandler):
    @authenticated
    def post(self):
        try:
            entry = self.friendfeed.post_entry(
                body=self.request.get("body"),
                to=self.request.get("to"))
            out = {'error' : 0}
        except:
            out = {"redirect" : '/oauth/authorize'}
        self.response.headers['Content-Type'] = 'application/json'
        jsonData = out
        self.response.out.write(simplejson.dumps(jsonData))
        return
...

application = webapp.WSGIApplication([
    (r"/oauth/callback", OAuthCallbackHandler),
    (r"/oauth/authorize", OAuthAuthorizeHandler),
    (r"/oauth/check", OAuthCheck),
    (r"/a/entry", EntryHandler),
])

El código es bastante mejorable (no hay gestión de errores y no se hace nada con la variable entry) pero hace lo que quiero: publica una entrada en FriendFeed.

Autentificando

Para publicar entradas en FrinedFeed necesito estar autentificado por lo cual quiero que la aplicación solo muestre el formulario de entrada de datos si el usuario esta autentificado. Para hacer esto hago una llamada asíncrona al servidor al cargar la aplicación que me mire si el usuario esta o no autentificado

function checkFfLogin() {
    $.post("/oauth/check", {}, function (data) {
        if (data.redirect) {
            logged = false;
        } else {
            if (data.ok && data.ok==1) {
                logged = true;
            }
        }

        if (logged == true) {
            $('#login').html('');
        } else {
            $('#login').html("<a href='" + data.redirect + "'><img border='0' src='/sign-in-with-friendfeed.png' alt='Sign in with FriendFeed'></a>");
        }
        populateChanels();
    }, "json");
}

Esto se complementa con la parte del servidor.

class OAuthCheck(webapp.RequestHandler):
    @authenticated
    def post(self):
        cookie_val = parse_cookie(self.request.cookies.get("FF_API_AUTH"))
        try:
            key, secret, username = cookie_val.split("|")
            self.response.headers['Content-Type'] = 'application/json'
            jsonData = {"ok" : 1}
            self.response.out.write(simplejson.dumps(jsonData))
        except:
            self.response.headers['Content-Type'] = 'application/json'
            jsonData = {"redirect" : '/oauth/authorize'}
            self.response.out.write(simplejson.dumps(jsonData))
        return

Cambio de canal

Ya solo queda que el usuario pueda cambiar de canal.

var selectedChanel;
function start(group) {
    $('#body').val('');
    selectedChanel = group;
    if (t) {
        clearTimeout(t);
    }
    $('#ff').html('<img src="/ajax-loader.gif" alt="loader"/>... cargando ' + chanels[group]);

    list(group);
}

FIN

Pues bien la aplicación ya esta operativa. Entiendo que tiene bastantes lagunas pero funciona y e cumplido las premisas que me había propuesto: Un par de tardes para hacerla y otra más para escribir la entrada en el blog y subirla al appengine.

Enlace a la aplicación: http://tv-sms.appspot.com/
Código fuente: http://code.google.com/p/gam-tvsms/

Things I miss in PHP: Function decorators

The problem:

I want to build a set of classes accessible via REST interface. One example here with ZF. My idea is allow users to access via REST to my set of classes. It’s no so difficult. But the problem appears when I want to set an authorization engine to my REST web service. Public functions can be called but sometimes user must be authenticated (with a trusted cookie).

I have my class:

class Lib_Myclass
{
    public function publicFunction($var1)
    {
        return "Hello {$var1}";
    }

    public function privateFunction($var1)
    {
        return "private Hello {$var1}";
    }
}

publicFunction can be called without login but privateFunction only after login

I have done similar with python and Google App Engine. With python I use decorators and its very clean and easy

class Myclass:
    def publicFunction(self, var1):
        return "Hello %s" % var1

    @private
    def privateFunction(self, var1):
        return "private Hello %s" % var1

And I define my decorator ‘private’:

def private(funcion):
    def _private(*list_args):
        if isValidUser():
            valor = funcion(*list_args)
        else:
            value = ('You must be logged')
        return value
    return _private

Really simple and clean. But in PHP I don’t know how to do it in a simple way. I don’t want to do reflections in every call and check if the function is allowed or not.

My solution:

Instead of having one class with public and private functions I divide it in two classes: one for the public function and another one for the private (private here means logged sessions not private keyword in PHP).
I also create two empty Interfaces PublicAccess and PrivateAccess and using PHP’s instanceof I can throw an exception if user without login tries to call any function in a class with the interface PrivateAccess

interface PublicAccess{}
interface PrivateAccess{}

class Lib_Myclass1 implements PublicAccess
{
    public function publicFunction($var1)
    {
        return "Hello {$var1}";
    }
}

class Lib_Myclass2 implements PrivateAccess
{

    public function privateFunction($var1)
    {
        return "private Hello {$var1}";
    }
}

I prefer the python’s solution

An idea for calling PostgreSQL’s stored procedures with PDO

As far as I know PDO doesn’t allow to call directly the  PostgreSQL’s strored procedures. That’s not a problem. We can create a SQL and call a stored procedures as simple sql.

Imagine we have a stored procedure in the schema called ’schemaName’ with the name ‘method1′.

CREATE OR REPLACE FUNCTION schemaName.method1(param1 numeric, param2 numeric)
  RETURNS numeric AS
$BODY$
BEGIN
   RETURN param1 + param2;
END;
$BODY$
  LANGUAGE 'plpgsql' VOLATILE
  COST 100;

The way of call it is something like this:

$conn = new PDO($dsn, $user, $password);
$conn->beginTransaction();
$stmt = $this->prepare("SELECT * FROM schemaName.method1(?, ?)");
$stmt->execute(1, 2);
$stmt->setFetchMode(PDO::FETCH_ASSOC);
$out = $stmt->fetchAll();
$conn->commit();

An idea for doing the same in a more clean way is:

$conn = new MyPDO($dsn, $user, $password);
$conn->beginTransaction();
$out = $conn->setSchema('schemaName')->method1(1, 2);
$conn->commit();

That’s only an approach. I haven’t think a lot about it but that’s OK as a first approach.

And now the class I’ve created extending PDO to obtain the above interface.

The trick is in __call function. Using __call I have dynamic functions in my MyPDO class and I will suppose every functions will be stored procedures.

class MyPDO extends PDO
{
    private $_schema = null; 

    /**
     * Set Schema
     *
     * @return MyPDO
     */
    public function setSchema($_squemaName)
    {
        $this->_schema = $_squemaName;
        return $this;
    } 

    function __call($method, $arguments)
    {
        $_params = array();
        if (count($arguments)>0) {
            for ($i=0; $i<count($arguments); $i++) {
                $_params[] = '?';
            }
        } 

        $stmt = $this->prepare("SELECT * FROM {$this->_schema}.{$method}(" .
            implode(', ', $_params) .  ")");
        $stmt->execute($arguments);
        $stmt->setFetchMode(PDO::FETCH_ASSOC);
        return $stmt->fetchAll();
    }
}

Moving singleton and factory patterns to Abstract with php 5.3

I have built a backend library. I have a tree of classes and i want to use singleton and factory patterns to my class set. Easy isn’t it?

My dummy class

class Lib_Myclass
{
    public function function1($var1)
    {
        return $var1;
    }
}

new instance of my class

$obj = new Lib_Myclass;
$obj->function1('hi');

Ok. Nice OO tutorial Gonzalo, but I want to use factory pattern so:

class Lib_Myclass
{
    static function factory()
    {
        return new Lib_Myclass;
    }
    public function function1($var1)
    {
        return $var1;
    }
}

Lib_Myclass::factory()->function1('hi');

Now imagine you have a lot of classes. You must create over and over the factory function in every classes. OK you can create an abstract class and extend all classes with your abstract class but …

abstract class AbstractClass
{
    static function factory()
    {
        return new Lib_Myclass; // <---- what's the name of the class?
    }
}

Whats is the name of the class you will use when creating the new object? You can use __CLASS__ but it only works when you are in the parent class. In an extended class __CLASS__ points to the name of the abstract class and not the parent one. With php < 5.3 there are some tricks for doing it but those tricks are very ugly. In php 5.3 we have get_called_class. So now we can create an abstract class with or factory and singleton implementations and extend our classes without adding any extra code over and over again

abstract class AbstractClass
{
    static $_instance = null;

    static function singleton()
    {
        if (is_null(self::$_instance)) {
            self::$_instance = self::factory();
        }
        return self::$_instance;
    }

    static function factory()
    {
        $class = get_called_class();
        return new $class;
    }
}

And now or class:

class Lib_Myclass extends AbstractClass
{
    public function function1($var1)
    {
        return $var1;
    }
}

And use patterns:

<?php
Lib_Myclass::factory()->function1('hi');

Lib_Myclass::singleton()->function1('hi');

Clean and simple. I like it.

Should we use our own frameworks in a production system?

Some days ago I read an article called why every developer should write their own framework. It’s an interesting post and everybody are agree with the author. Build a framework is a good way to learn other frameworks. You face problems and you give solutions to those problems similar than other framework’s ones. It’s also good see the code of other frameworks to see the differences. Definitely I am fully agree with Brandon.

But my question is more complicated. Should we use our own frameworks in a production system? I think the answer is not as clear as the fact of building or own framework. I will try to answer (or at least give my answer) in two scenarios. One as a developer and another one as an IT Manager.

Answer for a developer

As a developer is a difficult answer. We are agree build our own framework is a good exercise but use it in a production system?. When you build your own framework you are the biggest expert in your framework. You don’t need to read books to learn your framework. You need to read books to take ideas and learn more things from another ones but not on your framework. That’s a good point. But you must code a lot. You must code solutions that other frameworks has done already implemented. So your own framework speeds up the learning curve (when the framework is done or at least done) but slows down the developing time when you need to code core functions. If your framework is finished and it works under all situations and all possible future possibilities had been taken into account, the use of your own framework is the best idea. But your own framework will be never finished. It will be always under construction so the answer is not easy. I also don’t like to develop solutions for non-existing problems. I prefer to develop the solution when the problem appear. I like think in possible problems and think in a possible solution but not code the solution until the problem appear. Maybe I can can use a sandbox to develop some ideas but not add them to the framework. so at least for me my frameworks are never finished.

As a developer another question appear. Does my framework improve my resume. I’m not a person who learn things only to improve my resume but, like or not, resume is important. If you are an expert in Zend, Cake or Symfony you will have maybe more possibilities getting a job. It will be very difficult to find a job offer looking for an expert in your own framework. isn’t it? But If you build your own framework and if you are in touch with the frameworks of the market (you read articles, books, and you play a bit with them) it will be very easy to adapt yourself to another exiting framework.

Answer for an IT Manager

If you are an IT Manager and you need to choose between custom-made framework and existing one I will give you two possible answer. the sort one and the long one. The sort one is use an existing one. If you need to build an IT team it will be easy to hire developers if you chose an existing framework. Zend, Cake, Symfony are good frameworks. They are open source and is easy find developers. Of course you must chose only open source frameworks. They are good enough, widely used and free. Free as free beer. Something very important nowadays.

Now the long answer: Depend on your team. If you don’t have a team and you need to hire use the sort answer and use an existing one. But if you have a team the answer is not as easy as if you don’t have it. Home made framework means the knowledge is in-house. Home made framework means you don’t need to paid any external consultant to solve anything or to add any new functionality. But home-made framework means you cannot do those things because nobody outside your team knows your framework. If you need to hire more developers you will need to train them and probably the documentation of your framework will be very poor (or maybe it doesn’t exits) so the answer is difficult.

Another important fact is: When we build our own things, those things are something like our children and we work with them in a different way than if our boss imposes those things to us as a mandatory things.

Conclusion

There are two possible answer: yes and not. Use one or another according to your personal situation. Probably you will chose a wrong answer but remember its better to choose one than stay without doing anything.

Soy rico

Lunes 7:00. Suena el despertador. Normalmente cuando suena el despertador es el peor momento del día. Hoy encima me ha sentado especialmente mal (lunes, vacaciones recientes, …). Llego a la oficina, miro el correo y me encuentro con esto:

Hello,

I am Kun Chun, a legal practitioner. A defunct client of mine, that bears the same last name as yours, passed away due a heart-related condition on January 12th 2005. His heart condition was as a result of the death of all the members of his kinsfolk in the tsunami disaster on the 26th December 2004 in Sumatra Indonesia.

I am contacting you to seek your consent to present you as a successor to my late client, who has a deposit of $19million. This will be executed under a legal arrangement that will protect you against any legal ramification. If this business proposition offends your moral values, do accept my apology.

I can be reached on: xxx@xxx.xx

Best regards,

Kun Chun

Pues eso: un pobre hombre se muere en el tsunami de Sumatra en el 2004. Ese pobre hombre (descanse en paz) se apellidaba igual que yo (mira tu que casualidad) por lo que paso a ser su sucesor heredando una fortuna de 19 millones de dólares (ole). Y yo sin saberlo. Menos mal que Kun Chun se ha percatado de ello y me lo hace saber mandandome un email.

Internet es genial. Hoy he ganado 19 millones de dólares sin hacer nada. Entiendo que al amigo Kun tendré que pagarle algo pero pida lo que pida se lo merecerá. Menos mal que este correo ha pasado el filtro antispam porque de otra manera no me habría enterado.