Getting N Random Elements out of an Iterator – RandomIterator

Considering there is an Iterator or Traversable with an unknown number of elements, I wondered if it is possible to get one or more random iterations out of it.

PHP on Google App Engine – Quick First Review

This is fresh out of the news (Ars; Wired; WHIR; VB), and what Google wants to offer looks like a very cool package. So if you hadn’t had the time to view the video, here is a quick summary and some first comments. Please mind that this is really fresh.

It’s basically PHP 5.4 in a “hardened edition”^[1]. I think starting with PHP 5.4 on going live is a great achievement here, so this gets my PHP Applicatoratores Badge.

Also the extensions offered so far give a nice outlook, e.g. you find ZLlib and GD in there for example so you don’t hit a show stopper that fast. I name those two specifically because they became a burden with the CV-Backlog example application installment on Heroku.

However, this selection of extensions has somewhat to be improved. Default PHP extensions like iconv which actually need to be active to have other extensions properly work are not yet part of the Google PHP Runtime. To give a practical example with this iconv one, as it has got the DOM extensions activated but not the iconv extension, there is not much to deal with character encodings for our beloved DOMDocument:

$doc = new DOMDocument();
$doc->loadXML('<?xml version="1.0" encoding="Windows-1252"?><root />');
$doc->documentElement->appendChild($doc->createElement('füürüü', 'ärööö'));
echo $doc->saveXML();

This little example leaves you with a recoding related error, and no, users don’t love these (and sadly many PHP developers won’t even understand it either):

DOMDocument::saveXML(): output conversion failed due to conv error, bytes 0xFC 0xFC 0x72 0xFC
<?xml version="1.0" encoding="Windows-1252"?>
<root><f

So this needs feedback from the community I think to get the rough edges out. That one here is to make the faux-pas to not deliver PHP with iconv. But I think by the spirit of the video, these things are expected to be cleaned out after they get reported to the PHP Runtime Team @ Google (Ref: Issue 9340: ICONV PHP extension support).

For your encryption needs, this is not that harsh, even mcrypt is missing openssl is available.

phpinfo() in Google App Engine

However extension are not everything (and obviously something more easily to fix I assume) and these other parts are quite well solved for the PHP worlds in PHP Runtime for Google App Engine. I would say this is where it actually already shines with the now available preview.

That is: The mysql libraries work out of the box with Google CloudSQL, a Mysql 5.5 compliant managed database server on the App Engine cloud.

/* FIXME this needs a PDO example */
$connection = mysql_connect(
    $host = ":/cloudsql/project_name:instance_name", $username, $password
);

So this is quasi with zero code-changes to get an existing application run that uses any of the three Mysql client libraries (mysql_*, mysqli_*, PDO).

Similarly easy have been the problems of file storage solved. Like with other cloud-appserver-platforms, in Google App Engine you can’t change application files (and yes, your live-application deployment should not even expect to do so [yes I look at you WordPress]). However you need a place to store files to, for example for file-uploads. Google offers here something named Google Cloud Storage and it’s easily integrate with PHP via stream-wrappers. This equally should allow to port a PHP app with none or very little code-changes and especially with easy to be done changes:

$text = file_get_contents("gs://my_bucket/shakespeare.txt");

So this is really nifty, because this comes out of the box. And btw, both these features work with the development server for App Engine, so you can develop and test your apps.

Another goodie App Engine comes with is Memcached (The video has a WordPress demo that runs Batcache and this worked out of the box for example, WordPressians know what this means). And there’s Task Queue, a goodie (and fully integrated for PHP tasks now) that allows you to schedule long-taking actions inside App Engine. And there are more services in App Engine.

Summary

This Post is just a little summary of the video and some little tests I could run so far, so really a quick review. I think the package Google offers here is heading right into the right direction for todays PHP developers that are looking for a cloud platform serving PHP needs. It’s good to see more alternatives here and if you’re interested for more starting from 27:30 Jason Cartwright shows step-by-step what they did to put a Drupal based real-life site on it, which I think should show pretty well about the pros/cons such a deployment has but also how it’s done.

Compared with Heroku for PHP, I’d say Google stands out because they really put some love into getting PHP as runtime there – for the out-of-the-box experience. So I can only give the best wishes for their launch with this quick review and hope for a good working together with the PHP community. Compared with Heroku again this is necessary because different to Heroku where you can actually compile what you need, this is not possible with Google App Engine. You’re bound to the Runtime they “allow you to use”. As you know, Vendor Lock-In can have multiple Angles, so take care for directions you want to head to and try before you buy.

I will for sure give this a test-drive and let you know how it worked out. Maybe the CV-Backlog sample application with memcached temp-storage?

Resources

• Google App Engine for PHP – Limited Preview Sign-Up (Google)
• Interactive server-side PHP shell for Google App Engine (Google Demo)
• Google I/O 2013 – A New Language for App Engine (Youtube Video)
• Getting Started: PHP (Google Developers Documentation)
• PHP Overview on Google App Engine (Google Developers Documentation)

Jetbrains also fired-up some support for it close to the launch, a plugin for Phpstorm:

• Support for Google App Engine PHP in PhpStorm (15 May 2013 by Mikhail Vink)
• Getting Started with PhpStorm as Google App Engine PHP IDE
• Getting Started with Google App Engine PHP in PhpStorm (Youtube Video)

---

[1] Google did not specify what “hardened” actually means, I would say this translates well that the codebase has been slightly changed in a willing to improve it so you *might* find some differences in behaviour at some edges, but I have no further information about this so far.

Update: There is a list of disabled PHP functions and the /e modifer (eval) from preg_replace has been removed.

Event-driven PHP – Igor Wiedler

Nice talk by Igor at TakeOff 2013: Event-driven PHP – Igor Wiedler.

The Greatest PHP Value

Just two days ago I asked a PHP-quiz-question in the chatroom on Stackoverflow, something along the lines:

PHP: Which one is greatest?

PHP has a comparison operator to compare if one value is greater than the other (>). Which one of the three values INF, array() and (object) array() is the greatest?

1. Infinity – INF
2. Array – array()
3. Object – (object) array()
4. undefined

and it really is a PHP Quiz. The undefined means that you can not find out if a single one is the largest. If you want to guess your own, you should not read further yet.

---

Even the Array is greater than Infinity in PHP, the Array is less than an Object. Now you could say that Object then must be the greatest, however the Object is less than Infinity.

So between these three values you can not formulate any expression in PHP that describes the greatest one therefore the answer is undefined.

// http://eval.in/15219
$inf    = INF;
$array  = array();
$object = (object) $array;

var_dump($object > $array  and $object > $inf  ); # bool(false)
var_dump($array  > $object and $array  > $inf  ); # bool(false)
var_dump($inf    > $object and $inf    > $array); # bool(false)

So maybe some of them is equal to another? No not as well .

So what are the practical implications of this? A simple example that is impacted by that is the sort() function. When a sorting behavior that does not do any type-conversion is used (default), then the result is based on the order in the array and not on the values. That might be unexpected.

As always when I’m puzzled with PHP I write a more or less well formulated question on Stackoverflow about it and ping NikiC. Because he not only answers the question but also makes me learn a lot of new terms and is so firm with PHP internals. See for yourself: PHP Type-Juggling and (strict) Greater/Lesser Than Comparisons which has got quite some traction already.

And if you’re using an older PHP version, the answer might not be undefined. Until PHP 5.1.6 Objects were greater than Infinity like Arrays are but less than Arrays allowing a clear winner: Array is greatest!

Read On: Comparison operators – PHP Sadness #52 – Very detailed and impressive description and visualization of comparison in PHP.

The Annotated XML Specification

The Annotated XML Specification (by Tim Bray; 1998)

Late Static Bindings in PHP Callbacks

No idea if this is ever useful, but just found this not documented in the PHP manual so far (and the Callbacks entry looks already chaotic so I don’t edit it right now):

You can write callbacks of static class methods as a string in the form of "classname::methodname". This should be commonly known, an example of that is:

// http://3v4l.org/vdhJq

echo Module::work('hello world'), "\n";              # HELLO WORLD
echo call_user_func('Module::work', 'hello world');  # HELLO WORLD

abstract class Module
{
    public static function work($it) {
        return strtoupper($it);
    }
}

What I stumbled over and which I didn’t know is that since PHP 5.3 it is also possible to reference the current class with three additional strings for the classname placeholder above:

1. "self::methodname" – use the current class.
2. "parent::methodname" – use the parent class.
3. "static::methodname" – use the class that was initially called at runtime.

Here an example that uses a callback function inside the module class to do some string replacements with the "static" string:

// http://3v4l.org/SsaZg

echo Module::work('hello world.'), "\n";  # [[[hello]]] [[[world]]].
echo MyModule::work('hello world.');      # {{{hello}}} {{{world}}}.

abstract class Module
{
    public static function work($it) {
		return preg_replace_callback('~\w+~', 'static::replace', $it);
    }

    protected static function replace($matches) {
        return sprintf('[[[%s]]]', $matches[0]);
    }
}

abstract class MyModule extends Module
{
    protected static function replace($matches) {
        return sprintf('{{{%s}}}', $matches[0]);
    }

}

Sidenote: The array callback notation works for that as well:

    preg_replace_callback('~\w+~', array('static', 'replace'), $it);

This came a bit unexpected to me because in another feature introduced in PHP 5.3: A call of static class methods with a variable for the classname PHP:

// http://3v4l.org/IWQtr

echo Module::work('hello world'), "\n";  # HELLO WORLD

abstract class Module
{
    public static function work($it) {
        $module = 'Module';
        return $module::strtoupper($it);
    }

    private static function strtoupper($it) {
		return strtoupper($it);
    }

}

Unexpected because it does not allow to use the strings "self", "parent" or "static" inside that variable:

// http://3v4l.org/p0RQv

echo Module::work('hello world'), "\n";  # Fatal error: Class 'self' not found

abstract class Module
{
    public static function work($it) {
        $module = 'self';
        return $module::strtoupper($it);
    }

    private static function strtoupper($it) {
		return strtoupper($it);
    }

}

Last but not least, there is this very special callback, I don’t have any name for. What it does is working exactly like the "self::methodname" callback:

    
    preg_replace_callback('~\w+~', array('self',   'self::replace'),   $it);
                                         // like self - http://3v4l.org/B6Lbe

    preg_replace_callback('~\w+~', array('static', 'static::replace'), $it);
                                       // like static - http://3v4l.org/adTgD

    preg_replace_callback('~\w+~', array('static', 'self::replace'),   $it);
                                       // like static - http://3v4l.org/GaCVT

    preg_replace_callback('~\w+~', array('self',   'static::replace'), $it);
    // like "final" (invalid callback in LSB context) - http://3v4l.org/oBeNN

So much for the magic static and self-self life in callbacks and class method calls. PHP is not well known for language consistency and I put that one as well into the book. One should not exploit this static feature anyway and a module class should be abstract final probably but sadly, PHP does not know that.

This post is just another write-up for the know-your-language department and not a recommendation at all.

---

Edit: Parts of my first conclusion on self-self were wrong, it does not do any LSB, what I thought was LSB was in the first static::replace example was wrong. I also made that example more distinct.

SymfonyLive San Francisco 2012 – David Zuelke – Designing HTTP Interfaces and RESTful Web Services

Dive Into HTML5 (by Mark Pilgrim with contributions from others)

New Year – New Attacks on your Freedoms

This year is business as usual, join the protests on February 23. #idp13

The Noun Project

Read On: Visualizing Change: An Interview with The Noun Project (16 Jul 2012; by Valerie Casey)

Interview with John Allspaw of Etsy

PSF: Python trademark at risk in Europe: We need your help!

python

Please spread the word and help: Python trademark at risk in Europe: We need your help! – it’s easy to fight trademark issues if there is clear evidence – by many of us – what the name means for you or your company. Because trademark can’t top real-life . But it’s some legal hassle, so your support is needed.

How foreach actually works – PHP internals nicely explained by NikiC

SimpleXML Type Cheatsheet

A SimpleXMLElement can represent many different things, from an element, to a list of attributes or childelements.

Sometimes it’s good to know how to find out, especially with the magic the extension comes with. The following is a table with the different types and an expression for it, $element is the Simplexml element:

Type	Test
Element	$element->xpath('.') == array($element)
Attribute	$element[0] == $element and $element->xpath('.') != array($element)
Attributes	$element->attributes() === NULL
Elements	$element[0] != $element and $element->attributes() !== NULL
Single	$element[0] == $element
Empty List	$element[0] == NULL
Document Element	$element->xpath('/*') == array($element)

PHP Version Info: PHP 5.2.2 or higher needed.

Usage Example

Normaly dom_import_simplexml comes to the rescue to upgrade from SimpleXML to DOMDocument. It works for nearly any SimpleXMLElement object, but not for an empty attribute or element list. To prevent the warning, check:

$isEmpty = $element[0] == NULL;
$DOMNode = $isEmpty ? NULL : dom_import_simplexml($element);

Common Xpath Cheats

SimpleXML can be pretty limited, some of the missing features can be worked-around before upgrading to DOMDocument by making use of Xpath:

What	Xpath	Note
Parent Element	..	Works also for attribute
Document Element	/*

The SimpleXMLElement::xpath() method returns an array by default. The array can only contain element or attribute nodes, nothing else. On an empty list SimpleXMLElement the method returns NULL. A FALSE is returned in case the Xpath expression had an error. So you can safely cast to an array for streamlining. It returns an empty array if there is no result then.

Usage Example

Obtaining the document element which represents the SimpleXMLElement returned by new, simplexml_load_file() or simplexml_load_string(). For example if $element is an element somewhere in the XML but the full document should be output:

list($root) = $element->xpath('/*');
$root->asXML();

SimpleXML Line Separator

Is "\n" (LF, \x0A, 10), always. When the document element is output asXML() the first line is the XML declaration. The last line is empty (newline at the end of file).

---

See as well

How to tell apart SimpleXML objects representing element and attribute?
In SimpleXML, how can I add an existing SimpleXMLElement as a child element?

https://bugs.php.net/bug.php?id=52858

Just found a nice PDF (Kudo @jens-erat /via) about XPath worth to share the link along:

Xpath PDF – Lecture: XML Technologies Winter 2012/13 – Dr. Christian Grün

Google PDF Preview shorturl in case your browser does not like to open it (my Google Chrome had a hickup with it, Curl/Opera is fine): http://goo.gl/nqYTZ

PHP Autoload Invalid Classname Injection

There are many ways to attack an application, many are working by injecting some malicious data hoping to trigger a deserved action in the end.

Most of these are possible when input data is not properly sanitized. This can have connotation in the programming language used, a famous PHP example is with register globals for example or the famous magic quotes.

These are fairly old stories and nowadays using a recent PHP version does not come with these rough edges so you’re normally more safe. However, this also means you need to look more specifically for areas of your code where injection is still possible.

Due to some recent work I needed to review code for autoloader invalid classname injection (see class autoloading). I did not found much concrete information online which functions/code are able to trigger this. Somewhat a pity, so first of all here is a list of functions which are (many by default) triggering autoloading:

call_user_func()
call_user_func_array()
class_exists()
class_implements()
class_parents()
class_uses()
get_class_methods()
get_class_vars()
get_parent_class()
interface_exists()
is_a()
is_callable()
is_subclass_of()
method_exists()
property_exists()
spl_autoload_call()
trait_exists()

All but one of these functions filter the classname parameter (the first character if "\" – and only the first – is removed). Only the function (spl_autoload_call()) passes the parameter in raw – that is the first character always unchanged.

This list is likely not complete (e.g. getting objects from the database with PDO/Mysqli and classnames), feel free to add your findings as well via comments or your own posting. Some of these functions – depending on PHP version – can not be used for injection by default.

Further potentially exploitable code is instantiating the ReflectionClass or making use of variable classname instantiation via new like new $classname;. The latter would result in a fatal error, however if an autoloader could be exploited via injection, the exploit would work before the fatal error is triggered.

Interestingly, the unserialize() function with PHP’s default unserializer can not be used to inject invalid classnames, unserialisation fails in that case highlighting the first mismatching character offset in a class / namespace name per a warning. PHP just refuses to parse those while unserializing. Howver, take care here for anything that has it’s own serialization and __sleep / __wakeup can be injection points on their own.

Less surprisingly, the get_object_vars() function does not work for injection because it needs an object, not a class. Sometimes you don’t know with PHP, so I included it in my review.

Defense In Depth at Classloader Level

So what is a valid classname anyway? Back in 2010 when I wrote PHP Syntax Regulary Expressed I noted that the syntax of namespaces remains undefined in PHP documentation. Reading PHPs source at least let one assume that it’s the same as for the classname plus the namespace separator (I write assume because it’s all at the tokenizer/parser level depening on what T_STRING stands for).

So expressing a valid fully qualified classname (with optionally prefixed by a namespace separator) as PCRE regular expression could look like (two variants):

\\\\?(?>[a-zA-Z_\x7f-\xff][a-zA-Z0-9_\x7f-\xff]*\\\\)*[a-zA-Z_\x7f-\xff][a-zA-Z0-9_\x7f-\xff]*
\\\\?(?>[a-zA-Z_\x7f-\xff][a-zA-Z0-9_\x7f-\xff]*\\\\?)+

Inside an auto-loader, this can be quickly checked with preg_match():

$pattern = '(^\\\\?(?>[a-zA-Z_\x7f-\xff][a-zA-Z0-9_\x7f-\xff]*\\\\?)+$)';
$valid   = (bool) preg_match($pattern, $classname);

One might argue that because a prefix of the namespace separator \ is invalid and most of PHP function calls filter it away anyway, that it is more strict to exclude it from the regular expression:

$pattern = '(^(?>[a-zA-Z_\x7f-\xff][a-zA-Z0-9_\x7f-\xff]*\\\\?)+$)';
$valid   = (bool) preg_match($pattern, $classname);

Next to a regular expression I’m not aware of any way to validate a classname in PHP. I just filed a feature request #64185 because one function that actually could be able to do that is not yet able so: is_callable() with the $syntax_only parameter set to true.

Take care and let me know if you find some more functions worth to look when reviewing the code.

---

Image Credits: Citations from Along the River During Qingming Festival, 18th century remake of a 12th century original by Chinese artist Zhang Zeduan; via Trialsanderrors

Debug with Xdebug on the Commandline with Phpstorm and Git Bash

Enable Phpstorm Xdebug listening:

$ export XDEBUG_CONFIG="idekey=PHPSTORM"

Press the listen button inside Phpstorm to actually listen:

To disable Phpstorm Xdebug processing in Bash again:

$ unset XDEBUG_CONFIG

This (PHPSTORM) is the default Phpstorm IDE-Key. You don’t need to disable listening after the variable is unset.

Should work with any other UNIXoide shell as well.

See as well:

• Debugging php-cli scripts with xdebug and netbeans?
• Configuring PhpStorm + XDebug

EPL/GPL Commentary – Life at Eclipse (6 Apr 2010; by Mike Milinkovich)

The Open-By-Rule Benchmark

The case against Singletons in WordPress

PHP Bug #63769 file:// protocol does not support percent-encoded characters

PHP: Bug #63769 file:// protocol does not support percent-encoded characters