Getting N Random Elements out of an Iterator – RandomIterator

hakre - random iterator

Considering there is an Iterator or Traversable with an unknown number of elements, I wondered if it is possible to get one or more random iterations out of it.

Continue reading

Posted in Developing, Hakre's Tips, PHP Development, Pressed | Tagged , , , , , , | Leave a comment

The SimpleXMLElement Magic Wonder World in PHP

PHP’s Simplexml ships with a lot of magic to simplify access to an XML documents element and attribute node values. Some criticize this and suggest to use the DOM library instead. The DOM library on the other hand, even it can do everything tend to be known with an XML document, it’s pretty verbose – and yes that’s some critique with XML as well, the verbosity. Sure there are many nice libraries around the DOM library and wrapping it and one of these libraries again is Simplexml.

From a data-type perspective, the SimpleXMLElement is quite an interesting one actually, literally I mean figuratively -what not. It’s something like a hierarchical data-structure. One that comes with it’s own query method via the xpath() method. It can be iterated, traversed, nodes added and leafs unset as if it would be an array or an std class. And it comes with a serializer built in – into XML – in both directions.

From it’s internals, it’s fully backed from C code below from libxml, it’s also pretty fast and perhaps also fine with the memory (at least I hope).

It speaks Unicode in the popular UTF-8 encoding you know from the web and if you need to, it can even convert to other encodings.

And one of it’s magic properties is that it’s such a class of classes in PHP that can be casted from one class to another. This works by converting one (subclass of a) SimpleXMLElement to another subclass of it by sending it through DOM (the besaid sister-library):

$foo  = new Foo("<doc/>");
$via  = dom_import_simplexml($foo);
$cast = simplexml_import_dom($via, 'Bar');
var_dump(get_class($cast)); # string(3) "Bar"

This is actually not only true for SimpleXMLElement but also to the node-classes in a DOMDocument to a certain degree but this post is about SimpleXMLElement so just saying.

I have to say it: With so much simplification and magic, there is a price to pay and there are limitations, too. The constructor is final, so you can’t override it. No way :). This hinders you in terms of “classic” object inheritance. One path out is to decorate the elements, but even I did this in the past, it doesn’t feel equally well as well. It might also be more work as first thought. But most often, extending SimpleXMLElement just more to sugar-in some methods, so it’s often not worth for a full-feature decoration. So this is a limitation. ERR_TOO_MUCH_MAGIC comes to mind.

And some argue as for the data-structure you can’t use it as array or object store as all class-properties or array-indexes represent either XML element or attribute nodes only accepting scalar types (actually stringy values).

Storing Arrays and Objects in a SimpleXMLElement

Let me elaborate on that last point a little. It’s normally not possible to store array or object data inside a SimpleXMLElement. As you couldn’t serialize it as XML, by default it’s fordidden to do:

class Foo extends SimpleXMLElement

$foo  = new Foo("<doc/>");
# Warning: It is not yet possible to assign complex types to properties
$foo->bar = $foo;

If you now think that creating a private field and assigning the data to the private field would be a solution, it will teach you about another limitation: there are no private fields with a SimpleXMLElement. It’s field are all exposed XML nodes so all you can store there are strings.

But wouldn’t it be nice to actually be able to store some objects therein? Let’s elaborate a bit on the internals which is how I discovered some nice properties of the document model in PHP and it’s use from within Simplexml.

The SimpleXMLElement is somewhat a shell around some other object only. It perhaps can be describben as a Flyweight (as in the pattern), an interface of factory and object manager of the underlying document nodes. And the document again can be represented as a DOMNode which again is a shell/interface around the underlying document node managed by libxml. This is the underlying structure of not only the SimpleXMLElements but also the tree structure of the DOM. The PHP SimpleXML/DOM extensions manage all these document nodes nicely for us.

If it is now possible to turn a SimpleXMLElement into a DOMNode it is then – because of the object model in PHP with the dynamic properties (every object in PHP is actually somewhat an array/hash) – possible to assing data to a document node without creating a new element as it would be the case on level of SimpleXMLElement:

class Foo extends SimpleXMLElement
    function setData($data) {
        $element = dom_import_simplexml($this);
        $element->data = $data;

$foo = new Foo("<doc/>");

This does actually work, but the data won’t yet persist. What is necessary to keep the dynamic property data here in memory within the DOM is to add as circular reference to the DOMNode, let’s call that one circref in this example. It’s then possible to write and read the data:


        function setData($data) {
            $element = dom_import_simplexml($this);            
            $element->data    = $data;
            $element->circref = $element;

        function getData() {
            $element = dom_import_simplexml($this);
            return $element->data;


The usage example demonstrates that it is now possible to store (or attach) an object to the document node accessed via Simplexml:

$foo = new Foo("<doc/>");


# class Foo#1 (1) {
#   public $bar =>
#   class Foo#2 (0) {
#   }
# }

For those who hate Simplexml but even read until here: As already written earlier, this same principle works with pure DOMDocument / DOMNode as well. Just in case you want to re-use the node based data-structure and you need to add (object) information to it. All you need is the circular reference to keep the association between the data and the node in memory. And it’s really within the same document:

# obtaining the data via DOMDocument
$doc = dom_import_simplexml($foo)->ownerDocument;

$bar = $doc->getElementsByTagName('bar')->item(0);

As you can imagine same applies for xpath queries – both via DOMXPath or SimpleXMLElement.

Posted in Developing, Hakre's Tips, PHP Development, Pressed, The Know Your Language Department, Uncategorized | Tagged , , , , | Leave a comment

Kubuntu Default Browser

The system-settings don’t reflect the whole picture in Kubuntu. Settings can be done via the command-line. Here exemplary to chromium:

gvfs-mime --set x-scheme-handler/http chromium-browser.desktop
gvfs-mime --set x-scheme-handler/https chromium-browser.desktop

I had an issue with this for quite some time, I finally found the answer via “opening web link from evolution uses wrong browser”.

Posted in Hakre's Tips, Pressed, Uncategorized | Leave a comment

Installing PDF Printer under Kubuntu with AppArmor and Samba

It actually should be as little as installing cups-pdf (CUPS-PDF (Ubuntuusers German)):

$ sudo apt-get install cups-pdf

Then opening Printers in system preferences did show the new printer.

However printing a test-page didn’t work. I then just followed the tail of the syslog:

# tail -f /var/log/syslog

to find out that app-armor blocked something:

Feb  9 09:47:15 producer25 kernel: [43208.761736] type=1400 audit(1423471635.886:111): apparmor="DENIED" operation="connect" profile="/usr/lib/cups/backend/cups-pdf" name="/run/samba/winbindd/pipe" pid=14195 comm="cups-pdf" requested_mask="rw" denied_mask="rw" fsuid=0 ouid=0

With this error message at hand it was easy to find a bug-report which offered a solution that worked for me directly:

add this to your profile (in the cups-pdf section):

/run/samba/winbindd/pipe rw,

then do this:

$ sudo apparmor_parser -r /etc/apparmor.d/usr.sbin.cupsd

After that I restarted cups.d and apparmor.d and it directly worked.

Printing the PDF

PDF files are stored under ~/PDF by default. If you print large files, it takes some time until the PDF file is written in full. Unless totally done, you (normally) can’t open it successfully. There is no notification given (like with compressing files for example) when the print job as been done and the PDF is fully written. There is also no temporary name. I didn’t look into the configuration of Cups-PDF deeply as I only wanted to get it to the run quickly, so just sharing the experiences, it’s probably worth a feature-request when I find the time.

Posted in Hakre's Tips | Tagged , , , | Leave a comment

Hide Folders in PhpStorm Project Pane

How can I hide a directory in a PHPStorm project, for example .sass-cache I just asked myself this morning.

Searching online didn’t reveal it to me and there are numerious options that were close but not about hiding the directory from the project panel.

Finally a co-worker pointed me into the right direction, there are two things to do under circumstances:

  1. Exclude the directory: Mark the directory itself as excluded (Right-Click -> Mark Directory As -> Excluded). It might then already disappear
  2. Hide excluded files: Tick the project pane option (under the Gear-Wheel symbol) and un-tick Show Excluded Files to hide excluded directories

and that’s it.

However: Showing the .idea folder doesn’t work this way (WI-26391).

Posted in Developing, Hakre's Tips, Pressed | Tagged | Leave a comment

Just linked: Learning OOP in PHP

Link | Posted on by | Tagged , | Leave a comment

Download NextGen Gallery WordPress Gallery with Wget

For a gallery download with Wget made by NextGen Gallery (at least this is what I read from then nggpage=2 query string), I had success with:

wget -nc -nd -A '*.jpg' -R 'thumbs_*,index.html*' \
    -I /wp-content/gallery/ -r -l 1 \

My example gallery with with all jpeg files. The switches are documented in the GNU Wget Manual.

Posted in Hakre's Tips, Pressed | Tagged , , , | Leave a comment

Commandline Source Fixes Foo

From time to time I need to cleanup source trees. Today I needed to do that again, here are some command liners to get some work done.

If you’re on Windows, all you need to do to get these running is to install git for windows which has git bash and all the commands used in these examples.

Convert Line Endings / Line Separators

First of all it’s good to review if the find command-line actually finds the file looking for. E.g. to exclude some directories (here exemplary .git for git version control and .idea for Phpstorm and other Idea IDEs) and then list the file extensions that would be find:

find . \( -name '.git' -o -name '.idea' \) -prune -o \
    -type f -printf '%f\n' | awk -F . '{print $NF}' | sort -u

Example: List of file extensions

$ find . \( -name '.git' -o -name '.idea' \) -prune -o \
>     -type f -printf '%f\n' | awk -F . '{print $NF}' | sort -u

This shows it’s save to operate on these. Lets ensure all line-endings are unix and not dos:

find . \( -name '.git' -o -name '.idea' \) -prune -o \
    -type f -exec dos2unix -bUvt {} \; 2>&1

This executes dos2unix (here in test-mode, remove -t switch to apply changes) on each file (redirecting stderr to stdout so it’s easier to grep or less). Dos2unix allows more conversions, use dos2unix --help more more info.

Something new I tried today was to apply such a command only onto files that have been touched by the last commit. In a clean staging area after that commit, I could apply dos2unix with the help of git diff-tree and a Bash loop:

git diff-tree --no-commit-id --name-only -r HEAD \
    | while read line ; do dos2unix -bUvt "$line" ; done

So instead of the find operation, I create a list of files to operate on with git and then a while read line ; do ; done loop invokes the command.

\ No newline at end of file

Another common change to apply is to add newlines at the end of files. Some background information about why a newline at the end of file is useful is given in Sanitizing files with no trailing newline (May 2010; by waldner). This one was not so easy for me to find as I wanted to invoke it again via find, but I finally made it working like a charm with the help of sed and the nice Gnu extension of -i (edit file in place) it has – to great extend because of How to add a newline to the end of a file? (Unix & Linux SE):

find . \( -name '.git' -o -name '.idea' -o -name 'vendor' \) -prune -o \
    -type f -exec grep -Iq . {} \; -exec sed -i -e '$a\' {} \;

this does not produce any output, but you can review the changes then with git diff. All those \ No newline at end of file should be gone then.

As usual, keep a backup before running modifications over a whole directory tree automated. Take care to not traverse into directories where you don’t want to.

Posted in Developing, Hakre's Tips, Pressed, Tools | Tagged , , , , , , | Leave a comment

Composer Clear Cache

Composer The Cache

It’s one of the best kept secrets of popular PHP dependency manager Composer: How to flush composers cache.

You normally don’t need it, however if you create some composer.json and you want to put it to a test, this can be useful to know. Or let’s imagine your composer cache grows some gigabytes large. Here are two ways:

First, you can just nuke it from above:

$ rm -rf "`composer config cache-dir`"

(if you’re using Windows, use git-bash.)

The second way is to tell Composer where to find the cache via environment variables:

$ COMPOSER_CACHE_DIR=/dev/null composer install

This second method is perhaps better if in your tests you don’t want to influence the whole system.

The third but not yet available method is to make use of a composer command. This is discussed in the feature request Add command to clear composer cache which also exchanges some arguments pro and con. But I though I spare that for this little TLDR; type of blog-post.

Update 5 Jun: Just in a recent Pull-Request “Add clear cache command” (#3034) such a concrete command has been offered to merge by David Neilsen.

Update 14 Jul: See as well If all else fails Slide of Using composer correctly (confoo) (26 Feb 2014 by Igor Wiedler).

Posted in Developing, Hakre's Tips, PHP Development | Tagged , , , , , | Leave a comment

Some nice line-up for the know your language department: PHP turtles – Turtles all the way down.

Link | Posted on by | Tagged | Leave a comment

XPath Null Byte Injection in PHP

Back in July this year, in Mitigating XPath Injection Attacks in PHP I was writing about how to properly quote a string in PHP’s Xpath 1.0.

The code presented there was based on the assumption that the resulting expression is binary safe.

However that was too shortsighted because Xpath in PHP can be attacked using null-byte-injection. The PHP extension does cut-off the string at the first null-byte, allowing you to truncate an expression early.

/*/user[name = 'Mirza']/secret<NUL>]/location

Technically XML covers the full Unicode repertoire excluding the surrogate blocks FFFE and FFFF and excluding most US-ASCII control characters (those below space), only Tab, Line-Feed (LF) and Carriage-Return (CR) are allowed in XML.

This is also the reason when you need to safely transport binary data with XML, that you need to encode it, for example in base64 (See base64Binary primitive XML datatype), because otherwise the XML would be broken resulting in data-loss.

Back to the mentioned XPath injection attacks and how to mitigate them. If an injected string is able to cut-off at the first null-byte position, the quoting as described does not work stable any longer. An attacker can break out of it by injecting a null-byte. The impact is not very high, because of the quoting that xpath_string() applies, injecting a null-byte will result in a Unfinished literal warning.

However when data is injected not as string with the help of xpath_string(), null-bytes do still play against you in PHP Xpath. As those are not valid anyway in XML and therefore no text or identifiers can contain it, you can safely reject or sanitze null-bytes further up in the input processing. For example as Suhosin can do.

So better keep in mind to verify incoming (Unicode) data your application accepts. Even valid Unicode, it might not always be appropriate.

See Also:

Posted in Hakre's Tips, PHP Development, Pressed, Surviving the Internet | Tagged , , , , , , , | Leave a comment

Devil’s Dictionary of Programming

Link | Posted on by | Leave a comment

Professional Webdevelopers At Work – Yahoo Mail Endless Redirect Demonstration

So familiar with these 1996 Web-Technologies but not having the time to care in these rushing 201x days as this two minute documentary of an endless redirect-chain shows. Thanks to random URL parameters used to prevent ancient caching woes in combination with cookies – but failing to test if cookies actually work on the target domain. (Recordered 2013-10-26)

Posted in Professional Webdevelopers At Work | Tagged , , , , , , , | Leave a comment – GitHub mirrors of every plugin in the plugin repository

Just a little follow up to Your Guide to Composer in WordPress as I was stumbling over while surfing (and equally short just for the log):

Bryan Petty (tierra) was so kind to mirror the WordPress Source/Development branch on Github, it is here: tierra/wordpress. While I was stumbling over it, I also discovered another project he is involved in: – GitHub mirrors of every plugin in the plugin repository.

The Plugin Developer Guide has some details how it works. Full source of it is also available on Github: WordPress Plugins GitHub Mirror Application.

Interestingly there is also WordPress Plugin Tests but I didn’t had the time to review it, it perhaps makes sense.

Aside | Posted on by | Tagged , , , , , | Leave a comment

Ircmaxell’s Rambling On Internals

Ircmaxell’s Rambling On Internals raises a very important point about the use of RFCs in the PHP community and the problem they have been introduced as a tool to only negotiate – not solve – the problems of the PHP Internals list.

His arguments are as always pretty weighted and need more voice, so read and spread the word. He has my support and it’s a loss for PHP and Internals as a whole should stop sitting on their legs.

I’m not so good with arguing, so I prefer to share my opinion because that is the least I can do to not let this pass unnoticed (which would be an even bigger mistake):

In my personal opinion Pierre and Stas suck most (now I said it). And that is my personal opinion. Pierre weights harder because I’ve met him in person and he has a split tongue, all he feared in that discussion was totally untrue and never happened. All he promised to do otherwise didn’t happen either. So only hot air for nothing, just for the sake of influencing others for technical arguments – not reality. Ircmaxell on the other hand not only explains he is also a do-er.

From Stas I’m just getting ill by reading that much text only because he has the time to write so many emails whole day long. I wish he would go down with us in the swamps of the PHP tag on Stackoverflow, perhaps that will envision him. It would be a benefit for the PHP community as a whole and he wouldn’t have so much time writing emails in Internals. Now that’s a productive suggestion I’d say.

I know it’s hard to run a project, especially for years and with a big userbase. But seeing Ircmaxell leaving with no further action from the Internals community itself is not excused by that.

Just my 2 cents, share yours.

See as well: I don’t understand PHP beaurocracy but I do understand Anthony Ferrara!.

Posted in Linked, Pressed | Tagged , | Leave a comment

Free The Cuban Five! 12. September 2013

Especially for our German speaking visitors: Unterstützen und Verbreiten –

And for our US-visitors: Miami, FL, Sept. 8 | Washington, DC, Sept. 12 (White House) | Washington, DC, Sep 13 (University of the District of Columbia Law School)

Posted in Linked, Pressed, Uncategorized | Tagged , | Leave a comment

Greeting, Greetings and the GreetingFactory

Just stumbled over: If you ask Is this correct object oriented programing in php? and then get an answer from Gordon, well, see for yourself. (via)

Posted in Linked, Pressed | Tagged , | 1 Comment

The Sky. The Universe. The Missing Unit Tests

Out there in the Universe. Now comparisons get Epic Pictures when it comes to WordPress and Unit Tests: Beta Sagittae. via The Loop.

Reminded me of Development By The Numbers – Slides (May 2013; by ircmaxell) having also nice comparisons for numbers.

Posted in Code Smells, Hacking The Core, Linked, PHP Code Smells, Pressed | Tagged , , , , , | Leave a comment

Your Guide to Composer in WordPress

Your Guide to Composer in WordPress and there is WordPress Packagist. I only knew about Composer Installers (incl. WordPress ones) so far.

Aside | Posted on by | Tagged , , | Leave a comment

The Negative Influence of WordPress on PHP

The current The TIOBE Programming Community Index for July 2013 shows an increase for PHP, gaining grounds fast and as an ongoing trend over the last year:

If compared to January 2013, PHP is the fastest climber with an increase of +1.64% […]. The major driver behind PHP’s popularity seems to be the new PHP Zend Framework that was released in September 2012.

It clearly shows the stamina and power PHP as a programming language has, with the two recent milestones of the two popular PHP 5.4 and 5.5 releases. Those new releases are a key driver for next generation frameworks like the bespoken Zend Framework 2.

PHP is strong standing against negative influences popular but legacy PHP applications put onto it, namely and most foremost the most popular of all these: WordPress. WordPress is continuously bringing down PHP since years as Google Trends shows:

As this Google Trends graph revals, WordPress popularity is constantly hurting PHP

As this Google Trends graph revals, WordPress popularity is constantly hurting PHP

So how long can PHP resist against this bad influence? What will happen when those two lines cross? Will the world as we know it fall apart?

You probably have come to the conclusion that comparing two independent statistics allows you to draw all kind of crazy assumptions – so do I. What has been outlined above is pure irony as you might have already noticed (but it’s said that irony does not work well in the internet, so you probably didn’t even notice).

Manuel Lemos (Google Profile) from yesterday was spreading his opinion that WordPress has made PHP popular – not PHP and not any PHP Frameworks.

Well, make your own mind, what I just wanted to show is that running wild assumptions normally does work even badlier than irony in the internets. And comparing two totally unrelated statistics (“small lies, big lies, …”) shows more about your own opinion than anything else.

My 2 cents.

Posted in Hakre's Tips, Linked, Pressed, Surviving the Internet | Tagged , , , , , | 2 Comments

Atomic deploys at Etsy

Link | Posted on by | Tagged , , | Leave a comment