Iterating over Multiple Iterators at Once

PHP’s SPL has two build-in Iterators that deal with multiple iterators at once: AppendIterator and MultipleIterator. In this posting I’ll cover both a bit:

• AppendIterator will put one iterator after the other so it get’s like a long queue or string. (PHP 5 >= 5.1.0)
• MultipleIterator iterates over all attached iterators at once, so to say in parallel. (PHP 5 >= 5.3.0)

The AppendIterator

As appending is straight forward, it does not need much description, just some example to get into the materia:

$iterator1 = new ArrayIterator(range('A', 'C'));
$iterator2 = new ArrayIterator(range('D', 'F'));

$multiple = new AppendIterator();
$multiple->append($iterator1);
$multiple->append($iterator2);

foreach ($multiple as $key => $value) {
    $iteratorIndex = $multiple->getIteratorIndex();
    printf(" * (#%d)[%s] => %s\n", $iteratorIndex, $key, $value);
}

The output is pretty self explanary:

 * (#0)[0] => A
 * (#0)[1] => B
 * (#0)[2] => C
 * (#1)[0] => D
 * (#1)[1] => E
 * (#1)[2] => F

There is not much fancy stuff here. You need to use append() for each iterator you want to append. Additionally it’s possible to obtain the inner iterator that is currently iterated over with getInnerIterator() similar to other standard iterators. Also it’s possible to get all iterators via getArrayIterator(). If any of those iterators are modified while iterating over them, they become changed. That’s also usual for objects and iterators are just objects. So really nothing fancy here, a bit more advanced is the MultipleIterator which comes next.

The MultipleIterator

The MultipleIterator is build similar to the AppendIterator, but iterators are attached instead of appended:

$multiple = new MultipleIterator();
$multiple->attachIterator($iterator1);
$multiple->attachIterator($iterator2);

As it iterates over multiple iterators at once, things aren’t that straight forward. For example, the index or key of an element: Which one is it? By default, MultipleIterator will return an Array for both the current keys and values. As PHP does only allows an integer value or string as a “key”, the following example spits a warning:

Warning: Illegal type returned from MultipleIterator::key()

foreach ($multiple as $key => $value) {
    printf(" * [%s] => %s\n", $key, $value);
}

Because by default MultipleIterator returns an invalid key (array), the warning appears and it is converted to 0. To obtain keys as well, the key() method needs to be called:

foreach ($multiple as $value) {
    $key = $multiple->key();
    printf(" * [%s] => %s\n", implode(', ', $key), implode(', ', $value));
}

As both $key and $value are an Array using implode here visualized what happens behind the scene:

 * [0, 0] => A, D
 * [1, 1] => B, E
 * [2, 2] => C, F

Note that implode hides the information about the keys of those returned arrays. To make them visible I used a small helper function called sfmta to display the array information properly:

foreach ($multiple as $value) {
    $key = $multiple->key();
    printf(" * key(%s)  =>  value(%s)\n", sfmta($key), sfmta($value));
}

/**
 * convert array to string (single-line style)
 * @param array $array
 * @return string
 */
function sfmta(array $array) {
    $glue = ', ';
    foreach($array as $key => &$element) {
        $element = sprintf('[%s] => %s', $key, $element);
    }
    unset($element);
    return implode($glue, $array);
}

The output now reveals the keys of the array for key and value as well:

 * key([0] => 0, [1] => 0)  =>  value([0] => A, [1] => D)
 * key([0] => 1, [1] => 1)  =>  value([0] => B, [1] => E)
 * key([0] => 2, [1] => 2)  =>  value([0] => C, [1] => F)

This shows, that by default the keys are the zero-based position. This can be changed by using one of the MultipleIterator::MIT_KEYS_... flags:

• MultipleIterator::MIT_KEYS_NUMERIC is the default one, it will return a numeric index (position), as usual 0-based -OR-
• MultipleIterator::MIT_KEYS_ASSOC which will take the key value from the subiterator.

To make use of MultipleIterator::MIT_KEYS_ASSOC, the iterators needs to be attached with an integer or string value for the info parameter of attachIterator() (the second parameter). If set to MIT_KEYS_ASSOC that associated information will be taken. Example:

$multiple = new MultipleIterator(MultipleIterator::MIT_KEYS_ASSOC);
$multiple->attachIterator($iterator1, 'foo');
$multiple->attachIterator($iterator2, 'bar');

foreach ($multiple as $value) {
    $key = $multiple->key();
    printf(" * key(%s)  =>  value(%s)\n", sfmta($key), sfmta($value));
}

As for the first iterator 'foo' and for the second 'bar' is used for the info parameter and the assoc-flag is set, the output now contains those as keys:

 * key([foo] => 0, [bar] => 0)  =>  value([foo] => A, [bar] => D)
 * key([foo] => 1, [bar] => 1)  =>  value([foo] => B, [bar] => E)
 * key([foo] => 2, [bar] => 2)  =>  value([foo] => C, [bar] => F)

If you don’t set the info parameter for one or more of the attached iterators and you make use of MIT_KEYS_ASSOC you’ll get an exception as the data is missing (NULL):

Fatal error: Uncaught exception ‘InvalidArgumentException’ with message ‘Sub-Iterator is associated with NULL’

Next to this key related behaviour, a question is what happens if one of the iterators is larger than the other? like A, B instead of A, B, C for the first iterator? By default the MultipleIterator will only return if all iterators are valid. The following output shows this, the last [2] => F element of the second iterator is just gone:

 * key([0] => 0, [1] => 0)  =>  value([0] => A, [1] => D)
 * key([0] => 1, [1] => 1)  =>  value([0] => B, [1] => E)

This behaviour can be changed with another pair of flags, one of the two MultipleIterator::MIT_NEED... ones:

• MultipleIterator::MIT_NEED_ALL is the default one, it will invalidate the iteration if not all iterators are valid any longer -OR-
• MultipleIterator::MIT_NEED_ANY will as long as at least one iterator is (still) valid.

A short MultipleIterator::MIT_NEED_ANY example:

$iterator1 = new ArrayIterator(range('A', 'B'));
$iterator2 = new ArrayIterator(range('D', 'F'));

$multiple = new MultipleIterator(MultipleIterator::MIT_NEED_ANY);
$multiple->attachIterator($iterator1);
$multiple->attachIterator($iterator2);

foreach ($multiple as $value) {
    $key = $multiple->key();
    printf(" * key(%s)  =>  value(%s)\n", sfmta($key), sfmta($value));
}

Output:

 * key([0] => 0, [1] => 0)  =>  value([0] => A, [1] => D)
 * key([0] => 1, [1] => 1)  =>  value([0] => B, [1] => E)
 * key([0] => , [1] => 2)  =>  value([0] => , [1] => F)

The “missing” values in the output are actually NULL. As array keys can not be NULL, it’s safe to check with those if a sub-iterator did provide a value or not.

There is another pair of flags, that one controls what will be taken as keys.

All flags are listed in the PHP Manual as well. They can be changed while iterating with the setFlags() and getFlags() returns the current flags.

Different to AppendIterator it’s not possible with this iterator to actually retrieve the list of iterators that are attached. As you can already see with the naming differences append() vs. attachIterator() both iterators are not streamlined with each other strictly, probably because they have been implemented in different times (PHP 5.1 and PHP 5.3). And there are other differences as well: AppendIterator internally uses an ArrayObject to store all iterators, while MultipleIterator uses an SplObjectStorage. The first allows read access to the list, the second doesn’t.

The “lost” DualIterator

As the last paragraph might sound a bit smart-ass, it’s actually some info I got from Marcus Börger’s blog who is responsible for (large parts of) PHP’s SPL: MultipleIterator for PHP. He also refers to a DualIterator which does not ship with PHP. A PHP variant of the DualIterator can be found in Börger’s SVN repository on Google Code.

Read On: Last time I was having “Some PHP Iterator Fun” by iterating multiple times over the same Iterator (28 Feb 2012). And a selective iteration over multiple iterators can be found in Merge two or more ArrayIterators by one of their values.

---

Backlinks: Hakre’s Blog: Iterating over Multiple Iterators at Once (by Chris Cornutt/PHPDeveloper.org; 16 Apr 2012) and Site News: Popular Posts for the Week of 04.20.2012.