ZF2018-01: URL rewrite vulnerability

ZF2018-01: URL Rewrite vulnerability

zend-diactoros (and, by
extension, Expressive),
zend-http (and, by extension,
Zend Framework MVC projects),
and zend-feed (specifically, its
PubSubHubbub sub-component) each contain a potential URL rewrite exploit. In
each case, marshaling a request URI includes logic that introspects HTTP request
headers that are specific to a given server-side URL rewrite mechanism.

When these headers are present on systems not running the specific URL rewriting
mechanism, the logic would still trigger, allowing a malicious client or proxy
to emulate the headers to request arbitrary content.

Action Taken

In each of the affected components, we have removed support for the specific
request headers. Users can provide support within their applications to
re-instate the logic if they are using the specific URL rewrite mechanism; users
are encouraged to filter these headers in their web server prior to any rewrites
to ensure their validity.

The patch resolving the vulnerability is available in:

  • zend-diactoros, 1.8.4
  • zend-http, 2.8.1
  • zend-feed, 2.10.3

Zend Framework MVC, Apigility, and Expressive users will receive relevant
updated components via composer update.

We highly recommend all users of affected projects update immediately.

Acknowledgments

The Zend Framework team thanks the following for identifying the issues and
working with us to help protect its users:

Source: Zend security feed

Zend Framework/Homestead Integration

Last year, we wrote about using Laravel Homestead with ZF projects.
Today, we contributed some changes to Homestead to simplify setting it up to
serve Apigility, Expressive, and Zend Framework projects.

As of version 7.6.0, Homestead users can now define sites with any of the
following "type" values:

  • apigility
  • expressive
  • zf

When one of these values is used, Homestead will setup the nginx instance used
by Homestead to properly to work with the project.

Getting started

Much of what we detailed last year is still true:

  • You will need to add the laravel/homestead box to Vagrant: vagrant box add laravel/homestead.

  • You will likely want to use the vagrant-hostsupdater
    plugin to Vagrant to facilitate mapping the VM IP address and server name to
    your system hosts file: vagrant plugin install vagrant-hostsupdater.

  • You will need to temporarily add the laravel/homestead package as a
    development dependency to your application: composer require --dev laravel/homestead.

  • You will need to use the Homestead tooling to create a Vagrantfile and
    Homestead.yaml configuration file: ./vendor/bin/homestead make.

Configuring Homestead

Once you have your Homestead.yaml file created, you can edit it. The two things
we need specifically are:

  • A folder mapping the application root directory to a directory in the vagrant
    image.
  • A site definition.

Generally, the folder mapping is already present, and will look something like
the following:

folders:
  - map: /home/username/dev/application
    to: /home/vagrant/code

If you want the Homestead.yaml to be portable, however, you can tell it to map
the current directory, and not a fully qualified path:

folders:
  - map: .
    to: /home/vagrant/code

Next, we’ll look at the site definition. After you first run homestead make,
you should have the following:

sites:
  - map: homestead.test
    to: /home/vagrant/code/public

Let’s change this a bit. First, we’ll give a new site name, then a site type
(I’ll use "expressive" here, but you can change this to "apigility" or "zf"
based on your application), and we’ll enable Z-Ray.

sites:
  - map: expressive.test
    to: /home/vagrant/code/public
    type: expressive
    zray: "true"

Yes, the correct value for the zray setting is "true"; see this issue for
details
.

From here, we can finally get running:

$ vagrant up

If you are not using the vagrant-hostsupdater plugin, you’ll need to add an
entry to your system hosts file:

192.168.10.10 expressive.test

Fin

We’re hoping having this support in place will allow Zend Framework zend-mvc,
Apigility, and Expressive developers to create and share development
environments easily between their teams. If you have additional features you
would like enabled by default (e.g., auto-detection of ZF, Apigility, and
Expressive applications by homestead make, additional default nginx
configuration, etc.), be sure to swing by the
Slack or
forums and ask!

I want to extend a hearty thank you to Joe Ferguson
for helping me provide the integration, and guiding me through the
contribution process for Homestead.

Source: Zend feed

PHP 7.2 Support!

With Expressive 3 complete, we were able
to turn our sights on another important initiative: PHP 7.2 support across all
components and Apigilty modules.

The short story is: as of today, that initiative is complete! If you are using
the Zend Framework MVC framework, Expressive, or Apigility, or any of the ZF
components standalone, you should be able to perform a composer update to get
versions that support PHP 7.2.

The full story is much longer.

How we got there

The PHP project does a pretty stellar job of preserving backwards compatibility.
Some might say they do it to a fault, being averse to any changes that might
cause breakage for users, even if the change fixes bad behavior on the part of
the language.

Interestingly, there have been a ton of initiatives to tighten up the language
and have it behave more predictably. Unfortunately, we, and a number of projects
on which we depend, were bit by some of these efforts that went into PHP 7.1 and
7.2.

One in particular was problematic.

Let’s say you have a class such as the following:

class SomeContainer
{
    public function get($name, array $options = null)
    {
    }
}

Next, we’ll have an extension to that class that overrides that method and
changes the default value:

class AContainerExtension extends SomeContainer
{
    public function get($name, array $options = [])
    {
    }
}

These should be fine, right? Wrong.

Starting in 7.1.0, the above emits an E_WARNING due to incompatible
signatures. This is because PHP 7.1 adds nullable types, and considers the
first signature equivalent to a nullable array.

The problem is that PHPUnit, on seeing an E_WARNING, creates an error status
for the test in which it is raised.

There were a number of other minor changes such as deprecated APIs that also
affected our code, often leading to unexpected test errors. Technically, the
code likely could run, but not without emitting deprecation notices and/or
warnings — and our goal is to run cleanly, so that users can see only the
warnings pertinent to their own application code.

On top of this, a number of PHPUnit classes exhibited similar behaviors, which
meant that under PHP 7.2, with the versions of PHP we were using, we could not
verify that our code could work under that version.

The upshot for us is that testing against 7.2 wasn’t as easy as just adding
another PHP version to the test matrix. We also had to find a set of different
PHP versions that we could test against (for instance, PHPUnit 6 and 7 require
PHP 7 versions, but we also still support PHP 5.6 in many of our components and
modules), figure out how to get Travis-CI to install a PHPUnit version
appropriate to the PHP version we were testing in, and ensure that the PHPUnit
features we were using worked across all PHPUnit versions against which we might
test.

Thankfully, we had a secret weapon: Michał Bundyra (@MichalBundyra on twitter).
Michał spent a fair bit of time this past year developing increasingly effective
approaches to this sort of problem, and, once the 7.2 initiative was announced,
jumped in and created patches for almost every single component and module we
ship.

Seriously, this was work above and beyond anything I can reasonably expect of
a volunteer. Go thank him, already!

Our approach

The approach Michał developed involves two things. First, a range of known-good
PHPUnit dependencies, and, second, a set of configuration for Travis-CI that
will allow installing the appropriate dependencies.

First, we use the following constraints for PHPUnit in our composer.json files
when both PHP 5.6 and PHP 7 versions are required:

"phpunit/phpunit": "^5.7.21 || ^6.3 || ^7.1",

These allow us to use the 5.7 series for PHP 5.6, and either the 6.3 or 7.1
series when under other PHP versions.

We also commit our composer.lock file. I’ll show why in a moment.

Next, we use configuration similar to the following with Travis-CI:

sudo: false

language: php

cache:
  directories:
    - $HOME/.composer/cache

env:
  global:
    - COMPOSER_ARGS="--no-interaction"

matrix:
  include:
    - php: 5.6
      env:
        - DEPS=lowest
    - php: 5.6
      env:
        - DEPS=locked
        - LEGACY_DEPS="phpunit/phpunit zendframework/zend-code"
    - php: 5.6
      env:
        - DEPS=latest
    - php: 7
      env:
        - DEPS=lowest
    - php: 7
      env:
        - DEPS=locked
        - CS_CHECK=true
        - LEGACY_DEPS="phpunit/phpunit-mock-objects phpspec/prophecy zendframework/zend-code"
    - php: 7
      env:
        - DEPS=latest
    - php: 7.1
      env:
        - DEPS=lowest
    - php: 7.1
      env:
        - DEPS=locked
        - TEST_COVERAGE=true
    - php: 7.1
      env:
        - DEPS=latest
    - php: 7.2
      env:
        - DEPS=lowest
    - php: 7.2
      env:
        - DEPS=locked
    - php: 7.2
      env:
        - DEPS=latest

before_install:
  - if [[ $TEST_COVERAGE != 'true' ]]; then phpenv config-rm xdebug.ini || return 0 ; fi

install:
  - travis_retry composer install $COMPOSER_ARGS --ignore-platform-reqs
  - if [[ $LEGACY_DEPS != '' ]]; then travis_retry composer update $COMPOSER_ARGS --with-dependencies $LEGACY_DEPS ; fi
  - if [[ $DEPS == 'latest' ]]; then travis_retry composer update $COMPOSER_ARGS ; fi
  - if [[ $DEPS == 'lowest' ]]; then travis_retry composer update --prefer-lowest --prefer-stable $COMPOSER_ARGS ; fi
  - stty cols 120 && composer show

script:
  - vendor/bin/phpunit
  - if [[ $CS_CHECK == 'true' ]]; then vendor/bin/phpcs ; fi

Let’s break that down.

We set up a few things up front for all builds: we’re using dockerized php
jobs (sudo: false, language: php), we’re caching composer metadata between
builds (which greatly speeds up the installation process!), and defining our
default composer arguments.

From there, we define our test matrix. Each job in the matrix includes:

  • The PHP version we are testing against.
  • Environment variables for that specific build.

You’ll notice we have three jobs for each PHP version, corresponding to the
following environment variables:

  • DEPS=lowest
  • DEPS=locked
  • DEPS=latest

These variables are indicating how we want to install dependencies:

  • locked indicates we want to use those specified in the composer.lock file.
  • lowest means we want to test against the lowest stable dependencies we allow.
  • latest indicates we want to test against the latest available dependences we
    allow.

This approach allows us to determine:

  • When we start using features from a library that are not present in the
    earliest version we have indicated we support. If the lowest tests fail, we
    likely either need to change what part of a third-party API we are consuming,
    or bump the minimum supported version of that dependency.

  • When a library has introduced a BC break in a more recent release than we
    tested against previously. In such cases, we can try and find a way to make
    our own usage of that library forwards-compatible with the new version; create
    an issue notifying the developer(s) of that library of the BC break; or change
    our dependencies to not allow the newer version.

Additionally, some jobs have more variables they define:

  • CS_CHECK will tell the job whether or not to run CS checks. (We also often
    define an env variable for running coverage reports.)

  • LEGACY_DEPS allows us to specify dependencies we need to update after
    initial installation. More on that in the coming paragraphs.

We also disable xdebug unless we’re running coverage reports. This speeds up
Composer operations as well as running unit tests. I have the before_install
script detailed above, but do not define any environments with the
TEST_COVERAGE variable set, nor demonstrate how we use it to run reports.

When we hit the install section is when the "magic" happens. The first thing
we do is an install from the lockfile. When we do so, we pass the
--ignore-platform-reqs option, as we cannot guarantee that the dependencies in
the lockfile will work for the current PHP version being used.

We then check to see if LEGACY_DEPS is non-empty. If so, we do a composer update --with-dependencies, passing the value of LEGACY_DEPS as the packages
to update. This allows us to use the lockfile on locked versions, but then get
platform-specific dependencies for libraries where we know that what’s in the
lockfile may not work on all platforms
.

Next, we check for DEPS=latest, running composer update, and DEPS=lowest,
running composer update --prefer-lowest --prefer-stable.

Finally, we display what dependencies were installed, along with their versions.
We use the construct stty cols 120 to set the display columns, as otherwise
composer will not detect a TTY, and spit out only the dependencies, with no
associated version.

The beauty of this approach is that we are able to use it almost verbatim across
our repositories, with only minor changes to which LEGACY_DEPS we need, and
which versions need them. Having multiple tests per version, spanning a range of
dependencies, has allowed us to identify and solve problems arising from
libraries we consume quickly.

This approach allowed us to run tests under PHP 7.2, fix any issues identified,
and finally release new versions that fully support PHP 7.2.

What’s next?

We have a number of initiatives we’re working on in the coming months:

  • Frank Brückner is working on a site
    refresh to both make the documentation and main sites more consistent in
    look-and-feel, as well as better support mobile browsers.

  • We continue to work on the Apigility on Expressive initiative. While many
    features were released with stable versions for Expressive 3, there’s still work
    to be done, including tooling support.

  • Aleksei Khudiakov has been working on a set of proposals
    for a PSR-7-based zend-mvc v4.

  • We want to work on tutorials and guides to help users make the most of
    Expressive, as well as migrate to Expressive from zend-mvc.

If you want to help out with any of these initiatives:

Source: Zend feed

Async Expressive? Try Swoole!

When we were finalizing features for Expressive 3,
we had a number of users testing using asynchronous PHP web servers. As a
result, we made a number of changes in the last few iterations to ensure that
Expressive will work well under these paradigms.

Specifically, we made changes to how response prototypes are injected into
services.

Response prototypes?

What’s the problem?

In an async system, one advantage is that you can bootstrap the application
once, and then respond to requests until the server is shutdown.

However, this can become problematic with services that compose a response
prototype in order to produce a response (e.g., authentication middleware that
may need to produce an "unauthenticated" response; middleware that will produce
a "not found" response; middleware that will produce a "method not allowed"
response; etc.). We have standardized on providing response prototypes via
dependency injection, using a service named after the interface they implement:
PsrHttpMessageResponseInterface.

If a particular service accepts a response instance that’s injected during
initial service creation, that same instance will be used for any subsequent
requests that require it. And that’s where the issue comes in.

When running PHP under traditional conditions — php-fpm, the Apache SAPI,
etc. — all requests are isolated; the environment is both created and torn
down for each and every request. As such, passing an instance is perfectly safe;
there’s very little chance, if any, that any other service will be working with
the same instance.

With an async server, however, the same instance will be used on each and every
request. Generally, manipulations of PSR-7
message instances will create new instances, as the interfaces they implement
are specified as immutable. Unfortunately, due to technical limitations of the
PHP language, we were unable to make the body of response messages
immutable. This means that if one process writes to that body, then a
subsequent process — or even those executing in parallel! — will
receive the same changes. This can lead to, in the best case scenario,
duplicated content, and, in the worst, provide incorrect content or perform
information leaking!

To combat these situations, we modified the PsrHttpMessageResponseInterface
service we register with the dependency injection container: it now returns not
an instance of the interface, but a factory capable of producing an instance.
Services should compose this factory, and then call on it each time they need to
produce a response. This fixes the async problem, as it ensures a new instance
is used each time, instead of the same instance.

(Additionally, this change helps us prepare for the upcoming PSR-17, which
describes factories for PSR-7 artifacts; this solution will be compatible with
that specification once complete.)

Why async?

If asynchronous systems operate so differently, why bother?

There’s many reasons, but the one that generally gets the attention of
developers is performance.

We performed benchmarks of Expressive 2 and Expressive 3 under both Apache and
nginx, and found version 3 received around a 10% improvement.

We then tested using Swoole. Swoole is a PHP
extension that provides built-in async, multi-threaded input/output (I/O)
modules; it’s essentially the I/O aspects of node.js — which allow you to
create network servers and perform database and filesystem operations —
but for PHP.

A contributor, Westin Shafer, has written a module
for Expressive 3 that provides an application wrapper for Swoole

that is exposed via a CLI command. We ran our same benchmarks against this, and
the results were astonishing: applications ran consistently 4 times faster
under this asynchronous framework, and used fewer resources!

While performance is a great reason to explore async, there are other reasons as
well. For instance, if you do not need the return value of an I/O call (e.g., a
database transaction or cache operation), you can fire it off asynchronously,
and finish out the response without waiting for it. This can lead to reduced
waiting times for clients, further improving your performance.

We have had fun testing Swoole, and think it has tremendous possibilities when
it comes to creating microservices in PHP. The combination of Expressive and
Swoole is remarkably simple to setup and run, making it a killer combination!

Notes on setting up Swoole

The wshafer/swoole-expressive package requires a version 2 release of the
Swoole extension.

However, there’s a slight bug in the PECL installer whereby it picks up the
most recent release as the "latest", even if a version with greater stability
exists. As of the time of writing, version 1.10.2 of Swoole was released after
version 2.1.1, causing it to be installed instead of the more 2.X version.

You can force installation of a version by appending the version you want when
invoking the pecl command:

$ pecl install swoole-2.1.1

The version must be fully qualified for it to install correctly; no partials
(such as swoole-2 or swoole-2.1 will work.

Source: Zend feed

Expressive 3!

Yesterday, we tagged and released Expressive 3!

Expressive 3 provides a middleware microframework.

Create a new Expressive application using Composer:

$ composer create-project zendframework/zend-expressive-skeleton

The installer will prompt you for your choice of:

  • Initial application architecture (minimal, flat, modular)
  • Which dependency injection container you would like to use.
  • Which routing library you would like to use.
  • Which templating library you would like to use, if any.
  • Which error handling library you would like to use, if any.

From there, it creates a new project for you, and allows you to get started
developing immediately.

You can read more in our quick start,
and may want to check out our command line tooling
to see what we provide to make development even faster for you!

What are the features?

Expressive 3 embraces modern PHP, and requires PHP 7.1 or higher. Strong
type-hinting, including return type hints, make both our job and your job
easier and more predictable. The ability to use all modern PHP features helps us
deliver a solid base for your application.

Expressive 3 provides full support for the PSR-15 (Middleware and Request
Handlers) standard
. We believe strongly
in supporting standards, to the extent that this release also drops direct
support for the "double-pass" middleware style

we have supported since version 1.0

Expressive 3 massively refactors its internals as well. In fact, the majority of
the code in the zend-expressive package was removed, moved to other existing
packages where it had a better semantic affiliation1,
or extracted to new packages2. This base
package now mainly handles coordinating collaborators and providing a
user-friendly interface to creating your application pipeline and routes.3

Expressive 3 provides more command line tooling and tooling improvements in
order to make developing your application easier. We added a command for
creating factories for existing classes (factory:create).4
The middleware:create command now creates a factory for the
middleware generated. We added support for creating request handlers5,
complete with factory generation and registration, as well as template support.6

Finally, we recognize that Expressive has changed massively between versions 1
and 3, while simultaneously keeping its primary API stable and unchanged.
However, to help users find the information they need for the version they run,
we have rolled out versioned documentation, with each version providing only
information specific to its release cycle:

The most recent version will always be present in the primary navigation, with
links to other versions present as well.

New components!

We have several new components that provide features for Expressive — or
any PSR-15 framework you may be using! These include:

We have a number of other packages in the works around authentication,
authorization, and data validation that we will be releasing in the coming weeks
and months; stay tuned for announcements!

What about upgrading?

We have prepared a migration document
that covers new features, removed features, and a list of all changes.

Additionally, we have provided migration tooling
to aid you in your migration from version 2 to version 3. The tool will not
necessarily give you a fully running application
, but it will take care of
the majority of the changes necessary to bump your application to version 3,
including setting up appropriate dependencies, and updating your bootstrapping
files to conform to the new skeleton application structure.

If you need assistance, you can find community help:

What’s next?

We have been working on a number of API-related modules for Expressive (and any
PSR-15 applications) since last summer, with a number of components already
completed, and others close to completion. We plan to finalize these in the next
few months.

Thank You!

We extend a hearty thank you to everyone who tested the various pre-releases and
provided feedback. Additionally, we are singling out the following individuals
who provided significant contributions to the Expressive 3 project:

  • Enrico Zimuel provided a ton of feedback and
    critique during the design phase, and was a driving force behind many of the
    API usability decisions.

  • Rob Allen did a workshop at SunshinePHP, right as we
    dropped our initial alpha releases, and provided feedback and testing for much
    of our tooling additions.

  • Frank Brückner provided ongoing feedback
    and review of pull requests, primarily around documentation; he is also
    responsible for a forthcoming rewrite of our documentation theme to make it
    more responsive and mobile-friendly.

  • Daniel Gimenes provided feedback and ideas as
    we refactored zend-stratigility; he is the one behind package-level utility
    functions such as ZendStratigilitydoublePassMiddleware(),
    ZendStratigilitypath(), and more.

  • Witold Wasiczko provided the majority of the
    rewrite of zend-stratigility for version 3. He can be celebrated for removing
    over half the code from that repository!

In addition to these people, I want to extend a personal thank you to the
following people:

  • Geert Eltink has helped maintain Expressive v2, and
    particularly the various routers and template engines, making them ready for
    v3 and testing continually. As a maintainer, I was able to rely on him to take
    care of merges as we finalized the releases, and was pleasantly surprised to
    wake up to new releases several times when he fixed critical issues in our
    alpha and RC releases.

  • Michał Bundyra provided a constant stream of
    pull requests related to quality assurance (including ongoing work on our phpcs
    extension!), as well as critical review of incoming patches. He spearheaded
    important work in the refactoring process, including changes to how we handle
    response prototypes, and critical fixes in our routers to address issues with
    how we detect allowed methods for path route matches. We synced each and every
    single day, often arguing, but always coming to consensus and plowing on.

If you get a chance, reach out to these contributors and thank them for the
release!

Footnotes

  • 0: The Expressive ecosystem makes
    use of many other standards as well, including
    PSR-7 HTTP Messages,
    PSR-11 Container, and
    PSR-13 HTTP Links.

  • 1: As an example, the routing,
    dispatch, and "implicit methods" middleware were all moved to the
    zend-expressive-router
    package, as they each work with the router and route results.

  • 2: Request generation, application
    dispatch, and response emission were all moved to a new package,
    zend-httphandlerrunner.

  • 3: These refactors led to a net
    removal of code across the board, vastly simplifying the internals. This
    will lead to ease of maintenance, greater stability, and, based on benchmarks
    we’ve been performing, 10% better performance and less system resource usage.

  • 4: factory:create uses PHP’s
    Reflection API in order to determine what dependencies are in place in order to
    generate a factory class; it also registers the class and factory with the
    container!

  • 5: In previous Expressive versions,
    we referred to "actions", which were any middleware that returned a response
    instead of delegating to another layer of the application. PSR-15 calls such
    classes request handlers. Our tooling provides an action:create command,
    however, for those who prefer the "action" verbiage.

  • 6: The command creates a template
    named after the handler created; it uses the root namespace of the class to
    determine where to put it in the filesystem. Additionally, it alters the
    generated request handler to render the template into a zend-diactoros
    HtmlResponse!

Source: Zend feed

Expressive 3.0.0RC2 released

This week, we’ve worked on backports from Expressive 3 to Expressive 2, and, in
the process, identified a few issues with how the routing package handles
implicit HEAD and OPTIONS requests. As a result, we’ve just released
3.0.0rc2:

  • https://github.com/zendframework/zend-expressive-skeleton/releases/3.0.0rc2

What are "implicit" HEAD and OPTIONS requests?

Implicit HEAD and OPTIONS requests are requests using those methods made to
routes that do not explicitly define them; in other words, if no routes for a
given path include the HEAD or OPTIONS methods.

We provide a way for router implementations to flag a routing failure as being
due to requesting a method that is not explicitly allowed. We also provide
middleware for providing responses to HEAD and OPTIONS requests under those
conditions, as well as separate middleware for simply reporting that a method is
not allowed.

Getting started with RC2

To start a new project based on 3.0.0rc2, use
Composer to create a new project:

$ composer create-project "zendframework/zend-expressive-skeleton:3.0.0rc2"

If you want to install to a custom directory name, use the following instead:

$ composer create-project zendframework/zend-expressive-skeleton {your directory} 3.0.0rc2

Once installed, you can follow the same instructions as for RC1.

Updating from RC1

Updating from RC1 requires a few manual steps.

Prior to upgrading, you will need to do the following:

$ composer require "zendframework/zend-diactoros:^1.7.1"

Then run:

$ composer update

Once done, you will need to make one change to your config/pipeline.php.

Locate the following line:

$app->pipe(MethodNotAllowedMiddleware::class);

Cut the line, and paste it following the line reading:

$app->pipe(ImplicitOptionsMiddleware::class);

This change is necessary due to how each of these middleware inspect the routing
result and act on it. If MethodNotAllowedMiddleware operates before the
Implicit*Middleware, it will detect a 405 condition. Moving it after those
middleware allow them to intercept for HEAD and OPTIONS requests.

Roadmap

We still have a number of tasks to accomplish before the stable 3.0.0 release.
In particular:

  • We need to provide full documentation for the v3 release.

  • We will be issuing a 2.2 release with:

    • Deprecations, based on the v3 changes.
    • Backports of select v3 changes in order to aid migration.
    • See the following for full details: https://discourse.zendframework.com/t/roadmap-expressive-2-2/504
  • We need to document migration from v2.2 to v3, and potentially provide
    automated tooling.

  • We anticipate users may still find bugs in the RC, and will be actively
    incorporating bugfixes before the stable release.

Our target date is still 15 March 2018, but we need your help! Help by testing
the RC2 skeleton and providing your feedback. As we prepare the v2.2 release,
help by testing tooling and applying our migration documentation to your
applications, and let us know what works and what doesn’t. If you find features
that are not documented, let us know by filing issues or asking questions in
our Slack.

We look forward to the stable release, and the positive impact PSR-15 will have
on the PHP ecosystem!

Source: Zend feed

Expressive 3 Alpha 3

Today, we pushed the final changes and fixes that culminated in the
Expressive Installer and Skeleton 3.0.0alpha3
release
!

The alpha releases have a ton of great features; keep reading to find out more!

Alpha 1 and Alpha 2

We released 3.0.0alpha1 on Tuesday, 6 February 2018, and 3.0.0alpha2 on
Wednesday, 7 February 2018. While they were usable, there were a few issues we
discovered that we felt should be addressed before a public announcement.
3.0.0alpha3 represents a stable, testable release.

Installation

Currently, we do not have a completed migration path for 3.0; this is our work
in the coming weeks. Additionally, there may yet be changes coming as we get
your feedback. As such, we recommend:

  • Install a fresh project.
  • Do not use the alpha release(s) in production!

To create a new project based on 3.0.0alpha3, use
Composer:

$ composer create-project "zendframework/zend-expressive-skeleton:3.0.0alpha3"

This will install the skeleton, and then start prompting you for specific
components you want to install, including choice of
PSR-11 container, choice of router,
choice of template engine, and choice of error handler. Generally speaking, we
recommend the default values, except in the case of the template engine (which
defaults to none; choose the engine you’re most comfortable with).

Once your selections are made, the skeleton will install dependencies; when it
is complete, you can enter the newly created directory to begin development:

$ cd zend-expressive-skeleton

Alternate directory

You can specify an alternate directory when calling composer create-project;
when you do, you can also specify the specific version separate from the root
package:

$ composer create-project zendframework/zend-expressive-skeleton expressive 3.0.0alpha3

Creating middleware

Version 3 of Expressive will work with
PSR-15 (HTTP server request handlers)
middleware and request handlers only. You will be writing these to create
your application.

Other supported types

Expressive 3 also supports other types of middleware definitions, though they
are not recommended:

  • Callable middleware using the same signature as PSR-15. These can be piped
    and routed to directly.
  • Callable double-pass middleware; these must be decorated using the
    ZendStratigilitydoublePassMiddleware() utility class — which also
    requires a PSR-7 response prototype.

The skeleton project now ships with zend-expressive-tooling
by default, and maps its expressive command as a composer command:

$ composer expressive help

To create your first middleware:

$ composer expressive middleware:create "AppXClacksOverheadMiddleware"

This will create the class AppXClacksOverheadMiddleware, and tell you where
it has been created. You can then edit it:

<?php
namespace App;

use PsrHttpMessageResponseInterface;
use PsrHttpMessageServerRequestInterface;
use PsrHttpServerMiddlewareInterface;
use PsrHttpServerRequestHandlerInterface;

class XClacksOverheadMiddleware implements MiddlewareInterface
{
    public function process(ServerRequestInterface $request, RequestHandlerInterface $handler) : ResponseInterface
    {
        $response = $handler->handle($request);
        return $response->withHeader('X-ClacksOverhead', 'GNU Terry Pratchett');
    }
}

Once your middleware is created, register it in the container as an invokable,
via the config/autoload/dependencies.global.php file:

'dependencies' => [
    'invokables' => [
        AppXClacksOverheadMiddleware::class => AppXClacksOverheadMiddleware::class,
    ],
],

Finally, register it in your config/pipeline.php file:

// likely an early statement, before routing
$app->pipe(AppXClacksOverheadMiddleware::class);

You’ve just created your first middleware!

Creating handlers

PSR-15 defines two interfaces. In the previous section, we demonstrated
implementing the MiddlewareInterface. That interface references another, the
RequestHandlerInterface.

Internally, we provide one that maintains the middleware pipeline and the state
within the pipeline such that calling handle() advances to the next
middleware.

However, there’s another place handlers are of interest: for routes.

Most often, when you create a route, the class you write to handle the route
will generate a response itself, and never need to delegate to another handler.
As such, you can write handlers instead!

Like middleware, the tooling provides a command for creating handlers:

$ composer expressive handler:create "AppHandlerHelloWorldHandler"

This will create a RequestHandlerInterface implementation using the given
name, and then tell you where on the filesystem it created it.

For this example, we’ll assume you’re using zend-diactoros (as it is used in the
Expressive skeleton by default), and we’ll create a handler that generates a
ZendDiactorosResponseHtmlResponse. Open the file, and edit the contents to
look like the following:

<?php

namespace AppHandler;

use PsrHttpMessageResponseInterface;
use PsrHttpMessageServerRequestInterface;
use PsrHttpServerRequestHandlerInterface;
use ZendDiactorosResponseHtmlResponse;

class HelloWorldHandler implements RequestHandlerInterface
{
    public function handle(ServerRequestInterface $request) : ResponseInterface
    {
        return new HtmlResponse('<h1>Hello, world!</h1>');
    }
}

Like the XClacksOverhead middleware, We’ll register this with the container as
an invokable, via the file config/autoload/dependencies.global.php:

'dependencies' => [
    'invokables' => [
        AppHandlerHelloWorldHandler::class => AppHandlerHelloWorldHandler::class,
    ],
],

Finally, we’ll route to it via your config/routes.php file:

$app->get('/hello', AppHandlerHelloWorldHandler::class, 'hello');

You’ve just created your first handler!

Handlers and route-specific pipelines

If you have used Expressive before, you may recall that the various routing
methods allowed middleware and middleware pipelines previously. This is still
true! The only difference with version 3 is that we also allow request
handlers.

In fact, you can add handlers into your middleware pipelines as well! If
we wanted the XClacksOverheadMiddleware to only be in that specific route,
we could write it as follows:

$app->get('/hello', [
    AppXClacksOverheadMiddleware::class,
    AppHandlerHelloWorldHandler::class,
], 'hello')

The only caveat is that handlers always return a response, which means they
should always be the last item in a pipeline.

Test it out!

You can use the following command to fire up the PHP built-in web server:

$ composer serve

The command works on Windows and macOS; for Linux users, it will work as long as
you have PHP 7.1.14 and later or 7.2.2 and later installed. For those on earlier
versions, use the following:

$ php -S 0.0.0.0:8080 -t public/ public/index.php

Once you have, try hitting your new route: http://localhost:8080/hello

You should get your HTML content as defined above! If you introspect the request
in your browser (or using a CLI tool such as cURL or HTTPie), you’ll also see a
X-Clacks-Overhead header from the middleware you created!

Roadmap

Hitting our first alpha releases is a huge milestone, and the culmination of
many months of development. We’re very excited about the results. In both
Stratigility (our middleware foundation library) and Expressive, we have
drastically reduced the amount of code, while providing essentially the same
feature set (and, in many cases, expanding that feature set!).

In the coming weeks, we’ll be developing a final version 2 minor release, 2.2.0,
as well as working on documentation and migration tooling. The main goal of the
2.2 release will be to mark deprecated features, provide forward compatible
alternatives, and provide tooling to help you migrate to those alternatives.

We also have a few planned features for Expressive 3 to complete. We’re working
on changes to the zend-component-installer to allow whitelisting packages with
configuration providers, so that users will not need to be prompted during
initial installation to inject configuration for packages we already know about.
We also plan to develop tooling for creating and registering factories based on
a given class, and updating the handler:create and middleware:create
factories to generate and register factories as well. Also, recently we released
version 3 of the zend-di package, and we’re considering integrating it by
default when zend-servicemanager is configured, to provide auto-wiring of
dependencies.

This may sound like quite a bit, but much of it is already in progress, and our
plan is to have a stable release by no later than 15 March 2018.

In the meantime: install and test 3.0.0alpha3! Kick its tires, and let us know
what works, and, more importantly, what doesn’t work, so we can provide you a
stable, exciting 3.0.0 release!

Source: Zend feed

Expressive 3 Preview

Last week, the PSR-15 working group voted to start its review
phase
.
PSR-15 seeks to standardize server-side request handlers and middleware, and
both Stratigility and Expressive have been implementing draft specifications
since their version 2 releases. Entering the review phase is an important
moment: it means that the working group feels the specification is stable and
ready for adoption. If, after the review period is over, no major changes are
required, the specification can be presented to the PHP-FIG core committed for a
final acceptance vote, at which point it will be frozen and ready for mass
adoption.

Our plan is to have Stratigility and Expressive follow the new specification in
its final form. To that end, we have been executing on a plan to prepare all our
projects that work with PSR-15 to adopt the latest round of changes.

That work is ready today!

What has changed in PSR-15?

The latest round of changes to the specification prior to entering the review
period were as follows:

  • The namespace of the draft specification was changed from
    InteropHttpServerMiddleware to InteropHttpServer. These will therefor
    become PsrHttpServer once the specification is accepted.

  • The DelegateInterface was renamed to RequestHandlerInterface, and
    the method it defines renamed to handle().

  • The MiddlewareInterface‘s second argument to process() was updated to
    typehint against RequestHandlerInterface.

  • The package shipping the interface was split into two,
    http-interop/http-server-handler and http-interop/http-server-middleware;
    these will become psr/http-server-handler and psr/http-server-middleware,
    respectively, once the package is accepted. The http-server-middleware
    packages depend on the http-server-handler packages.

These changes, of course, are not backwards compatible, and our attempts to
write a polyfill library were ultimately unsuccessful. As a result, we decided
to bump the major version of all libraries currently depending on the draft
specification.

What we have done

Our approach in updating the various packages was as follows:

  • We created a new release branch named after the next major release. For
    instance, if a library is currently issuing v2 releases, we created a
    release-3.0.0 branch.
  • We updated the branch aliases defined in the composer.json for the package
    as follows, on all branches:

    • The master branch points to the current minor release. As an example, for a
      package with a current stable 2.3.1 version, the branch alias became
      "dev-master": "2.3.x-dev".
    • If a development branch already exists, we updated similarly to the master
      branch. For the above example, the branch alias would read "dev-develop": "2.4.x-dev".
    • The new release branch is then mapped to the upcoming major version:
      `"dev-release-3.0.0": "3.0.x-dev".
  • On the release branches, we updated dependencies as follows:
    • PHP dependencies became simply ^7.1 (per our decision posted in
      June
      ).
    • References to http-interop/http-middleware packages were changed to
      "http-interop/http-server-middleware": "^1.0.1".
    • References to packages that have corresponding release branches were updated
      to have their constraints point to the appropriate development release branch.
      As an example, "zendframework/zend-expressive-router": "^3.0.0-dev".

These changes ensure users can install the new development versions of packages
by feeding an appropriate development constraint.

You’ll note that we bumped the minimum supported PHP version in these packages
as well. Because we were doing that, we also decided to make use of PHP 7.1
features. In particular:

  • Scalar and return type hints.
  • Nullable and void types.
  • Null coalesce.
  • strict_types where it simplifies validation of scalars (which turns out to
    be almost everywhere).

For packages that define interfaces, this meant that we also needed
corresponding major version bumps in packages that implement those interfaces.
This affected the router and template implementations in particular.

If you want a complete list of what was updated, you can visit the burndown
list in the forums
.

How YOU can test

This is all very nice and technical, but how can YOU test out the new versions?

Install the development version of the Expressive skeleton!

$ composer create-project "zendframework/zend-expressive-skeleton:3.0.x-dev" expressive-3.0-dev

This will create the skeleton project, with your selected functionality, in a
directory named expressive-3.0-dev. From there, you can start developing!

When you do, be aware of the following:

  • Middleware must now implement InteropHttpServerMiddlewareInterface:

    namespace YourModule;
    
    use InteropHttpServerMiddlewareInterface;
    use InteropHttpServerRequestHandlerInterface;
    use PsrHttpMessageResponseInterface;
    use PsrHttpMessageRequestHandlerInterface;
    
    class YourMiddleware implements MiddlewareInterface
    {
        public function process(
            ServerRequestInterface $request,
            RequestHandlerInterface $handler
        ) : ResponseInterface {
        }
    }
    

    Note: vendor/bin/expressive middleware:create will create these correctly
    for you with its 1.0.0-dev release!

  • If you want to delegate handling to the next middleware, you will now use the
    $handler, and call its handle() method:

    $response = $handler->handle($request);
    
  • If you want to use one of the optional Expressive packages, such as
    zend-expressive-session, you will need to require it using a development
    constraint. For instance:

    $ composer require zendframework/zend-expressive-session:^1.0.0-dev
    

    Note the use of the semantic pin (^), as well as the -dev suffix; both are
    necessary for composer to identify the development release.

Regarding the last point, the following is a list of all packages with
development release branches, along with the corresponding version you should
use when requiring them while testing:

Package Version
zend-expressive ^3.0.0-dev
zend-expressive-aurarouter ^3.0.0-dev
zend-expressive-authentication ^1.0.0-dev
zend-expressive-authentication-oauth2 ^1.0.0-dev
zend-expressive-authorization ^1.0.0-dev
zend-expressive-csrf ^1.0.0-dev
zend-expressive-fastroute ^3.0.0-dev
zend-expressive-flash ^1.0.0-dev
zend-expressive-helpers ^5.0.0-dev
zend-expressive-plastesrenderer ^2.0.0-dev
zend-expressive-router ^3.0.0-dev
zend-expressive-session ^1.0.0-dev
zend-expressive-skeleton ^3.0.0-dev
zend-expressive-template ^2.0.0-dev
zend-expressive-tooling ^1.0.0-dev
zend-expressive-twigrenderer ^2.0.0-dev
zend-expressive-zendrouter ^3.0.0-dev
zend-expressive-zendviewrenderer ^2.0.0-dev
zend-problem-details ^1.0.0-dev
zend-stratigility ^3.0.0-dev

In most cases, unless you are extending classes we provide, your existing code
should just work with the new packages once you update your middleware to the
new signatures.

Updating an existing application

Updating an existing application requires a bit more effort. You will need to
manually edit your composer.json to update the constraints for each of the
above packages to match what is in the table. Additionally, if you see
references to either http-interop/http-middleware or
webimpress/http-middleware-compatibility, you will need to remove those.
You will also need to add the following two lines to the file:

"minimum-stability": "dev",
"prefer-stable": true

Once done with the composer.json changes, run composer update to pick up
the changes. If you encounter any issues, run rm -Rf composer.lock vendor, and
then execute composer install.

Finally, you will need to update any middleware in your application to
implement the new interface. Ensure you have zend-expressive-tooling
installed, and install it if you do not, using the ^1.0.0-dev constraint
(composer require --dev "zendframework/zend-expressive-tooling:^1.0.0-dev").
Once you do, run:

$ ./vendor/bin/expressive migrate:interop-middleware

What’s next?

If you run into things that do not work, report them on the appropriate issue
tracker.

Once PSR-15 is finalized, our plan is to go through and update each package
depending directly on it to point to the new PHP-FIG sponsored packages, and
update import statements throughout our code appropriately. We’ll then likely
issue a beta release for folks to test against one last time.

In the meantime, we’ll also be looking at other changes we may want to make. New
major version breaks should happen only rarely going forward, and we may want to
make a few more changes to help improve quality, simplify maintenance, and
increase usability before we make the final release. As we do, we’ll update you
here on the blog.

Want some ebooks on ZF and Expressive?

We collated our posts from the first half of 2017 into two ebooks:

  • Zend Framework 3 Cookbook, which covers usage of a couple dozen ZF
    components, within zend-mvc and Expressive applications, as well as
    standalone.
  • Expressive Cookbook, which covers features of Expressive and middleware
    in general.

You can get them free with registration on the zend.com
website
.

Source: Zend feed

A new release of zend-db

Today, we released zend-db 2.9.0!
This is our first new feature release in over 18 months, and contains
7 bug fixes, 6 new features, numerous unit test additions, and many
documentation improvements.

zend-db is an important component of many PHP projects, and we know that its
support is crucial for many people. As such, we allocated a number of weeks to
triaging the various open issues and patches (more than 50) to ensure we would
provide a stable release.

The release contains the following changes:

Added

  • #216 added AFTER support
    in ALTER TABLE syntax for MySQL.
  • #223 added support for
    empty values set with the IN predicate.
  • #271 added support for
    dash characters in MySQL identifiers.
  • #273 added support for
    implementing an error handler when used with db2_prepare.
  • #275 added support for
    LIMIT OFFSET for db2.
  • #280 added the version
    DSN parameter for the pdo_dblib extension.

Fixed

  • #205 fixes whitespace
    issues in ORDER BY syntax.
  • #224 fixes how parameters
    are bound to statements in the PDO adapter. PDO has a restriction on parameter
    names of [0-9a-zA_Z_]; as such, the driver now hashes the parameter names
    using md5() in order to ensure compatibility with other drivers.
  • #229 fixes SSL support
    in the mysqli adapter.
  • #255 fixes an edge case
    when using ResultSet with array values (versus objects).
  • #261 fixes how the
    Firebird adapter attempts to retrieve the last generated value so as to
    prevent exceptions being raised.
  • #276 and
    #287 provide fixes to
    enable usage of the component with PHP 7.2.

We also dropped support for PHP 5.5 (EOL last year) and HHVM; zend-db 2.9 and
above now only support PHP 5.6 and PHP 7+ releases.

Future of zend-db

We are planning a 3.0 release of zend-db release sometime in 2018. This new major version
will contain new features sucha as extended DDL support
for different database vendors (currently, most support targets MySQL), and support
for SEQUENCE.
Additionally, that release will drop support for PHP versions older than 7.1.

If you want to contribute to zend-db, you are more than welcome! For more
information, read the Zend Framework contribution guide.

Special thanks

A special thanks to the following zend-db contributors (in no particular order):

We also extend thanks to our community review team for their efforts in making
this release of zend-db possible.

Source: Zend feed

Emitting Responses with Diactoros

When writing middleware-based applications, at some point you will need to emit
your response
.

PSR-7 defines the various interfaces
related to HTTP messages, but does not define how they will be used.
Diactoros defines several
utility classes for these purposes, including a ServerRequestFactory for
generating a ServerRequest instance from the PHP SAPI in use, and a set of
emitters, for emitting responses back to the client. In this post, we’ll
detail the purpose of emitters, the emitters shipped with Diactoros, and some
strategies for emitting content to your users.

What is an emitter?

In vanilla PHP applications, you might call one or more of the following
functions in order to provide a response to your client:

  • http_response_code() for emitting the HTTP response code to use; this must
    be called before any output is emitted.
  • header() for emitting response headers. Like http_response_code(), this
    must be called before any output is emitted. It may be called multiple times,
    in order to set multiple headers.
  • echo(), printf(), var_dump(), and var_export() will each emit output
    to the current output buffer, or, if none is present, directly to the client.

One aspect PSR-7 aims to resolve is the ability to generate a response
piece-meal, including adding content and headers in whatever order your
application requires. To accomplish this, it provides a ResponseInterface with
which your application interacts, and which aggregates the response status code,
its headers, and all content.

Once you have a complete response, however, you need to emit it.

Diactoros provides emitters to solve this problem. Emitters all implement
ZendDiactorosResponseEmitterInterface:

namespace ZendDiactorosResponse;

use PsrHttpMessageResponseInterface;

interface EmitterInterface
{
    /**
     * Emit a response.
     *
     * Emits a response, including status line, headers, and the message body,
     * according to the environment.
     *
     * Implementations of this method may be written in such a way as to have
     * side effects, such as usage of header() or pushing output to the
     * output buffer.
     *
     * Implementations MAY raise exceptions if they are unable to emit the
     * response; e.g., if headers have already been sent.
     *
     * @param ResponseInterface $response
     */
    public function emit(ResponseInterface $response);
}

Diactoros provides two emitter implementations, both geared towards standard PHP
SAPI implementations:

  • ZendDiactorosEmitterSapiEmitter
  • ZendDiactorosEmitterSapiStreamEmitter

Internally, they operate very similarly: they emit the response status code, all
headers, and the response body content. Prior to doing so, however, they check
for the following conditions:

  • Headers have not yet been sent.
  • If any output buffers exist, no content is present.

If either of these conditions is not true, the emitters raise an exception.
This is done to ensure that consistent content can be emitted; mixing PSR-7 and
global output leads to unexpected and inconsistent results. If you are using
middleware, use things like the error log, loggers, etc. if you want to debug,
instead of mixing strategies.

Emitting files

As noted above, one of the two emitters is the SapiStreamEmitter. The normal
SapiEmitter emits the response body at once via a single echo statement.
This works for most general markup and JSON payloads, but when returning files
(for example, when providing file downloads via your application), this strategy
can quickly exhaust the amount of memory PHP is allowed to consume.

The SapiStreamEmitter is designed to answer the problem of file downloads. It
emits a chunk at a time (8192 bytes by default). While this can mean a bit more
performance overhead when emitting a large file, as you’ll have more method
calls, it also leads to reduced memory overhead, as less content is in memory
at any given time.

The SapiStreamEmitter has another important feature, however: it allows
sending content ranges.

Clients can opt-in to receiving small chunks of a file at a time. While this
means more network calls, it can also help prevent corruption of large files by
allowing the client to re-try failed requests in order to stitch together the
full file. Doing so also allows providing progress status, or even buffering
streaming content.

When requesting content ranges, the client will pass a Range header:

Range: bytes=1024-2047

It is up to the server then to detect such a header and return the requested
range. Servers indicate that they are doing so by responding with a Content-Range
header with the range of bytes being returned and the total number of bytes
possible; the response body then only contains those bytes.

Content-Range: bytes=1024-2047/11576

As an example, middleware that allows returning a content range might look like
the following:

function (ServerRequestInterface $request, DelegateInterface $delegate) : ResponseInterface
{
    $stream = new Stream('path/to/download/file', 'r');
    $response = new Response($stream);

    $range = $request->getHeaderLine('range');
    if (empty($range)) {
        return $response;
    }

    $size  = $body->getSize();
    $range = str_replace('=', ' ', $range);
    $range .= '/' . $size;

    return $response->withHeader('Content-Range', $range);
}

You’ll likely want to validate that the range is within the size of the file, too!

The above code emits a Content-Range response header if a Range header is in
the request. However, how do we ensure only that range of bytes is emitted?

By using the SapiStreamEmitter! This emitter will detect the Content-Range
header and use it to read and emit only the bytes specified by that header; no
extra work is necessary!

Mixing and matching emitters

The SapiEmitter is perfect for content generated within your application
— HTML, JSON, XML, etc. — as such content is usually of reasonable
length, and will not exceed normal memory and resource limits.

The SapiStreamEmitter is ideal for returning file downloads, but can lead to
performance overhead when emitting standard application content.

How can you mix and match the two?

Expressive answers this question by providing
ZendExpressiveEmitterEmitterStack. The class acts as a stack (last in,
first out), executing each emitter composed until one indicates it has handled
the response.

This class capitalizes on the fact that the return value of EmitterInterface
is undefined. Emitters that return a boolean false indicate they were unable
to handle the response
, allowing the EmitterStack to move to the next emitter
in the stack. The first emitter to return a non-false value halts execution.

Both the emitters defined in zend-diactoros return null by default. So, if we
want to create a stack that first tries SapiStreamEmitter, and then defaults
to SapiEmitter, we could do the following:

use PsrHttpMessageResponseInterface;
use ZendDiactorosResponseEmitterInterface;
use ZendDiactorosResponseSapiEmitter;
use ZendDiactorosResponseSapiStreamEmitter;
use ZendExpressiveEmitterEmitterStack;

$emitterStack = new EmitterStack();
$emitterStack->push(new SapiEmitter());
$emitterStack->push(new class implements EmitterInterface {
    public function emit(ResponseInterface $response)
    {
        $contentSize = $response->getBody()->getSize();

        if ('' === $response->getHeaderLine('content-range')
            && $contentSize < 8192
        ) {
            return false;
        }

        $emitter = new SapiStreamEmitter();
        return $emitter->emit($response);
    }
});

The above will execute our anonymous class as the first emitter. If the response
has a Content-Range header, or if the size of the content is greater than 8k,
it will use the SapiStreamEmitter; otherwise, it returns false, allowing the
next emitter in the stack, SapiEmitter, to execute. Since that emitter always
returns null, it acts as a default emitter implementation.

In Expressive, if you were to wrap the above in a factory that returns the
$emitterStack, and assign that factory to the
ZendDiactorosEmitterEmitterInterface service, then the above stack will be
used by ZendExpressiveApplication for the purpose of emitting the
application response!

Summary

Emitters provide you the ability to return the response you have aggregated in
your application to the client. They are intended to have side-effects: sending
the response code, response headers, and body content. Different emitters can
use different strategies when emitting responses, from simply echoing content,
to iterating through chunks of content (as the SapiStreamEmitter does). Using
Expressive’s EmitterStack can provide you with a way to select different
emitters for specific response criteria.

For more information:

Source: Zend feed

1 2 3 4