A Technical Discussion Prompted by PHP's filter_var()

0x01 Background

I have been working through PHP Security Calendar 2017; this is day two.

Day 2 - Twig

Can you spot the vulnerability?

TEXT

// composer require "twig/twig"
require 'vendor/autoload.php';

class Template {
  private $twig;

  public function __construct() {
    $indexTemplate = '<img ' .
      'src="https://loremflickr.com/320/240">' .
      '<a href="{{link|escape}}">Next slide »</a>';

    // Default twig setup, simulate loading
    // index.html file from disk
    $loader = new Twig\Loader\ArrayLoader([
      'index.html' => $indexTemplate
    ]);
    $this->twig = new Twig\Environment($loader);
  }

  public function getNexSlideUrl() {
    $nextSlide = $_GET['nextSlide'];
    return filter_var($nextSlide, FILTER_VALIDATE_URL);
  }

  public function render() {
    echo $this->twig->render(
      'index.html',
      ['link' => $this->getNexSlideUrl()]
    );
  }
}

(new Template())->render();

This challenge tests XSS. Most XSS arises at output sinks, for example echo $var when $var is controllable and unfiltered or insufficiently filtered.^[1]^[2]^[3]^[4]^[5]^[6]

Here, XSS occurs because code inside the tag is insufficiently filtered, allowing the javascript: pseudo-protocol.

0x02 Analysis

The code is short. It defines a Template class with three methods: __construct(), getNextSlideUrl(), and render().

__construct() loads the template, getNextSlideUrl() validates the URL, and render() outputs it. The key lies in two filters:

Twig's escape filter
filter_var() URL Validation

See Twig's official documentation for the escape filter:

TEXT

escape uses the PHP native htmlspecialchars function for the HTML escaping strategy.

The escape filter is essentially a wrapper around htmlspecialchars; the underlying protection still comes from htmlspecialchars.

PHP

htmlspecialchars(string, flags, character-set, double_encode)

htmlspecialchars converts special characters to HTML entities. It can prevent some SQL injection and XSS attacks.

Some XSS payloads do not require special characters. Now examine filter_var():

PHP

filter_var(variable, filter, options)

TEXT

filter_var($nextSlide, FILTER_VALIDATE_URL);

The nextSlide value enters filter_var(), which checks whether it is a valid URL. URL validation has many bypasses. Because htmlspecialchars is also present, payloads requiring quotes or angle brackets are excluded.

The official solution is:

TEXT

?nextSlide=javascript://comment%250aalert(1)

NextSlide receives the value

TEXT

javascript://comment%250aalert(1)

Echoing this value inside the tag produces XSS through the following flow:

It first enters the <a> tag:

TEXT

<a href='javascript://comment%250aalert(1).'>Next slide »< /a>

// is the comment marker; %25 is a percent sign; % followed by 0a becomes a newline. This creates a standalone alert(1) line and executes alert successfully.

0x03 Example

TEXT

// index.php
<?php
$url = $_GET['url'];
if(isset($url) && filter_var($url, FILTER_VALIDATE_URL)){
    $site_info = parse_url($url);
    if(preg_match('/sec-redclub.com$/',$site_info['host'])){
        exec('curl "'.$site_info['host'].'"', $result);
        echo "＜center>＜h1>You have curl {$site_info['host']} successfully!＜/h1>＜/center>
              ＜center>＜textarea rows='20' cols='90'>";
        echo implode(' ', $result);
    }
    else{
        die("＜center>＜h1>Error: Host not allowed＜/h1>＜/center>");
    }

}
else{
    echo "＜center>＜h1>Just curl sec-redclub.com!＜/h1>＜/center>＜br>
          ＜center>＜h3>For example:?url=http://sec-redclub.com＜/h3>＜/center>";
}

TEXT

?>
// f1agi3hEre.php
<?php
$flag = "HRCTF{f1lt3r_var_1s_s0_c00l}"
?>

The challenge is difficult to understand without source, but obvious once the code is read.

Read the URL parameter from GET; it must satisfy filter_var's FILTER_VALIDATE_URL rules.

It must also contain a Linux command that exec() can use to read f1agi3hEre.php.

Many techniques bypass filter_var; see the references below.

The payload is:

TEXT

?url=hello://";ls;";sec-redclub.com/

The host value is clearly

TEXT

";ls;";sec-redclub.com

Combined with exec, the result is equivalent to:

TEXT

exec(ls,$result);
exec(sec-redclub.com,$result);
echo implode(' ', $result);

The final payload for reading the flag is:

TEXT

? url=hello://";cat<f1agi3hEre.php;";sec-redclub.com/

0x03 An Interesting Detail

Other solutions also appeared during testing, such as:

TEXT

?url=demo://%22;ls;%23;sec-redclub.com:80/

But local testing fails:

I suspected a PHP-version difference. My local system used PHP 7.1 while the blog used PHP 5.x, so I reproduced it on the blog:

The bypass succeeds. Why? The first suspect is the PHP built-in implementation, so inspect filter_var in PHP 5.x:

TEXT

/* {{{ proto mixed parse_url(string url, [int url_component])
   Parse a URL and return its components */

PHP_FUNCTION(parse_url)
{
        char *str;
        int str_len;
        php_url *resource;
        long key = -1;

        if (zend_parse_parameters(ZEND_NUM_ARGS() TSRMLS_CC, "s|l", &str, &str_len, &key) == FAILURE) {
                return;
        }

        resource = php_url_parse_ex(str, str_len);
        if (resource == NULL) {
                /* @todo Find a method to determine why php_url_parse_ex() failed */
                RETURN_FALSE;
        }

        if (key > -1) {
                switch (key) {
                        case PHP_URL_SCHEME:
                                if (resource->scheme != NULL) RETVAL_STRING(resource->scheme, 1);
                                break;
                        case PHP_URL_HOST:
                                if (resource->host != NULL) RETVAL_STRING(resource->host, 1);
                                break;
                        case PHP_URL_PORT:
                                if (resource->port != 0) RETVAL_LONG(resource->port);
                                break;
                        case PHP_URL_USER:
                                if (resource->user != NULL) RETVAL_STRING(resource->user, 1);
                                break;
                        case PHP_URL_PASS:
                                if (resource->pass != NULL) RETVAL_STRING(resource->pass, 1);
                                break;
                        case PHP_URL_PATH:
                                if (resource->path != NULL) RETVAL_STRING(resource->path, 1);
                                break;
                        case PHP_URL_QUERY:
                                if (resource->query != NULL) RETVAL_STRING(resource->query, 1);
                                break;
                        case PHP_URL_FRAGMENT:
                                if (resource->fragment != NULL) RETVAL_STRING(resource->fragment, 1);
                                break;
                        default:
                                php_error_docref(NULL TSRMLS_CC, E_WARNING, "Invalid URL component identifier %ld", key);
                                RETVAL_FALSE;
                }
                goto done;
        }

        /* allocate an array for return */
        array_init(return_value);

    /* add the various elements to the array */
        if (resource->scheme != NULL)
                add_assoc_string(return_value, "scheme", resource->scheme, 1);
        if (resource->host != NULL)
                add_assoc_string(return_value, "host", resource->host, 1);
        if (resource->port != 0)
                add_assoc_long(return_value, "port", resource->port);
        if (resource->user != NULL)
                add_assoc_string(return_value, "user", resource->user, 1);
        if (resource->pass != NULL)
                add_assoc_string(return_value, "pass", resource->pass, 1);
        if (resource->path != NULL)
                add_assoc_string(return_value, "path", resource->path, 1);
        if (resource->query != NULL)
                add_assoc_string(return_value, "query", resource->query, 1);
        if (resource->fragment != NULL)
                add_assoc_string(return_value, "fragment", resource->fragment, 1);
done:
        php_url_free(resource);
}
/* }}} */

The PHP 7.1 implementation of filter_var is:

TEXT

* {{{ proto mixed parse_url(string url, [int url_component])
   Parse a URL and return its components */

PHP_FUNCTION(parse_url)
{
        char *str;
        size_t str_len;
        php_url *resource;
        zend_long key = -1;

        if (zend_parse_parameters(ZEND_NUM_ARGS(), "s|l", &str, &str_len, &key) == FAILURE) {
                return;
        }

        resource = php_url_parse_ex(str, str_len);
        if (resource == NULL) {
                /* @todo Find a method to determine why php_url_parse_ex() failed */
                RETURN_FALSE;
        }

        if (key > -1) {
                switch (key) {
                        case PHP_URL_SCHEME:
                                if (resource->scheme != NULL) RETVAL_STRING(resource->scheme);
                                break;
                        case PHP_URL_HOST:
                                if (resource->host != NULL) RETVAL_STRING(resource->host);
                                break;
                        case PHP_URL_PORT:
                                if (resource->port != 0) RETVAL_LONG(resource->port);
                                break;
                        case PHP_URL_USER:
                                if (resource->user != NULL) RETVAL_STRING(resource->user);
                                break;
                        case PHP_URL_PASS:
                                if (resource->pass != NULL) RETVAL_STRING(resource->pass);
                                break;
                        case PHP_URL_PATH:
                                if (resource->path != NULL) RETVAL_STRING(resource->path);
                                break;
                        case PHP_URL_QUERY:
                                if (resource->query != NULL) RETVAL_STRING(resource->query);
                                break;
                        case PHP_URL_FRAGMENT:
                                if (resource->fragment != NULL) RETVAL_STRING(resource->fragment);
                                break;
                        default:
                                php_error_docref(NULL, E_WARNING, "Invalid URL component identifier " ZEND_LONG_FMT, key);
                                RETVAL_FALSE;
                }
                goto done;
        }

        /* allocate an array for return */
        array_init(return_value);

    /* add the various elements to the array */
        if (resource->scheme != NULL)
                add_assoc_string(return_value, "scheme", resource->scheme);
        if (resource->host != NULL)
                add_assoc_string(return_value, "host", resource->host);
        if (resource->port != 0)
                add_assoc_long(return_value, "port", resource->port);
        if (resource->user != NULL)
                add_assoc_string(return_value, "user", resource->user);
        if (resource->pass != NULL)
                add_assoc_string(return_value, "pass", resource->pass);
        if (resource->path != NULL)
                add_assoc_string(return_value, "path", resource->path);
        if (resource->query != NULL)
                add_assoc_string(return_value, "query", resource->query);
        if (resource->fragment != NULL)
                add_assoc_string(return_value, "fragment", resource->fragment);
done:
        php_url_free(resource);
}

The main changes are:

The trailing parameter in RETVAL_STRING(..., 1) was removed. What effect does that have? The official documentation explains:

strdup() is a common C library function that copies a string into newly allocated storage.

Returning to the original question: does the extra 1 affect filter_var?

My result is that it has no effect.

RETVAL_STRING(..., 1) can become RETVAL_STRING(...), while RETVAL_STRING(..., 0) can become RETVAL_STRING(...); efree(...). The difference is whether the string is reallocated.

What causes the same payload to behave differently? Check the local MySQL version:

version 8.0.

The preliminary conclusion is that the MySQL version is responsible.

I also built it in a virtual machine with MySQL 5.5:

The environment is Windows, so ls has no visible effect. filter_var was clearly bypassed; otherwise it would fail as on my local machine.

TEXT

Error: Host not allowed

Locally, replace the # comment marker with --:

The bypass also succeeded, but it was unclear why no file list appeared (Any explanation?). Interested readers can test whether the differing result is caused by the MySQL version or another factor. I did not investigate further due to time constraints; additional findings are welcome.