Replicating PHPSESSID and srctoken session authentication with mitmproxy

A month ago, I wanted to automate queries to a website that is using the PHPSESSID cookie to keep track of sessions. I struggled a lot and couldn't find any documentation covering the behaviour I was observing. But yesterday I finally found a solution !

In hope it could help others, I'm going to expose my findings here. I won't detail all my initial attempts and only focus on how to use the amazing mitmproxy command.

First, some exposition : the PHPSESSID cookie is used by PHP to keep track of sessions. It is generated when first accessing the website and sent to the client in the initial response headers. For the session to "stick", the client must include that cookie in every later request to the server.

But on the server side, the PHPSESSID cookie has an expiration date (in my case after 20 minutes). So in order to automate queries to the website I had to find a way to automatically extract and reuse that cookie.

Now, quoting the official documentation, mitmproxy is an interactive console program that allows traffic flows to be intercepted, inspected, modified and replayed.

So first, it's a traffic inspection tool, like Fiddler, wireshark or tcpdump. To enable it, there are only 2 steps :

• start the proxy so it listens on 0.0.0.0:8080 : mitmproxy --host. The interactive window opened should be empty, you can get the list of available commands with ?.
• configure your browser to use this adress as a proxy to access the Internet.

Now you can browse to the website you want to interract with, and mitmproxy will record the traffic "flows" generated.

Once you are done, save the record in a file : w a traffic.mitm. And exit : q y.

To experience the full capabilities of mitmproxy, launch the command again without parameters and re-open & replay the "traffic flows" file with c traffic.mitm. The arrow keys will let you navigate between the flows, and you'll be able to selectively replay a flow with r or inspect one by pressing Enter. In the flow view, you'll see the request headers and can switch to the response details by pressing Tab.

Not only mitmproxy let you replay recorded traffic, you can also programmatically modify your requests using scripts: https://docs.mitmproxy.org/stable/addons-scripting/.

That's an awesome feature, and a few weeks ago I was able in no time to write a basic script that recorded the PHPSESSID cookie generated on the first request to the website, and inject it in the following requests.

But that did not work. Given the mostly empty HTML responses generated by the server, the bare cookie wasn't enough for the session to properly "stick" and the website ro recognize me... :(

What I only realized yesterday is that I missed one key element: the srctoken input value. This doesn't seem like a widely used method, but at each request my PHP website was generating a form input tag with a different predefined value. This form parameter was then sent in the next POST request query url.

To explain more graphically, my initial mental model of the PHPSESSID cookie exchange was the following :

But this is a more complete picture :

With that final bit of information I was able to craft the following mitmproxy script :

from __future__ import print_function
import re
from libmproxy.protocol.http import decoded

def log(string):
    with open('./mitmproxy.log', 'a') as output_log:
        print(string, file=output_log)

def response(context, flow):
    log("Entering 'response' hook")
    if 'Set-Cookie' in flow.response.headers:
        for cookie in flow.response.headers['Set-Cookie']:
            if not 'PHPSESSID' in cookie:
                continue
            # storing values in the 'context' as globals wouldn't persist
            context.phpsessid = re.search('PHPSESSID=(.*);', cookie).group(1)
            log('New PHPSESSID cookie set: {}'.format(context.phpsessid))
    with decoded(flow.response):
        match = re.search('<input id="srctoken" name="srctoken" type="hidden" value="(.*)" />', flow.response.content)
        if match:
            context.srctoken = match.group(1)
            log('srctoken found: {}'.format(context.srctoken))

def request(context, flow):
    log("Entering 'request' hook")
    if not hasattr(context, 'phpsessid'):
        log("ERROR: no PHPSESSID extracted - aborting")
        return
    phpsessid_str = 'PHPSESSID={}'.format(context.phpsessid)
    if 'Cookie' in flow.request.headers:
        cookie_substituted = None
        def process_cookie(cookie):
            if not 'PHPSESSID' in cookie:
                return cookie
            cookie_substituted = cookie
            return re.sub('PHPSESSID=[^;]*', phpsessid_str, cookie)
        flow.request.headers['Cookie'] = [process_cookie(cookie) for cookie in flow.request.headers['Cookie']]
        if cookie_substituted:
            log('Substituted {} in existing cookie {}'.format(phpsessid_str, cookie_substituted))
        else:
            flow.request.headers['Cookie'].append(phpsessid_str)
            log('Added cookie {}'.format(phpsessid_str))
    else:
        flow.request.headers['Cookie'] = [phpsessid_str]
        log('Added unique cookie {}'.format(phpsessid_str))
    if hasattr(context, 'srctoken') and "application/x-www-form-urlencoded" in flow.request.headers["content-type"]:
        form = flow.request.get_form_urlencoded()
        log("Modifying srctoken: {} -> {}".format(form["srctoken"], [context.srctoken]))
        form["srctoken"] = [context.srctoken]
        flow.request.set_form_urlencoded(form)

And then use it to replay my full traffic flow :

$ mitmproxy -s sticky_phpsession.py -c traffic.mitm