Most recent entries from all

CVE-2021-43854 — Inefficient Regular Expression Complexity in nltk

Affected: nltk

NLTK (Natural Language Toolkit) is a suite of open source Python modules, data sets, and tutorials supporting research and development in Natural Language Processing. Versions prior to 3.6.5 are vulnerable to regular expression denial of service (ReDoS) attacks. The vulnerability is present in PunktSentenceTokenizer, sent_tokenize and word_tokenize. Any users of this class, or these two functions, are vulnerable to the ReDoS attack. In short, a specifically crafted long input to any of these vulnerable functions will cause them to take a significant amount of execution time. If your program relies on any of the vulnerable functions for tokenizing unpredictable user input, then we would strongly recommend upgrading to a version of NLTK without the vulnerability. For users unable to upgrade the execution time can be bounded by limiting the maximum length of an input to any of the vulnerable functions. Our recommendation is to implement such a limit.

GHSA-f8m6-h2c7-8h9x — Inefficient Regular Expression Complexity in nltk (word_tokenize, sent_tokenize)

Affected: PyPI: nltk

### Impact The vulnerability is present in [`PunktSentenceTokenizer`](https://www.nltk.org/api/nltk.tokenize.punkt.html#nltk.tokenize.punkt.PunktSentenceTokenizer), [`sent_tokenize`](https://www.nltk.org/api/nltk.tokenize.html#nltk.tokenize.sent_tokenize) and [`word_tokenize`](https://www.nltk.org/api/nltk.tokenize.html#nltk.tokenize.word_tokenize). Any users of this class, or these two functions, are vulnerable to a Regular Expression Denial of Service (ReDoS) attack. In short, a specifically crafted long input to any of these vulnerable functions will cause them to take a significant amount of execution time. The effect of this vulnerability is noticeable with the following example: ```python from nltk.tokenize import word_tokenize

n = 8 for length in [10**i for i in range(2, n)]: # Prepare a malicious input text = "a" * length start_t = time.time() # Call `word_tokenize` and naively measure the execution time word_tokenize(text) print(f"A length of {length:<{n}} takes {time.time() - start_t:.4f}s") ``` Which gave the following output during testing: ```python A length of 100 takes 0.0060s A length of 1000 takes 0.0060s A length of 10000 takes 0.6320s A length of 100000 takes 56.3322s ... ``` I canceled the execution of the program after running it for several hours.

If your program relies on any of the vulnerable functions for tokenizing unpredictable user input, then we would strongly recommend upgrading to a version of NLTK with…

PYSEC-2021-859

Affected: PyPI: nltk