AVID-2023-V016
Vulnerability from avid – Published: 2023-03-31 – Updated: 2023-03-31 ATLAS Case Study| Artifact | Type |
|---|---|
| MathGPT (https://mathgpt.streamlit.app/) | System |
| URL | Label |
|---|---|
| https://atlas.mitre.org/studies/AML.CS0016 | Achieving Code Execution in MathGPT via Prompt Injection |
| https://arxiv.org/abs/2103.03874 | Measuring Mathematical Problem Solving With the MATH Dataset |
| https://arxiv.org/abs/2110.14168 | Training Verifiers to Solve Math Word Problems |
| https://lspace.swyx.io/p/reverse-prompt-eng | Reverse Prompt Engineering for Fun and (no) Profit |
| https://research.nccgroup.com/2022/12/05/explorin… | Exploring prompt-based attacks |
{
"affects": {
"artifacts": [
{
"name": "MathGPT (https://mathgpt.streamlit.app/)",
"type": "System"
}
],
"deployer": [
"MathGPT (https://mathgpt.streamlit.app/)"
],
"developer": []
},
"credit": null,
"data_type": "AVID",
"data_version": "0.2",
"description": {
"lang": "eng",
"value": "The publicly available Streamlit application [MathGPT](https://mathgpt.streamlit.app/) uses GPT-3, a large language model (LLM), to answer user-generated math questions.\n\nRecent studies and experiments have shown that LLMs such as GPT-3 show poor performance when it comes to performing exact math directly[\u003csup\u003e\\[1\\]\u003c/sup\u003e][1][\u003csup\u003e\\[2\\]\u003c/sup\u003e][2]. However, they can produce more accurate answers when asked to generate executable code that solves the question at hand. In the MathGPT application, GPT-3 is used to convert the user\u0027s natural language question into Python code that is then executed. After computation, the executed code and the answer are displayed to the user.\n\nSome LLMs can be vulnerable to prompt injection attacks, where malicious user inputs cause the models to perform unexpected behavior[\u003csup\u003e\\[3\\]\u003c/sup\u003e][3][\u003csup\u003e\\[4\\]\u003c/sup\u003e][4]. In this incident, the actor explored several prompt-override avenues, producing code that eventually led to the actor gaining access to the application host system\u0027s environment variables and the application\u0027s GPT-3 API key, as well as executing a denial of service attack. As a result, the actor could have exhausted the application\u0027s API query budget or brought down the application.\n\nAfter disclosing the attack vectors and their results to the MathGPT and Streamlit teams, the teams took steps to mitigate the vulnerabilities, filtering on select prompts and rotating the API key.\n\n[1]: https://arxiv.org/abs/2103.03874 \"Measuring Mathematical Problem Solving With the MATH Dataset\"\n[2]: https://arxiv.org/abs/2110.14168 \"Training Verifiers to Solve Math Word Problems\"\n[3]: https://lspace.swyx.io/p/reverse-prompt-eng \"Reverse Prompt Engineering for Fun and (no) Profit\"\n[4]: https://research.nccgroup.com/2022/12/05/exploring-prompt-injection-attacks/ \"Exploring prompt-based attacks\""
},
"impact": {
"avid": {
"lifecycle_view": [
"L06: Deployment"
],
"risk_domain": [
"Security"
],
"sep_view": [
"S0403: Adversarial Example"
],
"taxonomy_version": "0.2"
}
},
"last_modified_date": "2023-03-31",
"metadata": {
"vuln_id": "AVID-2023-V016"
},
"problemtype": {
"classof": "ATLAS Case Study",
"description": {
"lang": "eng",
"value": "Achieving Code Execution in MathGPT via Prompt Injection"
},
"type": "Advisory"
},
"published_date": "2023-03-31",
"references": [
{
"label": "Achieving Code Execution in MathGPT via Prompt Injection",
"type": "source",
"url": "https://atlas.mitre.org/studies/AML.CS0016"
},
{
"label": "Measuring Mathematical Problem Solving With the MATH Dataset",
"type": "source",
"url": "https://arxiv.org/abs/2103.03874"
},
{
"label": "Training Verifiers to Solve Math Word Problems",
"type": "source",
"url": "https://arxiv.org/abs/2110.14168"
},
{
"label": "Reverse Prompt Engineering for Fun and (no) Profit",
"type": "source",
"url": "https://lspace.swyx.io/p/reverse-prompt-eng"
},
{
"label": "Exploring prompt-based attacks",
"type": "source",
"url": "https://research.nccgroup.com/2022/12/05/exploring-prompt-injection-attacks"
}
],
"reports": null
}
Drag nodes to arrange them. Double-click a related vulnerability to open it, or a source to visit its link. The Physics button on the left tunes the layout and the simulation; the legend at the bottom left hides or shows nodes by kind and by sighting type.
Sightings
| Author | Source | Type | Date | Other |
|---|
Nomenclature
- Seen: The vulnerability was mentioned, discussed, or observed by the user.
- Confirmed: The vulnerability has been validated from an analyst's perspective.
- Published Proof of Concept: A public proof of concept is available for this vulnerability.
- Exploited: The vulnerability was observed as exploited by the user who reported the sighting.
- Patched: The vulnerability was observed as successfully patched by the user who reported the sighting.
- Not exploited: The vulnerability was not observed as exploited by the user who reported the sighting.
- Not confirmed: The user expressed doubt about the validity of the vulnerability.
- Not patched: The vulnerability was not observed as successfully patched by the user who reported the sighting.
Vulnerability timeline
Sightings and recorded dates, grouped by day · UTC
How durations are calculated
Following the VTEM framework (Black Hat USA 2015), time to patch starts when the vendor or a coordinator was informed, total time to patch at discovery, and disclosure to patch at public disclosure; vendor response time runs from the report to its acknowledgement. Time to exploit and time unpatched start at the earliest discovery or report, falling back to public disclosure. Unpatched time ends at the first recorded patch, or the calculation time when no patch is recorded. Exploitation to patch shows the interval between those two events; a published proof of concept is a public exploit, not exploitation.
Each result shows its source dates. A negative interval means the end event came first; when date-only inputs leave the order open, the result says so. Date-only inputs are approximate. The earliest evidence wins: KEV report dates and sightings are labelled as such. Records describing another vulnerability are listed but never calculated from, and a PoC alone does not establish exploitation. Missing patch evidence does not prove that no patch exists. Filters below change the event list only.
Loading timeline…
The approach is described in our paper Mapping CVEs to MITRE ATT&CK Techniques: A Curated Gold-Set Classifier and the Limits of LLM-Assisted Label Expansion.
Browse all ATT&CK techniques and the vulnerabilities related to each.
Related by attack behaviour
Vulnerabilities whose description is nearest to this one in the vector space of the CIRCL/vulnerability-attack-technique-biencoder model. This is a similarity search over the bi-encoder space (plain cosine), not a classification, and it has no measured accuracy.