ghsa-cpr5-fxjg-jcr6
Vulnerability from github
Published
2024-02-28 09:30
Modified
2024-02-28 09:30
Details

In the Linux kernel, the following vulnerability has been resolved:

nvmet-tcp: fix incorrect locking in state_change sk callback

We are not changing anything in the TCP connection state so we should not take a write_lock but rather a read lock.

This caused a deadlock when running nvmet-tcp and nvme-tcp on the same system, where state_change callbacks on the host and on the controller side have causal relationship and made lockdep report on this with blktests:

================================ WARNING: inconsistent lock state 5.12.0-rc3 #1 Tainted: G I

inconsistent {IN-SOFTIRQ-W} -> {SOFTIRQ-ON-R} usage. nvme/1324 [HC0[0]:SC0[0]:HE1:SE1] takes: ffff888363151000 (clock-AF_INET){++-?}-{2:2}, at: nvme_tcp_state_change+0x21/0x150 [nvme_tcp] {IN-SOFTIRQ-W} state was registered at: __lock_acquire+0x79b/0x18d0 lock_acquire+0x1ca/0x480 _raw_write_lock_bh+0x39/0x80 nvmet_tcp_state_change+0x21/0x170 [nvmet_tcp] tcp_fin+0x2a8/0x780 tcp_data_queue+0xf94/0x1f20 tcp_rcv_established+0x6ba/0x1f00 tcp_v4_do_rcv+0x502/0x760 tcp_v4_rcv+0x257e/0x3430 ip_protocol_deliver_rcu+0x69/0x6a0 ip_local_deliver_finish+0x1e2/0x2f0 ip_local_deliver+0x1a2/0x420 ip_rcv+0x4fb/0x6b0 __netif_receive_skb_one_core+0x162/0x1b0 process_backlog+0x1ff/0x770 __napi_poll.constprop.0+0xa9/0x5c0 net_rx_action+0x7b3/0xb30 __do_softirq+0x1f0/0x940 do_softirq+0xa1/0xd0 __local_bh_enable_ip+0xd8/0x100 ip_finish_output2+0x6b7/0x18a0 __ip_queue_xmit+0x706/0x1aa0 __tcp_transmit_skb+0x2068/0x2e20 tcp_write_xmit+0xc9e/0x2bb0 __tcp_push_pending_frames+0x92/0x310 inet_shutdown+0x158/0x300 __nvme_tcp_stop_queue+0x36/0x270 [nvme_tcp] nvme_tcp_stop_queue+0x87/0xb0 [nvme_tcp] nvme_tcp_teardown_admin_queue+0x69/0xe0 [nvme_tcp] nvme_do_delete_ctrl+0x100/0x10c [nvme_core] nvme_sysfs_delete.cold+0x8/0xd [nvme_core] kernfs_fop_write_iter+0x2c7/0x460 new_sync_write+0x36c/0x610 vfs_write+0x5c0/0x870 ksys_write+0xf9/0x1d0 do_syscall_64+0x33/0x40 entry_SYSCALL_64_after_hwframe+0x44/0xae irq event stamp: 10687 hardirqs last enabled at (10687): [] _raw_spin_unlock_irqrestore+0x2d/0x40 hardirqs last disabled at (10686): [] _raw_spin_lock_irqsave+0x68/0x90 softirqs last enabled at (10684): [] __do_softirq+0x608/0x940 softirqs last disabled at (10649): [] do_softirq+0xa1/0xd0

other info that might help us debug this: Possible unsafe locking scenario:

   CPU0
   ----

lock(clock-AF_INET); lock(clock-AF_INET);

*** DEADLOCK ***

5 locks held by nvme/1324: #0: ffff8884a01fe470 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xf9/0x1d0 #1: ffff8886e435c090 (&of->mutex){+.+.}-{3:3}, at: kernfs_fop_write_iter+0x216/0x460 #2: ffff888104d90c38 (kn->active#255){++++}-{0:0}, at: kernfs_remove_self+0x22d/0x330 #3: ffff8884634538d0 (&queue->queue_lock){+.+.}-{3:3}, at: nvme_tcp_stop_queue+0x52/0xb0 [nvme_tcp] #4: ffff888363150d30 (sk_lock-AF_INET){+.+.}-{0:0}, at: inet_shutdown+0x59/0x300

stack backtrace: CPU: 26 PID: 1324 Comm: nvme Tainted: G I 5.12.0-rc3 #1 Hardware name: Dell Inc. PowerEdge R640/06NR82, BIOS 2.10.0 11/12/2020 Call Trace: dump_stack+0x93/0xc2 mark_lock_irq.cold+0x2c/0xb3 ? verify_lock_unused+0x390/0x390 ? stack_trace_consume_entry+0x160/0x160 ? lock_downgrade+0x100/0x100 ? save_trace+0x88/0x5e0 ? _raw_spin_unlock_irqrestore+0x2d/0x40 mark_lock+0x530/0x1470 ? mark_lock_irq+0x1d10/0x1d10 ? enqueue_timer+0x660/0x660 mark_usage+0x215/0x2a0 __lock_acquire+0x79b/0x18d0 ? tcp_schedule_loss_probe.part.0+0x38c/0x520 lock_acquire+0x1ca/0x480 ? nvme_tcp_state_change+0x21/0x150 [nvme_tcp] ? rcu_read_unlock+0x40/0x40 ? tcp_mtu_probe+0x1ae0/0x1ae0 ? kmalloc_reserve+0xa0/0xa0 ? sysfs_file_ops+0x170/0x170 _raw_read_lock+0x3d/0xa0 ? nvme_tcp_state_change+0x21/0x150 [nvme_tcp] nvme_tcp_state_change+0x21/0x150 [nvme_tcp] ? sysfs_file_ops ---truncated---

Show details on source website

To clipboard 

{
  "affected": [],
  "aliases": [
    "CVE-2021-47041"
  ],
  "database_specific": {
    "cwe_ids": [],
    "github_reviewed": false,
    "github_reviewed_at": null,
    "nvd_published_at": "2024-02-28T09:15:40Z",
    "severity": null
  },
  "details": "In the Linux kernel, the following vulnerability has been resolved:\n\nnvmet-tcp: fix incorrect locking in state_change sk callback\n\nWe are not changing anything in the TCP connection state so\nwe should not take a write_lock but rather a read lock.\n\nThis caused a deadlock when running nvmet-tcp and nvme-tcp\non the same system, where state_change callbacks on the\nhost and on the controller side have causal relationship\nand made lockdep report on this with blktests:\n\n================================\nWARNING: inconsistent lock state\n5.12.0-rc3 #1 Tainted: G          I\n--------------------------------\ninconsistent {IN-SOFTIRQ-W} -\u003e {SOFTIRQ-ON-R} usage.\nnvme/1324 [HC0[0]:SC0[0]:HE1:SE1] takes:\nffff888363151000 (clock-AF_INET){++-?}-{2:2}, at: nvme_tcp_state_change+0x21/0x150 [nvme_tcp]\n{IN-SOFTIRQ-W} state was registered at:\n  __lock_acquire+0x79b/0x18d0\n  lock_acquire+0x1ca/0x480\n  _raw_write_lock_bh+0x39/0x80\n  nvmet_tcp_state_change+0x21/0x170 [nvmet_tcp]\n  tcp_fin+0x2a8/0x780\n  tcp_data_queue+0xf94/0x1f20\n  tcp_rcv_established+0x6ba/0x1f00\n  tcp_v4_do_rcv+0x502/0x760\n  tcp_v4_rcv+0x257e/0x3430\n  ip_protocol_deliver_rcu+0x69/0x6a0\n  ip_local_deliver_finish+0x1e2/0x2f0\n  ip_local_deliver+0x1a2/0x420\n  ip_rcv+0x4fb/0x6b0\n  __netif_receive_skb_one_core+0x162/0x1b0\n  process_backlog+0x1ff/0x770\n  __napi_poll.constprop.0+0xa9/0x5c0\n  net_rx_action+0x7b3/0xb30\n  __do_softirq+0x1f0/0x940\n  do_softirq+0xa1/0xd0\n  __local_bh_enable_ip+0xd8/0x100\n  ip_finish_output2+0x6b7/0x18a0\n  __ip_queue_xmit+0x706/0x1aa0\n  __tcp_transmit_skb+0x2068/0x2e20\n  tcp_write_xmit+0xc9e/0x2bb0\n  __tcp_push_pending_frames+0x92/0x310\n  inet_shutdown+0x158/0x300\n  __nvme_tcp_stop_queue+0x36/0x270 [nvme_tcp]\n  nvme_tcp_stop_queue+0x87/0xb0 [nvme_tcp]\n  nvme_tcp_teardown_admin_queue+0x69/0xe0 [nvme_tcp]\n  nvme_do_delete_ctrl+0x100/0x10c [nvme_core]\n  nvme_sysfs_delete.cold+0x8/0xd [nvme_core]\n  kernfs_fop_write_iter+0x2c7/0x460\n  new_sync_write+0x36c/0x610\n  vfs_write+0x5c0/0x870\n  ksys_write+0xf9/0x1d0\n  do_syscall_64+0x33/0x40\n  entry_SYSCALL_64_after_hwframe+0x44/0xae\nirq event stamp: 10687\nhardirqs last  enabled at (10687): [\u003cffffffff9ec376bd\u003e] _raw_spin_unlock_irqrestore+0x2d/0x40\nhardirqs last disabled at (10686): [\u003cffffffff9ec374d8\u003e] _raw_spin_lock_irqsave+0x68/0x90\nsoftirqs last  enabled at (10684): [\u003cffffffff9f000608\u003e] __do_softirq+0x608/0x940\nsoftirqs last disabled at (10649): [\u003cffffffff9cdedd31\u003e] do_softirq+0xa1/0xd0\n\nother info that might help us debug this:\n Possible unsafe locking scenario:\n\n       CPU0\n       ----\n  lock(clock-AF_INET);\n  \u003cInterrupt\u003e\n    lock(clock-AF_INET);\n\n *** DEADLOCK ***\n\n5 locks held by nvme/1324:\n #0: ffff8884a01fe470 (sb_writers#4){.+.+}-{0:0}, at: ksys_write+0xf9/0x1d0\n #1: ffff8886e435c090 (\u0026of-\u003emutex){+.+.}-{3:3}, at: kernfs_fop_write_iter+0x216/0x460\n #2: ffff888104d90c38 (kn-\u003eactive#255){++++}-{0:0}, at: kernfs_remove_self+0x22d/0x330\n #3: ffff8884634538d0 (\u0026queue-\u003equeue_lock){+.+.}-{3:3}, at: nvme_tcp_stop_queue+0x52/0xb0 [nvme_tcp]\n #4: ffff888363150d30 (sk_lock-AF_INET){+.+.}-{0:0}, at: inet_shutdown+0x59/0x300\n\nstack backtrace:\nCPU: 26 PID: 1324 Comm: nvme Tainted: G          I       5.12.0-rc3 #1\nHardware name: Dell Inc. PowerEdge R640/06NR82, BIOS 2.10.0 11/12/2020\nCall Trace:\n dump_stack+0x93/0xc2\n mark_lock_irq.cold+0x2c/0xb3\n ? verify_lock_unused+0x390/0x390\n ? stack_trace_consume_entry+0x160/0x160\n ? lock_downgrade+0x100/0x100\n ? save_trace+0x88/0x5e0\n ? _raw_spin_unlock_irqrestore+0x2d/0x40\n mark_lock+0x530/0x1470\n ? mark_lock_irq+0x1d10/0x1d10\n ? enqueue_timer+0x660/0x660\n mark_usage+0x215/0x2a0\n __lock_acquire+0x79b/0x18d0\n ? tcp_schedule_loss_probe.part.0+0x38c/0x520\n lock_acquire+0x1ca/0x480\n ? nvme_tcp_state_change+0x21/0x150 [nvme_tcp]\n ? rcu_read_unlock+0x40/0x40\n ? tcp_mtu_probe+0x1ae0/0x1ae0\n ? kmalloc_reserve+0xa0/0xa0\n ? sysfs_file_ops+0x170/0x170\n _raw_read_lock+0x3d/0xa0\n ? nvme_tcp_state_change+0x21/0x150 [nvme_tcp]\n nvme_tcp_state_change+0x21/0x150 [nvme_tcp]\n ? sysfs_file_ops\n---truncated---",
  "id": "GHSA-cpr5-fxjg-jcr6",
  "modified": "2024-02-28T09:30:38Z",
  "published": "2024-02-28T09:30:38Z",
  "references": [
    {
      "type": "ADVISORY",
      "url": "https://nvd.nist.gov/vuln/detail/CVE-2021-47041"
    },
    {
      "type": "WEB",
      "url": "https://git.kernel.org/stable/c/06beaa1a9f6e501213195e47c30416032fd2bbd5"
    },
    {
      "type": "WEB",
      "url": "https://git.kernel.org/stable/c/60ade0d56b06537a28884745059b3801c78e03bc"
    },
    {
      "type": "WEB",
      "url": "https://git.kernel.org/stable/c/906c538340dde6d891df89fe7dac8eaa724e40da"
    },
    {
      "type": "WEB",
      "url": "https://git.kernel.org/stable/c/999d606a820c36ae9b9e9611360c8b3d8d4bb777"
    },
    {
      "type": "WEB",
      "url": "https://git.kernel.org/stable/c/b5332a9f3f3d884a1b646ce155e664cc558c1722"
    }
  ],
  "schema_version": "1.4.0",
  "severity": []
}


Log in or create an account to share your comment.




Tags
Taxonomy of the tags.




Sightings

Author Source Type Date

Nomenclature

  • Seen: The vulnerability was mentioned, discussed, or seen somewhere by the user.
  • Confirmed: The vulnerability is confirmed from an analyst perspective.
  • Exploited: This vulnerability was exploited and seen by the user reporting the sighting.
  • Patched: This vulnerability was successfully patched by the user reporting the sighting.
  • Not exploited: This vulnerability was not exploited or seen by the user reporting the sighting.
  • Not confirmed: The user expresses doubt about the veracity of the vulnerability.
  • Not patched: This vulnerability was not successfully patched by the user reporting the sighting.