Neovim 0,11: call to plines_win() may cause Nvim exits with no error

[lyokha]

Neovim version (nvim -v | head -n1)

NVIM v0.11.0

Operating system/version

Linux LLF2TDVV6 5.15.0-43-generic #46-Ubuntu SMP Tue Jul 12 10:30:17 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

How to reproduce the issue

I was not sure how to title this. Actually, I just wanted to update treesitter code for compatibility with Neovim 0.11. Here is a trivial diff:

diff --git a/lua/ufo/provider/treesitter.lua b/lua/ufo/provider/treesitter.lua
index d440075..83e379b 100644
--- a/lua/ufo/provider/treesitter.lua
+++ b/lua/ufo/provider/treesitter.lua
@@ -1,5 +1,6 @@
 local bufmanager = require('ufo.bufmanager')
 local foldingrange = require('ufo.model.foldingrange')
+local utils = require('ufo.utils')
 
 ---@class UfoTreesitterProvider
 ---@field hasProviders table<string, boolean>
@@ -107,7 +108,12 @@ local function iterFoldMatches(bufnr, parser, root, rootLang)
         for id, nodes in pairs(match) do
             local m = metadata[id]
             if m and m.range then
-                local type = nodes.type and nodes:type() or nil
+                local type
+                if utils.has11() then
+                    type = nodes[1].type and nodes[1]:type() or nil
+                else
+                    type = nodes.type and nodes:type() or nil
+                end
                 node = MetaNode:new(m.range, type)
             elseif type(nodes) ~= "table" then
                 -- old behaviou before 0.11
@@ -180,6 +186,9 @@ function Treesitter.getFolds(bufnr)
         self.hasProviders[ft] = false
         error('UfoFallbackException')
     end
+    if utils.has11() then
+        parser:parse()
+    end
 
     local ranges = {}
     local ok, matches = getCpatureMatchesRecursively(bufnr, parser)
diff --git a/lua/ufo/utils.lua b/lua/ufo/utils.lua
index b2c211a..8bf9c74 100644
--- a/lua/ufo/utils.lua
+++ b/lua/ufo/utils.lua
@@ -5,6 +5,18 @@ local fn = vim.fn
 local cmd = vim.cmd
 local uv = vim.loop
 
+---
+---@return fun(): boolean
+M.has11 = (function()
+    local has11
+    return function()
+        if has11 == nil then
+            has11 = fn.has('nvim-0.11') == 1
+        end
+        return has11
+    end
+end)()
+
 ---
 ---@return fun(): boolean
 M.has10 = (function()

Note that according to the release notes they did:

• Query:iter_matches() correctly returns all matching nodes in a match instead of only the last node. This means that the returned table maps capture IDs to a list of nodes that need to be iterated over. For backwards compatibility, an option all=false (only return the last matching node) is provided that will be removed in a future release.

• vim.treesitter.get_parser() and vim.treesitter.start() no longer parse the tree before returning. Scripts must call LanguageTree:parse() explicitly.

  local p = vim.treesitter.get_parser(0, 'c')
  p:parse()

Hence the changes.

I use ufo to fold code blocks in markdown files on startup. It looks like

In Neovim 0.11 they, unfortunately, removed delimiters of code blocks with conceal_lines but I customized the query and returned the view.

The issue I found is that sometimes, once in 6-7 runs, nvim abruptly exits with succesfull error code ($? is 0) just after ufo successfully have rendered the folds. This happens in a blink, but I see that the folds get rendered.

My first thought was that this is related to

• Treesitter folding is now calculated asynchronously.

But this wasn't the case as I tried to use vim.g._ts_force_sync_parsing = true but nvim still exited. I began to trace ufo using log.error() and found that nvim exits during repetitive calls in

            size = lsizeObj:size(i)

This line calls pline_win() via LFFI object. I tried to replace this by LNonFFI in code: the exits have gone. I returned to LFFI and finally came to this:

diff --git a/lua/ufo/wffi.lua b/lua/ufo/wffi.lua
index 591edad..7125a9a 100644
--- a/lua/ufo/wffi.lua
+++ b/lua/ufo/wffi.lua
@@ -19,7 +19,8 @@ end
 ---@return number
 function M.plinesWin(winid, lnum)
     local wp = findWin(winid)
-    return C.plines_win(wp, lnum, true)
+    local n = C.plines_win(wp, lnum, true)
+    return n
 end
 
 ---

Surprisingly to me, the exits have gone. I cannot explain how this affects the behaviour as I do not know Lua internalls well. Perhaps this is a transitory bug or something related to tail recursion. I never experienced this in Neovim 0.10, so it could be a bug of Neovim 0.11.

Expected behavior

No unexpected exits.

Actual behavior

Nvim exits unexpectedly sometimes. There is a solution (or maybe workaround), see above.

[kevinhwang91]

As my understanding, changing below code can make the issue gone?

diff --git a/lua/ufo/wffi.lua b/lua/ufo/wffi.lua
index 591edad..7125a9a 100644
--- a/lua/ufo/wffi.lua
+++ b/lua/ufo/wffi.lua
@@ -19,7 +19,8 @@ end
 ---@return number
 function M.plinesWin(winid, lnum)
     local wp = findWin(winid)
-    return C.plines_win(wp, lnum, true)
+    local n = C.plines_win(wp, lnum, true)
+    return n
 end
 
 ---

Look like luajit issue. What's lua =jit.version output? Have you seen any trace from coredumpctl before the changing?
How about change return C.plines_win(wp, lnum, true) to return tonumber(C.plines_win(wp, lnum, true))?

[lyokha]

As my understanding, changing below code can make the issue gone?

Yes.

What's lua =jit.version output?

:lua =jit.version
version      version_num                                                                                                                                                                      
"LuaJIT 2.1.1741730670"

Have you seen any trace from coredumpctl before the changing?

Thank you for the tip about coredumpctl. Yes, there are core dumps on this:

           PID: 1721969 (nvim)
           UID: 1000 (alexeyradkov)
           GID: 1000 (alexeyradkov)
        Signal: 11 (SEGV)
     Timestamp: Sat 2025-03-29 13:01:28 +04 (1min 34s ago)
  Command Line: nvim --embed README.md
    Executable: /usr/local/bin/nvim
 Control Group: /user.slice/user-1000.slice/session-5.scope
          Unit: session-5.scope
         Slice: user-1000.slice
       Session: 5
     Owner UID: 1000 (alexeyradkov)
       Boot ID: 43df24e91aa44e3b956c754890457fbd
    Machine ID: e290fd21df2f4899abc4af2a8fe7f2e1
      Hostname: LLF2TDVV6
       Storage: /var/lib/systemd/coredump/core.nvim.1000.43df24e91aa44e3b956c754890457fbd.1721969.1743238888000000.zst (present)
     Disk Size: 4.1M
       Message: Process 1721969 (nvim) of user 1000 dumped core.
                
                Found module /home/alexeyradkov/.local/share/nvim/plugged/nvim-treesitter/parser/c.so with build-id: 7fac446249936638e8b410c7761a84f5e5408812
                Found module /home/alexeyradkov/.local/share/nvim/plugged/nvim-treesitter/parser/nginx.so with build-id: c79bed0d0ab2ceff983bd3e6a9dc5f464e591edb
                Found module /home/alexeyradkov/.local/share/nvim/plugged/nvim-treesitter/parser/markdown.so with build-id: 3b13a52866eedbcfb2544fe2ee1ea860306ab800
                Found module linux-vdso.so.1 with build-id: 9331511ef5fa11346249a031fc4f45a9beada3fe
                Found module markdown_inline.so with build-id: 91b9b0769bd1e2d21bd9d19dd476ee560ee60645
                Found module libstdc++.so.6 with build-id: e37fe1a879783838de78cbc8c80621fa685d58a2
                Found module ld-linux-x86-64.so.2 with build-id: e4de036b19e4768e7591b596c4be9f9015f2d28a
                Found module libc.so.6 with build-id: cd410b710f0f094c6832edd95931006d883af48e
                Found module libgcc_s.so.1 with build-id: e3a44e0da9c6e835d293ed8fd2882b4c4a87130c
                Found module libm.so.6 with build-id: 7d8778fca8ea4621b268cc03662855d0cd983439
                Found module nvim with build-id: 75bd58648fdde4df88b3e1d99d44e61f132acc8e
                Stack trace of thread 1721969:
                #0  0x000055e3b579ccd1 lj_snap_restore (nvim + 0x45dcd1)
                #1  0x000055e3b5783a41 trace_exit_cp (nvim + 0x444a41)
                #2  0x000055e3b578f69c lj_vm_cpcall (nvim + 0x45069c)
                #3  0x000055e3b57850e8 lj_trace_exit (nvim + 0x4460e8)
                #4  0x000055e3b57912af lj_vm_exit_handler (nvim + 0x4522af)
                #5  0xffffffffffffffff n/a (n/a + 0x0)

I guess though that this doesn't really show the cause.

How about change return C.plines_win(wp, lnum, true) to return tonumber(C.plines_win(wp, lnum, true))?

Yes, this fixes the issue.

[kevinhwang91]

bqf also use C.plines_win(wp, lnum, true),
kevinhwang91/nvim-bqf#159 says crash after 86046c, I guess it's related to conceal_line, please printC.plines_win(wp, lnum, true) and provide more info. Reproducing steps may help.

[lyokha]

Unlikely this is related to conceal_lines, because I deleted them from queries in order to restore folds a-la Neovim 0.10 as shown on the image in the first message.

I made this change:

diff --git a/lua/ufo/utils.lua b/lua/ufo/utils.lua
index b2c211a..bfe0792 100644
--- a/lua/ufo/utils.lua
+++ b/lua/ufo/utils.lua
@@ -291,7 +303,8 @@ function M.evaluateTopline(winid, line, lsizes)
         local lnum = M.foldClosed(winid, i)
         log.info('lnum:', lnum, 'i:', i)
         if lnum == -1 then
-            size = lsizeObj:size(i)
+            log.error(lnum. i, lsizeObj:size(i))
+            size = 1
         else
             size = 1
             i = lnum

got the crash, error log doesn't contain interesting sizes:

[25-03-29 17:33:47] [ERROR] utils.lua:306 : -1 207 1
[25-03-29 17:33:47] [ERROR] utils.lua:306 : -1 206 1
[25-03-29 17:33:47] [ERROR] utils.lua:306 : -1 205 1
[25-03-29 17:33:47] [ERROR] utils.lua:306 : -1 204 1
[25-03-29 17:33:47] [ERROR] utils.lua:306 : -1 203 1
[25-03-29 17:33:47] [ERROR] utils.lua:306 : -1 202 1
[25-03-29 17:33:47] [ERROR] utils.lua:306 : -1 201 1
[25-03-29 17:33:47] [ERROR] utils.lua:306 : -1 165 1
[25-03-29 17:33:47] [ERROR] utils.lua:306 : -1 164 1
[25-03-29 17:33:47] [ERROR] utils.lua:306 : -1 163 1
[25-03-29 17:33:47] [ERROR] utils.lua:306 : -1 162 1
[25-03-29 17:33:47] [ERROR] utils.lua:306 : -1 161 1
[25-03-29 17:33:47] [ERROR] utils.lua:306 : -1 160 1
[25-03-29 17:33:47] [ERROR] utils.lua:306 : -1 155 1
[25-03-29 17:33:47] [ERROR] utils.lua:306 : -1 154 1
[25-03-29 17:33:47] [ERROR] utils.lua:306 : -1 153 1
[25-03-29 17:33:47] [ERROR] utils.lua:306 : -1 152 1


[25-03-29 17:33:56] [ERROR] utils.lua:306 : -1 207 1
[25-03-29 17:33:56] [ERROR] utils.lua:306 : -1 206 1
[25-03-29 17:33:56] [ERROR] utils.lua:306 : -1 205 1
[25-03-29 17:33:56] [ERROR] utils.lua:306 : -1 204 1
[25-03-29 17:33:56] [ERROR] utils.lua:306 : -1 203 1
[25-03-29 17:33:56] [ERROR] utils.lua:306 : -1 202 1
[25-03-29 17:33:56] [ERROR] utils.lua:306 : -1 201 1
[25-03-29 17:33:56] [ERROR] utils.lua:306 : -1 165 1
[25-03-29 17:33:56] [ERROR] utils.lua:306 : -1 164 1
[25-03-29 17:33:56] [ERROR] utils.lua:306 : -1 163 1
[25-03-29 17:33:56] [ERROR] utils.lua:306 : -1 162 1
[25-03-29 17:33:56] [ERROR] utils.lua:306 : -1 161 1
[25-03-29 17:33:56] [ERROR] utils.lua:306 : -1 160 1


[25-03-29 17:34:05] [ERROR] utils.lua:306 : -1 207 1
[25-03-29 17:34:05] [ERROR] utils.lua:306 : -1 206 1

I suppose this is because crash happens as soon as getting lsizeObj:size(i) and it doen't reach printing. OTOH, crash happens on different lines. I looked into file and found that lines of crashes (152, 160, 206) are located outside holds as expected (as soon as lnum is -1).

Another interesting fact. Firstly I made this change:

diff --git a/lua/ufo/wffi.lua b/lua/ufo/wffi.lua
index 591edad..6bd2b8f 100644
--- a/lua/ufo/wffi.lua
+++ b/lua/ufo/wffi.lua
@@ -5,6 +5,7 @@ local M = {}
 local utils
 local C
 local ffi
+local log = require('ufo.lib.log')
 
 local CPos_T
 
@@ -19,6 +20,7 @@ end
 ---@return number
 function M.plinesWin(winid, lnum)
     local wp = findWin(winid)
+    log.error(C.plines_win(wp, lnum, true))
     return C.plines_win(wp, lnum, true)
 end

The crashes have gone i this case as well.

[lyokha]

Here are my ufo settings:

  local ufo_ft_map = {
    lsp = { 'c', 'cpp', 'go', 'haskell', 'perl', 'rust' }
  }

  local ufo_virt_text = function(virtText, lnum, endLnum, width, truncate)
    local newVirtText = {}
    local suffix = (' 󰏢 %d '):format(endLnum - lnum)
    local sufWidth = vim.fn.strdisplaywidth(suffix)
    local targetWidth = width - sufWidth
    local curWidth = 0
    for _, chunk in ipairs(virtText) do
      local chunkText = chunk[1]
      local chunkWidth = vim.fn.strdisplaywidth(chunkText)
      if targetWidth > curWidth + chunkWidth then
        table.insert(newVirtText, chunk)
      else
        chunkText = truncate(chunkText, targetWidth - curWidth)
        local hlGroup = chunk[2]
        table.insert(newVirtText, {chunkText, hlGroup})
        chunkWidth = vim.fn.strdisplaywidth(chunkText)
        -- str width returned from truncate() may be less than 2nd argument,
        -- need padding
        if curWidth + chunkWidth < targetWidth then
            suffix = suffix .. (' '):rep(targetWidth - curWidth - chunkWidth)
        end
        break
      end
      curWidth = curWidth + chunkWidth
    end
    table.insert(newVirtText, {suffix, 'FoldAnno'})
    return newVirtText
  end

  require'ufo'.setup {
    provider_selector = function(bufnr, filetype, buftype)
      if buftype == 'nofile' then
        return ''
      end
      local function contains(tbl, val)
        if tbl then
          for i = 1, #tbl do
            if tbl[i] == val then
              return true
            end
          end
        end
        return false
      end
      local ret = ''
      if contains(ufo_ft_map.disabled, filetype) then
        return ''
      elseif contains(ufo_ft_map.lsp, filetype) then
        ret = { 'lsp', 'treesitter' }
      else
        ret = { 'treesitter', 'indent' }
      end
      vim.keymap.set('n', 'zR', require'ufo'.openAllFolds,
                     { buffer = true })
      vim.keymap.set('n', 'zM', require'ufo'.closeAllFolds,
                     { buffer = true })
      vim.keymap.set('n', 'zr', require'ufo'.openFoldsExceptKinds,
                     { buffer = true })
      vim.keymap.set('n', 'zm', require'ufo'.closeFoldsWith,
                     { buffer = true })
      return ret
    end,
    fold_virt_text_handler = ufo_virt_text,
    close_fold_kinds_for_ft = {
      default = { 'comment', 'imports' },
      pandoc = { 'fenced_code_block' },
      rst = { 'directive' }
    },
    preview = {
      win_config = {
        border = 'none',
        winhighlight = 'NormalFloat:Folded,FloatBorder:Folded',
        winblend = 0
      }
    }
  }

  vim.keymap.set('n', 'zK', function()
    require'ufo'.peekFoldedLinesUnderCursor()
  end)

(I wrap this into lua <<EOF ... EOF in my init.vim file.

I'm getting crashes on README.md from https://github.com/lyokha/nginx-easy-context.

I also use this trick:

  -- Register filetype pandoc as language markdown
  vim.treesitter.language.register('markdown', 'pandoc')

to combine markdown with the rich set of features from the vim-pandoc plugin.

[kevinhwang91]

diff --git a/lua/ufo/wffi.lua b/lua/ufo/wffi.lua
index 591edad..01f28fe 100644
--- a/lua/ufo/wffi.lua
+++ b/lua/ufo/wffi.lua
@@ -88,6 +88,8 @@ local function init()
         int plines_win_nofill(win_T *wp, linenr_T lnum, bool winheight);
     ]])
     CPos_T = ffi.typeof('pos_T')
+    jit.off(findWin, true)
+    jit.off(M.plinesWin, true)
 end
 
 init()

If this fix the issue, please comment out jit.off(findWin, true) or jit.off(M.plinesWin, true) to find out which function optimized by jit cause the issue.

[lyokha]

Yes, jit.off fixed the issue. Surprisingly, in any combination: I made many runs to make sure that jit.off(findWin, true) and jit.off(M.plinesWin, true) being set separately made the crashes have gone.

[kevinhwang91]

Thanks, fixed now.

[lyokha]

Thanks! So you just turned jit off for everything?
What do you think about applying changes related to treesitter in Neovim 0.11 (the patch I showed in the 1st message)? I can send PR if you prefer.

[lyokha]

Btw wrapping call to C.plines_win() in tonumber() also worked well.

[kevinhwang91]

Thanks! So you just turned jit off for everything?

Just turn off the expose functions in wffi.

What do you think about applying changes related to treesitter in Neovim 0.11 (the patch I showed in the 1st message)? I can send PR if you prefer.

Yes, but you should check out the latest code about treesitter provider, your provided code is stale.

Btw wrapping call to C.plines_win() in tonumber() also worked well.

It doesn't make a lot of sense, it's just hacking the jit, it doesn't avoid the bug. Turning off the optimization of functions directly is the most direct solution. We do not know what optimizations jit has done, it is a black box for us.