Debugging a seg fault

[BradHutchings]

Sometimes, I feel like a script kiddy. Like now. LOL.

Inspired by llamafile, but needing to be able to work with the latest and greatest small models as they became available, I forked llama.cpp and gave it a Cosmo treatment with cosmocc-4.0.2.zip,

https://github.com/BradHutchings/Mmojo-Server

The convenience of having one binary output for x86 / ARM and Linux / macOS / Windows is amazing! So is packaging support files in the APE zip. Since llama.cpp adds CPU and GPU support at compile time, I take a bit of a performance hit, but it's quite livable with small models and small devices.

I have run into a seg fault when loading some models (IBM Granite, Meta Llama, OpenAI gpt-oss) on x86, but not ARM. My best not very good or informed effort with gdb is telling me that it happens in the Cosmo lib:

common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
[New LWP 1407]
[New LWP 1408]

Thread 1 "mmojo-server" received signal SIGSEGV, Segmentation fault.
0x00000000013a8c38 in _mm_load_si128 (__P=0x0) at ./third_party/intel/emmintrin.internal.h:536
warning: 536    ./third_party/intel/emmintrin.internal.h: No such file or directory
(gdb) bt
#0  0x00000000013a8c38 in _mm_load_si128 (__P=0x0) at ./third_party/intel/emmintrin.internal.h:536
#1  memchr_sse (s=0x0, c=0 '\000', n=0) at libc/intrin/memchr.c:42
#2  0x0000004100000041 in ?? ()
#3  0x0000000000000000 in ?? ()
(gdb)

I think I've set environment variables close to right to get a good stack trace, but you can see it's not a great stack trace on the app.

export CC="cosmocc -I$(pwd)/cosmocc/include -L$(pwd)/cosmocc/lib -O0 -mdbg -g"
export CXX="cosmocc -I$(pwd)/cosmocc/include \
    -I$(pwd)/cosmocc/include/third_party/libcxx \
    -L$(pwd)/cosmocc/lib -L$(pwd)/openssl -O0 -mdbg -g"
export AR="cosmoar -mdbg"

So two questions...

1. Are there any plans for an update to cosmocc on the horizon which might address this issue? Is there anything I can do to help with that?
2. Any debugging hints (e.g. compiler flags) so I could offer better information, or maybe fix something in llama.cpp? I can code is all sorts of languages, but I don't play with debugging with gdb much.

Thanks!

-Brad

[BradHutchings]

Let me add something that should have been obvious to me if I'd dug as deeply as a I could with the stack trace above.

There is a reason it doesn't crash on ARM, but does on x86. That is what is what is effectively a bounds check by the for loop in memchr_pure:

static inline const unsigned char *memchr_pure(const unsigned char *s,
                                               unsigned char c, size_t n) {
  size_t i;
  for (i = 0; i < n; ++i) {
    if (s[i] == c) {
      return s + i;
    }
  }
  return 0;
}

Meanwhile, memchr_sse goes straight into dereferencing the NULL *s.

I can probably find where to fix this in llama.cpp by checking that pointer before calling memchr. But the default C library memchr seems more forgiving than Cosmo's SSE implementation.

---

EDIT: I wasn't quite right on ARM running memchr_pure, but ARM is not running memchr_sse.

[BradHutchings]

grep tells me there are no direct called to memchr from the llama.cpp source. There are several from openssl, which I need to build with Cosmo to get a working llama-server.

Getting a full stack crawl would be very helpful in fixing this outside Cosmo.

[BradHutchings]

After spending the better part of my day messing with this, I'd like to put this line at the beginning of memchr_sse in libc/intrin/memchr.c:

 if (n == 0) return 0;

That should fix the problem I'm seeing in the least intrusive way.

But for the life of me, I can't figure out what to copy where so I can use my built version of Cosmo. I figured out:

make
make aarch64

But what I need to copy to my .cosmo folder to make my build available?

[BradHutchings]

I figured it out and am closing this discussion.

#1439