Lua Notes

from discord:

Ideally a beginner in Nelua would start coding just like he would code in Lua using tables, then as his knowledge grows he would gradually learn how to makes things more efficient by using records and C like features, and maybe disabling GC. That is the vision I had for the language, were you can start easy for quick prototyping and day by day learn more advanced stuff, but it's not there yet. Right now the language is more for semi-advanced programmers because you may have to deal with custom types, allocations and pointers. But it's already a lot usable for me and for some people to replace C/C++.

nice things about nelua

1. very reasonable influences: Lua, C, Nim, Odin, Zig. It's trying to be a BetterC for Lua people, with Lua-like tastefulness in its reliable feature set.
2. specifically I quite like
   • Lua's syntax, where defer print(1) print(2) end on one line is fine, and with a very hard to misuse syntax to drop parens, and I like multiple return values for go/odin-style error handling.
   • C's semantics, with straightforward value and pointer types, with none of C++'s rr-values, references, std::forward, move semantics, 300 ways to initialize a variable most of which are wrong, etc.
   • Nim's FFI and C interaction, very much on display here
   • Nim's and Zig's metaprogramming and having a powerful language at compiletime
   • Odin and Zig's focus on manual memory management with allocators, although I'm not sure what specific influences there are
3. it's a manual-memory-management language that offers custom allocators while having a GC where that's appropriate, and as a safety net.
4. the generated C is remarkably easy to read and plays well with C tooling like gdb, valgrind, ltrace, and C compiler extensions
5. the reasonable C also help it get quicker builds from tcc and more reliable cross-compiling with zig
6. nelua itself is easy and cheap to build. No LLVM. No weird dependencies. Builds from scratch in 3 minutes on a single-core RISC V machine
7. it also seems to compile reasonably quickly. On a slightly complex unix filter ported to a few languages on underpowered arm64 linux,

   language	fast build	prod build	build remarks
   nelua	1.5s	5s	tcc, gcc
   v	2.3s	over a minute	no flags (tcc), -prod
   go	0.5s	27s	gc, tinygo
   nim	8.6s	17s	no tcc because nim can't build with it on this platform; -d:release

a Lua preprocessor

There are ancient languages that offer the same language at both compile-time and runtime: Forth, Common Lisp.

There are more recent languages that do this: D, Nim, Zig, Rust.

There's a lot to say about the compile-time character of these languages, but to stick with Nelua: it is very unusual for having a powerful compile-time language that is very different from the runtime language. Lua has dynamic typing, is GC'd, does everything with tables, has a completely different performance profile, etc. Nelua is also different from all above languages in that Lua really is a preprocessor: it can't call your Nelua functions, and its own functions can't be called at runtime.

Nelua is also different from most of the above languages (more, the 'recent' ones) in that Lua has impressive responsibilities and is part of normal programming - for example, generics are a complete product of Lua. Every standard library collection is defined in a big Lua function that gets specialized (and memoized) by another Lua function. The easiest way to write a generic function is to take an auto parameter and then ask, in Lua, which concrete type you've received. With Lua you can ask (as the 'arg' library does) "is anyone actually using this variable?" to decide whether to emit the code that populates that variable. Many normal tasks that you would expect the compiler to do, Lua actually does. It is very powerful and it is also very distinct in code (it's in separate files or in special brackets) and in programs (there's no Lua in a Nelua binary).

Maybe, it's actually a bad design constraint for the compile-time language to be the 'same' language as the runtime language. Maybe this constraint limits the value that programmers could be getting out of such languages. Maybe the recent languages are designed around too negative a reaction to abuses of the C Preprocessor, or are too narrowly interested in specific utility (like CTFE as a performance trick).

Anyway, it's a neat aspect of Nelua's design, and it's not simply "a powerful macro system" as a feature.

memory management

Manual memory management has an unpleasant dichotomy of stack (fast-allocating, trivially managed memory, but static in size and small) vs. heap (slow-allocating, tedious and error-prone to manage memory, but dynamic in size and large). GC is a solution that takes the hassle out of the heap option, with some runtime implications. A borrow checker is a solution that prevents errors with the heap option, with some heavy program-structure implications.

Custom allocators break the dichotomy and give you lots of options with different implications for your programs. You can have fast allocations and trivial management provided you clean everything up at the same time (arena allocator), and you can have fast allocations and never need to clean up small amounts of data with short lifetimes (temp allocator), and you can have an allocator that always fails (nil allocator), and more.

In addition to custom allocators, nelua has defer and scoped <close> variables. (Not having exceptions is also a plus, but there are some threats to add them...)

Also, the 'no gc' option is a real one. You can write a program against the GC, then disable the GC, then gradually fix the program to stop leaking with normal memory analysis tools like valgrind. 'No gc' is neither about picking your poison among automatic memory management nor about choosing to write in a completely incompatible manner. I've gone down the belated-nogc route and I had to replace f(allocatingf()) with local s <close> = allocatingf() f(s) in a few cases, and it wasn't too bad.

2 months later, during Advent of Code

This contest exposed a lot of dissatisfactions with Nelua in its current state. I had boat loads of errors due to accidental tautologies by testing against strings like 'a' instead of bytes like 'a'_u8. I had a function return a boolean because it was typed as returning bool, silently ignoring my later changing the code in that function to return a number. I had tons of off-by-one errors due to mixed indexing types. I had memory corruption errors. That error messages don't include line numbers constantly had me re-running with -d and then pulling up the C code to map the error back to the error in my code, and I sometimes made mistakes with this mapping. I was constantly having to write little utilities to dump out data that would be printed usefully by default in most languages. I sometimes had to bisect my code to find missing 'end' keywords that resulted in an error uselessly pointing to the bottom of the file. Sometimes I had a preprocessor error that usefully pointed to the top of the file. Under normal circumstances and when you know what you're doing, Nelua is not at all painful to write, but when already stressed there's a huge cost from the poor error responses.

I expect this will improve, especially with the high-level goal of the language - that people should be able to write Lua-like code and then gradually make it lower level. It's also very normal for less developed tooling to work fine for skilled users (like the designer) and only expose sharp edges to clumsy users (like someone trying to learn the language). This was precisely my biggest annoyance with Nim, that I'd forget syntax and then have strange "accepts invalid" errors with it.

My thoughts on Nelua's positives haven't changed. It's still simple, it's still very easy to write efficient code, and still very easy to predict how that code translates to C, and the resulting code is easy to follow.

I also started to mitigate some of the pain: I relied on the GC more, I used nelua-batteries' json and inspector libraries to dump data, and I got better at finding the problem with gdb output.

But if you're writing experimental code you may do a lot better with another language.