Lua Notes: lua minimalism

lunar minimalism

printing in different languages
how minimalistic is Lua?
how minimalistic is Nelua?

Let's talk about some different languages first.

odin

This is /bin/false:

package main

import "core:os"

main :: proc() {
	os.exit(1)
}

And this is 'hello world':

package main

import "core:fmt"

main :: proc() {
	fmt.println("Hellope!")
}

/bin/false compiles to 36KB, or 28KB when size-optimized, and 23KB when also stripped.

'hello world' compiles to 277KB, or 169KB when size-optimized, and 155KB when also stripped.

Why the jump in binary size? core:fmt is a very convenient library that can print every possible type in the language in a nice way, and it pulls in a ton of heavy-weight code:

import "core:math/bits"               
import "core:mem"
import "core:io"
import "core:reflect"
import "core:runtime"
import "core:strconv"
import "core:strings"
import "core:time"
import "core:unicode/utf8"
import "core:intrinsics"

Meanwhile, a D hello world is just shy of a megabyte with dmd, 3.1MB with static libphobos and gdc, and manages to look much smaller with dynamic phobos.

Perl

In Perl, this is what happens when you print a hash table:

$ perl -le 'my %h = qw(a 1 b 2); print %h'
b2a1

This useless output is the new and improved version! The previous useless output had internal data about hash table buckets. Perl people use different modules, only some of them provided with perl, to print hash tables. I usually use Data::Dumper:

$ perl -MData::Dumper -le 'my %h = qw(a 1 b 2); print Dumper(\%h)'
$VAR1 = {
          'b' => '2',
          'a' => '1'
        };

This is in contrast to Nim, Clojure, Go, etc. But not Lua:

lua

$ luajit -e 'print({a=1, b=2})'
table: 0x7f5380608150
$ nelua -i '## print({a=1, b=2})'
table: 0x7f0186796c00

Lua has the inspect library to print out tables:

$ nelua -i '## print(inspect({a=1, b=2}))'
{
  a = 1,
  b = 2
}

and... Data::Dumper and 'inspect' are so much better than most built-in language facilities. For trivial stuff like this it hardly matters, but for extremely deep and nested data structures it really helps to have a depth limit, or circular-reference detection:

$ nelua -i '## local circle = {a=1, b=2}  circle["circle"] = circle  print(inspect(circle))'
<1>{
  a = 1,
  b = 2,
  circle = <table 1>
}

contrast:

user=> (def a [(atom nil)])
#'user/a
user=> (reset! (first a) a)
[#object[clojure.lang.Atom 0x4ced35ed {:status :ready, :val [#object[clojure.lang.Atom 0x4ced35ed {:status :ready, :val [#object[clojure.lang.Atom 0x4ced35ed {:status :ready, :val [#object[clojure.lang.Atom 0x4ced35ed {:status :ready, :val [#object[clojure.lang.Atom 0x4ced35ed ...this keeps going for a while

how minimalistic is Lua?

In some ways, it isn't. Lua has the full set of dynamic-language types: numbers, strings, arrays (tables), and hash tables (tables again), with automatic memory management. It has not just 'string' literals, not just "string" literals, not just [[string]] literals, but many variations on that last format. It has coroutines, string matching, closures, extensive use of local variables (that's more than Forth, OK?), as well as powerful metamethods, a useful stdlib, and language support for iterators.

But Lua wasn't arrived at through a desire to keep the feature-count low, and it's not an esoteric programming language that's intended to be a challenge to work with. The minimalism is not an end of its own, but is purposeful: to keep the implementation of Lua small, portable, and useful as an extension language.

Something I like to say about Lua is that it is what Python claims to be. It's small, simple, and easy to learn. Python is praised for all these things, but Python has never once stopped growing as a language. Today Lua still doesn't have += operators -- and neither did Python at first release, but they were added almost immediately anyway. On page xiii of Programming in Lua, Lua is said to be an excellent 'glue' language. Python also enjoyed praise as a glue language, but Python never had its design constrained by that use.

More remarks from Programming in Lua - the next page, xiv:

Lua's extensibility is so remarkable that many people regard Lua not as a language, but as a kit for building domain-specific languages.
Lua is a simple and small language. It has few (but powerful) concepts.
When we talk about portability, we are talking about running Lua on all platforms we have ever heard about

how minimalistic is Nelua?

Nelua is constrained by many of the same attitudes as Lua. Serious portability, minimal dependencies, being small - as constraints for the Nelua compiler. And for resulting programs: readable generated C code and minimal extraneous runtime code. Even an isatty() dependency for the resulting executables, that they have to perform before reasonably deciding to emit ANSI color codes, is unwanted.

For example, the Nelua 'hello world':

print 'Hello, world!'

compiles to a 104-line C file, and then a 16KB binary. The C file consists of a long C Preprocessor header and then just this:

/* ------------------------------ DECLARATIONS ------------------------------ */
typedef struct nlstring nlstring;
typedef uint8_t* nluint8_arr0_ptr;
struct nlstring {
  nluint8_arr0_ptr data;
  uintptr_t size;
};
NELUA_STATIC_ASSERT(sizeof(nlstring) == 16 && NELUA_ALIGNOF(nlstring) == 8, "Nelua and C disagree on type size or align");
static void nelua_print_1(nlstring a1);
static int nelua_main(int argc, char** argv);
/* ------------------------------ DEFINITIONS ------------------------------- */
void nelua_print_1(nlstring a1) {
  if(a1.size > 0) {
    fwrite(a1.data, 1, a1.size, stdout);
  }
  fputs("\n", stdout);
  fflush(stdout);
}
int nelua_main(int argc, char** argv) {
  nelua_print_1(((nlstring){(uint8_t*)"Hello, world!", 13}));
  return 0;
}
int main(int argc, char** argv) {
  return nelua_main(argc, argv);
}

Nelua builds a struct with a string and a length, and then the debugging print(), specialized to printing a single string, pulls out the length and writes it, then puts a newline, then flushes stdout, and then the program exits with success.

This program can get as small as 6KB:

$ nelua --cc='zig cc --target=x86_64-linux-musl' -P nogc -s --cflags=-Oz -o hi -M hi.nelua
$ ls -lh hi|cut -d' ' -f5
6.2K
$ ldd hi
	not a dynamic executable

Here's Nim's hello world:

echo "Hello, world!"

This compiles to three C files, one being 102KB of system.nim, which builds to a 67KB binary. The main C file is smaller, just 90 lines, and after a very short C Preprocessor header, is all of this:

typedef struct NimStrPayload NimStrPayload;
typedef struct NimStringV2 NimStringV2;
struct NimStrPayload {
	NI cap;
	NIM_CHAR data[SEQ_DECL_SIZE];
};
struct NimStringV2 {
	NI len;
	NimStrPayload* p;
};
typedef NimStringV2 tyArray__nHXaesL0DJZHyVS07ARPRA[1];
N_LIB_PRIVATE N_NIMCALL(void, echoBinSafe)(NimStringV2* args_p0, NI args_p0Len_0);
N_LIB_PRIVATE N_NIMCALL(void, nimTestErrorFlag)(void);
N_LIB_PRIVATE N_NIMCALL(void, atmdotdotatsdotdotatsdotdotatsnimatsnimminus2dot0dot0atslibatssystemdotnim_Init000)(void);
N_LIB_PRIVATE N_NIMCALL(void, NimMainModule)(void);
static const struct {
  NI cap; NIM_CHAR data[13+1];
} TM__qPm73pJ2y20X84Q1UM9cicw_3 = { 13 | NIM_STRLIT_FLAG, "Hello, world!" };
static NIM_CONST tyArray__nHXaesL0DJZHyVS07ARPRA TM__qPm73pJ2y20X84Q1UM9cicw_2 = {{13, (NimStrPayload*)&TM__qPm73pJ2y20X84Q1UM9cicw_3}}
;

N_LIB_PRIVATE void PreMainInner(void) {
}

N_LIB_PRIVATE int cmdCount;
N_LIB_PRIVATE char** cmdLine;
N_LIB_PRIVATE char** gEnv;
N_LIB_PRIVATE void PreMain(void) {
#if 0
	void (*volatile inner)(void);
	inner = PreMainInner;
	atmdotdotatsdotdotatsdotdotatsnimatsnimminus2dot0dot0atslibatssystemdotnim_Init000();
	(*inner)();
#else
	atmdotdotatsdotdotatsdotdotatsnimatsnimminus2dot0dot0atslibatssystemdotnim_Init000();
	PreMainInner();
#endif
}

N_LIB_PRIVATE N_CDECL(void, NimMainInner)(void) {
	NimMainModule();
}

N_CDECL(void, NimMain)(void) {
#if 0
	void (*volatile inner)(void);
	PreMain();
	inner = NimMainInner;
	(*inner)();
#else
	PreMain();
	NimMainInner();
#endif
}

int main(int argc, char** args, char** env) {
	cmdLine = args;
	cmdCount = argc;
	gEnv = env;
	NimMain();
	return nim_program_result;
}

N_LIB_PRIVATE N_NIMCALL(void, NimMainModule)(void) {
{
	echoBinSafe(TM__qPm73pJ2y20X84Q1UM9cicw_2, 1);
	nimTestErrorFlag();
}
}

This

saves argc/argv/env (which Nelua's 'arg' library only does if the variable is actually used)
calls PreMain which runs system.nim's initializer
has some nested volatile function pointer dance
finally gets to my code at NimMainModule

calls echoBinSafe against an unreadable constant

(now in system.nim) locks stdout (if built with threads)

does all of this to write the string:

NimStringV2* s;
NI i;
s = (NimStringV2*)0;
i = ((NI)0);
{
        while (1) {
                size_t T4_;
                NI TM__Q5wkpxktOdTGvlSRo9bzt9aw_99;
                if (!(i < args_p0Len_0)) goto LA3;
                if (i < 0 || i >= args_p0Len_0){ raiseIndexError2(i,args_p0Len_0-1); goto BeforeRet_;
                }                    
                s = (&args_p0[i]);
                T4_ = (size_t)0;
                T4_ = fwrite(((void*) (nimToCStringConv((*s)))), ((size_t) ((*s).len)), ((size_t)1), stdout);
                (void)(T4_);
                if (nimAddInt(i, ((NI)1), &TM__Q5wkpxktOdTGvlSRo9bzt9aw_99)) { raiseOverflow(); goto BeforeRet_;
                };
                i = (NI)(TM__Q5wkpxktOdTGvlSRo9bzt9aw_99);
        } LA3: ;
}

writes a newline
flushes stdout
unlocks stdout (if built with threads)

does some error thing

Here's just the unreadable constant:

typedef struct NimStrPayload NimStrPayload;
typedef struct NimStringV2 NimStringV2;
struct NimStrPayload {
        NI cap;
        NIM_CHAR data[SEQ_DECL_SIZE];
};
struct NimStringV2 {
        NI len;
        NimStrPayload* p;
};
typedef NimStringV2 tyArray__nHXaesL0DJZHyVS07ARPRA[1];
static const struct {   
  NI cap; NIM_CHAR data[13+1];
} TM__qPm73pJ2y20X84Q1UM9cicw_3 = { 13 | NIM_STRLIT_FLAG, "Hello, world!" };
static NIM_CONST tyArray__nHXaesL0DJZHyVS07ARPRA TM__qPm73pJ2y20X84Q1UM9cicw_2 = {{13, (NimStrPayload*)&TM__qPm73pJ2y20X84Q1UM9cicw_3}}
;

Now, the point of this isn't to beat up on Nim. Nim does not have the readability of its intermediate C as a constraint, and goals that aren't even targeted are very easy to miss. The point is just: this is a lot of stuff. Part of Nelua's minimalism is to not have this stuff.