stz

the hidden parameter

the hidden parameter
Photo by Stefan Steinbauer / Unsplash

STZ is multithreaded. Therefore globals that aren't const are very bad. We still need a way to conveniently pass around some 'global like' information though. Let's consider the most important one - memory management.

There's a school of thought that says you have two allocators - a heap allocator and a temporary allocator. The idea being that you allocate your intermediate data with the temp allocator and allocate the final product with the heap allocator.

I'd like to borrow a page out of the world of Garbage Collection and instead extract the data from an allocator we want to keep and copy it in to the parent allocator. This is effectively the mark and sweep algorithm except on a much more constrained data set and once the sweep is done, the move is complete, the rest of the data goes byebye.

As such there's only ever one active allocator running at any time. We can pass it around and use it:

// concatenate two strings together in to a new string
[a concat: b allocator: alloc]
[String, String, Allocator -> result: String |
  result length: a length + b length.
  result data: (alloc allocate: result length.
  a copy> result.
  b copy> result[a length:]].

But who wants to pass around an allocator everywhere? We want to stick it in the 'global like' information. As such we need a magic context variable that we pass around behind the scenes. A hidden parameter.

We can change the signature to have a default that pulls the current allocator from the hidden parameter context:

[a concat: b allocator: alloc]
[a: String, b: String,
 alloc: Allocator = context allocator -> result: String |
 ...]

Because allocator is an optional parameter, we have just defined two methods: concat:allocator: and concat:. We can now utilise the easier call with the context allocator:

hello-world := 'hello' concat: 'world'.

Back to the original goal of creating intermediate objects though. Let's write how we think it should work and see if that makes sense:

context push-allocator: {Allocator|} --- context pop-allocator.
hello := 'hello'.
space := ' '.
world := 'world'.
temp1 := hello concat: space.
result := temp1 concat: world alloc: context parent-allocator.

Well that's not convenient - yet. May be if we pop and declare what we want to extract:

context push: {Allocator|}
temp1 := 'hello' concat: ' '.
temp2 := temp1 concat: 'world'.
hello-world := context pop: temp2.

This is better. It allows us to separate concerns:

[self make-hello-world] [_ -> return: String |
  temp1 := 'hello' concat: ' '.
  temp2 := temp1 concat: 'world'.
  return temp2].

context push: {Allocator|}.
hello-world := context pop: self make-hello-world.

We can probably make this cleaner too by using a block to create something in the context of a new allocator:

temp-alloc: Allocator.
hello-world := temp-alloc while: [self make-hello-world].

This feels right to me. We're no longer focused on using the context object or the allocator on the context. All that gets hidden away nicely.

The hidden parameter makes this code feel almost magical. Despite being a low level language we could almost fool ourselves in to believing there's automatic memory management going on.

Other globals we might care about could be entropy for a random number generator; or perhaps the random number generator itself. For now I see no reason to batten down exactly what's on that context. One thing for sure though is that the current memory allocator will be there.

Read more