Hopper Rework: mmap prototyping

A while back I spoke about cernan to the Rust Meetup in San Francisco. I kindly mentioned in my spiel that hopper – the disk-backed MPSC that allows cernan to cope with overload and keep a constrained memory profile – was overly complicated internally. I still stand by that. Nestled inside hopper is private.rs which holds a single struct that all senders and receivers have to synchronize on, plus individually do a complicated dance to make sure they keep on top of hopper's current queue file. (That's a whole thing.)

Post-talk, uh, talk and working my way through Herlihy and Shavit's The Art of Multiprocessor Programming has inspired me to take another crack at hopper's internals. The key goals I have are:

• fine-grained, atomic synchronization
• mmap'ing queue files

I started this work – kinda – in hopper itself and wrote down a notes map for myself, here. Already I think this set of notes is wrong, plus the implementation is... half baked. Regarding the notes, shaving a bit off the size doesn't make sense and if you go elsewhere in the codebase you'll notice I call out the confusion of measuring in terms of bytes versus bits. I feel like there's promise here but the constraint of working in an existing codebase is a little much. Hell, I haven't even run the code I've written in hopper already. The feedback loop of run, explore, run is too hard to get going.

To that end – and in the spirit of sharing in-progress exploration of a problem – I'm opening up blt/mmap_comm. The idea here is the same one, ultimately, that I have for hopper: some threads make work, one thread consumes it. It's just teensy and a little more open to fiddling.

There's only one commit in the repository so far and all that's done there is two threads – one called 'sender', one 'receiver' – bounce work through an mmap'ed file. The sender is writing a 64-bit integer which the receiver pulls and sums into its own storage. The sender likewise accumulates into its own storage. To avoid the receiver running off the end of the rails, the sender accumulates an atomic counter indicating to the receiver that there's value to read. Before exiting, the sender decrements a counter indicating how many senders are still alive. If the receiver hits the condition where there are no more senders and no more reads it exits. When the main thread un-blocks from its joins it compares the sum of the sender and the sum of the receiver. If sums match, we win.

For the most part it won't match -- the synchronization is non-functional -- but maybe you'll get lucky?

Header image: KSC-20170322-PH_SWW01_0046, modified by author