Cetra's Thoughts

The State of Allocators in 2026

Unknown — Wed, 11 Mar 2026 00:00:00 +0000

The Allocators RFC</a> is about to reach a decade milestone of it being accepted</a>, but with no stabilisation.

With the 2025 State of Rust Survey</a> indicating ~12% would be unblocked by stabilisation and another ~27% would have improved code, it still seems to be something that a significant portion of rust users want.

I figured it was time to take a lay of the land and work out what the current state of allocators is, where the blockers are, and any outstanding but critical issues remain. I'm hoping this will spur some activity and breathe new life into one of the older RFCs

The Current State</h2> The allocator trait, while implemented for nightly</a>, is still unstable. It has been defined as a trait for a good 6 years or more, without many changes to the core methods of the trait.
Currently, the trait implementation contains two simple required methods:
pub unsafe trait Allocator { fn allocate(&self, layout: Layout) -> Result<NonNull<[u8]>, AllocError>; unsafe fn deallocate(&self, ptr: NonNull<u8>, layout: Layout); // ...other provided methods... }</code></pre> Implementing this trait, you can then use this with std container types such as Box</code> or Vec</code> to return values allocated by your custom allocator:
let mut vec: Vec<i32, MyAllocator> = Vec::new_in(MyAllocator);</code></pre> The Vec</code> container now has two generic types, Vec<T, A></code>, the type T</code> it's a container for, and the A</code> allocator itself (In fact, if you dig deep enough, you can see that Vec<T></code> is just a special cased version with a global allocator, i.e, Vec<T, A = Global></code>). You need to keep around the allocator in the Vec</code> because whenever you want to resize, truncate or drop the memory, you need to use the same allocator to do so, otherwise it's almost impossible not to hit Undefined Behaviour, if you are fiddling around with another allocator's memory. In practice this field with a Zero Sized Type (ZST) like Global</code> this does get inline compiled away, and vec is the standard ptr</code>, len</code>, capacity</code>, but for other allocators that need state, this means increasing the size of Vec</code> to include an allocator field. The same with Box<T></code> as well, this is a Box<T, A></code> with Box<T, A = Global></code> by default. Although Box</code> is much simpler in terms of memory allocations/chagnes, it's only really ever allocated when created, no resizes, truncations etc... but you still need to keep the allocator around to deallocate. So, for such a simple trait definition, it seems like consensus should be easy right? Just two methods and there is already most of the grunt work done on nightly to support it. However: even this simple of a definition still has some outstanding questions unanswered.
Current Blockers</h2>As of writing this article, the rust allocator working group repo has 75 issues open</a>, with the roadmap not touched in a long time, and probably due for some cleanup. Thom Chiovoloni wrote a few years ago</a> about the allocator trait and some of the blockers that exist, and even now a lot of the original referred issues have not been closed. There has been many</a> discussions</a> on reddit and other places</a> since. Now in early 2026, it seems like consensus still hasn't been reached. While a number of those existing issues stand, here's my opinion on what some of the main ones are. The list isn't exhaustive, but I think covers some of the "bigger" style problems of the trait so far. To be clear: I'm not strongly advocating the inclusion of any of these changes, my hope is to stimulate a bit of discussion. Zero Sized Allocations</h3> Currently, the trait allows the Layout</code> to be defined as zero sized, which back in 2020 was known to cause issues with zero sized types</a>. What happens is that the allocator contract, allows the same pointer for zero sized allocations, since we aren't actually "allocating" anything. Then this might mean that returning the same pointer for different things could be considered Undefined Behaviour (although I'm not sure it strictly is). Not only that: this breaks pointer equality as well. This means you need to special case this in all of your allocator implementations, i.e, impl Allocator for MyAllocator { fn allocate(&self, layout: Layout) -> Result<NonNull<[u8]>, AllocError> { if layout.size() == 0 { return Ok(NonNull::dangling()); } // do actual allocations here } unsafe fn deallocate(&self, ptr: NonNull<u8>, layout: Layout) { if layout.size() == 0 { return; } // do actual deallocations here } }</code></pre> And so every implementation of allocator will need code similar to this example, or at least consider zero sized allocations, or wastefully allocate space just to uphold the contract. So does it make sense to keep Layout</code> as is? One possible fix is to introduce a NonZeroLayout</code> type which enforces that allocations must actually have a size. This foregoes some of the footguns around zero sized allocations, but it would introduce complexity for any trait consumers. This is a small change to the trait interface: /// Our new non zero layout (other constraints left out for brevity) pub struct NonZeroLayout { size: NonZeroUsize, align: NonZeroUsize, } pub unsafe trait Allocator { fn allocate(&self, layout: NonZeroLayout) -> Result<NonNull<[u8]>, AllocError>; unsafe fn deallocate(&self, ptr: NonNull<u8>, layout: NonZeroLayout); }</code></pre> Sidenote: If we look at malloc</code> in C</code> how it behaves, it's allowed to return a null pointer</a> (among other things). However, rust's allocate</code> return is always NonNull</code> and so I think that enforcing the NonZeroLayout</code> balances out these argument types nicely. Context and Rust for Linux</h3> In order to unblock things, and probably because it's a different use case (panics are more severe in the kernel), the rust for linux team has pressed on ahead without requiring allocators to be stable. It's worth studying the shape they came up with, because I feel as if the trait was stabilised as is, it still wouldn't be sufficient for their use case. Their allocator trait</a>, as it stands is slightly similar to the existing trait, but introduces a few extra args: pub unsafe trait Allocator { const MIN_ALIGN: usize; // Required method unsafe fn realloc( ptr: Option<NonNull<u8>>, layout: Layout, old_layout: Layout, flags: Flags, nid: NumaNode, ) -> Result<NonNull<[u8]>, AllocError>; // Provided methods fn alloc( layout: Layout, flags: Flags, nid: NumaNode, ) -> Result<NonNull<[u8]>, AllocError> { ... } unsafe fn free(ptr: NonNull<u8>, layout: Layout) { ... } }</code></pre> Essentially the realloc</code> does the heavy lifting of alloc</code> and free</code> as well since they are provided, but it's otherwise similar to the existing trait. The interesting takeaway here is the Flags</code> struct and NumaNode</code>: both imply that not all allocation requests are equal. In fact in the kernel there are different scenarios for what should happen with memory allocations</a>. This means that, there should be some way to provide context for a given allocation. This is discussed in wg #138</a> and a bit of a sketch of a trait is included: pub unsafe trait Allocator { type Ctx: Copy; fn allocate(&self, ctx: Self::Ctx, layout: Layout) -> Result<NonNull<[u8]>, AllocError>; unsafe fn deallocate(&self, ptr: NonNull<u8>, layout: Layout); }</code></pre> While I think the ship has already sailed on rust kernel allocators, it does feel context is a strong consideration for inclusion in the base trait. Splitting the Traits</h3> One of the goals of the allocator trait is to handle more "esoteric" style allocations. While we want to handle standard system style alloctors like glibc</code>, jemalloc</code> and mimalloc</code>, we also want to support other styles like garbage collected allocators and bump allocators. Taking the bump allocator as an example, they deallocate all memory at once, rather than individually. So essentially a deallocation for an individual value is a no-op. But as mentioned earlier, a Box</code> keeps around two generic arguments, Box<T, A></code>. While this is fine for ZST allocators like Global</code>, compiling down to a single pointer, if you keep around a field for the allocator, which is never actually going to be used to deallocate, then you are increasing the size of Box</code> for no reason. One solution discussed in the working group is to split the traits into two</a>: for allocation and deallocation. This looks something like this: pub unsafe trait Deallocator { unsafe fn deallocate(&self, ptr: NonNull<u8>, layout: NonZeroLayout); } pub unsafe trait Allocator: Deallocator { fn allocate(&self, layout: Layout) -> Result<NonNull<[u8]>, AllocError>; }</code></pre> And assuming that you have a way of converting allocators to other deallocators, then, using the example from zakarumych in the issue, you can do something like: // used to keep the lifetime around struct BumpDealloc<'a> { _marker: PhantomData<&'a Bump>, } unsafe impl<'a> Deallocator for BumpDealloc<'a> { fn deallocate(&self, _ptr: NonNull<u8>, _layout: Layout) {} } impl<'a> From<&'a Bump> for BumpDealloc<'a> { fn from(_bump: &'a Bump) -> Self { BumpDealloc { _marker: PhantomData } } } fn foo<'a>(bump: &'a Bump) { // currently includes `&'a Bump` in the `Box`, making it bigger than a pointer let box: Box<u32, &'a Bump> = Box::new_in(42, bump); // now it's pointer sized let box: Box<u32, Bumped<'a>> = box.into(); assert_eq!(size_of_val(&box), size_of::<usize>()); }</code></pre> There are suggestions that this could work without the split of the trait. For example: struct BumpDealloc<'a>(PhantomData<&'a Bump>); unsafe impl Allocator for BumpDealloc<'_> { fn allocate(&self, _: Layout) -> Result<NonNull<[u8]>, AllocError> { panic!("cannot allocate through BumpDealloc") } unsafe fn deallocate(&self, _: NonNull<u8>, _: Layout) {} }</code></pre> However, having this as a runtime panic feels like it goes against the spirit of rust. So it's a trade off of keeping the trait simple, or allowing this sort of expressivity. I have flipped back and forth on this one whether it's worthwhile to split the traits. The only strong case in the issue is the no-op deallocator, or an deallocator that takes up less space than the allocator. So essentially you need an allocator always anyway, and the deallocator is a special pairing. If there were other use cases that would benefit, then maybe it is worth considering. I think that as long as people are not prevented from doing what they want, then we should keep the trait simple. Associated Error Type</h3> Allocations can fail, and the existing trait returns a Result</code> with AllocError</code> as the error type. This is a ZST error which doesn't include a whole lot of information about why the allocation failed, just that it did. It would be nice if an allocation fails, some more contextual information is included, so that the callee can recover from any issues. This is discussed in wg #23</a> and wg #106</a> with some use cases around why it would be useful to provide custom error types. With associated errors, the trait would look like: pub unsafe trait Allocator { type Error: core::error::Error; fn allocate(&self, layout: Layout) -> Result<NonNull<[u8]>, Self::Error>; unsafe fn deallocate(&self, ptr: NonNull<u8>, layout: Layout); // ...other provided methods... }</code></pre> There is obviously alternatives to this. For instance, if context was a thing, you could provide an allocation number on allocations, so you could get back the error out of band. Or you could lookup a last error in threadlocal etc.. but obviously this is prone to errors if another allocation happens in between. But all of these feel a little less _nice than using the existing Result</code> type for what it's designed for. However, one issue around associated types is around making it easily dyn compatible. If you had an associated error type, you would need to ensure you are using dyn Allocator<Error = AllocError></code> everywhere rather than dyn Allocator</code>. I don't think this is a big deal, and the std lib could provide some helper plumbing to make it seamless, but nothing that would prevent dyn compatible allocators. The Store API Alternative</h3> There exists an alternative to the allocator trait, called the Store API</a> which has seen some momentum behind it. This would unlock a few more use cases above & beyond the existing trait, especially around putting things on the stack. But (and this is just my personal hot take which is not worth much!) I think it's over-engineered and suffers from the second system effect</a>, when we haven't even got the first system out yet. It has a higher & more complex API surface area, introducing 5 traits</a>, and abstracting away pointers to handles feels like unnecessary indirection. I found it a bit more difficult to follow, trying to imagine porting a simple allocator to use it, compared to the existing trait. I feel like there is a world where we can extend beyond the base trait to support some of the better bits of the Storage API, but it feels like an even bigger uphill battle to stabilise in lieu of the existing trait. i.e, let's take the best bits and stabilise them separately, if we can. A Stable Alternative</h3> With the age of this feature, it's inevitable that there will be some experimentation of a bridge solution, while waiting for stabilisation. The allocator_api2</code></a> crate exists to fill this gap. This crate mirrors the current trait definition in nightly, allowing use of the allocator trait, as it stands, on stable. It is surprisingly a popular crate with over 250 million downloads on crates.io, and an optional dependency of crates like hashbrown</code></a> and bumpalo</code></a> The Cost of Monomorphisation</h2> This isn't actually an existing issue, but more a discussion around some of the ramifications of the existing design. What I mean by the cost of monomorphisation, apart from the code generation angle, is the developer experience. Having a new generic argument on all containers will cause a lot of churn at call sites. For example, right now, Vec<T></code> is generic against the value itself, and nothing else. I'm sure there are millions of functions out there that have a signature like: fn do_something(input: Vec<String>) { // something with the vec }</code></pre> However, if we want out function to support custom allocated vecs, we now need to add trait bounds on the function signature: fn do_something<A: Allocator>(input: Vec<String, A>) { // something with the vec }</code></pre> Not only that, mixing and matching vecs with different allocators also becomes a little painful, as Vec<T, A> != Vec<T, B></code>. You can imagine this would be a nightmare to adjust everywhere. And so we need some guidance, or a solution for it. I did a small investigation around how other languages handled this, focusing on Zig and C++, since both support custom allocators. Now, I am not an expert of either language, so there is a good chance I am wrong in some of this understanding. But the gist is that, it feels like dynamic dispatch is not as bad as it seems here. Zig's Allocator</h3> I wanted to look at how allocators worked in Zig, as the sentiment of using Zig is that it provides greater flexibility around memory allocations. So what does it do that rust doesn't, and how does it do it? The Allocator</a> is a struct with two pointers: one for the state, and the other to a VTable of const fns: // not really defined like this (but for illustrations only) pub const Allocator = struct { ptr: *anyopaque, // opaque pointer to allocator state vtable: *const VTable, // pointer to constant function table }; pub const VTable = struct { alloc: *const fn (*anyopaque, usize, Alignment, usize) ?[*]u8, resize: *const fn (*anyopaque, []u8, Alignment, usize, usize) bool, remap: *const fn (*anyopaque, []u8, Alignment, usize, usize) ?[*]u8, free: *const fn (*anyopaque, []u8, Alignment, usize) void, };</code></pre> This is a type erased, dynamic dispatch style allocator. You would think that this would be slower than static dispatch, but in fact Nical wrote a blog</a> that suggests that this isn't always the case: sometimes the code bloat from monomorphisation outweighs the performance benefits of static dispatch. And besides, it does look like LLVM can, in most cases, rewrite them as static calls anyway</a>, so you get to have your cake & eat it too. How could we get something similar in rust? Well, with the trait as it exists now, we could just use dyn Allocator</code> that essentially provides the same style dynamic dispatch, and use that as the Allocator</code> bound in function calls: fn do_something(input: Vec<T, Box<dyn Allocator>>) { // something with the vec }</code></pre> So with rust, we do have options to go down the dynamic dispatch path already, given the trait definition. Obviously object-safety, etc.. comes into it, but it's not completely out of the realm of possibility C++ Allocator story</h3> C++ is an older language, and so you expect it to have some warts, moreso than rust. C++ does have some of the same monomorphisation issues that rust will potentially have: different allocators means that the container type is different, but besides templates</a> being a solution, C++17 introduced polymorphic memory resources</a>, or pmr to address this. I.e, the following C++ code would error auto vec1 = std::vector<int,allocator1>(); auto vec2 = std::vector<int,allocator2>(); auto vec = vec1; vec = vec2;</code></pre> pmr solves this with type erasure, in similar fashion to zig and dyn</code> in rust (albeit probably not identical under the hood): auto vec1 = std::pmr::vector<int>(&resource1); auto vec2 = std::pmr::vector<int>(&resource2); auto vec = vec1; vec = vec2;</code></pre> So C++ started off down the static route, and moved towards dynamic dispatch. Obviously you can still use the older container types, but it might be fitting that we pay a bit more attention to dynamic dispatch with allocators, since other languages are tending towards them. If we had some LLVM trickery to be able to statically inline in the dynamic case (if it doesn't already exist in rust), as in Zig, we might not end up paying as much for it, or it could be free. Three Steps Forward</h2> So as you can see, almost 10 years on, there is still some outstanding decisions to be made. The inertia of getting this stabilised has definitely slowed down in recent years, but there is definitely still a need to think about this. I like to frame it as boiling down to a few possible outcomes: Wait for a better alternative to emerge or some language features</li> Stabilise the trait as-is: no further changes to the API and let's go for it</li> Work a little bit more on the trait and then stabilise</li> </ol> Wait for a better alternative</h3> This is essentially the current state of play and, unless there is something to help with the current level of inertia, in all likelihood we could still be here in another decade! I am not that pessimistic, however, and think that we can definitely solve it if there is some attention placed on it. Stabilise as-is</h3> The trait, as it stands, is actually already useful and, in fact with the allocator_api2</code> crate is already useable. And considering that the trait hasn't changed all that much recently, it might be an indication that it's ready for stabilisation. However, this could be because it's on nightly, meaning not many consumers of the API exist currently. So the risk of stabilising it means that there is a missed use case. I am partially in favour of this direction. This would eliminate the rust for linux use case and other esoteric cases, but they have pressed on without it anyway. Work a little bit more on the trait</h3> Coming back to the 75 open issues</a> and some of the main blockers mentioned, we could adjust the trait slightly so that it supports some extra needed features. I was a little strategic in selecting the blockers to focus on, as they are all composable together. Omitting the trait split, but including: the context, non-zero layout and error associated type, you could envisage a trait looking like this: pub unsafe trait Allocator { type Ctx: Copy; type Error: core::error::Error; fn allocate( &self, ctx: Self::Ctx, layout: NonZeroLayout ) -> Result<NonNull<[u8]>, Self::Error>; unsafe fn deallocate(&self, ptr: NonNull<u8>, layout: NonZeroLayout); // ...other provided methods... }</code></pre> I'm not implying this should be the final shape, but I think something along these lines is within the realms of possibility. Besides blockers mentioned above, there are some unsoundness issues like the Box noalias</code></a> that might be resolved already, and a few other issues that might need some triage. I think my approach would be: Divide issues into stabilisation blockers, design trade-offs, and miscellaneous/post stabilisation</li> For the stabilisation blockers that don't really require much change of the design, if at all, these should be worked through</li> For the design trade-offs, a decision one way or another should be made, and incorporated in or not</li> For the miscellaneous/post stabilisation issues, they can remain open if they are not critical.</li> </ul> I feel like working through the issues in the repo is the best path forward. Conclusions</h2> As it stands, custom allocators are still not in a state to stabilise, and it would be good to stabilise them soon. I haven't really touched on why they are useful, since I feel that the benefits are pretty clear. My own interest in it comes from working on the rust logfire team</a>, contributing to datafusion</a>, building, and dealing with memory in production systems</a> that love to crash when they don't have memory. All of these things I am working on will benefit from custom allocators, and unblock me in some cases. I want to have the tools available to wrangle allocations where it counts. I am hoping with this article that it at least spurs some discussion and breathes new life into custom allocators.

Creating MCP Servers in Rust Unknown — Sun, 01 Feb 2026 00:00:00 +0000 MCP servers are useful tools that allow AI agents the ability to interact with applications in a prescribed fashion. I wanted to understand how they put together so I could work out where/how they are useful. One of the things I found when first investigating them, is that there aren't that many MCP servers that are written in rust, most are either NodeJS or Python. So this is my journey of how I went about creating todo-mcp</code>: a rust based todo list MCP server, with local state synchronisation and desktop/android app. Todo MCP is open source, and available at https://github.com/cetra3/todo-mcp</a> MCP Servers and Requirements</h2> The very first challenge I ran into was that, if you have a number of agent windows open, they will all spawn separate MCP processes. If you are integrating with a remote web server, this isn't much of an issue, as they will be essentially wrappers over API calls to a service that can handle multiple clients. However, if you wanted a more "local-first" approach, this becomes a little hard without some state synchronisation between processes. There isn't any central server to ask state for, and more often than not, when I think of a todo to write, I don't want to have to be at home, or even connected to any network. I also wanted to have a "companion app" open so that I could see my todo lists without having to use an MCP server at all, and mobile support, so I could use it when I am out of the house. I had experimented already with automerge</code></a> and its associated API in mpad</code></a> to make a multiplayer text pad, and thought it would be a good fit for this project. We could use the same strategy here: use automerge CRDTs and persist to disk periodically and then when another connection to another process is made (either on the same device or on another device), then it could reconcile changes made, and update the state accordingly. Automerge and CRDTs</h2> Automerge</a> uses Conflict-free Replicated Data Types (CRDTs)</a> to ensure data synchronisation without a central authority. A CRDT is a data structure that will ensure "eventual consistency" to state, with some smarts around merging multiple disparate sources. An example of a CRDT (one of the more simpler ones) is a Grow Only set. Say you have two separate processes that have a set of values that are shared: Process A has: - Value A</code></pre> Process B has: - Value B - Value C</code></pre> To merge/reconcile this, you can see the obvious choice is: - Value A - Value B - Value C</code></pre> The most important thing is that the output state converges. If Process A sends an update before Process B, or vice versa, the result is the same. This type of CRDT, however, is quite simplistic, and doesn't allow you to do more complex state changes. You can imagine an extension to this would be to add another set for "removed" values, so you could remove the values from the set. I.e, if starting from the last merge an update is sent like: Remove: - Value B</code></pre> Then the state is: Added: - Value A - Value B - Value C Removed: - Value B</code></pre> & the merged state is: - Value A - Value C</code></pre> This is sometimes called a Two Phase Set. It allows you to add & remove values from a set, but once a value is removed, it can't be re-added back. To extend this again to allow items to be added/removed multiple times, you could add a timestamp to each of the values, when they were added or removed, and they are added/removed based upon which set the value is in, and the newest timestamp (AKA Last Write Wins). You can see as this gets more complex, a lot of bookkeeping is required. For different types of data, like text, lists, objects, there are a number of different types of CRDTs available, that can sometimes be composed together to create complex state shapes. That's where automerge</code> comes in. automerge</code> provides a number of CRDTs you can use to build out your state: Composite nested CRDTs such as lists, objects and text</li> Primitive CRDTs such as counters, numbers, booleans</li> </ul> Autosurgeon</h3> CRDTs are a good tool to use, and automerge</code> provides some great foundations for using them. However, like manually writing JSON structures is a pain, and we derive with serde</code></a> most of the time, we can use autosurgeon</code></a> to automatically derive a rust structure, converting it into something that can be used with automerge</code> CRDTs So for a todo</code> like structure, we can write: use autosurgeon::{Reconcile, Hydrate}; #[derive(Reconcile, Hydrate)] struct TodoList { todos: Vec<Todo> } #[derive(Reconcile, Hydrate)] struct Todo { title: String, completed: bool }</code></pre> And use the hydrate/reconcile methods in autosurgeon</code> to manage state changes for us. This is not without challenges, however, and seeding new processes can be tricky. When writing mpad</code> I encountered a bit of an issue around this</a>, which has not been resolved as of this blog. However I have managed to work around it for now. Sending State via Multicast</h2> I elected to use Multicast to send state changes to other instances. Multicast is (normally) a LAN protocol, that allows 1 or more processes the ability to register for packets to the same IP address/Port combo (Within the specific Multicast IP range of 224.0.0.0/4</code>). With a bit of setup when creating a UDP socket, you can retrieve packets destined to the ip/port pairing. The OS will send an IGMP group request, which hopefully your router/switch will respect, and start sending packets to your socket. Sending packets is pretty easy, you just send them to the same ip/port as things are listening. However, UDP does not have the same guarantees as TCP, and with multicast TCP can't be used. Simple Multicast Protocol</h3> I've elected to create a pretty bare-bones protocol for use with UDP for sending packets from "sites" (i.e, instances of a process). Essentially, given any payload to send over the network it will: Chunk it down to fit into the standard Ethernet MTU (which is normally 1500 bytes but I capped it 1400 for a bit of headroom)</li> Send the site id, which is randomly generated by each process on startup, I could use the sender ip/port but that doesn't deal with reconnections/network changes.</li> Send a sequence number, the current payload number being sent</li> Send a total length of the sequence num</code>, which is how many packets will be sent for this payload</li> Send an index number (what packet number this is for the given sequence)</li> Finally the payload</li> </ul> With this we have semi-reliable network communications. It hasn't been widely tested with packet loss etc.. but it's enough to get started. Some improvements in the future we could do NACK</code> style handling, whereby if we missed a sequence, we can request the full state from that site etc.. but for now this was enough to work reliably well in testing. Already when a "site" reconnects due to network changes, it resends its full state for reconciliation each time in the first message. Choosing a GUI Framework</h2> One design decision I made was that I wanted a little companion app around seeing and interacting with Todos. I also wanted support for using it on my android phone, so a cross platform UI was necessary. Slint: The First Attempt</h3> Slint is a rust first cross platform GUI framework with some nice editor plugins and some good primitives to get you started. I wrote an initial version using slint, and was happy with how lightweight it was and would still consider it for new projects. However I ran into two bugs when doing some lightweight QA: Undo/Redo functionality did not update the TextInput</code>, so there was no real way to get the text from the GUI and feed it into my CRDT: https://github.com/slint-ui/slint/issues/9205</a>. For this one I raised a PR and got it fixed</a>. Text Editing on Android was not working. With garbled text and other weirdness: https://github.com/slint-ui/slint/issues/9240</a>. Turns out because I wasn't using the standard android keyboard, this caused issues. I tried a good half a day to try and fix this, by looking at the companion Java helper</a> that is used to manage inputs, but gave up in the end. While I still think Slint is perfect for this, these issues made it seem like a bit of an uphill battle to implement something pretty simple. Most of the guts of this application isn't in UI, so I can swap it out for any frame work. So for now, I moved onto something I had been meaning to try for a while: Dioxus</a>. Dioxus: Embracing Webview</h3> Dioxus is cross platform, but it is not native, rather it uses a webview to display the UI. There are some great time savers like hot reloading, and support for Tailwind CSS is pretty easy. Plus, I have coded enough websites to understand the quirks of that ecosystem enough to know I could get myself out of trouble. It took me far less time to get things up and running, and initial testing showed that there weren't any major flaws. One feature I added immediately was the ability to have multiple todo lists in a collapsible fashion: The one downside is memory usage: it is essentially embedding a full browser on Desktop and so uses a lot more RAM and resources. There are some potential avenues for addressing this, such as an experimental native renderer</a> in the works, so hopefully this will be a good solution in the future. MCP Server Libraries</h2> There is first class support for rust MCP servers via the rmcp</code></a> crate. However, this was also not without issues. When running an MCP process within Zed</a>, it failed to run. Turns out that some of the schema definitions were wrong and didn't match the spec. I raised a PR and got this fixed as well: https://github.com/modelcontextprotocol/rust-sdk/pull/377</a> With rmcp</code> you can use derive macros to build your MCP server: #[tool_router] impl TodoMcp { ... #[tool(description = "Get all todo lists")] async fn get_todos( &self, Parameters(params): Parameters<GetListParams>, ) -> Result<Json<TodoListsResponse>, McpError> { let state = self.todo_state.read()?; let response: TodoListsResponse = (&*state).into(); Ok(Json(response)) } ...</code></pre> Then serving it is pretty easy (I only use stdio mode for MCPs): pub async fn run_mcp() -> Result<(), Error> { let todo_mcp = TodoMcp::new().serve(stdio()).await?; todo_mcp.waiting().await?; Ok(()) }</code></pre>Claude Code Hook Integration</h2> With Claude Code, while you can instrument an MCP server and run it, I wanted it to feel a little more "native" in that whenever there was a todo list in any agent session, I wanted that to be published. That way I could open my companion app on my phone, or desktop, and watch the todo list updates in real time. I could go make a coffee and wait for the todos to be completed before checking back in. To accomplish that, I made a small integration with Claude Code's hook system</a>: </video> Because hooks execute a command (like a bash command) and exit, I wanted this to be pretty dumb and simple: Receive the hook JSON from stdin</li> Reconcile the todo list & work out a good name</li> Add/Update the todo in the list when things happen in claude</li> </ul> Then I can add this to ~/.claude/settings.json</code> to have it invoked: { "hooks": { "PostToolUse": [ { "matcher": "TaskCreate|TaskUpdate", "hooks": [ { "type": "command", "command": "todo-mcp hook" } ] } ] } }</code></pre> This mostly works, but needs a bit more testing. The one thing is that it's hard to get a nice name for a "claude session" without a bit of song and dance, and so I have elected just to include the current working dir as a title of the list. The other problem was sometimes Claude skips creating todos and essentially adds them completed. So for this I revert back to reading it from the todo structure in ~/.claude/todos/</code> which seems to work and allows me to reconcile changes. Obviously a bit more testing is still needed, as I have sometimes seen it not act the way I want, and sometimes claude does not even create todo lists at all, so it's currently an open question how to address that. Dealing with Network Changes</h2> When testing out this app with my phone, I went shopping and marked off my shopping on the list as I went around the store. When I got home, I noticed that the list was now out of sync, and I had to restart the app to get things going again. Not an ideal situation. What I wanted to happen was that when a network changed, I wanted to restart the multicast protocol tasks in a way that allowed me to reconcile when I came back home. So I found the great netwatcher</code></a> crate, that allows me to register a callback to fire whenever there are any network changes. This required a bit of re-architecting and decoupling of the multicast socket to the state of the app, so that we are able to continue off where we left. While it is cross platform, I had to end up creating a custom AndroidManifest.xml</code> for the extra ACCESS_NETWORK_STATE</code> permission, which isn't included in dioxus by default</a>. Then it was a matter of testing it. For desktop I just turned on/off the wifi with iwctl</code> which would allow me to simulate network failures. And for android, I could just enable/disable wifi from settings. However, with android there is a state where your app is loaded, but it isn't currently on screen. In this state, the app will continue to run, but can't use any network sockets, returning a "permission denied" error. To work around this, I simply retry connecting every 10 seconds or so. However, there is probably a better solution, but requires a bit more investigation. I am hopeful that in newer versions of Dioxus, this will be made a lot easier, and so I'm waiting for 0.8</code> to see if there is an easy solution to solve it. Claude Code and Frustration Driven Development</h2> I wanted to give Claude Code a good rinse out when making this project. A lot of the discourse around AI is extremely positive, or entirely negative, there doesn't seem to be as much nuance. So while I started with a base that I wrote, when adding new features and testing ideas I first prompted claude to help plan out and implement them. 9 times out of 10, unless it was a simple change, I found the code that it wrote was both alien and hard to follow. If this was another colleague's PR I would sit down and work with them together to get it to a good state. But with Claude you can't do that, it never learns above the baseline, and when that context window blows out, or a new session is created, we're back to square one. It felt like, when reading Claude code changes, the entire thing was a giant code smell. It's hard to put my finger on why it was bad, it just didn't seem like it was idiomatic, electing to take shortcuts and otherwise not understand the context of what was happening. This gets compounded the more features and complexity your application has. For small things, like writing tests, and doing small refactors, this was great, but for bigger things it fell down pretty quickly. So what I ended up doing is something I am coining Frustration Driven Development or FDD for short. This is when you have code that is written, that does what you want, just not in an ideal fashion, and so out of frustration, you rewrite it completely. There are sections of code I haven't adjusted much from Claude's initial output, and reviewing some of the source code of the initial release I can already see changes/cleanup I'd like to make. Since this is a bit of a hobby project, I don't have much time to dedicate to making this pristine and in a state that I would be proud of. So I guess that's a small win for AI: I don't know if I would've finished it in this state without using Claude Code. I hope that when I do get round to it, I can use some FDD to help work through some of it. Wrapping Up</h2> Todo MCP was a lot of fun to write and I have been using it as my daily driver currently, replacing google keep for my todo lists. I actually end up using it more for the todo functionality rather than MCP/Claude integrations, so it's sort of evolved into something more than just an MCP server in my eyes. There were a few learnings around writing an application like this in rust, and I feel that the ecosystem around this is still very green (i.e, slint text issues & rmcp not working with zed at all). I had to raise a few PRs and a few issues to get to where I am at, and there is still lots of missing documentation/ways to do things. I'm hopeful though that in the ensuing months/years, this will not be as big of an issue. One last thing is around the publishing of this app. There isn't a great story currently around using rust-based MCPs, for other languages you have uvx</code> and npx</code>, so I might end up making a wrapper to support either, just so it's a lot easier for people to try out. For now a simple cargo install todo-mcp</code> will get you going. A tale of two lengths Unknown — Sun, 11 May 2025 00:00:00 +0000 They say there are 2 hard problems in computer science, and I've hit all 3 of them.... This is a blog post about a memory problem we were hitting in production with a service written in rust, and the tools used to resolve it. The Problem</h2> Object Storage, whether S3 or something else, does not have the best performance for latency in comparison to local disk. There can be a lot of overhead to retrieving files. Alongside this, repetitive calls to the same files means that you will have multiple requests to the same data. One solution is to have a local cache of files sitting close to wherever you are using them. That way, for regularly requested files, you can reduce latency and decrease the amount of requests to the backing object store. For a possible solution, we recently moved to using foyer</code></a>, touted as a hybrid cache, as it provides not only a memory cache, but can be used with local disk storage as well. For really hot files, they would be served in memory. For colder files, they could be read from a local ephemeral ssd, so still providing some speedup even if evicted from memory. This looked like a great fit for our use case. Things looked like they were going OK, until the cache had been running for a day or so. We noticed that the memory limits we were setting were not honoured, and we were seeing pod restarts with the reason set to OOMKilled</code></a>. Something about how we configured the cache was not right. Cache Layout</h3> In our setup, we don't want to actually cache entire files, just the ranges of bytes that have been requested. Our request pattern means there is never any overlap between ranges, and the exact same ranges are going to be requested repeatedly. So our Cache Key essentially looks like: struct CacheKey { path: String, start: usize, end: usize, }</code></pre> And our Cache Value is standard Bytes</code></a>, giving us a cache that looks a bit like this: struct ByteCacheAppState { write_cache: HybridCache<CacheKey, Bytes>, }</code></pre> Of course with a cache we can't have unbounded memory usage. So we configure a weighter that uses the length of bytes in memory (plus a memory_size</code> method for the key to account for the path string + two usizes, even if it's insignificant: we want to be as accurate as possible!): ... .with_weighter(|key, value| key.memory_size() + value.len()) ...</code></pre> The Bytes</code> struct is a bit of a special struct that is used in a lot of crates. It's essentially an Arc<[u8]></code> but has some magic to deal with other backing implementations. It's cheaply cloneable: the underlying memory isn't cloned, just a reference count is updated. You can also take a subslice into it as well, so that you can have a view into a portion of the backing bytes, without reallocating. Byte Request</h3> When we get a request for a range of bytes, we want to either return that ranges bytes if it's in the cache (either memory or disk), or read from object storage, storing that in the cache, and returning the bytes. With foyer</code> you can use the fetch</code></a> method to do just that: let cache_entry = write_cache .fetch(cache_key, || async move { let mut req = read_from_store(&path, range, &store).await?; let mut buffer = BytesMut::with_capacity(byte_len); while let Some(bytes) = req.stream.try_next().await? { buffer.extend_from_slice(&bytes); } Ok(bytes.freeze()) }) .await?;</code></pre> And so our cache is pretty simple. We keep around slices of files that have been requested on the predicted chance that they will be requested again. Troubleshooting</h2> Since we were running into memory limit situations, we started troubleshooting what could possibly be wrong. The first thing that we tried, was to reduce the limits that we gave to the cache to be much less than the memory limits of the pod, to see if it would plateau albeit at a higher limit. This seemed to have worked but was still a bit variable in our memory usage. And we still didn't know where the extra memory usage was coming from. We really needed to see what was going on with the heap. One of the advantages of garbage collected languages is that you can really easily introspect the heap. For rust this was traditionally a bit harder. However, there are now some awesome tools we can use to investigate memory allocations. One we ended up using to troubleshoot was the jemalloc_pprof</code></a> crate. Profiling with jemalloc_pprof</code></h3> One way to profile memory usage is to have jemalloc as the global allocator, set some settings to turn on profiling, and then use jemalloc_pprof</code> to dump out the heap profile. The cache is a simple axum</code></a> webserver, so we ended up creating a request endpoint we could use to read the allocation profile when the cache was warmed up. Here's how you can add this to your own project, if you wanted to follow along: Firstly, add tikv-jemallocator</code> and the jemalloc_pprof</code> crate to Cargo.toml</code> with a couple of enabled features: tikv-jemallocator = { version = "0.6.0", features = ["profiling", "unprefixed_malloc_on_supported_platforms"] } jemalloc_pprof = {version = "0.7.0", features = ["symbolize"]}</code></pre> Secondly, enable this as the global allocator in main.rs</code>, with some exported settings (see the jemalloc_pprof</code> readme</a> for an explanation): #[global_allocator] static GLOBAL: tikv_jemallocator::Jemalloc = tikv_jemallocator::Jemalloc; #[allow(non_upper_case_globals)] #[export_name = "malloc_conf"] pub static malloc_conf: &[u8] = b"prof:true,prof_active:true,lg_prof_sample:19\0";</code></pre> Thirdly, make a simple axum request handler to return the heap: async fn get_heap() -> Result<Response<Body>, Error> { let mut prof_ctl = jemalloc_pprof::PROF_CTL.as_ref().unwrap().lock().await; let pprof = prof_ctl .dump_pprof()?; Ok(Response::builder().status(200).body(pprof.into())?) }</code></pre> Lastly, Run the program and get some allocations happening. Getting Results</h3> You can use curl to save the heap locally: curl http://127.0.0.1:8000/path_to_get_heap > heap.pprof</code></pre> Then you can use pprof</code></a> to serve the results as a webpage: pprof -http=:8080 heap.pprof</code></pre>Heap Results</h2> With the cache service instrumented, we ran up an instance with a limit of 10GB</code> RAM for the cache. We then warmed the cache up to the point we were seeing it go above that limit. Then we downloaded the heap, and investigated it. Tell me if you can spot the issue in the flame graph: </a> What this profile is showing us is a couple of things: about 2/3 come from our cache fetch</code> implementation using BytesMut</code>, and the other comes from re-hydrated values from the foyer disk cache. No memory leaks it seems. However: the amount of bytes that our heap has in total is 17GB</code> which is almost twice the amount of memory we have configured in the cache. BytesMut</code> and Vec</code></h3> When you use BytesMut::with_capacity</code>, this uses Vec</code> as a backing store</a>: pub fn with_capacity(capacity: usize) -> BytesMut { BytesMut::from_vec(Vec::with_capacity(capacity)) }</code></pre> Within Vec</code>, there are two lengths. There is the len()</code> which is how many initialised values it has. Then there is capacity()</code>, which is how much it's asked the allocator for. Normally this capacity()</code> is expanded as a power of two when adding values and is always greater than or equal to the length of entries. This can mean the in-memory size is not always what you'd expect. So we have two lengths to consider: len()</code> and capacity()</code>, but only capacity()</code> gives a more accurate account of memory usage. Bytes</code> and capacity()</code></h3> There is no capacity()</code> method on Bytes</code>, given that its backing could be a number of different implementations. This isn't great for our use case here, where we want to account for the memory usage more accurately. Changing To Vec<u8></code></h2> Since we can't use capacity()</code> on Bytes</code>, one troubleshooting step was that we tried using Vec</code> directly. This won't give us cheaply cloneable bytes, but it will at least allow us to verify that this is where the extra memory usage comes from. So we have a cache implementation like: struct ByteCacheAppState { write_cache: HybridCache<CacheKey, Vec<u8>>, }</code></pre> With a weighter like: ... .with_weighter(|key, value| key.memory_size() + value.capacity()) ...</code></pre>The New Profile</h3> OK, running again with a 10GB</code> limit, let's have a look at the new flame graph profile: </a> OK, that looks a lot better! The Actual Bug</h2> So it looks like the difference between len()</code> and capacity()</code> is our culprit. We can use this implementation, or use another struct that allows us to read actual capacity. Problem solved right? However, this is a tale of two lengths! Vec::with_capacity</code></h3> After a bit of digging into the way allocating works on a Vec</code>, if you create a Vec</code> with a given capacity, it shouldn't reallocate until the length is greater than the initial capacity, even if it's not a power of two. Within our fetch()</code> method we are setting the capacity explicitly, so we should only allocate once per cache entry, and not reallocate. In the documentation</a> it's implicitly stated that with_capacity</code> does not guarantee that capacity()</code> and len()</code> will be equal, i.e, even in the given the example it's written so: let mut vec = Vec::with_capacity(10); assert_eq!(vec.len(), 0); assert!(vec.capacity() >= 10);</code></pre> Here that sneaky little >=</code> sign indicates that the capacity does not have to be equal to the value given by with_capacity</code>. But, in practice currently, if you use with_capacity</code> it will ask the allocator</a> for that exact amount and won't reallocate until there are more entries than the initial capacity: let mut vec = Vec::with_capacity(10); assert_eq!(vec.len(), 0); assert_eq!(vec.capacity(), 10); for i in 0..10 { vec.push(i); } assert_eq!(vec.len(), 10); assert_eq!(vec.capacity(), 10);</code></pre> This brings us back to our initial implementation: let mut buffer = BytesMut::with_capacity(byte_len); while let Some(bytes) = req.stream.try_next().await? { buffer.extend_from_slice(&bytes); }</code></pre> We should only be allocating once during the call to with_capacity</code>. Both len()</code> and capacity()</code> should be equal and equal to our given capacity byte_len</code>. Why are they not equal and why does it, looking at the heap profile, look like there is a reallocation happening? Inclusive and Exclusive Ranges</h3> We're using standard HTTP Range requests</a> for requesting byte ranges. In rust parlance, they are Inclusive Ranges. I.e, start..=end</code></pre> Instead of Exclusive ranges. I.e, the standard Range</code></a> start..end</code></pre> And essentially everything in rust is expecting a Range</code> instead of RangeInclusive</code></a>. But when making http range requests we need to convert the other way. And so to go from exclusive to inclusive: let start = range.start(); let end = range.end() - 1; let inclusive = start..=end;</code></pre> And to go from inclusive to exclusive: let start = range.start(); let end = range.end() + 1; let exclusive = start..end;</code></pre>Cache Keys and Inclusive Ranges</h3> Our CacheKey</code> implementation, we store the start & end as an exclusive range. fn to_cache_key(path: &str, range: RangeInclusive<usize>) -> CacheKey { CacheKey { path: path.to_string(), start: range.start(), end: range.end() - 1, } }</code></pre> Then we use this cache key to ask how much we should allocate: let byte_len = cache_key.end - cache_key.start;</code></pre> Then in our backing request to our object storage, we were converting it back into an inclusive range by: let range = RangeRequest { start: cache_key.start, end: cache_key.end + 1 };</code></pre> Can you spot the issue with the code above? What is wrong with our implementation here? If you go back and re-read how to convert from range to the other, you might catch it. That's right: the conversion in the above code is backwards. We were converting it the wrong way! The byte_len</code> was always two bytes short. Meaning that the BytesMut</code> would always reallocate. To make matters worse: even though our byte_len</code> was not the right length, the two errors cancelled eachother out. We ended up inadvertently sending the correct request to the object store, our downstream services got the byte ranges they wanted, and the bug went undetected. That is, until we reach our assigned memory limits... Adding some debug assertions in the code it was easy to make our existing tests fail when they ran: let mut buffer = BytesMut::with_capacity(byte_len); while let Some(bytes) = req.stream.try_next().await? { buffer.extend_from_slice(&bytes); } debug_assert_eq!(bytes.len(), byte_len); debug_assert_eq!(bytes.capacity(), byte_len);</code></pre>The Fix</h2> So the fix in the end was simple: do a proper inclusive/exclusive conversion, which would mean that we only allocate as much as we need. Converting to the exclusive range cache key: fn to_cache_key(path: &str, range: RangeInclusive<usize>) -> CacheKey { CacheKey { path: path.to_string(), start: range.start(), end: range.end() + 1, } }</code></pre> Converting back to the inclusive range request: let range = RangeRequest { start: cache_key.start, end: cache_key.end - 1 };</code></pre> With this in place, everything worked without having to stop using Bytes</code>, and preventing further deep changes. Conclusions & Thoughts</h2> While rust is great at managing memory for you, there are situations where you need to keep a close eye on how much memory is being used. A cache service is a great example of that. What would have been great from the start of the language, and definitely too late to change now, would be designing some of the standard library with fallible allocations from the get go. If we had an API shaped for that, we would be able to catch errors like this much easier. Yes there are now methods like try_reserve</code> on Vec</code>, but downstream abstractions like BytesMut</code> do not have the same methods. Thinking this through: for a language that prides itself on explicit behaviour, this feels like one area where it could be less implicit. In fairness, there could be more structure in place in the cache implementation to prevent this sort of bug from appearing, like using types more strongly to represent ranges etc.. If nothing else, it goes to show that sometimes bugs can be deceptively simple to fix, and having the tools to profile applications is vitally important. Creating your own custom derive macro Unknown — Sun, 23 Jul 2023 00:00:00 +0000 One of the things I tackled early on with when creating DiveDB</a> was trying to import a lot of metrics from dive logs. At regular intervals, normally every 10 seconds, a dive computer will record metrics about the dive, such as depth, tank pressure, water temperature etc... While this isn't a huge amount of entries, nearly every SQL framework I used for it was painfully slow at importing this type of data. The only crate that I managed to get to work with reasonable performance was tokio-postgres</a>. Itself being lightweight: it doesn't include a lot of features you would expect from a full blown SQL framework like diesel</a> or SQLx</a>. I was OK with that, I don't mind writing a bit of raw SQL, but one thing I thought was missing was some sort of way to create a struct from a database row. So I set out to make a really simple trait to hydrate a struct from a row. The obvious candidate for this is a procedural macro. Writing procedural macros has been written about before, but I haven't found a terse enough guide that I was happy with. I wanted to document my approach in the odd chance it helps other people. At the very least I have something to refer to if I need to do this again! Aim of the Macro</h2> The Aim of the macro is to be able to write #[derive(MyTrait)]</code> and have it fill in the trait implementation at compile time. In this case I will be deriving a simple trait that looks like this: use tokio_postgres::Row; use anyhow::Error; pub trait FromRow { /// hydrate a struct from a database `Row` fn from_row(row: Row) -> Result<Self, Error> where Self: std::marker::Sized; }</code></pre> There are a few things I have made the decision not to handle to keep things simple: SQLx is already has some great macros for validating against a live schema. I don't want to reimplement this here.</li> I don't want to pay the cost of column index lookups for named fields. Instead, we will use indices and expect the queries we write return fields in correct order.</li> </ul> So if I have a struct like this: pub struct DiveMetric { pub time: i32, pub depth: f32, pub pressure: Option<f32>, pub temperature: Option<f32>, }</code></pre> It should automatically derive the trait like so: impl FromRow for DiveMetric { fn from_row(row: Row) -> Result<Self, Error> { Ok(Self { time: row.try_get(0)?, depth: row.try_get(1)?, pressure: row.try_get(2)?, temperature: row.try_get(3)?, }) } }</code></pre>Crate Layout</h2> You will need at minimum 2 crates to do this: One crate for the macro itself. I.e, it has proc-macro = true</code> in the Manifest, I called this macro</code></li> One crate has the definitions of the trait. I have called this core</code>. This crate will include the macro as a dependency</li> Any extra crates that require this can depend on core</code></li> </ul> Procedural Macro Crate</h2> A Procedural Macro Crate is built at compile time and can influence your source code, pretty much by injecting source code in the form of "tokens" in-line with the rest of your code. The TokenStream</code></a> interface is very very bare minimum. You get a list of these tokens and it's up to you to work out what they mean. Luckily there is a crate out there, syn</code></a>, for parsing a TokenStream</code> into a higher level structure. Think reflection but at compile time. So firstly, let's set up the crate manifest: [package] name = "divedb_macro" version = "0.1.0" edition = "2021" [lib] proc-macro = true [dependencies] syn = { version = "2", features = ["full", "parsing"] }</code></pre> This is a standard crate, with the only difference is in that lib</code> section: we tell it this is a proc-macro crate. Then, we are going to create a derive macro with the same name as the trait, i.e, FromRow</code>. We will just create this in lib.rs</code> the entrypoint to our crate. The scaffolding we have to lay out is a method that looks something like this: use proc_macro::TokenStream; #[proc_macro_derive(FromRow)] pub fn derive_from_row(input: TokenStream) -> TokenStream { // do something with the token stream here }</code></pre>Parsing the TokenStream</h3> First step is to convert a TokenStream</code> into a structure we can work with. We'll use syn</code> to do the heavy lifting here. There are some helper methods to parse out as a DeriveInput</code></a> struct: let input = parse_macro_input!(input as DeriveInput);</code></pre> Looking at DeriveInput</code>, there is the Data</code></a> enum, which is an enum of 3 different types, Struct</code>, Enum</code> or Union</code>. We only care about structs, and don't want to derive it for anything else. So what we can do is an if let</code> match on the variant we want, and otherwise error out with some sort of message that will hopefully remind you when you derive it later on. There are a few ways to error out & most require a span</code>: the portion of the TokenStream</code> that relates to the error. I.e, where the red squiggly line will show up. You can also panic!</code> if you are shortcutting massively, but I wouldn't recommend it. I've elected to do a catchall in the case it's not in a structure that I want to support. Using syn::Error</code></a>, we can provide it a span, and a message. Since our method needs to return a TokenStream</code>, not a syn::Error</code>, we can convert the error by injecting a compile_error!()</code> in the form of a TokenStream</code> by using the to_compile_error()</code></a> function: #[proc_macro_derive(FromRow)] pub fn derive_from_row(input: TokenStream) -> TokenStream { let input = parse_macro_input!(input as DeriveInput); // Do something with the fields here // Catchall if we don't match on the structure we don't want TokenStream::from( syn::Error::new( input.ident.span(), "Only structs with named fields can derive `FromRow`" ).to_compile_error() ) }</code></pre> A lot of the examples for proc macros I've seen introduce more methods and structure here, but for brevity I've kept it all in the one method. With our high level structure out of the way, we can do an if let</code> match on finding a struct: if let syn::Data::Struct(ref data) = input.data { // we know it's a struct }</code></pre> You'll find that there are actually 3 variants of a struct: Named field structs, Unnamed (tuple) structs and unit structs: struct Named { field: String } struct Unnamed(String); struct Unit;</code></pre> It might be important if you are fleshing out a derive macro to deal with those other two cases. For this macro we only care about the first variant, the name field struct, not the other two. So we match on that: if let Fields::Named(ref fields) = data.fields { // Deal with a named-field struct }</code></pre>Emitting our TokenStream</h3> From here, we want to iterate over the fields, and for each field emit a bit of code like: <field_name>: row.try_get(<field_idx>)?,</code></pre> However: we can't just send a String</code> back from our method, we need to provide a TokenStream</code>. This is where the second handy crate comes in: quote</code></a>. We can use the quote!</code> macro to generate code snippets in a mostly-readable fashion and output them as a TokenStream</code>. One thing to note: quote!</code> will return a proc_macro2</code> TokenStream</code>, so you can either call into()</code> or use TokenStream::from</code>. So let's iterate over our fields, and for each field, return a snippet, a TokenStream</code>, which we can combine later: let field_vals = fields.named.iter().enumerate().map(|(i, field)| { // grab the name of the field let name = &field.ident; quote!(#name: row.try_get(#i)?) });</code></pre> We're using the standard iter().enumerate()</code> here to get out an index number of the field and the field.ident</code> to reflect the name of the field. As an alternative, if instead you wanted to use column names instead of column indices, you could adjust this easily: let field_vals = fields.named.iter().map(|field| { let name = field.ident.as_ref().unwrap(); let name_string = name.to_string(); quote!(#name: row.try_get(#name_string)?) });</code></pre> Ok, now we have a list of fields, and for each field, we have a snippet of source code, the next is to combine them and implement the FromRow</code> trait. Once again we can use quote!</code> to do this. Another thing worth noting is that it's as if the proc macro literally injects a bit of source code in-line where you call derive. This means that if you are going to refer to any structs/crates/modules etc.. it makes things a lot easier to refer to them via their full path. I.e, don't put Row</code>, put ::tokio_postgres::Row</code> instead. With that in mind, here's our simple quote!</code> statement: let name = input.ident; quote!( impl ::divedb_core::FromRow for #name { fn from_row(row: ::tokio_postgres::Row) -> Result<Self, ::anyhow::Error> { Ok(Self { #(#field_vals),* }) } })</code></pre> Most of this is self-explanatory. The only weird/interesting thing is this part: #(#field_vals),*</code></pre> This will just iterate over our field vals, and include each one. Putting it all together, we get a pretty terse, but powerful proc macro: use proc_macro::TokenStream; use quote::quote; use syn::{parse_macro_input, DeriveInput, Fields}; /// This derives the `FromRow` trait for structs /// Requires that the query is in field order, as it just uses row indices #[proc_macro_derive(FromRow)] pub fn derive_from_row(input: TokenStream) -> TokenStream { // Parse it as a proc macro let input = parse_macro_input!(input as DeriveInput); if let syn::Data::Struct(ref data) = input.data { if let Fields::Named(ref fields) = data.fields { let field_vals = fields.named.iter().enumerate().map(|(i, field)| { let name = &field.ident; quote!(#name: row.try_get(#i)?) }); let name = input.ident; return TokenStream::from(quote!( impl divedb_core::FromRow for #name { fn from_row(row: tokio_postgres::Row) -> Result<Self, anyhow::Error> { Ok(Self { #(#field_vals),* }) } })); } } TokenStream::from( syn::Error::new( input.ident.span(), "Only structs with named fields can derive `FromRow`", ) .to_compile_error(), ) }</code></pre>Core Crate</h2> The core crate implements the trait itself & also exports the macro as well. If the macro & trait are the same name, you can cheat and pub use</code> the macro crate so you don't always have to import the macro & trait separately. Otherwise the core lib.rs</code> is rather simple: use anyhow::Error; // Export our derive macro here. pub use divedb_macro::*; use tokio_postgres::Row; /// Really simple ORM for `tokio_postgres` pub trait FromRow { /// hydrate a struct from a database `Row` fn from_row(row: Row) -> Result<Self, Error> where Self: std::marker::Sized; ... }</code></pre>Standard Crate</h2> Now we have the macro crate, the core crate, we can simply use</code> this to derive on our own types: use divedb_core::FromRow; #[derive(FromRow)] pub struct DiveMetric { pub time: i32, pub depth: f32, pub pressure: Option<f32>, pub temperature: Option<f32>, }</code></pre>Expanding the Macro</h3> Remember when I said that these macros pretty much inject more source code in line? Well you can see that intermediate output rather easily by using cargo expand</code></a>. This will print out the results of the macro expansion and show what code is actually being generated & a lot more. If you are writing something super tricky, it really helps to see this output & makes a great tool for debugging procedural macros. I.e, for our above DiveMetric</code> struct, using column indices: impl divedb_core::FromRow for DiveMetric { fn from_row(row: tokio_postgres::Row) -> Result<Self, anyhow::Error> { Ok(Self { time: row.try_get(0usize)?, depth: row.try_get(1usize)?, pressure: row.try_get(2usize)?, temperature: row.try_get(3usize)?, }) } }</code></pre> And for using the column name variant: impl divedb_core::FromRow for DiveMetric { fn from_row(row: tokio_postgres::Row) -> Result<Self, anyhow::Error> { Ok(Self { time: row.try_get("time")?, depth: row.try_get("depth")?, pressure: row.try_get("pressure")?, temperature: row.try_get("temperature")?, }) } }</code></pre>Conclusion</h2> As you see, it's not too onerous to make your own custom derives. There is a bit of boilerplate that is needed, but you can write code to write code and keep things simple. Without greater support for compile time reflection in rust, you can reach for macros to help you remove the repetitive tasks we face as programmers Synchronizing state with Websockets and JSON Patch Unknown — Sun, 29 Jan 2023 00:00:00 +0000 At formlogic</a>, we are using rust for a number of components within our tech stack, including: internal and external web applications, coordinating and logging machining workflows, parsing an industrial domain specific language (G-Code</a>), and more. One of the challenges we faced early on was we had state that needed to be synchronized between rust applications, browsers and the database itself. We wanted to make sure that everyone could subscribe to state changes and everyone could have the same view. We also wanted the ability to roll back state, but preserve history i.e, have an append only immutable event log, while also being able to extend the structure of our state if we choose to in the future. This blog will step through one possible solution for this challenge using a combination of WebSockets</a> for transport and JSON Patch</a> for describing changes to an arbitrary structure. To provide an example of this, we'll be building up everyone's favourite Todo Application: a list of todos with a mark to say whether they're completed or not. However, our version will ensure that all connected clients will, in close to real time, have updates shown. For those wanting to skip ahead and see the finished product, you can view the source here</a>. </video> What is JSON Patch</h2> JSON Patch</a> is an RFC standard for describing patch updates to an arbitrary JSON structure. For instance, if you had a JSON object that looked like this: { "name": "Hello" }</code></pre> And a patch that looks like this: [ { "op": "replace", "path": "/name", "value": "Hello World!" } ]</code></pre> Then applying this patch, your object will now look like this: { "name": "Hello World!" }</code></pre> Another way to think about it, is that JSON Patch can encode transitions of state, and can be considered a lightweight form of event sourcing</a>. There are a number of libraries available in all languages, including rust</a> and for browsers</a>, and as the format itself is just JSON, it makes it easy to save the patches themselves in jsonb</code> column on your database. JSON Patch does not, by itself, enforce any schema for the JSON document. There are some operations that can cause an error if applied incorrectly, such as out of bounds replacements in arrays, but nothing that enforces that the end result is what you are expecting. For that purpose we're going to leverage serde</code></a> for its great type safety, ensuring our state is in the shape we want. WebSockets and Updates</h2> With JSON Patch we have a stream of changes to a structure we can act upon, but how do we transport it? Well, since we want browsers to be a part of this as well, a great candidate is WebSockets</a>. WebSockets are a great way to stream things into a frontend without having the overhead of having a HTTP Connection per message. This channel can stay open for as long as the browser is open. We can receive also messages in near real time: only network latency and a bit of decoding act as a bottleneck. Luckily WebSocket support is not only a feature of backend frameworks, but also has clients suited to connecting as well. We'll only be investigating browsers in this blog for brevity, but one could easily expand it to use something like tokio-tungstenite</code></a> to consume our messages in rust. Rust Backend</h2> For brevity's sake we will choose axum</code> as the backend, and their websockets example</a> as the base. For this example to build by itself, there are a few dependent crates: axum = { version = "0.6.4", features = ["ws"]} tokio = { version = "1", features = ["full"] }</code></pre> The ws</code> feature adds in websocket support to axum, otherwise all we'll use the defaults For debug logs, we'll use tracing</code></a> with tracing-subscriber</code> to get things logged nicely to stderr: tracing = "0.1.37" tracing-subscriber = "0.3.16"</code></pre> And for our example we're going to use serde</code> and serde_json</code> for serialization, with json-patch</code> as mentioned above. We'll also include the futures</code> crate, as we'll see that comes in use later when we want to split the rx/tx of a websocket channel: serde = { version = "1.0", features = ["derive"] } serde_json = "1.0" futures = "0.3" json-patch = "0.3.0"</code></pre>The Main Method</h3> With the main method we want to instantiate some logging, set up our routes for our web app & then serve on a port. While the initial example includes some static file hosting, we will skip those routes for brevity and only include the websocket route. Our main function becomes quite concise: async fn main() { // Set a sensible default for logging to ensure we see something if std::env::var_os("RUST_LOG").is_none() { std::env::set_var("RUST_LOG", "websocket_jsonpatch=debug") } // Initialise the `fmt` subscriber which will print logs to stderr tracing_subscriber::fmt::init(); // Add in our application with the websocket handler let app = Router::new() .route("/ws", get(ws_handler)) // Add in a shared websocket state struct `WsState` .layer(Extension(Arc::new(WsState::new()))); // Listen on port 3333 for connections let addr: SocketAddr = "127.0.0.1:3333".parse().unwrap(); debug!("Listening for requests on {}", addr); // Start up the server axum::Server::bind(&addr) .serve(app.into_make_service()) .await .unwrap(); }</code></pre>The Todo Struct</h2> Let's start shaping our todo struct: We want to name a todo list itself, such as Shopping List</code> or Writing a blog about Rust</code>. This also gives us an easy thing to update in the structure.</li> We want a list of entries, which have a name and a boolean as to whether they are marked Completed</code> or not.</li> We want some convenience derives such as for serialization with serde and sensible defaults.</li> </ul> With that in mind, we have the main Todo</code> struct: #[derive(Serialize, Deserialize, Default, Debug, Clone)] pub struct Todo { // The main name of the todo list name: String, // The list of todos todos: BTreeMap<u32, TodoRow>, }</code></pre> And each Row has the following shape: #[derive(Serialize, Deserialize, Default, Debug, Clone)] pub struct TodoRow { // The name of the specific todo name: String, // Whether this todo is completed or not completed: bool, }</code></pre>A Note on using a BTreeMap</code> instead of a Vec</code></h4> An initial version of this blog included todos as a Vec<TodoRow></code>, but that was found to be quite noisy when clearing out completed todos. Refactoring this to use a BTreeMap</code> instead took not long at all, and is a testament to this design being malleable. Readers are invited to adjust this shape back to a Vec</code> or provide other fields/values to see how to make such a change. Making Changes</h3> If you have used the redux pattern</a> before, you would be familiar with use actions to mutate state. We'll build something similar here, to make it easier to describe the intent of the change and to have all the logic in one module. These actions are what are going to be sent from the browser or other clients. When actions are received, the todo state is changed, and we generate a diff between the old and new JSON, which is then sent over the wire back to the frontend in the form of JSON Patches. These changes are executed in serial by means of a held mutex, rather than allow concurrent updates to the state. We'll also only have one "todo" list, but could imagine an easy extension to have multiple different lists, each with their own changes. For concurrency without making changes serial, a CRDT</a> like automerge</code></a> could be a suitable candidate. In practice, we've found that, when given a low concurrent user count per JSON object, the simplicity and auditability of JSON patches is a great trade-off. Actions</h3> We're going to have a few actions that are available to use (all of which should be self-descriptive): #[derive(Deserialize)] #[serde(tag = "type")] pub enum TodoAction { Add { row: TodoRow }, ChangeName { name: String }, Update { index: u32, row: TodoRow }, Remove { index: u32 }, RemoveCompleted, }</code></pre> We are using a tag to make it easier to define these in TypeScript later on. Serde will use a field named type</code> to store the enum variant. With those actions, we can then use them to apply to our Todo</code> struct and update the state: impl Todo { pub fn apply(&mut self, action: TodoAction) { match action { TodoAction::Add { row } => { // Find the next available index let index = self.todos.keys().max().copied().unwrap_or_default() + 1; // Insert this into our map self.todos.insert(index, row); } TodoAction::Update { row, index } => { self.todos.insert(index, row); } // Change the name of the todo list TodoAction::ChangeName { name } => self.name = name, TodoAction::Remove { index } => { self.todos.remove(&index); } // Filter and remove all completed todo rows TodoAction::RemoveCompleted => self.todos.retain(|_, val| !val.completed), } } }</code></pre>WebSockets</h2> We are going to use websockets as our primary means of communication between the front and backend, providing a bi-directional channel with which we can send messages. Shared WsState</h3> In the main()</code> function, we added a reference counted WsState</code> struct via AddExtensionLayer</code>. This represents our shared state between all websocket sessions. Inside this struct, we are going to keep two things: The Todo</code> struct as per our modelling above inside a tokio Mutex</code></li> A list of websocket sinks</a> that we can send updates to. So when one session makes a change, it is reflected on all other sessions.</li> </ul> This struct looks like so: pub struct WsState { // Our main state, behind a tokio Mutex todo: Mutex<Todo>, // A list of sessions we will send changes to txs: Mutex<Vec<SplitSink<WebSocket, Message>>>, }</code></pre>Adding Sessions</h4> When we add a session, we want to send a full state to get it caught up, and then add it to our list: async fn add_session(&self, tx: SplitSink<WebSocket, Message>) { // Get a mutable lock of our transactions let mut txs = self.txs.lock().await; // Send the initial state update if let Err(err) = tx .send(Message::Text( // This method will not fail in "normal" operations so an `expect()` is OK here serde_json::to_string(&ServerMessage::Full { todo: &*self.todo.lock().await, }) .expect("Serialize Error"), )) .await { warn!("Could not send initial state update: {}", err); return; } // Add session to our list of sessions txs.push(tx); }</code></pre>Applying State Changes</h3> On WsState</code> we'll create a method called apply</code> which takes a TodoAction</code> and applies this to the state, then broadcasts changes to all connected sessions. We'll break this method down as there is a lot happening here. The signature of this method looks like this: async fn apply(&self, action: TodoAction) -> Result<(), Error> { ... }</code></pre> The first thing we need to do is grab a mutable reference to our existing state: let mut state = self.todo.lock().await;</code></pre> Then we want to get a copy of the state as it stands currently, but as a serde_json::Value</code> which we can use to diff: let existing_json = serde_json::to_value(&*state)?;</code></pre> After we have our existing_json</code>, we can use the Todo::apply()</code> method to mutate the value: // Apply the action to our todo list. This mutates it in place state.apply(action);</code></pre> With that applied, we can grab the new state as a Value</code> as well: // Serialize out the new JSON for diffing let new_json = serde_json::to_value(&*state)?;</code></pre> Then, we can use the diff</code></a> method from json_patch</code> to automatically generate the patches needed to go from the existing to the new state: // Get the changes using the `diff` method from `json_patch` let ops = diff(&existing_json, &new_json).0;</code></pre> With this, you'll notice we are quite agnostic to what the shape is of the state, and how the state changes. All we need to be able to do is serialize it before and after, and we can completely change the structure internally. A further excercise would be to generalize this method to accept anything that implements Serialize</code> Ok, we have a list of patches, let's print them out so we can inspect them: debug!("New Patches:{:?}", ops);</code></pre>Broadcasting Our Changes</h4> Generating changes to state is one half of this method, the other half is to broadcast this out to every session. We will first check whether there were any changes worth broadcasting, and then will serialize it to a string // If there are no changes, don't bother broadcasting if !ops.is_empty() { // Serialize it to a json string here let message = serde_json::to_string(&ServerMessage::Patch { ops })?; ... }</code></pre> The next thing we want to do is grab all of our sessions and lock them for the time being: let mut txs = self.txs.lock().await;</code></pre> Now, some session between now and the last broadcast could have gone away. What we are going to do is take all the sessions out of the existing vec, replacing it with an empty one using std::mem:take()</code>, and then try sending to each session. If there is no issue sending, we add it back to the list. If there is an issue sending, we will log the error but not add it back, to be dropped/cleaned up when the object is out of scope: // We take all the txs to iterate, and replace with an empty `Vec` for mut tx in mem::take(&mut *txs) { // If there is an issue sending a message we will warn about it if let Err(err) = tx.send(Message::Text(message.clone())).await { warn!("Client disconnected: {}", err); // If there is no issue sending, then we add it back to our `Vec` } else { txs.push(tx) } }</code></pre> And then we're done! To recap, when applying changes we first generate a list of patches to apply. We then send that list to all the connected sessions, cleaning up any erroneous sessions in the process. Axum Websocket Interface</h3> Axum provides a WebSocketUpgrade</code> struct, with which you can feed it a function (or closure) to start an async task when the websocket upgrade happens. We use this to feed in our WsState</code> value (behind an Arc</code>): async fn ws_handler( ws: WebSocketUpgrade, Extension(state): Extension<Arc<WsState>>, ) -> impl IntoResponse { debug!("New Websocket Connection"); ws.on_upgrade(|socket| handle_socket(socket, state)) }</code></pre>Server Messages</h3> To keep things simple we will have an enum of possible messages the server/backend can send to clients. There are going to be only two types: a list of patches to apply or a full todo struct. When clients first connect, we need to get them caught up with the current state of the world, so we send them the full state. #[derive(Serialize, Debug)] #[serde(tag = "type")] enum ServerMessage<'a> { Patch { ops: Vec<PatchOperation> }, Full { todo: &'a Todo }, }</code></pre> As per the TodoAction</code> we'll use a type</code> tag for ease of integration with TypeScript. Also notice that we will be able to send multiple patches in one message. This is for in the case of a change causing a cascade of patches, such as when removing completed items in the todo list. This ServerMessage</code> enum will be serialized on the wire as JSON. Websockets support standard text messages, as well as binary types. A further exercise would be to use a different encoding, such as CBOR</code></a>, but does make things a bit less human readable Handle Socket</h3> Our handle_socket</code> method is responsible for initiating new session & feeding that session into the WsState</code> struct. pub async fn handle_socket(socket: WebSocket, state: Arc<WsState>) { // Magic happens here }</code></pre> Firstly, we'll split up the websocket into a tx/rx pair. We'll have the handle_socket</code> hold onto the rx</code> part, and move the tx</code> into the WsState</code> value. If you have a struct that implements Stream</code> and Sink</code> you can use futures and StreamExt</code> to split this into two separate objects, with separate ownership of both. let (tx, mut rx) = socket.split();</code></pre> We'll then add this session to our shared state: state.add_session(tx).await;</code></pre> This method will fire off a ServerMessage::Full</code> with the current state, returning early if this fails Then, we simply loop any messages received from the client & apply them to our shared state. The type of messages we receive from the client will be our TodoAction</code> enum. To loop while there are messages, we can use the following let</code> binding: while let Some(Ok(msg)) = rx.next().await { ... }</code></pre> This loop will terminate if there are no more messages to receive (i.e, the stream is finished and returning None</code>), and if there are any errors receiving messages. Inside the loop, we decode the message as JSON, and throw a warning if it's a message we're not expecting. We then apply that message to our shared state. As above, the shared state will in turn broadcast out JSON patches to update the state: // Loop until there are no messages or an error while let Some(Ok(msg)) = rx.next().await { if let Message::Text(text) = msg { // Decode our message and warn if it's something we don't know about if let Ok(action) = serde_json::from_str::<TodoAction>(&text) { // Apply the state, which will broadcast out changes as a JSON patch if let Err(err) = state.apply(action).await { warn!("Error applying state:{}", err); } } else { warn!("Unknown action received:{}", text); } } }</code></pre> Ok! So we have the shape of our data, some actions to mutate the state and some scaffolding to wire up the websockets. What's next is to build the frontend. React Frontend</h2> We're going to build a companion browser application to use our backend. This will be built in react, but could be built using any framework, or even vanilla javascript. Vite</a> will be the build tool of choice, given it has a fast development feedback loop and supports React/TypeScript. For styling, we'll use spectre.css</a> to make it look a little better than the standard style and provide some responsivness. You can create a new vite project by running: yarn create vite front --template react-ts</code></pre> We'll want to run the rust backend in another terminal window with: cargo run</code></pre> And then we want to proxy all requests to /ws</code> to the backend which we can do by changing the vite.config.ts</code>: export default defineConfig({ plugins: [react()], server: { proxy: { "/ws": { target: "http://127.0.0.1:3333", ws: true, }, }, }, });</code></pre> With that in place, we can yarn dev</code> to get up a developer window. Todo Structure</h3> We want to have the todo structure imitate what's in the backend. For brevity here we'll just write them by hand, but there does exist tools to have these structures generated automatically (see the blog: Publishing Rust types to a TypeScript frontend</a>). Starting off with the Todo</code> struct itself: export interface Todo { name: string; todos: { [index: number]: TodoRow }; } export interface TodoRow { id: number; name: string; completed: boolean; }</code></pre> Then, we deviate a little bit with the actions. In the backend we have used a tagged enum with the type</code> property to define what action is in use. In typescript, we will use a Discriminated Union</a> pattern to define these actions in a compatible way: export interface Add { type: "Add"; row: TodoRow; } export interface ChangeName { type: "ChangeName"; name: string; } export interface Update { type: "Update"; row: TodoRow; index: number; } export interface Remove { type: "Remove"; index: number; } export interface RemoveCompleted { type: "RemoveCompleted"; } export type TodoAction = Add | ChangeName | Update | Remove | RemoveCompleted;</code></pre>Websocket Component</h3> For the Websocket component, we'll cheat a little bit and use some global variables to hold around both the websocket connection itself & the shared state. Using react hooks we'll integrate with components to allow state changes to come through. Firstly, we define our server messages as per the backend. We'll be using the same Discriminated Union pattern. interface Patch { type: "Patch"; // `Operation` from `fast-json-patch` module ops: Operation[]; } interface Full { type: "Full"; todo: Todo; } type ServerMessage = Patch | Full;</code></pre> Then, let's set up our global variables: let websocket: WebSocket | undefined; let todo: Todo | undefined;</code></pre> We use undefined</code> to indicate those values aren't initialised. Both the websocket & the todo itself are initialised at different times. Then, we'll create a setupWebsocket</code> function which will handle the connection & and plumbing needed to send and receive messages. We'll want to have a callback function to accept a new todo</code> object which we'll use to integrate into react hooks later. So with the the signature looks like the following: const setupWebsocket = (onTodoUpdate: (todo: Todo) => void) => { ... }</code></pre> Then we want to create a new WebSocket</code>, but we need to pass it a valid uri. Websockets use ws://</code> instead of http://</code> and wss://</code> instead of https://</code> for their URLs, so we'll split apart the current window.location</code> and use components of it to build our uri: const loc = window.location; const uri = `${loc.protocol === "https:" ? "wss:" : "ws:"}//${loc.host}/ws`; console.log(`Connecting websocket: ${uri}`); const connection = new WebSocket(uri);</code></pre> Ok, with our connection created we will need to listen to different events, and execute accordingly. Firstly, when the connection is opened, we'll log that its connected and then set our global websocket</code> variable equal to this connection: connection.onopen = () => { console.log("Websocket Connected"); websocket = connection; };</code></pre> Now, there is a case that a websocket connection drops while you have the page open. We want to recover from this somewhat gracefully, retrying the connection until it's successful. One way of doing this is to re-execute the setupWebsocket</code> method after a bit of time has passed, and that's what we'll do here, retry the setup after half a second has passed. If the websocket has closed gracefully, then we don't bother retrying: // If we receive a close event the backend has gone away, we try reconnecting in a bit of time connection.onclose = (reason) => { websocket = undefined; // https://developer.mozilla.org/en-US/docs/Web/API/CloseEvent if (reason.code !== 1000 && reason.code !== 1001) { console.error("Websocket connection closed", reason); setTimeout(() => { setupWebsocket(onTodoUpdate); }, 500); } };</code></pre> Then, if there is an error with the websocket we can log what that error is, then close it connection.onerror = (error) => { console.error("Error with websocket", error); connection.close(); };</code></pre> Ok with the websocket life cycle out of the way, now comes to the meat of our event loop. What we're going to do is listen for server events, and if either type is received, act accordingly: For the Full</code> server message, the todo</code> object will be replaced outright.</li> For the Patch</code>, we will use the fast-json-patch</code> module to mutate todo</code> in place, and then execute our callback:</li> </ul> Whatever the type of message, the full todo</code> object is provided to the callback, with the patch merging logic happening here, and shielded from our callback update. connection.onmessage = (message) => { // Parse the message via JSON, asserting what we'll get from the backend const msg = JSON.parse(message.data) as ServerMessage; switch (msg.type) { case "Patch": { // Mutate the todo state in place, and then send an update if (todo !== undefined) { let { newDocument: newTodo } = applyPatch(todo, msg.ops, false, false); onTodoUpdate(newTodo); todo = newTodo; } break; } case "Full": { // Send on the full todo state onTodoUpdate(msg.todo); todo = msg.todo; break; } } };</code></pre> Now, we're almost done with the websocket, we need to integrate it a tiny bit nicer with react and react hooks. Firstly, using the useEffect</code> hook, we'll expose the todo</code> state to a component which will rerender on object updates. You can provide a function starting with the keyword use</code> to have it part of of the react render lifecycle. We'll bind a call back to update a variable todo</code> handled with useState</code> when there are changes, and then return that variable state from the function to use in another component. export const useWebsocket = () => { // Keep our local state of the todo app to trigger a render on change let [todo, updateTodo] = useState<Todo>(); useEffect(() => { // Update our app state when changes are received setupWebsocket((msg) => { updateTodo(msg); }); // If the destructor runs, clean up the websocket return () => { if (websocket) { websocket.close(1000); } }; // The empty `[]` dependency list makes this `useEffect` callback execute only once on construction }, []); return todo; };</code></pre> Another way of looking at it is this function is purely glue code to make it easy to integrate with react. Lastly, we'll cheat again with our global variables and have a sendAction</code> method which will, if the websocket is up and running, send a TodoAction</code> to the backend: export const sendAction = (action: TodoAction): void => { if (websocket) { websocket.send(JSON.stringify(action)); } };</code></pre>React Components</h3> We now have our general shape of our todo app, and have our websocket connection abstracted away nicely it's time to build up components. We'll start with the main App</code> component. This will be our root component, and will be responsible for building other components based upon the state. The App</code> component is a perfect candidate to use our useWebsocket</code> hook we have defined. We'll add in a small "loading" indicator if the todo</code> state is undefined, otherwise we'll pass on our todo</code> state to a component: function App() { const todo = useWebsocket(); return ( <div className="container grid-lg"> <div className="columns"> <div className="column col-lg-12 todo-title"> <h1>Todo App Example</h1> </div> </div> {todo === undefined && <div className="loading loading-lg"></div>} {todo && <TodoComponent todo={todo} />} </div> ); }</code></pre>Todo Component</h4> Our TodoComponent</code> is responsible for some of the more "global" actions of our state, such as adding a new todo, changing the todo list name and clearing out completed todos. Let's start with a standard functional component that takes a todo</code> property as input: function TodoComponent({ todo }: { todo: Todo }) { return <></>; }</code></pre> Now, let's build in the todo.name</code> property. If we want users to be able to edit it, it should live in an input field: <input value={todo.name} /></code></pre> This will ensure the value is reflected, and since our todo</code> object is "hooked" in to react, it will update when it changes. The next step is to use an onChange</code> callback to the input to send actions to the backed on change. We want to fire the ChangeName</code> action, and so we can use the sendAction</code> function from the websocket module to trigger this change. <input value={todo.name} onChange={(ev) => { sendAction({ type: "ChangeName", name: ev.currentTarget.value, }); }} /></code></pre> With that in place, we have the bare minimum implemented to see state changes. Now would be a great time, if you have been following along, to fire up your IDE and see what happens when you type in that input field (make sure the backend is running!). Now, for each TodoRow</code> we'll create a TodoRowComponent</code>, which will be responsible for the state of that given row: { Object.entries(todo.todos).map(([index, row]) => ( <TodoRowComponent key={index} row={row} index={+index} /> )); }</code></pre> And underneath our the list, we will have two buttons. One that only displays if there are todos that are completed: Object.entries(todo.todos).some(([, val]) => val.completed) && ( <button className="btn" onClick={() => { sendAction({ type: "RemoveCompleted", }); }} > Remove Completed </button> );</code></pre> And one that adds an extra item to the list: <button onClick={() => { const entries = Object.entries(todo.todos); sendAction({ type: "Add", row: { name: "", completed: false, }, }); }} > Add Todo </button></code></pre>TodoRowComponent</h4> The TodoRowComponent will take a TodoRow</code> and the index</code> of that row, and display actions for the items in our todo list: function TodoRowComponent({ row, index }: { row: TodoRow; index: number }) { ... }</code></pre> We can add in a check box to mark things completed or not: <input type="checkbox" checked={row.completed} onChange={() => { sendAction({ type: "Update", row: { ...row, completed: !row.completed, index, }, }); }} /></code></pre> And an input for the name of the specific todo list: <input value={row.name} type="text" onChange={(ev) => sendAction({ type: "Update", row: { ...row, name: ev.currentTarget.value, index, }, }) } /></code></pre> And a button to remove that row: <button onClick={() => sendAction({ type: "Remove", index, }) } /></code></pre>Running and Debugging</h2> Ok, we now have a frontend wired up to a backend, with a websocket used for state updates. Let's change a few values around and see what is going across the "wire". The easiest way to do this, is via your browser's network tab, looking for the request to ws://localhost:3000/ws</code>. In chrome, the tab you're looking for is Messages</code>, and in firefox it's Response</code>. You can also use a tool like websocat</code></a> to view the messages in a terminal. Running this up, you'll see the initial message come through when subscribed: { "type": "Full", "todo": { "name": "", "todos": [] } }</code></pre> Then, making changes you will see the updates feed in. For instance if you click the Add Todo</code> button, you'll see an outbound action: { "type": "Add", "row": { "name": "", "completed": false } }</code></pre> Followed by a patch update from the server: { "type": "Patch", "ops": [ { "op": "add", "path": "/todos/0", "value": { "completed": false, "name": "" } } ] }</code></pre> Go ahead and try out other updates to see what patches are generated and get a feel for the JSON patch format. Conclusions</h2> Starting from the backend and then building out the frontend, we've used Websockets and JSON Patch to provide a platform of shared state updates. Using existing standards and combining them together makes for a powerful and solid foundation. With rust, we get some great concurrency primitives that we can rely on to ensure our logic is correct. We've demonstrated here that, we can provide a powerful framework, abstracting away our transport and state update logic, with room to grow. The code for this article is available in a source repository here</a>, so please feel free to tweak. Publishing Rust types to a TypeScript frontend Unknown — Tue, 13 Sep 2022 00:00:00 +0000 When building in a strongly-typed language such as rust it's a shame to have to throw this strictness away, especially when dealing with the wild west of frontend stacks. The bigger the project is, the more you want to keep things cohesive. This cohesion evaporates quickly when you need to pass language barriers. While normal JavaScript does not have types, we can utilise TypeScript</a> to share this type information and hopefully maintain some cohesion between languages. The scenario we're targeting here is writing a web backend in rust, and having those types/methods available easily for a TypeScript</code> enabled frontend. This article describes a few common methods we can bridge the two languages together using currently available tools and libraries. Most of them follow similar characteristics, but each approach has their own set of tradeoffs. This blog will primarily use actix-web</code></a> as the example for the web backend, as it is one I'm most familiar with, but the libraries used here do have support for other backends. Derive Macros</h2> Strictly speaking, all these methods use derive macros in some way, but what I am describing here is using derive macros to publish structs/objects as typescript definitions. These don't include function/methods or http requests (which will be discussed later on). Rust does not have runtime reflection built into the language & so relies on decorating types via macros to provide some sort of approximate "reflection". In terms of the two main derive macro libraries to choose from, we have ts-rs</code></a> and schemars</code></a>. Both of these libraries are integrated tightly with serde</code>. ts-rs</h3> The ts-rs</code></a> library will allow you to output typescript types for your existing objects: #[derive(TS)] #[ts(export)] struct User { user_id: i32, first_name: String, last_name: String, }</code></pre> The #[ts(export)]</code> attribute will export this type when you run cargo test</code> on your crate. By default this will be in the bindings/</code> directory, with a separate typescript file per object type: export interface User { user_id: number; first_name: string; last_name: string; }</code></pre> Alternatively, you can use the TS::decl()</code></a> method to export it as a String</code>. fn main() { println!("{}", User::decl()); }</code></pre>schemars</h3> Now, schemars</code></a> does not export to typescript directly, but will export to JSON Schema</a> which can be consumed & converted into typescript with the help of conversion tools</a>. Working similar to ts-rs</code>, you can derive JsonSchema</code> on your objects: #[derive(JsonSchema)] struct User { user_id: i32, first_name: String, last_name: String, }</code></pre> From there, you can print out the schema with the help the schema_for</code> macro & serde_json</code>: fn main() { let schema = schema_for!(User); println!("{}", serde_json::to_string_pretty(&schema).unwrap()); }</code></pre> With this schema in hand, you can use a tool like json-schema-to-typescript</code></a> to output it as typescript types: $ cargo run | npx json-schema-to-typescript</code></pre>/* tslint:disable */ /** * This file was automatically generated by json-schema-to-typescript. * DO NOT MODIFY IT BY HAND. Instead, modify the source JSONSchema file, * and run json-schema-to-typescript to regenerate this file. */ export interface User { first_name: string; last_name: string; user_id: number; [k: string]: unknown; }</code></pre>OpenAPI Generators</h2> While straight derive macros are great at defining types, you sometimes also want some type safety around the methods used. If your backend is a web server, it's great to auto-generate the appropriate typescript code for making HTTP API calls. Luckily, if you can generate an OpenAPI spec</a>, you can auto-generate clients for use in the frontend, all typed and ready to go! Depending on your backend web framework will dictate what support there are for this method. There is a push for the most common web frameworks to have OpenAPI support with varying states of maturity. Paperclip and Actix Web</h3> This method is one I am most familiar with, using paperclip</code></a> with an actix-web</code></a> backend. The paperclip</code></a> crate allows you to annotate structs & request methods to have it auto-generate an OpenAPI spec for you: use actix_web::{App, Error, HttpServer}; use paperclip::actix::{ api_v2_operation, web::{self, Json}, Apiv2Schema, OpenApiExt, }; use serde::{Deserialize, Serialize}; #[derive(Serialize, Deserialize, Apiv2Schema)] struct User { user_id: i32, first_name: String, last_name: String, } #[api_v2_operation] async fn echo_user(body: Json<User>) -> Result<Json<User>, Error> { Ok(body) } #[actix_web::main] async fn main() -> std::io::Result<()> { HttpServer::new(|| { App::new() .wrap_api() .service(web::resource("/echo").route(web::post().to(echo_user))) .with_json_spec_v3_at("/api/spec/v3") .build() }) .bind("127.0.0.1:8080")? .run() .await }</code></pre> Now, you can curl to retrieve the json spec: curl http://127.0.0.1:8080/api/spec/v3</code></pre>{ "openapi": "3.0.0", "info": { "title": "", "version": "" }, "paths": { "/echo": { "post": { "requestBody": { "content": { "application/json": { "schema": { "$ref": "#/components/schemas/User" } } }, "required": true }, "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/User" } } } } } } } }, "components": { "schemas": { "User": { "type": "object", "properties": { "first_name": { "type": "string" }, "last_name": { "type": "string" }, "user_id": { "type": "integer", "format": "int32" } }, "required": ["first_name", "last_name", "user_id"] } } } }</code></pre> However, if you're like me and would like it written out to a file rather than rely on a backend to be running, you can do so with a little bit of fiddling around. Firstly, we want to create a function to configure our services: fn config(cfg: &mut web::ServiceConfig) { cfg.service(web::resource("/echo").route(web::post().to(echo_user))); }</code></pre> Then we can adjust our main app to use this to configure actix web: App::new() .wrap_api() .configure(config) .with_json_spec_v3_at("/api/spec/v3") .build()</code></pre> And we can create another function that will write out the api spec to a file: pub fn dump_schema<P: AsRef<Path>>(path: P) { App::new() .wrap_api() .configure(config) .with_raw_json_spec_v3(|app, spec| { let mut file = File::create(&path).expect("Could not create a file"); file.write_all( serde_json::to_string_pretty(&spec) .expect("Could not serialize spec") .as_bytes(), ) .expect("Could not write schema"); app }); }</code></pre> The reason we keep it separate from the main App</code> builder inside the HttpServer</code> closure is that the builder will be called per worker & so as you can imagine it would write out the file multiple times on server startup. Now we can call it manually: dump_schema("openapi.json");</code></pre>Utoipa</h3> More general purpose and targeting a number of different web frameworks, the utoipa</code></a> library is similar in function to paperclip</code>, albeit with a little bit more boilerplate to get going. As before, a derive macro ToSchema</code> handles the straight type magic: #[derive(Serialize, Deserialize, ToSchema)] struct User { user_id: i32, first_name: String, last_name: String, }</code></pre> Decorating services is a little more cumbersome, as we need to specify the request & response types alongside a status. Here's an actix web example: #[utoipa::path(post, request_body = User, responses((status = 200, body = User)))] #[post("/echo")] async fn echo_user(body: Json<User>) -> Result<Json<User>, Error> { Ok(body) }</code></pre> You can see, that the User</code> type has to be repeated in both the request_body</code> and body</code> attributes. This could potentially mean that the generated OpenAPI specification does not match reality, so extra care is needed to ensure they match. It would be nice if the proc macro could work this out by itself rather than having to redefine them in attributes, but it would require tighter integration, something that might be added in the future. Generating a OpenAPI JSON file is a little bit easier than in paperclip. Firstly you combine all your components on a struct: #[derive(OpenApi)] #[openapi(components(schemas(User)), paths(echo_user))] struct ApiDoc;</code></pre> Then you can use some standard methods to write it out: std::fs::write( "openapi.json", &ApiDoc::openapi() .to_pretty_json() .expect("Should serialize"), )?;</code></pre>TypeScript from OpenAPI</h3> With an openapi.json</code> file you can use a tool such as openapi-generator-cli</code> to generate the requisite definitions: npx openapi-generator-cli generate -i openapi.json -g typescript</code></pre> This won't create purely typescript definitions, as there still needs to be a library to make requests and you may want to generate types closer to your frontend framework like using typescript-rxjs</code> etc... Normally this isn't run individually, but rather it is included inside package.json</code> of an existing frontend project. GraphQL</h2> GraphQL</a> can be used to publish types, albeit indirectly and does require you to modify your backend to support it. The types that can be represented in GraphQL are also quite restricted: no recursive types</a>, no shared types between input/output</a>, incomplete support for sum-types/unions</a> to name a few issues I've personally hit using it. If you're happy with the high level of opinionation GraphQL forces upon your code base, you are rewarded with some great tooling & infrastructure. The main two libraries for GraphQL on the backend are async-graphql</code></a> and juniper</code></a>. On the frontend, there is similar tooling</a> to OpenAPI for generating types & requests. You will, however, need to also write out some specific GraphQL queries, to utilise them. async-graphql</h3> As with the Derive Macro & OpenAPI generators you start by decorating your structs. We can use the SimpleObject</code> and InputObject</code> derive macros (in GraphQL you need separate types for input...): #[derive(Serialize, Deserialize, SimpleObject)] struct User { user_id: i32, first_name: String, last_name: String, } #[derive(Serialize, Deserialize, InputObject)] struct UserInput { user_id: i32, first_name: String, last_name: String, }</code></pre> Next, we want to define a Query</code> object which will dictate the methods available via GraphQL: pub struct Query; #[Object] impl Query { async fn echo_user(&self, user: UserInput) -> User { User { user_id: user.user_id, first_name: user.first_name, last_name: user.last_name, } } }</code></pre> Then we can set up a handler for graphql queries (leaving subscriptions and mutations empty for now): async fn graphql_handler( schema: web::Data<Schema<Query, EmptyMutation, EmptySubscription>>, request: GraphQLRequest, ) -> GraphQLResponse { schema.execute(request.into_inner()).await.into() }</code></pre> And a GraphiQL</a> endpoint to use a browser to craft graphql queries: async fn graphql_playground() -> HttpResponse { HttpResponse::Ok() .content_type("text/html; charset=utf-8") .body(GraphiQLSource::build().endpoint("/graphql").finish()) }</code></pre> We wrap the schema in web::Data</code> and add it as app_data</code>, then provide our two new services: #[actix_web::main] async fn main() -> std::io::Result<()> { let schema = Schema::build(Query, EmptyMutation, EmptySubscription).finish(); let schema_data = web::Data::new(schema); HttpServer::new(move || { App::new() .app_data(schema_data.clone()) .service(web::resource("/graphql").route(web::post().to(graphql_handler))) .service(web::resource("/playground").route(web::get().to(graphql_playground))) }) .bind("127.0.0.1:8080")? .run() .await }</code></pre> You can also write out the schema via the sdl()</code> method on Schema</code>: std::fs::write("example.graphql", schema.sdl())?;</code></pre>GraphiQL Playground</h4> Navigating to http://localhost:8080/playground</code> in a browser you will get the GraphiQL playground</a>: </a> juniper</h3> The juniper</code></a> crate in this context is very similar to async-graphql</code></a>. juniper</code> as a crate has been around a fair bit longer async-graphql</code> & did not start off as an async library. Both are very capable libraries, but I have found async-graphql</code> to be better documented, and more "modern". In terms of coding for our example, juniper</code> is very similar to async-graphql</code>, with derive macros on the types you want to define: #[derive(Serialize, Deserialize, GraphQLObject)] struct User { user_id: i32, first_name: String, last_name: String, } #[derive(Serialize, Deserialize, GraphQLInputObject)] struct UserInput { user_id: i32, first_name: String, last_name: String, } </code></pre> The query object is also very much a slot in: pub struct Query; #[juniper::graphql_object] impl Query { async fn echo_user(&self, user: UserInput) -> User { User { user_id: user.user_id, first_name: user.first_name, last_name: user.last_name, } } }</code></pre> Creating the schema is also similar: let schema = Schema::new(Query, EmptyMutation::new(), EmptySubscription::new());</code></pre> The handler method deviates slightly, but is mostly the same as above: async fn graphql( req: actix_web::HttpRequest, payload: actix_web::web::Payload, schema: web::Data<Schema>, ) -> Result<HttpResponse, Error> { graphql_handler(&schema, &(), req, payload).await }</code></pre> The juniper_actix</code> integration also provides a graphiql_handler</code> which you can use to mount the playground (v1</code> of graphiql</code> instead of v2</code> at this time of writing). Putting this together, we get almost an identical layout: #[actix_web::main] async fn main() -> std::io::Result<()> { let schema = Schema::new(Query, EmptyMutation::new(), EmptySubscription::new()); let schema_data = web::Data::new(schema); HttpServer::new(move || { App::new() .app_data(schema_data.clone()) .service(web::resource("/graphql").route(web::post().to(graphql))) .service( web::resource("/playground") .route(web::get().to(|| graphiql_handler("/graphql", None))), ) }) .bind("127.0.0.1:8080")? .run() .await }</code></pre> As with async-graphql</code> the graphql schema can be written out to disk using the as_schema_language()</code> method: std::fs::write("example.graphql", schema.as_schema_language())?;</code></pre>GraphQL to TypeScript</h3> Now, the next question is: we have our graphql schema, how do we get to typescript definitions? The easiest way I've found, from a maintenance perspective, is a little bit involved for this particular blog. The gist is that using GraphQL Code Generator</code></a> you can define your queries and then have frontend methods/types generated from them. This does require a bit of scaffolding in your frontend project to get working, but is rather painless when it is set up, besides having to write queries/mutations for all of your types. Conclusions</h2> As you can see there are a variety of ways of publishing rust types for use within TypeScript</code>. Having this process is as automated as possible is a great way to icnrease cohesion between front & backend. Whether these methods are right for you depends entirely on your use-case & what tradeoffs you're willing to make: Types Only: You can use direct translation tools such as ts-rs</code></a> or schemars</code></a> OpenAPI: You can go the OpenAPI route using paperclip</code></a> or utoipa</code></a> GraphQL: You can use GraphQL with async-graphql</code></a> or juniper</code></a> When building DiveDB</a>, I opted to use GraphQL due to its great frontend tooling & standardised introspection. In future blogs I hope to dive into some of the details of this decision. Implementing a Job queue with SQLx and Postgres Unknown — Fri, 26 Jun 2020 00:00:00 +0000 SQLx is a new async SQL Toolkit</a> for rust that is closer to standard SQL than a more opinionated ORM like Diesel</a>. I wanted to give it a bit of a test run and see how easy it would be to convert usage from tokio-postgres</a>. So as the next saga in the jobq series (part 1</a>, part 2</a>), we will be converting jobq</a> crate to use sqlx</code>. You can find the SQLx branch here: https://github.com/cetra3/jobq/tree/sqlx</a> As a little spoiler: I found it quite easy to adjust and a pleasure to use but found the documentation a little lacking. Object Relational Mappers (ORMs)</h2> ORMs definitely serve a purpose. ORMs provide an opinionated way to manage the database schema and craft queries in an easy way. Normally you construct queries using builders, or other language constructs, and get given the results back straight in your language types (structs in the case of rust). Migration is a lot easier as most offer a managed way to run migrations for you. You don't need to know SQL either, and can simply run functions in your language to build up queries to the database. Writing raw SQL in some ORMs is a second class citizen, which is where I have a problem. I personally find the cognitive load of having to wrangle the builder pattern to pull out SQL is not worth it. I am quite comfortable writing SQL and have done for many years, and so I have steered away from using highly opinionated ORMs in the past. In the world of Java I much prefer MyBatis</a> or Jdbi</a> over Hibernate</a>. In Rust, I am using standard tokio-postgres</a> over diesel</a> for the same reasons (that: and diesel async support is not there yet and doesn't look like it will be</a> anytime soon...). Now if you are not familiar with SQL this is definitely a different story, and may find you have a different opinion on how productive you are. That's fine too, as this is one reason ORMs exist: to make it easier to code. I would still recommend spending time grokking SQL, as it is quite a great skill to have in your toolbox. Diesel ORM</h3> Picking on diesel</a> for a second: here is their Complex Queries example: let versions = Version::belonging_to(krate) .select(id) .order(num.desc()) .limit(5); let downloads = version_downloads .filter(date.gt(now - 90.days())) .filter(version_id.eq(any(versions))) .order(date) .load::<Download>(&conn)?;</code></pre> Here is the same query in SQL: SELECT version_downloads.* WHERE date > (NOW() - '90 days') AND version_id = ANY( SELECT id FROM versions WHERE crate_id = 1 ORDER BY num DESC LIMIT 5 ) ORDER BY date</code></pre> Which one do you find more readable? I find the SQL easier to read. SQLx ORM</h3> I still consider SQLx</code> an ORM, but with SQL as a first-class citizen. Here's how you'd do the same with SQLx: let query = "SELECT version_downloads.* WHERE date > (NOW() - '90 days') AND version_id = ANY( SELECT id FROM versions WHERE crate_id = $1 ORDER BY num DESC LIMIT 5 ) ORDER BY date"; sqlx::query_as(query) .bind(&num) .fetch_all(&*self.pool);</code></pre> If your Version</code> struct derived sqlx::FromRow</code>, then this would all work nicely. Converting to SQLx</h2> Converting to SQLx was a mostly painless experience, and only very minimal changes</a> were needed for the jobq code base. In the end, only the structs & db files were needed to be adjusted, but I could reuse all of the query strings and sql building logic. As I am not starting with SQLx, some of the touted features I didn't use (like compile time verification</a>). There were also a few gotchas I ran into, but did not find it hard to work through them after a bit of reading. Migrations and Raw SQL</h3> I haven't implemented migrations properly in jobq: it's just a simple sql script</a> that is run at startup. SQLx does have utilities for this</a>, and is also quite strong on compile time verification</a> to make this more type safe. However: I just want to run the script at startup. It's simple & I can do it from standard postgres cli, so I should be able to it simply from rust as well. The sqlx::query</code> method will convert the SQL to a prepared statement, which does not play nice with such a query: sqlx::query(include_str!("setup.sql")).execute(&pool).await?;</code></pre> Here's the error you may get: cannot insert multiple commands into a prepared statement</code></pre> Instead, you can use the Executor</code> trait</a> which is implemented for &PgPool</code>. This does make it a little awkward to write having to have the extra brackets: (&pool).execute(include_str!("setup.sql")).await?;</code></pre>Mapping a postgresql query to rust structs</h3> sqlx has a query_as</code> method which allows you to convert returned result rows into rust structs, a bit like serde is used to serialize/deserialize data. This works by deriving sqlx::FromRow</code></a> on your structs: #[derive(Serialize, Deserialize, Debug, Clone, sqlx::FromRow)] pub struct Job { pub id: i64, pub username: String, pub name: String, pub uuid: Uuid, pub params: Value, pub priority: Priority, pub status: Status, }</code></pre> As I am using custom enums</a>, I also need to derive sqlx::Type</code></a> on them: #[derive(Serialize, Deserialize, Debug, Clone, sqlx::Type)] pub enum Status { Queued, Processing, Completed, Failed, }</code></pre> (Side note: custom enums may not be a hot idea for maintainability</a>) With that I can do away with the old get_jobs</code></a> method, as this is handled for me now by SQLx. I can then use the fetch_all</code></a> method to return is as a Vec</code>: pub(crate) async fn get_processing_jobs(&self) -> Result<Vec<Job>, Error> { let query = "select id, name, username, uuid, params, priority, status from jobq where status = 'Processing' order by priority asc, time asc"; Ok(sqlx::query_as(query).fetch_all(&*self.pool).await?) }</code></pre> Worthy of note: the fetch</code></a> method returns a futures Stream</code> to iterate through, making a much tighter integration with async Binding Parameters</h3> I didn't need to change SQL statements at all, but did need to adjust some of the binding parameters using the bind</code> builder pattern: pub(crate) async fn complete_job(&self, id: i64) -> Result<(), Error> { let query = "update jobq set status = 'Completed', duration = extract(epoch from now() - \"time\") where id = $1"; sqlx::query(query).bind(&id).execute(&*self.pool).await?; Ok(()) }</code></pre> This is different to tokio-postgres</code> accepting an array of parameters. I have ran into issues before when trying to pass dynamic arguments and trying to wrangle life times. Here is how I have worked around this in the past (not exactly pretty): for row in self.pool.get()?.query( &*query, params .iter() .map(|val| &**val as &(dyn ToSql + Sync)) .collect::<Vec<_>>() .as_slice(), )?</code></pre>Conclusions</h2> As you can see from the diff of the commits</a>, the conversion to use SQLx was an easy excercise. I have found SQLx quite a bit more lightweight and more to my liking than some of the heavier ORMs I've dealt with in the past. Being async first means that this will be a great fit for the ecosystem: I can see it becoming the defacto async story for databases in the future. One area which may need some improvement is the documentation & examples. As it is a comparitively young library with the first commit a year ago</a>, I assume that this will only get better with time. The README</a> and api docs</a> were enough for me to get into trouble. If you are looking to talk to a database with async rust, make sure you consider SQLx! Drawing SVG Graphs with Rust Unknown — Fri, 22 May 2020 00:00:00 +0000 I have just released the first version of lorikeet-dash</a> and part of that exercise was to find out a way to draw SVG graphs using rust. I thought my findings & approach may be useful for others looking to do the same thing and so I have documented the design evolution in this blog. Here's what the end result looks like in the lorikeet-dash</code></a> project: Defining the Graph</h2> First we want to define what a graph is. A minimal example of what a graph would have is: A List of Points</li> A Name/Description</li> A Colour for the line</li> </ul> So, in a rust struct, this would look like the following: #[derive(Clone, Debug)] pub struct Graph { pub name: String, pub points: Vec<Point>, pub colour: String } #[derive(Clone, Debug, Copy)] pub struct Point { pub x: f64, pub y: f64, }</code></pre> Let's add some initial helper methods, one to create a new graph, and another to add points to it: impl Graph { pub fn new(name: String, colour: String) -> Self { Graph { name, points: Vec::new(), colour, } } pub fn add_point(&mut self, x: f64, y: f64) { self.points.push(Point { x, y }); } }</code></pre>SVG Template</h2> An SVG image is an XML file, which means that we can use standard templating engines to generate SVG with some parameters. I am going to use tera</code></a> in this example, as lorikeet</code></a> already uses it. You could, however, use any template engine. Let's start with a simple SVG template, of which just simply draws the graph name: <?xml version="1.0" standalone="no"?> <svg width="100%" height="100%" viewBox="0 0 {{height + padding * 2}} {{width + padding * 2}}" preserveAspectRatio="xMidYMid meet" xmlns="http://www.w3.org/2000/svg" > <text x="{{width/2 + padding}}" y="{{padding / 2}}" font-family="-apple-system, system-ui, BlinkMacSystemFont, Roboto" dominant-baseline="middle" text-anchor="middle" font-size="18" fill="#74838f" font-weight="700" > {{name}} </text> </svg></code></pre> Let's visit some decisions I have made here: We are including a width</code> & height</code> for the viewbox. Having the SVG display at its natural resolution increases the clarity of the line, so we'll allow them to be adjusted when generating an SVG.</li> There is a padding</code> value to include around the graph so that axis labels and the name of the graph will fit on the SVG. To make things simple this can be hardset to a value such as 50</code>. We'll subtract that x 2 from the provided width/height.</li> We're using a big font-family</code> list which should display roughly similar fonts depending on your OS.</li> </ul> With our tera template svg ready, we can use the include_str</code> macro to have this compiled in: Tera::one_off(include_str!("graph.svg"), &context, true).expect("Could not draw graph")</code></pre>Drawing the Name</h3> We'll add a draw_svg(&self, width: usize, height: usize)</code> method to the Graph</code> struct which should start by drawing the name as per the above template. The arguments are simply the width & height to draw. To ensure that the viewbox of the svg comes out the right size, we'll subtract the padding from both x 2. To put it another way, the width</code> and height</code> become the width and height of the graph itself rather than the SVG. pub fn draw_svg(&self, width: usize, height: usize) -> String { let mut context = Context::new(); //hardset the padding around the graph let padding = 50; //ensure the viewbox is as per input let width = width - padding * 2; let height = height - padding * 2; context.insert("name", &self.name); context.insert("width", &width); context.insert("height", &height); context.insert("padding", &padding); Tera::one_off(include_str!("graph.svg"), &context, true).expect("Could not draw graph") }</code></pre> The Context</code> here is what tera uses to provide variables to a template. Anything that implements serde::Serialize</code> can be put into the context. At this moment we're providing a few strings and width/height parameters. This can be run to spit out an SVG: fn main() { let graph = Graph::new("Example".into(), "#8ff0a4".into()); println!("{}", graph.draw_svg(800, 400)); }</code></pre> Running this, should output the following SVG: Drawing the Points</h2> Let's fill up the graph with some dummy points so we have something to draw (note: we are assuming you'll add the points in order here): graph.add_point(1.0, 1.0); graph.add_point(2.0, 3.0); graph.add_point(3.0, 2.5); graph.add_point(4.0, 6.0); graph.add_point(5.0, 3.0);</code></pre>Scaling the Axis</h3> To keep things simple, we'll start from the origin & scale both the x and y axis by the max value of either. We'll pull out both of those values and use these to scale the points within the svg: let max_x = self .points .iter() .map(|point| point.x) .fold(0. / 0., f64::max); let max_y = self .points .iter() .map(|point| point.y) .fold(0. / 0., f64::max);</code></pre>Drawing a Line</h3> To draw a line between points in an SVG, you can use the path</code> element with draw commands</a>. A simple line would look like: <path d="M10 10 L15 15"/></code></pre> That is: move to point 10,10 and then draw a line to point 15,15. These points are in the SVG space, so we will need to translate our graph points onto this space, keeping in mind padding, width and height and scaling. Translating the points</h3> We want to draw a line between each of the points in the SVG but scaled to fit within the viewport. To add another complication, the chart origin is in the bottom left, whereas the SVG origin is the top left. For both the x and y axis, we want to scale down based upon the max value (width</code> and height</code> here are the chart width): x = x / max_x * width y = y / max_y * height</code></pre> For the x axis, we need to shift them along the right based upon padding, otherwise they will sit on the left side of the svg: x = x / max_x * width + padding</code></pre> For the y axis, we want to flip this based upon the height, to translate the origin, then add height + padding</code>: y = y / max_y * (height * -1.0) + height + padding</code></pre>Iterating through</h3> We have our points commands, now let's iterate through and convert them to a bunch of draw commands. First we will use M</code> to move to start drawing, then we will use L</code> to draw a line between each. We can use enumerate()</code> on the iterator to get whether this is the first or subsequent value, and then join it together as a big string: let path = self .points .iter() .map(|val| Point { x: (val.x / max_x * width as f64) + padding as f64, y: (val.y / max_y * (height as f64 * -1.0)) + (padding + height) as f64, }) .enumerate() .map(|(i, point)| { if i == 0 { format!("M {} {}", point.x, point.y) } else { format!("L {} {}", point.x, point.y) } }) .collect::<Vec<String>>().join(" ");</code></pre> This will output a path string like so: M 190 383.33333333333337 L 330 250 L 470 283.3333333333333 L 610 50 L 750 250</code></pre>Adding to the SVG</h3> We can add the path & colour to the Context: context.insert("path", &path); context.insert("colour", &self.colour);</code></pre> Then add the following to have it drawn in the SVG: <path stroke="{{colour}}" stroke-linejoin="round" d="{{path}}" stroke-width="2.0" fill="none" /></code></pre> If all goes well, the graph should start looking more like a graph: Adding in the Axis</h2> We need to add in some axis scale so we know what the graph values represent. I'm gonna cheat here & break up both the x and y axis the same amount of times each. We'll break up the axis 5 times each, but have this set in the context to be changes later: context.insert("lines", &5);</code></pre> Tera allows us to do a loop within the template, so we'll use that to add some lines and nudges. {% for i in range(end=(lines + 1)) %} ... draw some lines here {% endfor %}</code></pre> Let's set some variables within the template, based upon the current line we're drawing. These are the offsets for both the axis: {% set offset_x = padding + loop.index0/lines * width%} {% set offset_y = padding + loop.index0/lines * height%}</code></pre>Horizontal Lines</h3> We will draw 5 horizontal dashed lines, to give scale to the y value. This will start on the left side of the graph and continue to the end of the graph. We'll draw a solid horizontal line when y = 0</code> (keep in mind the axis is flipped in SVG), which means we skip drawing this for the last line: {% if loop.last == false %} <path stroke="#74838f" stroke-dasharray="10 6" stroke-width="0.5" d="M {{padding}} {{offset_y}} L {{width + padding}} {{offset_y}}" /> {% else %} <path stroke="#74838f" stroke-width="2" fill="none" d="M {{padding}} {{offset_y}} L {{width + padding}} {{offset_y}}" /> {% endif %}</code></pre> Now our graph will be a bit easier to read: Vertical Lines</h3> We will just put little 10px nudges in the axis rather than dotted vertical lines. In the existing for loop, we can add the following: <path stroke="#74838f" stroke-width="2.0" d="M {{offset_x}} {{height + padding}} L {{offset_x}} {{height + padding + 10}}" /></code></pre> This will create nudges where we'll put our axis labels: Axis Labels</h3> We can add in axis labels in the same for loop. For the y-axis, the loop starts at the top of the graph and goes down, so we will use the following formula to present the 1/5th value: (lines - loop.index0)/lines * max_y</code></pre> We can use the tera round</code> helper to round the value up, so there is not a lot of decimal places: {{((lines - loop.index0)/lines * max_y) | round}}</code></pre> Putting this together: <text x="{{padding - 5}}" font-family="-apple-system, system-ui, BlinkMacSystemFont, Roboto" y="{{offset_y}}" dominant-baseline="middle" text-anchor="end" font-size="12" fill="#74838f" font-weight="bold" > {{((lines - loop.index0)/lines * max_y) | round}} </text></code></pre> And quite similarly, the x-axis can be generated the same way (width y</code> being fixed to where the nudge starts & x</code> being the offset): <text x="{{offset_x}}" font-family="-apple-system, system-ui, BlinkMacSystemFont, Roboto" y="{{height + padding + 10}}" dominant-baseline="hanging" text-anchor="middle" font-size="12" fill="#74838f" font-weight="bold" > {{loop.index0/lines * max_x | round}} </text></code></pre> Putting this all together, we should have our labels in the right place: Bonus: Smooth Lines</h3> One trick I have found</a> is that you can make simple smoth lines by treating the points as a Catmull-Rom spline</a>, and then converting to Cubic Beziers</a>, which SVG will draw happily. Using this method, you can make your charts nice and smooth: Second Bonus: Scuba Dive Graph</h3> I have also used a similar method to plot a dive, displaying the Depth and Air on a split axis, with my maximum depth displayed: Conclusion</h2> We used tera, as a template engine, to generate an SVG chart with rust. Some basic geometry was needed, but the results show for themselves. I hope you have found this educational and gives you some ideas. Also: try out lorikeet-dash</a> and let me know what you think! Implementing a Job queue with Tokio-Serde Unknown — Wed, 13 May 2020 00:00:00 +0000 In the last blog</a> of this series, I implemented job queue with tmq</a>. I noted back then that tmq is great if you need to interact with other languages, but may be a little overkill if you are just using rust. I wondered what it'd take to build the job queue with a smaller library footprint, using something like tokio-serde</a> instead of tmq</a>. It was successful, and this blog will step through some of the changes needed. To follow along, these changes are on the tokio_serde</code> branch of the jobq repo</a>. If you are getting lost make sure you review the last blog</a> to see some of the logic & design tradeoffs. What is Tokio-Serde</h2> Tokio-Serde is a little glue library that allows you to serialise/deserialise objects on the wire. With the help of tokio-util's codec feature</a>, you can use Tokio-Serde to encode/decode bytes into discrete messages to be passed on the wire. From the Tokio-Serde documentation</a> using tokio-serde is 3 layers deep: tokio::net::TcpStream</code>: The raw Stream/Sink where we can get/send bytes</li> tokio_util::codec::Framed</code>: The ability to chunk bytes into discrete frames</li> tokio_serde::Framed</code>: The ability to take these frames & then serialize/deserialize them</li> </ul> The TCP Stream</h3> The TCP stream is your standard tcp connection. In tokio, you can either bind</code> or connect</code> to a TCP socket. With connect</code>, you will get a straight tcp stream, if successful: let tcp_stream = TcpStream::connect("127.0.0.1:17653").await?;</code></pre> With bind</code> you will get a listener, which will produce tcp streams when new connections come in: let mut listener = TcpListener::bind("127.0.0.1:17653").await?; while let Some(tcp_stream) = listener.try_next().await? { //do something with the tcp stream }</code></pre>The Frame</h3> Ok so we have bytes coming across the wire now from our standard TCP stream. As bytes come in, the tcp stream will be woken up so work can commence on it. But TCP streams do not have a beginning or end, they are just a stream of bytes. We need something to break up this stream of bytes into discrete objects. This is where tokio_util::codec::Framed</code> comes in: it chunks down bytes on the wire into frames. The tokio_util</code> crate comes with the Encoder</a>/Decoder</a> traits which allow you to implement this. tokio-util</code> also comes with a simple, Length Delimited Codec</a> which, on the wire looks like this: +----------+--------------------------------+ | length | payload | +----------+--------------------------------+</code></pre> When decoding, this will read the length header, allocate some memory and wait until the payload is fully delivered. If there are not enough bytes on the wire to do this, then it will be continuously polled until it is ready, keeping around a buffer. The semantics of Decoder</code> mean that if you return no bytes, then it's an indication there needs to be more to decode a frame successfully. When encoding, the whole thing is ready to go, so tack on the length, your payload and let tokio handle the rest. The Serde Frame</h3> We now have some raw bytes that have been bundled into frames which can be encoded/decoded from the raw bytes into something more meaningful. Rather than implement this by hand, we can rely on tokio-serde</code> with one of the existing codecs, such as JSON, CBOR, Bincode or Messagepack. Our existing job queue uses CBOR so we'll use that here as well. We can use the same messages we have previously used</a> but we will condense down the Client/Worker to just ClientMessage</code></a>: #[derive(Serialize, Deserialize, Debug)] pub enum ServerMessage { Hello(String), // Hello with client name Request(JobRequest), Completed(Job), Failed(Job, String), } #[derive(Serialize, Deserialize, Debug)] pub enum ClientMessage { Hello(String), Order(Job), Acknowledged(Job), }</code></pre> Putting these 3 layers together, we get a Stream</code>/Sink</code> we can send values to: let tcp_stream = TcpStream::connect(addr).await?; let length_delimited = tokio_util::codec::Framed::new(tcp_stream, LengthDelimitedCodec::new()); let connection = tokio_serde::Framed::new(length_delimited, Cbor::default());</code></pre> With that we have our transport that we want: length delimited cbor frames. On the wire this is not too different to ZeroMQ, but would probably shave a few bytes here and there. Now we need to implement the Server/Client components. Implementing a Server</h2> We want to accept connections from clients/workers and then route out messages to the appropriate connection. We will leave the server logic itself mostly untouched, and instead replace the tmq::router</code> with a new Router</code></a> struct which uses tokio-serde</code> The Router</h3> The Router</code></a> struct handles incoming client connections and allows our server logic to send messages based upon their name. We have multiple clients/workers that will be connecting to our jobq server so we will need to keep around some state to handle those connections. How will we do that? Well, a simple HashMap<String, Connection></code> is probably a good start. When a connection comes in, we want to add it to our map of known clients. If a connection drops off (i.e, returns Poll::Ready(None)</code> from the Stream</code>) then we can clean it from our known connections. If a new client with the same name comes in, we can just drop off the old client connection (alternatively, we could reject the new one). These semantics are obviously different from the ZeroMQ Router socket</a>, and not nearly as robust, but are good enough for a PoC. The router can be implemented as a Stream</code> which, when polled will check: Any pending tcp connections</li> Any pending clients that are in the middle of handshaking</li> Any pending messages from existing clients</li> </ul> This can be seen as a loop, whereby each time we hit poll_next</code> we will hit all 3 areas in case we have anything waiting for us. If there is nothing in one area, then we move onto the next one, and so on. We can do an if let Poll::Ready()</code> pattern here to skip things that aren't ready. Pending Clients</h4> How do we know the client name when connecting? Well we can implement a list of pending clients. When they send a ServerMessage::Hello</code> message, we can then put them in our known clients list. This is a pretty lightweight handshake, but accomplishes our goals. First, we convert the tcp stream from listener into a list of pending clients: let self_mut = &mut self.as_mut(); if let Poll::Ready(val) = Pin::new(&mut self_mut.listener).poll_next(cx) { match val { Some(Ok(tcp_stream)) => { let length_delimited = CodecFramed::new(tcp_stream, LengthDelimitedCodec::new()); let framed = Framed::new(length_delimited, Cbor::default()); self_mut.pending_clients.push(framed); } Some(Err(err)) => { error!("Error checking for new requests:{:?}", err); } None => { return Poll::Ready(None); } } }</code></pre> Now, with that checked off, we can then consult our pending clients. Note here I am swapping out the pending clients Vec</code> with an empty Vec</code> to avoid doubly borrowing as mutable. There may be an easier/more efficient way of doing this and would be interested in knowing what that may look like. We will repopulate the list as we go through, so we won't lose any pending clients, but one or two may drop off if they are not doing the handshake correctly or they have already disconnected. We'll also add the Hello</code> message to our messages to send as a Stream</code> response. That's our buffer</code> that we will be using. let mut new_pending = Vec::new(); mem::swap(&mut self_mut.pending_clients, &mut new_pending); for mut pending in new_pending.into_iter() { if let Poll::Ready(val) = Pin::new(&mut pending).poll_next(cx) { match val { Some(Ok(ServerMessage::Hello(name))) => { debug!("New Client connection from `{}`", name); self_mut.buffer.push(ServerMessage::Hello(name.clone())); self_mut.clients.insert(name, pending); } Some(Ok(msg)) => { warn!("Received unknown message during handshake:{:?}", msg); } Some(Err(err)) => { error!("Error checking for new requests:{:?}", err); } None => (), } } else { self_mut.pending_clients.push(pending); } }</code></pre>Existing Clients</h4> We'll do the same trick for the client HashMap</code>: swap it out for an empty one as we're processing. We'll put all received messages into an internal buffer, which we will use later to send the actual response from the stream. let mut new_clients = HashMap::new(); mem::swap(&mut self_mut.clients, &mut new_clients); for (name, mut client) in new_clients.into_iter() { match Pin::new(&mut client).poll_next(cx) { Poll::Ready(Some(Ok(val))) => { trace!("Received message from `{}`: {:?}", name, val); self_mut.buffer.push(val); self_mut.clients.insert(name, client); } Poll::Ready(None) => { //Finished debug!("Client `{}` disconnecting", name); } Poll::Ready(Some(Err(err))) => { //Error error!("Error from `{}`: {} Removing connection.", name, err); } Poll::Pending => { self_mut.clients.insert(name, client); } } }</code></pre>Returning Messages</h4> We've buffered up the returned messages into a Vec</code> and we can return one. But if there are more after popping we can let the waker know we may have more stuff to process: if let Some(val) = self_mut.buffer.pop() { if self_mut.buffer.len() > 0 { cx.waker().wake_by_ref(); } return Poll::Ready(Some(Ok(val))); } return Poll::Pending;</code></pre>Sending Messages</h3> Sending messages requires that you know the name of the client, but it is otherwise a simple wrapper: pub async fn send_message(&mut self, client: &str, msg: ClientMessage) -> Result<(), Error> { if let Some(connection) = self.clients.get_mut(client) { connection.send(msg).await?; } else { return Err(anyhow!("Client `{}` not connected!", client)); } Ok(()) }</code></pre> We can even expose a way to check if any clients are connected of that name, and simply not bother sending a message in this case: pub fn is_connected(&self, client: &str) -> bool { self.clients.contains_key(client) }</code></pre>Changes to the Server</h3> The Server</code> struct itself does not really change all that much. Instead of using a tmq::router</code> we'll use our new Router</code>. Although strictly not equivalent, for the purpose of the job queue, this is an OK analogue: let mut router = Router::new(&self.job_address).await?; while let Some(server_msg) = router.next().await { // existing server logic here }</code></pre>Implementing Clients</h2> Not much has changed here as well. We could get away with not using our own struct definition here at all and just set up a connection, but would be great if we had something we could plop in where the old tmq::dealer</code> was used. Dealer</h3> We implement a Dealer</code> struct which simply wraps a tokio_serde::Framed</code>: type ClientFramed = Framed< CodecFramed<TcpStream, LengthDelimitedCodec>, ClientMessage, ServerMessage, Cbor<ClientMessage, ServerMessage>, >; pub struct Dealer { connection: ClientFramed, } impl Dealer { pub async fn new<A: ToSocketAddrs>(addr: A) -> Result<Self, Error> { let tcp_stream = TcpStream::connect(addr).await?; let length_delimited = CodecFramed::new(tcp_stream, LengthDelimitedCodec::new()); let connection = Framed::new(length_delimited, Cbor::default()); Ok(Self { connection }) } }</code></pre> With the dealer, both receiving and sending marry up with Stream</code> and Sink</code> quite well, so we can implement that quite easily: impl Stream for Dealer { type Item = Result<ClientMessage, Error>; fn poll_next(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Option<Self::Item>> { let self_mut = &mut self.as_mut(); match Pin::new(&mut self_mut.connection).poll_next(cx) { Poll::Ready(Some(val)) => Poll::Ready(Some(val.map_err(|err| err.into()))), Poll::Ready(None) => Poll::Ready(None), Poll::Pending => Poll::Pending, } } }</code></pre>Worker Changes</h3> The worker doesn't change much, we call split</a> on the Dealer</code> struct to get our two pipes, but otherwise treat it as per normal: let (mut send_skt, recv) = Dealer::new(&job_address).await?.split();</code></pre>Running the Changes</h2> Running the branch we should see a lot of the same chatter we saw from the previous version</a>: 2020-05-13 11:13:02 DEBUG jobq::server > New: Job { id: 5500, username: "test_client", name: "test", uuid: 95182793-7a08-4329-a9f1-4be0c1f1d041, params: Null, priority: Normal, status: Queued } 2020-05-13 11:13:02 WARN jobq::server > Job failed: 5496, Reason: Simulating failure 2020-05-13 11:13:02 WARN jobq::server > Job failed: 5484, Reason: Simulating failure 2020-05-13 11:13:02 WARN jobq::server > Job failed: 5472, Reason: Simulating failure 2020-05-13 11:13:03 WARN jobq::server > Job failed: 5460, Reason: Simulating failure</code></pre>Conclusion</h2> Implementing a more barebones messaging system native in rust took not much time at all. While this solution is not nearly as robust as using ZeroMQ, it does provide a more building blocks approach to messaging, reducing external dependencies. I found the hardest portion was gluing everything together between the 3 main crates and getting the types right. Tokio-Serde could use a bit more fleshing out and does seem quite embryonic still, but it is enough to get your hands dirty. The initial design of the jobq made it easy to slot in another transport layer as well. All the structs & types were all ready and rolling to go, we just needed a couple of extra libraries to fill in the pieces. Lorikeet 0.11.0 - Upgrading to async Unknown — Tue, 21 Apr 2020 00:00:00 +0000 I have just spent some time doing an initial async version of lorikeet</a> now that the async/await syntax is stable and the ecosystem has caught up. The major blocker was reqwest, as this is used extensively in the http</code> test. This async version is available now as version 0.11.0</code>. You can also install the cli by running cargo install lorikeet</code>. What is Lorikeet</h2> Lorikeet is a command line tool and a rust library to run tests for smoke testing and integration testing. Lorikeet currently supports bash commands and simple HTTP requests along with system information (RAM, Disk & CPU). There is more information on the github readme</a> about how to write test files, including how to structure dependent tests and make assertions on the output. As a simple example, here's a test plan to check to see whether reddit is up, and then tries to login if it is: check_reddit: http: https://www.reddit.com regex: the front page of the internet login_to_reddit: http: url: https://www.reddit.com/api/login/{{user}} form: user: {{user}} passwd: {{pass}} api_type: json jmespath: length(json.errors) matches: 0 require: - check_reddit</code></pre> You can run this as a simple cli command, ensuring that you have a config.yml</code> looks something like: user: test pass: test</code></pre> Running it you will see the results: $ lorikeet -c config.yml test.yml - name: check_reddit pass: true output: the front page of the internet duration: 1416.591ms - name: login_to_reddit pass: true output: 0 duration: 1089.0276ms</code></pre>Skipping futures 0.1</h2> In retrospect, I am glad that I skipped out on rewriting lorikeet with the classic combinator style futures. The migration from standard blocking code to async code is much easier than writing from combinators. Having done the former with mpart-async</a>, it felt quite easy to sprinkle in a few async</code> and await</code> statements to get things all wired up. As an example, waiting for the request</a> used to be: let mut response = client .execute(request.build().map_err(|err| format!("{:?}", err))?) .map_err(|err| format!("Error connecting to url {}", err))?;</code></pre> Now, with async, things are not much different: let response = client .execute(request.build().map_err(|err| format!("{:?}", err))?) .await .map_err(|err| format!("Error connecting to url {}", err))?;</code></pre>Using tokio sync primitives</h2> Tokio has a few primitives that are very close to their std counterparts. Lorikeet now uses a tokio mutex</a> to record the outcome of each of the steps: steps.lock().await[index] = Status::Completed(outcome);</code></pre> Also using an unbounded channel</a> so each step can inform the main runner when they are finished: let (tx, mut rx) = unbounded_channel(); // later on.... if let Some(finished_idx) = rx.recv().await { ... }</code></pre> Interesting to note that the UnboundedSender</code> send()</code> method is not async and does not block either. Multipart File Uploads</h2> The new version of reqwest does not support file uploads</a> easily. You don't want to buffer the entire file in memory. Luckily I have had some experience on how to wire this up with mpart-async</a>, and can reuse some of the learnings there. This does require the tokio-util</a> crate, which bridges AsyncRead</code>/AsyncWrite</code> with Sink</code>/Stream</code>. The relevent glue section is as follows</a>: let file_name = path_struct .file .file_name() .map(|val| val.to_string_lossy().to_string()) .unwrap_or_default(); let file = File::open(&path_struct.file) .await .map_err(|err| format!("{:?}", err))?; let reader = Body::wrap_stream(FramedRead::new(file, BytesCodec::new())); form.part(key, Part::stream(reader).file_name(file_name)</code></pre>Detecting Blocking Code</h2> There is no annotation or warning whenever you are using blocking code in an async context. This is an issue especially when porting over older code which was previously blocking. Chances are I have missed a few sections that are blocking and will need to revisit them accordingly. Luckily, the fail case for blocking code is usually reduced throughput, down to the number of worker threads, so it is not cataclysmic if there are some blocking sections in your async code. Conclusion</h2> I was happy with the ease at which it was to convert existing sections to async by sprinkling a few keywords here and there. I am concerned that it is too easy to have blocking sections, so I hope there is some tooling/linting around that in the future. Please give lorikeet a go and try out some of the newer features from the readme</a>! Implementing a Job queue with Tokio, PostgreSQL & ZeroMQ Unknown — Thu, 09 Apr 2020 00:00:00 +0000 One of the challenges I have had with on premise solutions is the lack of reliable environments and constrained resources. Not only are you limited in your ability to control things, you need to ensure that things continue on in the case of failure. I was tasked with rewriting the job processing pipeline for my company's product, SchoolBench</a>, to ensure a greater level of robustness in the case of service or system failure. This article steps through an approach to this using async rust & the help of tokio ZeroMQ library: tmq</a> along with tokio-postgres</a>. As some excellent work has been put in bringing tmq to async, I thought it pertinent to look at reapproaching the pipeline using async code. I have published an initial prototype</a> of what the end result may look like, and will be stepping through the approach here, and some of the tradeoffs I made Why TMQ/ZeroMQ?</h2> For those not familiar with ZeroMQ</a>, it is a messaging library with a lot of features, from standard publish/subscribe to more elaborate arrangements. tmq</a> is a tokio</a> compatible binding that allows you to bridge ZeroMQ with the async world of rust. You would use ZeroMQ if you need to interact with other languages easily and don't want it to be too opinionated on what message formats are being sent. If you are purely using rust then this may be not be a great fit, but it does still provide you some great interprocess & internetwork capabilities that you won't get from just tokio. We use ZeroMQ in SchoolBench as it is a polyglot application and has a lot of components written in python & java as well as rust. It is lightweight enough not to cause overheads & complex enough for the use cases I have thrown at it so far. Prior Work</h2> There are some existing rust based job queues that do similar things and may be more suited to your own personal tastes: https://github.com/kureuil/batch-rs</a></li> https://github.com/badboy/oppgave</a></li> https://github.com/rusty-celery/rusty-celery</a></li> Probably lots more I haven't found</li> </ul> Most of them use some sort of message broker, such as RabbitMQ</a> or Redis</a>. What is a Job?</h2> It helps to explain what a job is. In the context of SchoolBench a job is a piece of work that may take some time to do (think seconds), is quite intensive, and may fail. Once an asset is saved into SchoolBench, lots of processes are kicked off in the background to fill in the blanks and provide information about the asset. Some examples of jobs are: Running Facial Recognition</a> on images using tensorflow</li> Generating Thumbnails and Watermarked renditions of images</li> Calculating whether a photo is a duplicate</a> of another photo and marking the set of duplicates</li> </ul> A lot of these are quite intensive & can sometimes fail in weird and wonderful ways: An image uploaded may be corrupted, so renditions may not run; There may be an issue running the neural net; The VM may be experience ballooning and fail to allocate memory, etc... So we need a way of describing a discrete piece of work which may happen some time in the future and may happen by a completely isolated process. Resource Contention</h3> We have very limited resources in an on premise environment. After providing a minimum recommendation it is important to tune for that worst case. This is different in a cloud environment where you can spin up guests in a work stealing fashion if you throw more money at your infrastructure. on premise is usually more fixed in terms of server and hardware allocations, and so we need to plan for that. This is one area where the job queue deviates from a lot of existing systems out there: we assume we're just running on the one set of CPUs and so there is a fixed number of active jobs at any given time. We also want a simple way to prioritise tasks as they are submitted, so that some jobs finish first, with higher value jobs such as thumbnail generation happening first. The job queue uses a simple 3-tier priority system to do this: High, Medium & Low. This simple system could mean if there are not enough resources to finish jobs then there is a chance that lower tier jobs may never get around to completing. In practice the sort of work loads we have seen this is not the case, but it is a possibility. In which case more processors would be needed to keep up with the load or another style of priority to be implemented (such as fair queueing). Persistence</h3> Jobs need to survive crashes and server restarts in a robust fashion. Existing job queues utilise an existing persistence layer/message broker such as Redis</a> or RabbitMQ</a>. I have chosen PostgreSQL as the persistence layer as it is performant enough, and is already in use for metadata storage. However, you could easily swap out another type of persistence with a bit of refactoring. The Overall Process</h2> The high level proces is as follows: A User Action triggers some sort of logic which requests a job. This could be an automatic rule somewhere or could be the a user specifically requesting a job to run</li> A server listens for requests and puts them into a queue</li> If there are jobs that can be processed then they are submitted to the individual worker who is responsible for finishing the job</li> When a job completes or fails, then it is marked as so with a duration on how long things take</li> Everytime a new event is triggered, such as a job in a queue or a job completed/failed, the server checks to see if it can mark jobs as processing</li> All job status updates & information is persisted to a db, so that things can pick up again if there is a failure & reports can be ran using standard SQL tools</li> </ul> Client, Server, Worker</h3> We split the roles of the job queue into 3 different types: Clients that submit requests to the server</li> The Server which does the main job queue loop</li> The Workers that receive jobs and do work, returning whether the work was completed (not the work outcome itself, just the status; we assume that the worker will update something out of band like another db or filesystem)</li> </ul> One possible thing to note is that there is only ever one server, but there can be as many workers and clients as allowed. The workers themselves can also be clients and submit requests if they want to. Communication</h3> The server, worker and clients all communicate via a ZeroMQ DEALER/ROUTER</code> socket. Clients and Workers use the DEALER</code> style socket to retrieve & send messages, but also identify themselves, and the server uses ROUTER</code>. The identity is included as the first ZeroMQ multipart frame when it is received and so can be used to route messages back to the appropriate place. The messages are serialised on the wire as CBOR</a> using serde_cbor</a>. CBOR was chosen because it allows for the message type to be flexible & there are plenty of implementations in other languages. JSON could be used as well, as arguably the support for JSON is much higher, but would mean larger message sizes. With serde it's pretty easy to adjust what serialisation is used, so some experimentation could be worthwhile. The Job Structs</h2> The Job</code> struct</a> is defined as follows (with the Status</code> & Priority</code> enums listed too): pub struct Job { pub id: i64, // ID of the Job so it can be tracked pub username: String, // Username of who submitted the job pub name: String, // Name of the job/worker, `rendition`, etc.. pub uuid: Uuid, // Unique ID of the asset pub params: Value,// Any parameters of the job in question pub priority: Priority, // The priority of the job pub status: Status,// The status of the job }</code></pre> The job has a current status which starts with Queued</code>, goes to Processing</code> when it is active and then marked as either Completed</code> or Failed</code>: pub enum Status { Queued, Processing, Completed, Failed, }</code></pre> The priority is a simple 3-tier system of High</code>, Normal</code> & Low</code>: pub enum Priority { High, Normal, Low, }</code></pre>Message types</h3> Messages are split up based upon who the destination is with a shared Hello</code> message that can be serialised as any enum: #[derive(Serialize, Deserialize, Debug)] pub enum ServerMessage { Hello, Request(JobRequest), Completed(Job), Failed(Job, String), } #[derive(Serialize, Deserialize, Debug)] pub enum ClientMessage { Hello, Acknowledged(Job), } #[derive(Serialize, Deserialize, Debug)] pub enum WorkerMessage { Hello, Order(Job), }</code></pre> The JobRequest</code> is very similar to a Job</code> but does not have an allocated id or status yet: pub struct JobRequest { pub name: String, pub username: String, pub uuid: Uuid, pub params: Value, pub priority: Priority, }</code></pre>The Params Value</h3> One thing to note is the params</code> is of the type serde_json::Value</code>. This is really to allow the greatest flexibility into what parameters are sent with each job & also to have it persist to PostgreSQL as a jsonb</code> column. Alternatively, a tighter enum</code> could be used for different job names if you knew ahead of time the jobs that will be requested. Serialisation helpers</h3> To reduce some of the repetitiveness, there is a serialisation helper which simply serialises to CBOR</code> for any struct that implements Serialize</code>: pub trait ToMpart { fn to_mpart(&self) -> Result<Multipart, Error>; fn to_msg(&self) -> Result<Message, Error>; } impl<T: serde::ser::Serialize> ToMpart for T { fn to_mpart(&self) -> Result<Multipart, Error> { let bytes = serde_cbor::to_vec(&self)?; Ok(Multipart::from(vec![&bytes])) } fn to_msg(&self) -> Result<Message, Error> { let bytes = serde_cbor::to_vec(&self)?; Ok(Message::from(&bytes)) } }</code></pre>Client Requests</h2> Clients can send requests by constructing job requests and sending them to the server. They just need to construct the Request</code> enum and send it on the wire: let (mut send, mut recv) = dealer(&Context::new()) .set_identity(b"test_client") .connect(&config.job_address)? .split::<Multipart>(); let job = JobRequest { name: "test".into(), username: "test_client".into(), params: Value::Null, uuid: Uuid::new_v4(), priority: PriorityLow, }; send.send(ServerMessage::Request(job).to_mpart()?).await?;</code></pre>Worker Processing</h2> For workers doing the work, they should do something & then return whether it's failed or completed. To make this easier, a Worker</code> trait is implemented. Since some work will be async & some not, we want our trait to be async-capable. At the writing of this article you can't have async</code> in trait definitions, but there is the async_trait</a> crate that allows you to decorate a trait definition and do what we're after. For errors we're going to use the anyhow</a> crate, but a more generic error approach could be used with a bit of a refactor. #[async_trait] pub trait Worker: Sized { const JOB_NAME: &'static str; async fn process(&self, job: Job) -> Result<(), Error> }</code></pre> The implementation of this trait means that you can do lots within the process</code> body and simplifies it. An original version had the process</code> method accept &mut self</code>, but that caused some contention when trying to process jobs concurrently. Instead you'll need some interior mutability if you need to provide any mutable references to &self</code>. As an example</a>, the test worker will simply wait for 100 milliseconds and then fail every 12th job based upon the id: impl Worker for TestWorker { const JOB_NAME: &'static str = "test"; async fn process(&self, job: Job) -> Result<(), Error> { delay_for(Duration::from_millis(100)).await; if job.id % 12 == 0 { return Err(anyhow!("Simulating failure")); } Ok(()) } }</code></pre>The Work Method</h3> There is another method, work</code>, on the trait</a> which sets up & hides some of the complexity away of getting a worker to listen for requests coming through. This is plumbing to make it easy to run a Worker</code>. async fn work(&self, job_address: &str) -> Result<(), Error> { ... } </code></pre> You can await</code> this method for anything that implements Worker</code> to have it process incoming requests: use jobq::worker::{Worker, TestWorker}; tokio::spawn(async move { if let Err(err) = TestWorker.work(&worker_config.job_address).await { error!("{}", err); } });</code></pre> Under the hood it uses the job address to communicate with a Server and glues up some requests Firstly, it creates a dealer socket</a>, setting the identity accordingly: let (mut send_skt, recv) = dealer(&Context::new()) .set_identity(job_name.as_bytes()) .connect(&job_address)? .split::<Multipart>();</code></pre> As ZeroMQ sockets are Send</code> but not Sync</code>, we spin up a background task connected by an unbounded</code> queue which allows the sender to be cloned. This allows both heartbeats to be sent every 10 seconds from a background task & also send results of processing: let (send, mut recv_skt) = unbounded::<ServerMessage>(); tokio::spawn(async move { while let Some(jobq_message) = recv_skt.next().await { if let Ok(msg) = jobq_message.to_mpart() { if let Err(err) = send_skt.send(msg).await { error!("Error sending message:{}", err); } } } });</code></pre> The 10 second heartbeat</a> is similiarly set up as a separate task: tokio::spawn(async move { loop { if let Err(err) = ping_sender.send(ServerMessage::Hello).await { error!("Error:{}", err); }; delay_for(Duration::from_millis(10000)).await; } });</code></pre> There is then a big combinator statement</a> which in effect listens for job requests & then runs them in parallel: recv.filter_map(|val| { match val .map_err(Error::from) .and_then(|msg| serde_cbor::from_slice(&msg[0]).map_err(Error::from)) { Ok(WorkerMessage::Order(job)) => return ready(Some(job)), Ok(WorkerMessage::Hello) => { debug!("Pong: {}", job_type); } Err(err) => { error!("Error decoding message:{}", err); } } return ready(None); }) .map(|job| (self.process(job.clone()), send.clone(), job)) .for_each_concurrent(None, |(status, mut send, job)| async move { let server_message = match status.await { Ok(()) => ServerMessage::Completed(job), Err(err) => ServerMessage::Failed(job, err.to_string()), }; if let Err(err) = send.send(server_message).await { error!("Error sending server message: {}", err); } }) .await;</code></pre> There is a bit of cloning here that could probably be avoided, but the overhead of cloning a job is not that big. The for_each_concurrent</code></a> allows the worker to run concurrently, and is one of the reasons the trait takes &self</code> rather than &mut self</code>. All of this is essentially generic on the Worker</code> trait, meaning this plumbing is handled for you when you call work(&address)</code>. Blocking Tasks</h3> Some tasks may be blocking, which is a no-no in the async world. If one of the async worker threads is blocked then it can't listen for more events & will slow things down. Luckily tokio does provide the spawn_blocking</code></a> helper for executing blocking work on a dedicated threadpool: let result = tokio::task::spawn_blocking(move || do_blocking_work(job)); result.await??;</code></pre> The double .await??</code> is because the JoinHandle</code> returned from spawn_blocking</code> itself could fail and the internal result from the closure could fail. To control how many threads are used for blocking operations, you need to manually build your tokio run time</a>, rather than use the tokio::main</code> macro. The the number of blocking threads is max_threads - core_threads</code>. For instance, if you wanted only 2 dedicated blocking threads: let mut rt = tokio::runtime::Builder::new() .threaded_scheduler() .enable_all() .core_threads(4) .max_threads(6) .build()?</code></pre>The Server Loop</h2> The server loop</a> is the main loop responsible for coordinating tasks & otherwise tasks are finished in the order they are supposed to. The server here responds to messages from the ZeroMQ socket & acts according to the message received. Setup Phase</h3> The setup phase</a> involves setting up a db handle, then querying the database for any outstanding processing jobs. This is to ensure that they don't become stuck as part of a server restart, but can mean that there may be duplicate jobs submitted. If your jobs provide the same outcome based upon the job at hand this isn't a big deal, so the workers should do work that is idempotent if at all possible. Once that is done, the server keeps an active count of jobs & increments/decrements them when jobs finish/fail or start. Main Loop</h3> The main loop</a> simply waits until a message is received, and acts accordingly: while let Some(msg) = recv.try_next().await? { // This is the `ROUTER` socket identity let client_name = &msg[0]; // This is the message let server_msg = serde_cbor::from_slice::<ServerMessage>(&msg[1]); match server_msg { ... } }</code></pre>Hello Message</h4> If a Hello</code> message is received then the server will send any outstanding processing jobs it knows about: //Drain out existing processing jobs let (jobs, outstanding): (Vec<Job>, Vec<Job>) = processing.into_iter().partition(|job| job.name == name); processing = outstanding; for job in jobs { send_job(&handle, job, &mut send).await?; }</code></pre> The processing</code> vec is only ever populated at server start</a>, and so should mostly be empty unless the server was restarted with jobs that were active. After checking all this it responds back with a Hello</code> Job Request</h4> If a Request</code> message comes in, the server adds this to the db</a> & then sends an Acknowledged</code> message back to the client</a> with the populated details. This doesn't yet start the job however, merely it sets the job in a Queued</code> state. Completed</h4> If a Completed</code> message comes in then the server marks it as completed</a> decrementing the active job count. Failed</h4> If a Failed</code> message comes in the server will mark the job as failed</a> and submit the reason with it, decrementing the active job count Error Deserialising</h4> There is a possibility that a connected client could send an invalid message, so we just log with a warn</code> level in this case, skipping the message: match server_msg { .... Err(err) => { warn!("Could not deserialize message:{}", err); } }</code></pre>Submit Tasks</h3> After the main loop matching against the message types, the server checks to see if there are any active slots & if so, queries the database for any queued jobs: // If we have less active tasks lets check the queued stuff if active < self.num { let jobs = handle .get_queued_jobs(self.num as i64 - active as i64) .await?; for job in jobs { send_job(&handle, job, &mut send).await?; active = active + 1; } }</code></pre> After this processing, the server then waits again for the next message. Database Persistence</h2> One of the requirements is ensuring that jobs stay around and are persisted somewhere. The prototype uses PostgreSQL</a> to do this. As this is async, we'll use tokio-postgres</a> to perform updates. tokio postgres has some nice docs</a> to get us started, so we will use their example to build a database handler</a>: use tokio_postgres::{NoTls, Error}; #[tokio::main] // By default, tokio_postgres uses the tokio crate as its runtime. async fn main() -> Result<(), Error> { // Connect to the database. let (client, connection) = tokio_postgres::connect("host=localhost user=postgres", NoTls).await?; // The connection object performs the actual communication with the database, // so spawn it off to run on its own. tokio::spawn(async move { if let Err(e) = connection.await { eprintln!("connection error: {}", e); } }); // Now we can execute a simple statement that just returns its parameter. let rows = client .query("SELECT $1::TEXT", &[&"hello world"]) .await?; // And then check that we got back the same string we sent over. let value: &str = rows[0].get(0); assert_eq!(value, "hello world"); Ok(()) }</code></pre>Enums and Postgres Types</h3> You can use the postgres_types</code></a> crate to save simple enums to a table. So the Priority</code> struct can be persisted to the db like so: CREATE TYPE "Priority" as enum ('High', 'Normal', 'Low');</code></pre>A Simple Migration</h3> Hopefully our schema never changes.. Right? Well, in the prototype it won't. So we can simply embed an SQL script</a> to run when the a db handler starts up</a>: pub struct DbHandle { client: Arc<Client>, } impl DbHandle { pub(crate) async fn new(url: &str) -> Result<Self, Error> { let (client, connection) = tokio_postgres::connect(&url, NoTls).await?; tokio::spawn(async move { if let Err(e) = connection.await { error!("connection error: {}", e); } }); client.batch_execute(include_str!("setup.sql")).await?; Ok(DbHandle { client: Arc::new(client), }) } ... }</code></pre> This setup.sql</code> script should be designed to run more than once, but skip the parts that it's already done. There are a couple tricks to do this, but obviously doesn't work if you are adding/removing columns from a table etc.. For the enum types, you can just check the pg_type</code> to make sure that they exist & if not add them: IF NOT EXISTS (SELECT 1 FROM pg_type WHERE typname = 'Status') THEN CREATE TYPE "Status" as enum ('Queued', 'Processing', 'Completed', 'Failed'); END IF</code></pre> With the table & indexes you can do a CREATE <blah> IF NOT EXISTS</code>: CREATE INDEX IF NOT EXISTS status_idx ON jobq ( status );</code></pre> This won't error out if the index/table already exists, meaning you're safe to run the setup.sql</code> multiple times! Getting jobs from the DB</h3> Querying for jobs is quite simple, you just select the columns you want and do a get()</code> or try_get()</code> on them: impl DbHandle { .... pub(crate) async fn get_queued_jobs(&self, num: i64) -> Result<Vec<Job>, Error> { let query = "select id, name, username, uuid, params, priority, status from jobq where status = 'Queued' order by priority asc, time asc limit $1"; let mut jobs = Vec::new(); for row in result { let id = row.try_get(0)?; let name = row.try_get(1)?; let username = row.try_get(2)?; let uuid = row.try_get(3)?; let params = row.try_get(4)?; let priority = row.try_get(5)?; let status = row.try_get(6)?; jobs.push({ Job { id, username, name, uuid, params, priority, status, } }); } Ok(jobs) } ...</code></pre> There are a number of other methods as complete_job</a>, fail_job</a> that handle the SQL stuff & make it simple for the Server to use. Keeping things persisted in the database means we can use standard SQL tools to introspect the job queue and see where it's at. Recording the job duration</h3> The duration of the job is recorded when the job is finished/failed</a>: pub(crate) async fn complete_job(&self, id: i64) -> Result<(), Error> { let query = "update jobq set status = 'Completed', duration = extract(epoch from now() - \"time\") where id = $1"; self.client.query(query, &[&id]).await?; Ok(()) }</code></pre> This could've been done inside rust as well using Instant</code> but would require a bit more state to be handled in the Server. I've elected here to use a PostgreSQL solution instead to lean on the db. Running the example</h2> The example starts the test worker, a server and a client to submit 500 simple jobs: for i in 0..500 { let priority = if i % 2 == 0 { Priority::High } else { Priority::Normal }; let job = JobRequest { name: "test".into(), username: "test_client".into(), params: Value::Null, uuid: Uuid::new_v4(), priority, }; send.send(ServerMessage::Request(job).to_mpart()?).await?; }</code></pre> Running with cargo run</code> you will see the output after some time, with the simulated failures: 2020-04-14 16:04:33 DEBUG jobq > Message:Acknowledged(Job { id: 2999, username: "test_client", name: "test", uuid: fe6e21fc-8064-4c9a-965d-b1d124f2416d, params: Null, priority: High, status: Queued }) 2020-04-14 16:04:33 DEBUG jobq::server > New: Job { id: 3000, username: "test_client", name: "test", uuid: d20b4e9b-477b-45ca-9a6a-1f6423378779, params: Null, priority: Normal, status: Queued } 2020-04-14 16:04:33 DEBUG jobq > Message:Acknowledged(Job { id: 3000, username: "test_client", name: "test", uuid: d20b4e9b-477b-45ca-9a6a-1f6423378779, params: Null, priority: Normal, status: Queued }) 2020-04-14 16:04:39 WARN jobq::server > Job failed: 3000, Reason: Simulating failure 2020-04-14 16:04:39 WARN jobq::server > Job failed: 2988, Reason: Simulating failure 2020-04-14 16:04:39 WARN jobq::server > Job failed: 2976, Reason: Simulating failure</code></pre> You can also run some simple stats directly against the db to see how long things have taken (hopefully ~100ms): select avg(duration) from jobq; avg ------------ 0.10634147 (1 row)</code></pre>Conclusions</h2> I hope for anyone new that this article will help dip their feet into the waters of rust and async to see what can be built. Taking a few libraries and gluing them together we can get a pretty decent, albeit rough, outline of a background job queue. The new async/await</code> ecosystem while embryonic already has the tools there to build some cool stuff. There will be a separate article around the release of tmq</code> version 0.2.0</code> and some of the changes that have been made, but thought it might be great to provide a good example of working code before getting aroudn to that. Handling Breaking API Changes Unknown — Mon, 03 Feb 2020 00:00:00 +0000 While upgrading dependencies (basically deleting Cargo.lock</code>) in a big rust project I hit an issue. A new commit</a> in serde_json</code> caused has upstream failure in another library, jmespath</a>, and possibly more crates. Because these structs and enums visibility has changed, this could be interpreted as a breaking change, depending on what you consider public. While jmespath</code> was doing the wrong thing here, using undocumented API, it got me thinking about what can library maintainers and library consumers do to ensure semver compatibility, and what tools are out there to assist both groups. What is Semantic Versioning</h2> Semantic versioning (semver) is one standard used for version numbers which appears to have gained the most traction. Since cargo and friends use semver to identify what versions to use, it is important to understand (excuse my pun) the semantics of what this means. The full documentation of semver is on https://semver.org/</a>, but for brevity there are 3 numbers: Major: Any breaking API changes</li> Minor: Additions of new features or functionality that are backwards compatible</li> Patch: Bugfixes</li> </ul> There is some interesting differences for libraries < 1.0.0</code> which I was not aware of for longer than I care to admit. Namely either minor or patch numbers can be changes as you see fit. I.e, 0.3.0</code> could be wildly different from 0.3.1</code>, including breaking changes. However: Cargo is more strict</a> than the semver documentation for < 1.0.0</code> libraries: This compatibility convention is different from SemVer in the way it treats versions before 1.0.0. While SemVer says there is no compatibility before 1.0.0, Cargo considers 0.x.y</code> to be compatible with 0.x.z</code>, where y ≥ z</code> and x > 0</code>. </blockquote> Why we need it</h2> We need some way of portraying changes to an API and, while it's not perfect, it appears to be a defacto standard which makes interoperability with other libraries and crates a lot easier. It also gives cargo a programmatic way of finding updated versions, when it's used appropriately. So it is one part advertising the intent of the version and another part ensuring that things will compile if you bump up certain numbers. API Evolution RFC</h2> The Semver website is very general in its language and isn't specific to rust. There is, however, a great RFC that has some excellent guidelines both for the language itself & for library maintainers: RFC 1105: API Evolution</a> Here are a couple of excerpts: Major change: going from stable to nightly</h4> Changing a crate from working on stable Rust to requiring a nightly is considered a breaking change. That includes using #[feature]</code> directly, or using a dependency that does so. Crate authors should consider using Cargo "features"</a> for their crate to make such use opt-in. </blockquote> Major change: renaming/moving/removing any public items.</h4> Although renaming an item might seem like a minor change, according to the general policy design this is not a permitted form of breakage: it's not possible to annotate code in advance to avoid the breakage, nor is it possible to prevent the breakage from affecting dependencies. </blockquote> This RFC has some great advice, and I wish it was more publicly known, but seems relatively obscure. The Non-Exhaustive Attribute</h2> Another RFC related to versioning is Non-Exhaustive</a>, which is a handy way of planning breaking changes ahead of time. It works with enums by forcing open ended match</code> statements, which means consumers of your enum won't break if a new variant is introducted. It also works with structs and ensures users consuming libraries can't create them. This was recently stabilised in Rust 1.40</a> so it's a rather new feature of the language (on a side note: it's not easy to find when a particular RFC is stabilised, it would be great to have this reflected in the RFC somewhere). As an example if you have an enum like this: #[non_exhaustive] pub enum Error { Message(String), Other, }</code></pre> You would need to match</code> against the enum like this: match error { Message(ref s) => ..., Other => ..., _ => ..., }</code></pre> Your library could then introduce a new variant, which wouldn't break the existing consumer: #[non_exhaustive] pub enum Error { Message(String), IOError, Other, }</code></pre>Other Tools</h2> There have been some tools to both manage breaking changes in the past, a lot of the things I found were unmaintained or not in active use. There might also be some great language features that I haven't discovered yet that crate authors are already using. The Semver Trick</h3> Documented here is the semver trick</a>, which touts to be able to assist in this with downstream dependencies. There is mention of a libcpocalypse</code>, which does not sound nice and may happen again if there is a major version increase</a> to libc</code>. I tried to find some information about where this had happened before but searching for the term circles back to this project and doesn't give a great overview of the pains users had to go through before. Rust Semverver</h3> rust-semverver</a> is a Google Summer of Code Project which checks semver compliance with rust crates by using a novel approach: The approach taken is to compile both versions of the crate to rlibs and to link them as dependencies of a third, empty, dummy crate. Then, a custom compiler driver is run on the said dummy and all necessary analysis is performed in that context, where type information and other resources are available. </blockquote> Semantic-RS</h3> An existing tool called semantic-rs</a> touts to be able to do this for your crates automatically. This doesn't appear to have any changes for over a year, but does have an active issue register, but no PRs have been accepted for a while. It is highly opinionated as well, and may not be appropriate to bolt on to an existing library. This looks like something you need to use from the beginning as well. Rust Breaking Changes</h3> This also doesn't appear to be maintained, but was an automatic listing of breaking changes to rust: https://killercup.github.io/bitrust/</a>. I do wonder though how many of those changes are internal, as I don't believe there have been many external breaking changes. Future Proofing</h3> There is a chapter of the Rust API Guidelines</a> which deals with how to future proof an API, but as with other tools I have found does not appear to be updated in quite a while</a>: Future Proofing: https://rust-lang.github.io/api-guidelines/future-proofing.html</a> Conclusion</h2> While there has been a historic push to get crates to 1.0.0</a>, There does not appear to be much in the way of modern tooling in order to help with this and the focus of previous efforts see them moving elsewhere. With more libraries reaching 1.0.0</code> I think it's important to introduce more safety nets for library authors and consumers. As an idea, we can potentially see whether a crate has breaking change through lints etc.. so why don't we do that when we publish to crates.io? There is a lot of room for improvement here, and as new users come on board and new companies look at adopting, it's important that the stability of semver is maintained. I still think the way that cargo handles this is much better than other ecosystems. Lessons learnt updating a library to std::future Unknown — Wed, 22 Jan 2020 00:00:00 +0000 With the new std::future</code> way of doing things and tokio</a> slowly reaching maturation, it's time to look at updating the libraries out there that are using the old ways. For one of my libraries, tmq</a>, a Tokio ZeroMQ library, there is some awesome work already done</a> to get this updated. But, I thought it pertinent to at least get my feet in the water to see how hard it would be, from a library maintainer perspective, to update to std::future</code>. For this effort, I chose my small library: mpart-async</a>. You can see the changes I have made by comparing the versions</a> here. This blog is a small collection of notes & gotches I found when porting code across. mpart-async</code> is a mostly agnostic library for client side multipart/form-data</code> requests. There are existing libraries out there, like multipart</a>, but I found the API a little unwieldy for my taste (That: and async support is still in alpha</a> according to the readme). I wanted something that worked, and was simple, but didn't offer an opinion on the web client. I use both the actix client</a> & hyper</a> to make multipart requests depending on the project. The challenge is that with multipart requests you mostly have fields & binary files. For binary files, you can appreciate, should not be buffered entirely in memory, but streamed out as the bytes become available. So mpart-async</code> works with multiple internal streams of files and also provides a convenience wrapper</a> (if using tokio) for sending files given a path. A New Example</h2> All these changes to support async fn</code> and is it actually easier to consume/use async libraries? Given the hyper</a> example, I would say yes. Compare the new example</a>: #[tokio::main] async fn main() -> Result<(), Error> { //Setup a mock server to accept connections. //.... let client = Client::new(); let mut mpart = MultipartRequest::default(); mpart.add_field("foo", "bar"); mpart.add_file("test", "Cargo.toml"); let request = Request::post("http://localhost:3000") .header( CONTENT_TYPE, format!("multipart/form-data; boundary={}", mpart.get_boundary()), ) .body(Body::wrap_stream(mpart))?; client.request(request).await?; Ok(()) }</code></pre> With the older one</a>: fn main() { // current_thread::Runtime can't be used because of the blocking file operations let mut rt = Runtime::new().expect("new rt"); //Setup a mock server to accept connections. //.... // Open `Cargo.toml` file and create a request let request = File::open("Cargo.toml") .map_err(|e| format!("{}", e)) .and_then(|file| { // A Stream of BytesMut decoded from an AsyncRead let framed = FramedRead::new(file, BytesCodec::new()); let mut mpart = MultipartRequest::default(); mpart.add_field("foo", "bar"); mpart.add_stream( "foofile", "Cargo.toml", "application/toml", framed.map(|b| b.freeze()), ); Request::post("http://localhost:3000") .header( CONTENT_TYPE, format!("multipart/form-data; boundary={}", mpart.get_boundary()), ).body(Body::wrap_stream(mpart)) .into_future() .map_err(|e| format!("{}", e)) }); // Send request let task = request.and_then(|request| { let client = hyper::Client::new(); client .request(request) .map_err(|e| format!("{}", e)) .and_then(|response| { response .into_body() .concat2() .map_err(|e| format!("{}", e)) .and_then(|body| { if let Ok(data) = str::from_utf8(&body) { println!("Response: {}", data); } Ok(()) }) }) }); rt.block_on(task).expect("request failed"); }</code></pre> While there are still a few warts (such as having to manually add the boundary header), I find the code a lot more readable and easy to follow, rather than your standard combinator paths. No more Error</code> Associated Type</h2> The most drastic change to the Stream</code> trait is that there is no longer an Error</code> associated type. The old Stream</code>/Future</code> traits had both Item</code> and Error</code>, as it assumed that streams were always going to be fallible. The new traits do away with the Error</code> associated type. Instead, if you want your Stream</code> to possibly be an error then you need to return a Result</code> as your Item</code> type. For the Stream</code> trait, the method you implement has changed to poll_next</code> and uses the std::task::Poll</code> enum as a return type. The Poll</code> enum did feel a little inside out when I started using it, but makes sense in terms of there being no Error</code> type. You don't return a Result<Async<Option>, _>...</code> you instead return a Poll<Option<Result<_,_>>></code>. Generally, this means if you wrote this: return Ok(Async::Ready(Some(bytes)))</code></pre> You instead will return: return Poll::Ready(Some(Ok(bytes)))</code></pre>StreamExt</code> traits and friends</h2> As a consequence of no Error</code> associated type, and an example of different ergonomics, if you are dealing with Result</code> items, you may want to use the TryStreamExt</a> trait instead of StreamExt</a>. They are basically the same trait but one deals with Result</code> a bit more nicely. I'm not sure whether a simpler solution will be possible with higher kinded types in the future, but for now it is a tiny bit more complex than futures 0.1. Interestingly, the index page</a> for stream docs doesn't currently mention that the TryStreamExt</code> trait exists & I only found out about it by asking questions in the discord chat</a>. But wait, there is also StreamExt</a> from tokio which is subtly different, but does as of 0.2.10</code> allow you to run collect()</code> on Result<Bytes, _></code> item streams. When would you use either StreamExt</code>? I would say that if you want to be more general, you should probably use the futures</code> implementation. Bearing in mind you can't run try_collect()</code></a> on Result<Bytes, _></code> at the moment due to an outstanding issue</a> with the bytes crate. Why does this matter? Well, hyper</a> passes around streams of Result<Bytes, _></code> when you are streaming a body in & out of a request. Working with this particular stream signature is a bit clunky still, but I'm sure that this will be resolved in due course. So we have: StreamExt</code> from futures</code> that works on Stream</code> that aren't Result</code></li> TryStreamExt</code> from futures</code> that works on Stream</code> that are Result</code> but doesn't work well with Result<Bytes,_></code> when trying to try_collect()</code></li> StreamExt</code> from tokio</code> that works with Result<Bytes,_></code> and allows you to collect()</code> to get the output but now means you have a dependency on tokio</code></li> </ul> This might end up confusing more than just me, and I do hope that it's simplified in the future. Luckily in the core of the API I don't need to use them, only for tests & filestream, which means I'm quite happy to depend on the tokio</code> implementation. Wrapping inner streams in Pin</code></h2> I got tripped up on this for a while, and needed to again ask in chat for the answer here. If you have an inner stream and you want to call poll_next</code> on it, you need to wrap it in a Pin</code>, otherwise the Stream</code> trait does not stick and you will get some gnarly error message with some weird suggestions: The my original shot at the code was: stream.stream.poll_next(cx)</code></pre> But it needs to be: Pin::new(&mut stream.stream).poll_next(cx)</code></pre> The compiler error message is not too helpful here, and insists I restrict S</code> by Stream</code>, even though it is already restricted by Stream</code>. error[E0599]: no method named `poll_next` found for type `S` in the current scope --> src/lib.rs:282:41 | 282 | match stream.stream.poll_next(cx) { | ^^^^^^^^^ method not found in `S` | = help: items from traits can only be used if the type parameter is bounded by the trait help: the following trait defines an item `poll_next`, perhaps you need to restrict type parameter `S` with it: | 254 | impl<E, S: futures::Stream + Stream> Stream for MultipartRequest<S> | ^^^^^^^^^^^^^^^^^^^^</code></pre> You need to make sure that your inner stream implements Unpin</code> as well to go down this path: impl<E, S> Stream for MultipartRequest<S> where S: Stream<Item = Result<Bytes, E>> + Unpin,</code></pre>Storing the results from an async fn</code> in a struct</h2> Some of the methods in tokio are returns from an async fn</code> , like the File::open</a> method, which I use for FileStream</code>. I found the answer on stack overflow</a> as to how to do this with std::future</code>, since async fn</code> returns an opaque type. For the older version, the OpenFuture was a concrete type: pub struct FileStream { inner: Option<FramedRead<File, BytesCodec>>, file: OpenFuture<PathBuf>, }</code></pre> The newer version is a return from an async fn</code> and is opaque, so we wrap it using Box::pin</code>: pub struct FileStream { inner: Option<FramedRead<File, BytesCodec>>, file: Pin<Box<dyn Future<Output = Result<File, Error>> + Send + Sync>>, }</code></pre> You can then instantiate this with: Box::pin(File::open(file.into()))</code></pre> Then it's easy to call passing on the context from an existing poll()</code>: self.file.as_mut().poll(cx)</code></pre>Notifying a task</h2> FileStream</code> has two stages. One is the future to open the file. The second is streaming out the bytes of the file. When the future resolves to open the file, I want to notify the context that it should be polled again to start streaming. The old way of doing this was task::current().notify()</code>: self.inner = Some(FramedRead::new(file, BytesCodec::new())); task::current().notify();</code></pre> The new way appears to be using cx.waker().wake_by_ref()</code> where cx</code> is the context received from the poll: self.inner = Some(FramedRead::new(file, BytesCodec::new())); cx.waker().wake_by_ref();</code></pre> I say appears because the test written works without wake_by_ref()</code> being called. Requires a bit more investigation here I think to know exactly what's going on. My simple test example</a> works either way, strangely. Conclusion</h2> It is not too onerous to convert to std::future</code> for an existing library. I would assume the leap from the old tokio-core</code> would be harder, as the changes feel mostly cosmetic here. The omission of the Error</code> associated type to me actually makes things less ergonomic and things a little more fragmented (as evidenced by 3 *StreamExt</code> traits..). I was an advocate of this initially, but there probably needs to be a bit more work in making this nicer. There are still a lot of libraries out there that will be required to be updated, a lot of old blogs that are no longer relevant, and a lot of exploration that needs to be done to see how the async ecosystem falls out. But considering the friction of updating is quite small, I am quitely optimistic! My Wishes for Rust 2020 Unknown — Wed, 30 Oct 2019 00:00:00 +0000 2019 was another huge year for rust, but instead of implementing the shiny new thing, it's been a focus on refining existing ideas. Having being using rust professionally for over 3 years, I feel I will very soon be regailing new rust developers of the times before NLL & async, where we had to play lexicon-fu</a> to get things like caches to work in a reasonable fashion, or using super dooper combinator chains and wrestling with futures types (Either::A/B</code> anyone?). So I'd like to make clear my acknowledgement that I am happy with the direction that the rust language is heading in, and these requests are really nice to haves. The Future of Futures</h2> The async/await is on track to be stabilised next month, which means it's not a goal for 2020, but we have to look back to the futures-rs! There is a whole ecosystem that now needs to play catchup and I hope that it does within the year 2020. I also really hope they start using the logo a bit more! Tokio version 1.0</code></h3> I would like to see tokio version 1.0</code> in 2020. I would also like the core of tokio to be trimmed down immensely for inclusion as a dependency. Just a small wish I know! I understand the reluctance to brand it as 1.0</code> but there is already a lot of churn from the move from tokio-core</code> last year. This would also signal to users that it is mostly stabilised, since async/await is dropping soon. I haven't been following development closely to know if there are any further blockers beyond this though. Alternatives to tokio</h3> I'd also like to see more than one library for async reach wide use. I'd like to be able to mix and match executors with library code, and choose an implementation. The promise of async/await should allow for this in theory, but we'll have to see what it's like in practice. This will both add a bit of healthy competition to the ecosystem but will also prevent future design decisions from library bias. I know that async-std</a> is a great contender here and I hope that one day I could use this as a (mostly) drop in replacement for running my futures. This might already be possible, but I'd have no idea how to integrate this to my existing libraries (i.e, tmq</a> and mpart-async</a>) in an agnostic fashion. Non-Async stuff</h3> Some more care needs to be put into the non-async world. There was a great thread about an alternative to reqwest</a> which doesn't require the entire async stack to run. I would also like to see the blending of async and sync code made easier. Sometimes you want to use a db pool such as diesel, and currently that is a very messy thing to do. It would be great if that bridge was seamless and doesn't shoot you in the foot (i.e, make it a compile time guarantee to ensure no blocking code in async). Chat and Community Channels</h2> I really enjoyed the IRC chat room, but as Mozilla is decommissioning it</a>, it feels like there is now a massive empty hole where a good resource use to be. This isn't really something that the rust team themselves can do anything about, but I also don't think fragmenting team discussions across a variety of chat platforms sets a good precedent for the greater rust community. I don't have any direct qualms about using discord (which is currently recommended on the community page</a>) but it would be preferable if whatever was chosen was both accessible and open. I'd love it to be matrix</a>, but riot</a> is a bit of a clunky client, and there isn't really a better alternative client. Discord has much more polish. (As an aside the fediverse stuff is pretty cool! mastodon</a> and pixelfed</a> are great apps and I wish we saw more rusty stuff in this space) Reflecting on Last Year's Wishes</h2> My last blog</a> I had 3 main areas of improvement that I would like to have seen a focus on. Compiler Times, IDE Support and RLS</h3> This is still in flux but appears to be a step in the right direction. move over rls</a> and welcome rust-analyzer</a>. It seems the IDE story is a lot better, but still a long way off. Who knew it could be so hard! IntelliJ has made strides this year in making their IDE much better. Still though, every few months I load up some of my bigger rust projects, only to have some weirdness show up. I feel like this is very much an aspirational goal though, IDEs for all languages continue and improve to get better with time. It's not ever going to be the ideal of an IDE, but it's trending towards it. Async/Await</h3> The Async/Await story definitely improved with the imminent release (as of this blog) of async</code> syntax landing on stable. While there was a substantial amount of dogfooding around the syntax, I feel that the language team has settled on an appropriate solution. It looks a bit weird at first, but the promise of having match</code> or others use the same syntax style is intruiging. Documentation Churn</h3> This is still an area of concern, but has been addressed mostly in some of the frameworks I've seen. I feel like this won't be fixed until all the ecosystem has stabilised around new language features and we land a few more 1.0</code> crates. It's going to be a hassle redoing examples with async keywords! Conclusion</h2> I am happy with the way things have progressed throughout 2019 and I feel that 2020 is going to be a massively exciting time for rust. I don't feel there is a need for another epoch just yet, considering we're still working through the current one. Let's work towards polishing the async ecosystem for this year, and hopefully we'll have a very compelling story for everyone else to get involved. Face Detection with Actix Web Unknown — Thu, 11 Apr 2019 00:00:00 +0000 Last article I wrote about how to use tensorflow with rust</a>. This time we're going to take what we've built on, and serve it as an HTTP API call. As Actix Web</a> is nearing its inevitable 1.0 release</a>, I thought it would be a great time to build something with it. This article assumes you have some knowledge about Futures</a> and how they work. I will do my best to explain in simpler terms, but understanding the futures ecosystem would be quite handy to help work through this article. For this, I suggest you start with tokio</a>. (Some people have suggested waiting for async/await and friends to land before diving into Futures. I think you should get your hands dirty now: async programming will always be challenging and the changes being introduced only affect ergonomics, not fundamentals) Once again for the impatient, you can find the reference code here on the actix-web</code> branch: https://github.com/cetra3/mtcnn/tree/actix-web</a> API Shape</h2> The API shape here is rather simple. We want to emulate what we've done on the command line: Submit a picture, have a picture returned. To make things interesting, we'll provide a way to return the bounding boxes as a JSON array too. For submitting binary data via HTTP, there are a few options off the top of my head: Simply submit the raw data as it stands</li> Use multipart/form-data</code></li> Encode it as a JSON submission</li> </ul> I think the easiest would be just the raw data so let's do that! Multipart could possibly also work, but what about the case when you have to handle multiple images? JSON Submission seems a bit wasteful, as you would inevitably have to convert binary data using base64 or similar. So our API looks like this: Submit POST request as a raw file submission</li> Run a session against mtcnn to extract faces</li> Either return Bounding Boxes as JSON, or an Image Overlay as a JPEG like the command line example</li> </ul> MTCNN as a struct</h2> In our last blog we just simply used the main</code> function to perform everything, but it's not going to fly with actix until we do a bit of a refactor. We want to encapsulate the mtcnn behaviour as a struct, something that can be passed around and moved. The end goal is to use it in the application state. Struct Shape</h3> Let's make our struct include everything we need: The Graph</code></li> The Session</code></li> Some of the Tensor</code> input params that don't change from request to request.</li> </ul> We'll start by creating a new file mtcnn.rs</code> and adding in the struct definition. use tensorflow::{Graph, Session, Tensor}; pub struct Mtcnn { graph: Graph, session: Session, min_size: Tensor<f32>, thresholds: Tensor<f32>, factor: Tensor<f32> }</code></pre> Ok, now we're just going to plop in the initiation stuff in a new()</code> method. As the creation of some of these values is not infallible, we'll return a Result</code>: pub fn new() -> Result<Self, Box<dyn Error>> { let model = include_bytes!("mtcnn.pb"); let mut graph = Graph::new(); graph.import_graph_def(&*model, &ImportGraphDefOptions::new())?; let session = Session::new(&SessionOptions::new(), &graph)?; let min_size = Tensor::new(&[]).with_values(&[40f32])?; let thresholds = Tensor::new(&[3]).with_values(&[0.6f32, 0.7f32, 0.7f32])?; let factor = Tensor::new(&[]).with_values(&[0.709f32])?; Ok(Self { graph, session, min_size, thresholds, factor }) }</code></pre>The Run Function</h3> (I'm going to race along here to get to the good parts, so if you get stuck or are unsure about what's going on, have a look at the original article</a> for an explanation of what's happening here.) We've added all the bits we need to run our session Let's create a method that does what we're asking the API to do: submit an image, return some bounding boxes: pub fn run(&self, img: &DynamicImage) -> Result<Vec<BBoxes>, Status> { ... }</code></pre> Once again, we are returning a Result</code> as the type, as there are some parts of the run</code> that can fail. We're going to use the Status</code> type as that is normally the error type returned Like our previous main function, we'll need to flatten the image input: let input = { let mut flattened: Vec<f32> = Vec::new(); for (_x, _y, rgb) in img.pixels() { flattened.push(rgb[2] as f32); flattened.push(rgb[1] as f32); flattened.push(rgb[0] as f32); } Tensor::new(&[img.height() as u64, img.width() as u64, 3]) .with_values(&flattened)? };</code></pre> Then we'll feed in all the relevant inputs. This is just the same as our previous main</code> function, but we're going to just borrow values from self</code> rather than create them for each run: let mut args = SessionRunArgs::new(); args.add_feed( &self.graph.operation_by_name_required("min_size")?, 0, &self.min_size, ); args.add_feed( &self.graph.operation_by_name_required("thresholds")?, 0, &self.thresholds, ); args.add_feed( &self.graph.operation_by_name_required("factor")?, 0, &self.factor, ); args.add_feed(&self.graph.operation_by_name_required("input")?, 0, &input);</code></pre> Next, let's grab the outputs we want: let bbox = args.request_fetch(&self.graph.operation_by_name_required("box")?, 0); let prob = args.request_fetch(&self.graph.operation_by_name_required("prob")?, 0);</code></pre>Running The Session</h3> Now we have all our args set up, we can run the session: &self.session.run(&mut args)?;</code></pre> Uh oh. We're getting a compiler error here... error[E0596]: cannot borrow `self.session` as mutable, as it is behind a `&` reference --> src/mtcnn.rs:68:10 | 36 | pub fn run(&self, img: &DynamicImage) -> Result<DynamicImage, Box<dyn Error>> { | ----- help: consider changing this to be a mutable reference: `&mut self` ... 68 | &self.session.run(&mut args)?; | ^^^^^^^^^^^^ `self` is a `&` reference, so the data it refers to cannot be borrowed as mutable</code></pre> Turns out that the Session::run()</code> function takes &mut self</code>. What can we do to resolve this: Make our run</code> function take &mut self</code> so we can mutate the field</li> Do some tricky interior mutability</li> Submit an issue on the tensorflow-rust crate</a> to see whether Session</code> really needs to take &mut self</code>.</li> </ul> We'll go with option 3! Update your Cargo.toml</code> to point to the git master, rather than the cargo version: tensorflow = { git = "https://github.com/tensorflow/rust"}</code></pre>Getting the Bounding Boxes</h3> This has not changed at all since our main</code> method. We grab the bounding boxes, put them in our handy dandy BBox</code> struct: //Our bounding box extents let bbox_res: Tensor<f32> = args.fetch(bbox)?; //Our facial probability let prob_res: Tensor<f32> = args.fetch(prob)?; //Let's store the results as a Vec<BBox> let mut bboxes = Vec::new(); let mut i = 0; let mut j = 0; //While we have responses, iterate through while i < bbox_res.len() { //Add in the 4 floats from the `bbox_res` array. //Notice the y1, x1, etc.. is ordered differently to our struct definition. bboxes.push(BBox { y1: bbox_res[i], x1: bbox_res[i + 1], y2: bbox_res[i + 2], x2: bbox_res[i + 3], prob: prob_res[j], // Add in the facial probability }); //Step `i` ahead by 4. i += 4; //Step `i` ahead by 1. j += 1; } debug!("BBox Length: {}, BBoxes:{:#?}", bboxes.len(), bboxes); Ok(bboxes)</code></pre> And that's our run</code> function finished. BBox Struct as JSON</h3> We're gonna want to return a JSON representation of the BBox struct. So let's add in serde_derive</code>: use serde_derive::Serialize; #[derive(Copy, Clone, Debug, Serialize)] pub struct BBox { pub x1: f32, pub y1: f32, pub x2: f32, pub y2: f32, pub prob: f32, }</code></pre>Drawing the Output Image</h3> We'll create a function that will take a list of bounding boxes and an input image, returning the output image: pub fn overlay(img: &DynamicImage, bboxes: &Vec<BBox>) -> DynamicImage</code></pre> This hasn't changed much either, but we are returning the image, rather than saving it to a file: //Let's clone the input image let mut output_image = img.clone(); //Iterate through all bounding boxes for bbox in bboxes { //Create a `Rect` from the bounding box. let rect = Rect::at(bbox.x1 as i32, bbox.y1 as i32) .of_size((bbox.x2 - bbox.x1) as u32, (bbox.y2 - bbox.y1) as u32); //Draw a green line around the bounding box draw_hollow_rect_mut(&mut output_image, rect, LINE_COLOUR); } output_image</code></pre> Ok, we're pretty much done with our Mtcnn</code> struct and functions! Could we break this down even further? Yeah definitely! But for now, I think this is all we need. We've encapsulated the behaviour and created a nice to use couple of functions. Our New Main Function</h2> Ok, so we're no longer using it as a CLI, but as a self-hosted web app. We're going to need to change the arguments our application takes since we no longer have input and output files. I think the only argument we should be taking initially is the listen address, and even then we should use a sensible default. So let's use the help of structopt</a> to make this pretty minimal boilerplate: #[derive(StructOpt)] struct Opt { #[structopt( short = "l", long = "listen", help = "Listen Address", default_value = "127.0.0.1:8000" )] listen: String, }</code></pre>Logging Framework</h3> Actix Web uses the log</a> crate to display errors and debug messages. Let's move on from using println!</code> and instead use log. I like using pretty_env_logger</a> as it prints different levels as a different colour, and we can have timestamps which is useful. pretty_env_logger</code> still uses environment vars. Let's cheat and set our RUST_LOG</code> var if none is provided then initiate our logger //Set the `RUST_LOG` var if none is provided if env::var("RUST_LOG").is_err() { env::set_var("RUST_LOG", "mtcnn=DEBUG,actix_web=DEBUG"); } //Create a timestamped logger pretty_env_logger::init_timed();</code></pre> This sets up DEBUG</code> level logs for our app and actix web, but allows us to change the log levels via environment variables still. Actix and State</h2> We have some state we need to pass to actix to use: our Mtcnn</code> struct and the run method. There are a number of ways you can give State to actix, but probably the easiest way is the App::data</code> method. As we are now entering a multithreaded world, we're going to have to think about what things are Send</code> and Sync</code>. Ok, so how do we share something between threads? Well, as a first step I would look to std::sync</code> to see what we need. Since we know that mtcnn's run</code> function does not need to mutate it, only requiring a reference to immutable self</code>, we can probably wrap it in an Arc</code>. If we had to mutate it, then it would probably require a Mutex as well, but we can avoid this if we use the master</code> branch of tensorflow-rust</code> as above. So let's create our Arc: let mtcnn = Arc::new(Mtcnn::new()?);</code></pre> Now, to instantiate the server: HttpServer::new(move || { App::new() //Add in our mtcnn struct, we clone the reference for each worker thread .data(mtcnn.clone()) //Add in a logger to see the requests coming through .wrap(middleware::Logger::default()) // Add in some routes here .service( ... ) }) .bind(&opt.listen)? // Use the listener from the command arguments .run()</code></pre> Ok, let's step through what we're doing: We first build a HttpServer</code></li> This takes a closure which should return an App</code>. This App</code> is instantiated for each thread the http server is running</li> We add our Arc<Mtcnn></code> using the data</code> method, and clone it for each thread listener.</li> We add a logger middleware.</li> We set up some routes with the service</code> function</li> Then we bind to a listen address and run</li> </ul> Handling a Request</h2> Actix Web is an Async framework and uses tokio. Our function is Synchronous and takes some time to complete. In other words, our request is blocking. Can we mix and match sync and async? Absolutely, but it is a little more cumbersome as you'll see. Function Signature and Extractors</h3> Actix 1.0 makes heavy use of Extractors, which is a way of providing functions of quite different shapes. You basically specify what you want your web function to receive, and actix will wire it up for you. Take care though: This does mean that some things won't be discovered wrong until runtime. A perfect example I had when I used the wrong type signature for the web::Data</code> argument. So what do we need to extract from our request? The bytes of the request body, and and also our mtcnn</code>: fn handle_request( stream: web::Payload, mtcnn: web::Data<Arc<Mtcnn>>, ) -> impl Future<Item = HttpResponse, Error = ActixError> { ... }</code></pre> We will use this type signature for mtcnn</code> a fair bit, so let's make a type alias for it: type WebMtcnn = web::Data<Arc<Mtcnn>>;</code></pre>Getting the Image from a Payload</h2> Ok, so we need a way of retrieving the image from a payload and returning a Future. The web::Payload</code> struct implements Stream</a> with Item</code> set to Bytes</a>. Well, we don't really care about the individual bytes we get from a stream, we want the whole lot to decode the image! So let's convert the Stream</code> into a Future</code>, and merge all the individual bytes we'll be getting into one big bucket of bytes. Sounds complicated, but luckily Stream has a method: concat2</a>. This is a pretty powerful combinator which allows us to join the results of individual Stream polls into one if the item implements Extend</a> (and some other traits) which Bytes</code> happens to. So this looks like: stream.concat2().and_then(....)</code></pre>Image Decoding and web::block</code></h3> Ok, second thing we need to sort out, if we're going to be decoding an image, that is probably going to block the thread until it's done. How long? Well if it's a huge image, it might take milliseconds! So we want to ensure that we're not blocking the http workers while this is happening. Luckily, actix web has a way of taking a blocking portion of code, and wrapping that as a future. Enter web::block</code>. We'll do our decoding in a cpu pool, bridging sync and async together: stream.concat2().and_then(move |bytes| { web::block(move || { image::load_from_memory(&bytes) }) })</code></pre> Ok, that is pretty succint: we take a stream, convert it into a future and a bundle of bytes, then use web::block</code> to decode the bytes into an image in a background thread and return the result. the load_from_memory</code> function already returns a Result, which means we can just use that as the return type. Balancing the Error Type</h3> So, our Item is converted to Bytes</code> to DynamicImage</code>, but we still haven't dealt with the error types yet and it won't compile. What should our error type be? Let's use actix_web::Error</code> as ActixError</code>: use actix_web::{Error as ActixError} fn get_image(stream: web::Payload) -> impl Future<Item = DynamicImage, Error = ActixError> { stream.concat2().and_then(move |bytes| { web::block(move || { image::load_from_memory(&bytes) }) }) }</code></pre> Ok, that is giving us few really gnarly errors when we try and compile it: error[E0271]: type mismatch resolving `<impl futures::future::Future as futures::future::IntoFuture>::Error == actix_http::error::PayloadError` --> src/main.rs:67:22 | 67 | stream.concat2().and_then(move |bytes| { | ^^^^^^^^ expected enum `actix_threadpool::BlockingError`, found enum `actix_http::error::PayloadError` | = note: expected type `actix_threadpool::BlockingError<image::image::ImageError>` found type `actix_http::error::PayloadError`</code></pre> There's a couple more that aren't listed. When you are combining streams, mapping them as futures, and otherwise trying to get some output from these combinators you are actually dealing with both the Item</code> type, and the Error</code> type. The ergonomics of this are not as advanced as the Result</code> type, where we can use the ?</code> operator to automatically adjust to the right error (if a conversion is available). When both ops::Try</code> and async/await</code> syntax stabilises, this might be a different story, but for now, we need to deal with error types. What we have instead is the magic™️ from_err()</code> method. This is basically the same as the ?</code> operator but for futures. We have two futures we're dealing with: our bundle of bytes from the stream, and the image from the blocking closure. We have 3 error types we're dealing with: the Payload error, the Image load from memory error, and the blocking error. Let's start by putting from_err()</code> on both of the futures: fn get_image(stream: web::Payload) -> impl Future<Item = DynamicImage, Error = ActixError> { stream.concat2().from_err().and_then(move |bytes| { web::block(move || { image::load_from_memory(&bytes) }).from_err() }) }</code></pre> That worked! There is enough conversions for our error types to get to where we need to get it. The ActixError</code> type has a few blanket conversions from common error types enough to be able to make this magic happen. Getting the bounding boxes from the image</h2> At the core of it, we need to run the following: mtcnn.run(&img)</code></pre> But we want this to run in a threadpool too: web::block(|| mtcnn.run(&img))</code></pre> Let's work through the function signature we'll need. At a minimum we're gonna need the image, and the mtcnn struct. Then we want to return a Vec</code> of BBoxes. We should try and keep our error types the same, so we'll use the ActixError type. The function signature looks like this: fn get_bboxes(img: DynamicImage, mtcnn: WebMtcnn) -> impl Future<Item = Vec<BBox>, Error = ActixError> </code></pre> We know that we'll need a from_err()</code> on the web::block</code> to convert the error type, and we'll have to use a move</code> to give the image to the closure: fn get_bboxes(img: DynamicImage, mtcnn: WebMtcnn) -> impl Future<Item = Vec<BBox>, Error = ActixError> { web::block(move || mtcnn.run(&img)).from_err() }</code></pre> But does this compile? Not yet: error[E0277]: `*mut tensorflow_sys::TF_Status` cannot be sent between threads safely --> src/main.rs:75:5 | 75 | web::block(move || mtcnn.run(&img)).from_err() | ^^^^^^^^^^ `*mut tensorflow_sys::TF_Status` cannot be sent between threads safely | = help: within `tensorflow::Status`, the trait `std::marker::Send` is not implemented for `*mut tensorflow_sys::TF_Status` = note: required because it appears within the type `tensorflow::Status` = note: required by `actix_web::web::block`</code></pre> tensorflow::Status</code>, which is the error type, can't be sent between threads. Let's just shortcut here, and convert the error to a String</code>: fn get_bboxes(img: DynamicImage, mtcnn: WebMtcnn) -> impl Future<Item = Vec<BBox>, Error = ActixError> { web::block(move || mtcnn.run(&img).map_err(|e| e.to_string())).from_err() }</code></pre> This allows us to move the Result</code> across thread boundaries, as String</code> does implement Send</code> Returning JSON BBoxes</h2> Ok, so we have 2 functions, one to get the image from the request, and one to get the bounding boxes. We're gonna want to return json HttpResponse</code>: fn return_bboxes( stream: web::Payload, mtcnn: WebMtcnn, ) -> impl Future<Item = HttpResponse, Error = ActixError> { // Get the image from the input stream get_image(stream) // Get the bounding boxes from the image .and_then(move |img| get_bboxes(img, mtcnn)) // Map the bounding boxes to a json HttpResponse .map(|bboxes| HttpResponse::Ok().json(bboxes)) }</code></pre> Cool! Let's put this in our App definition: HttpServer::new(move || { App::new() .data(mtcnn.clone()) .wrap(middleware::Logger::default()) // our new API service .service(web::resource("/api/v1/bboxes").to_async(return_bboxes)) }) .bind(&opt.listen)? .run()</code></pre> And let's run it, using curl</code> to submit a query to get some results back $ curl --data-binary @rustfest.jpg http://localhost:8000/api/v1/bboxes [{"x1":471.4591,"y1":287.59888,"x2":495.3053,"y2":317.25327,"prob":0.9999908}....</code></pre> Awesome! Using jmespath</a> to see we get our 120 faces back: $ curl -s --data-binary @rustfest.jpg http://localhost:8000/api/v1/bboxes | jp "length(@)" 120</code></pre>Returning an Overlay Image</h2> The other API call we want is to return an image with the bounding boxes overlayed. This is not much of a stretch, but the act of drawing boxes on an image is definitely a blocking action, so we'll need to do the same thing and send it to a thread pool. Let's wrap our overlay function, converting it into a Future: fn get_overlay(img: DynamicImage, bboxes: Vec<BBox>) -> impl Future<Item = Vec<u8>, Error = ActixError> { web::block(move || { let output_img = overlay(&img, &bboxes); ... }).from_err() }</code></pre> We're going to want to return a Vec</code> of u8</code> bytes so we can use this in the return body. So we'll need to allocate some buffer and instruct image</code> to write out a JPEG from the image: let mut buffer = vec![]; output_img.write_to(&mut buffer, JPEG)?; // write out our buffer Ok(buffer)</code></pre> Ok, so let's put our little function together and see if it compiles: fn get_overlay(img: DynamicImage, bboxes: Vec<BBox>) -> impl Future<Item = Vec<u8>, Error = ActixError> { web::block(move || { let output_img = overlay(&img, &bboxes); let mut buffer = Vec::new(); output_img.write_to(&mut buffer, JPEG)?; // write out our buffer Ok(buffer) }).from_err() }</code></pre> Not quite yet: we're missing a type annotation: error[E0282]: type annotations needed --> src/main.rs:82:5 | 82 | web::block(move || { | ^^^^^^^^^^ cannot infer type for `E`</code></pre> Why is there an issue with the type? Well, it relates to this line here: Ok(buffer) // What's the `Error` type here?</code></pre> At the moment, the only error type is from the write_to</code> method which is ImageError</code>. But this line here doesn't have an error type, and could be anything. There are 3 ways I can immediately think to handle this: Way Number 1: Declare the error type in web::block</code> web::block::<_,_,ImageError></code></pre> This looks more like a turbosubmarine than a turbofish! But it compiles! Way Number 2: Declare the Result type with as</code> Ok(buffer) as Result<_, ImageError></code></pre> Way Number 3: Use map</code> to return the buffer on success: output_img.write_to(&mut buffer, JPEG).map(|_| buffer)</code></pre> I think for readability, #2 is probably easiest. The web::block</code> function takes 3 type arguments which can be confusing on first read of the code. #3 is good too but I think it looks a bit strange. Our final method looks like: fn get_overlay(img: DynamicImage, bboxes: Vec<BBox>) -> impl Future<Item = Vec<u8>, Error = ActixError> { web::block(move || { let output_img = overlay(&img, &bboxes); let mut buffer = Vec::new(); output_img.write_to(&mut buffer, JPEG)?; // Type annotations required for the `web::block` Ok(buffer) as Result<_, ImageError> }).from_err() }</code></pre>The API call</h3> Ok, we have our little futures that we need to return bounding boxes and image overlays. Let's stitch this together and return a HttpResponse</code>: fn return_overlay( stream: web::Payload, mtcnn: WebMtcnn, ) -> impl Future<Item = HttpResponse, Error = ActixError> { //... magic happens here }</code></pre> Ok, first step is to get the image from the stream: get_image(stream)</code></pre> And Then once the future has resolved, we want to get the bounding boxes: get_image(stream).and_then(move |img| { get_bboxes(img, mtcnn) })</code></pre>Moving Images Around</h3> Now we want to get the image overlay. We have an issue though! we give the get_bboxes</code> future our image, and it returns a Vec of bboxes, consuming the image. There are a couple of options here. We could clone()</code> the image when we give it to bboxes, but that is duplicating memory. We could wait for Pin</code> and async</code>/await</code> to be finished and probably deal with it in an easier way then. Or we could adjust our get_bboxes</code> method to return a tuple of both the image and bounding boxes: fn get_bboxes( img: DynamicImage, mtcnn: WebMtcnn, ) -> impl Future<Item = (DynamicImage, Vec<BBox>), Error = ActixError> { web::block(move || { mtcnn .run(&img) .map_err(|e| e.to_string()) //Return both the image and the bounding boxes .map(|bboxes| (img, bboxes)) }) .from_err() }</code></pre> Making sure to update our return_bboxes</code> function too: fn return_bboxes( stream: web::Payload, mtcnn: WebMtcnn, ) -> impl Future<Item = HttpResponse, Error = ActixError> { get_image(stream) .and_then(move |img| get_bboxes(img, mtcnn)) .map(|(_img, bboxes)| HttpResponse::Ok().json(bboxes)) }</code></pre>Getting the Overlay</h3> It would be great if rust could desugar a tuple into command arguments. Unfortunately not for us, so we will need to create a small closure: //Create our image overlay .and_then(|(img, bbox)| get_overlay(img, bbox)) .map(|buffer| { // Return a `HttpResponse` here })</code></pre>Generating the Response</h3> Our HttpResponse</code> needs to wrap the buffer into a Http Request with the buffer as the body: HttpResponse::with_body(StatusCode::OK, buffer.into())</code></pre> Is that it? Well no, we have to set the content type header to be a jpeg: let mut response = HttpResponse::with_body(StatusCode::OK, buffer.into()); response .headers_mut() .insert(CONTENT_TYPE, HeaderValue::from_static("image/jpeg"));</code></pre> Ok now we can return the result: fn return_overlay( stream: web::Payload, mtcnn: WebMtcnn, ) -> impl Future<Item = HttpResponse, Error = ActixError> { get_image(stream) .and_then(move |img| { get_bboxes(img, mtcnn) }) .and_then(|(img, bbox) | get_overlay(img, bbox)) .map(|buffer| { let mut response = HttpResponse::with_body(StatusCode::OK, buffer.into()); response .headers_mut() .insert(CONTENT_TYPE, HeaderValue::from_static("image/jpeg")); response }) }</code></pre> And add that to our App</code> builder: HttpServer::new(move || { App::new() .data(mtcnn.clone()) //Add in our data handler //Add in a logger to see the requets coming through .wrap(middleware::Logger::default()) //JSON bounding boxes .service(web::resource("/api/v1/bboxes").to_async(return_bboxes)) //Image overlay .service(web::resource("/api/v1/overlay").to_async(return_overlay)) }</code></pre> Great! Let's run it: $ curl --data-binary @rustfest.jpg http://localhost:8000/api/v1/bboxes > output.jpg</code></pre> And we have our original overlay! </a> Conclusions</h2> We stepped through converting a CLI app into a HTTP service, dipping our toes into the brave new async world. As you can see, actix web</a> is a very versatile web framework. My interest in it was borne out of having all the features I need to build up web apps: multipart, thread pools, great efficiency. While it is hard to bridge the sync and async gap, it's not impossible. It would be great if there were some more ergonomic ways to do so, as I think a lot of developers struggle with this: I have seen a lot of questions around integrating with diesel and friends. If you are looking for more actix web examples, the evergrowing examples repo is your best bet: https://github.com/actix/examples</a> I look forward to seeing what the community builds in the future! Face Detection with Tensorflow Rust Unknown — Thu, 28 Mar 2019 00:00:00 +0000 One of the promises of machine learning is to be able to use it for object recognition in photos. This includes being able to pick out features such as animals, buildings and even faces. This article will step you through using some existing models to accomplish face detection using rust and tensorflow</a>. For the impatient among you, you can find the source code here: https://github.com/cetra3/mtcnn</a> We will be using a pre-trained model called mtcnn</a> for face detection (Note: training a new model is not something we'll be concerned with in this article). The Challenge</h2> We want to read the photo, have the faces detected, and then return an image with the bounding boxes drawn in. In other words, we want to convert this (Image used with permission of RustFest</a>, taken by Fiona Castiñeira</a>): </a> Into this: </a> Tensorflow and MTCNN</h2> The original MTCNN model was written using Caffe</a>, but luckily there is a number of tensorflow python implementations for mtcnn. I'm going to pick the following</a> as it is a straight conversion into a single graph model file. Firstly we want to add tensorflow rust</a> as a dependency. Here's a Cargo.toml to start: [package] name = "mtcnn" version = "0.1.0" edition = "2018" [dependencies] tensorflow = "0.12.0"</code></pre> What we're going to do is load a Graph</a> which is the pre-trained MTCNN, and run a session. The gist is that a Graph is the model used for computation, and a Session is one run</code> of the Graph. A lot more information about these concepts can be found here</a>. I like to think of the Graph as an artificial brain in a vat, just to get some great imagery when you start plugging in inputs and outputs. So let's start by grabbing the existing mtcnn.pb</a> model and trying to load it up. Tensorflow graphs are serialised out in protobuf format and can be loaded in using Graph::import_graph_def</code>. use std::error::Error; use tensorflow::Graph; use tensorflow::ImportGraphDefOptions; fn main() -> Result<(), Box<dyn Error>> { //First, we load up the graph as a byte array let model = include_bytes!("mtcnn.pb"); //Then we create a tensorflow graph from the model let mut graph = Graph::new(); graph.import_graph_def(&*model, &ImportGraphDefOptions::new())? Ok(()) }</code></pre> Running cargo run</code> we should see no errors: $ cargo run Compiling mtcnn v0.1.0 (~/mtcnn) Finished dev [unoptimized + debuginfo] target(s) in 0.89s Running `target/debug/mtcnn`</code></pre> Great! Looks like we can load the graph! StructOpt and the Command Line</h2> We're going to want to test out image generation, so let's use structopt</a> to take two arguments: input</code> and output</code>. If you haven't used structopt before: structopt is like combining clap</a> with serde</a>. The input</code> argument will be the path of an image file. The output</code> will be where we save the output image. So our struct looks like this: use std::path::PathBuf; use structopt::StructOpt #[derive(StructOpt)] struct Opt { #[structopt(parse(from_os_str))] input: PathBuf, #[structopt(parse(from_os_str))] output: PathBuf }</code></pre> The parse(from_os_str)</code> attribute will convert a string argument directly into a PathBuf</code> to save us some boiler plate We can then use this to get a struct with our command line arguments fn main() -> Result<(), Box<dyn Error>> { let opt = Opt::from_args(); .... }</code></pre>Loading Image Data</h2> We need to provide the tensorflow graph with our image data. So how do we do that? We use a Tensor</a>! Tensors represent data within our graph, it sort of reminds me of sending vertices to a GPU. You have a big slice of data, and send it in a format that tensorflow is expecting. The input</code> tensor in this graph is an array of floats, with dimensions: height</code> x width</code> x 3</code> (for 3 colour channels). Let's use the image</a> crate to load the image: let input_image = image::open(&opt.input)?;</code></pre> Next we want to convert this image into its raw pixels by using the GenericImage::pixels</code> function and send that to our graph. All multi-dimensional Tensor arrays are flat, and stored in row major order. The model uses BGR</code> instead of the traditional RGB</code> for colours, so we'll need to reverse the pixel values when we iterate through. Putting it all together: let mut flattened: Vec<f32> = Vec::new(); for (_x, _y, rgb) in input_image.pixels() { flattened.push(rgb[2] as f32); flattened.push(rgb[1] as f32); flattened.push(rgb[0] as f32); }</code></pre> This simply iterates through the pixels, adding them to a flattened Vec. We can then load this into a tensor, specifying the image height and width as arguments: let input = Tensor::new(&[input_image.height() as u64, input_image.width() as u64, 3]) .with_values(&flattened)?</code></pre> Great! we have loaded our image into a format that the graph understands. Let's run a session! Creating a tensorflow session</h3> Ok, we have a graph, the input image, but now we need a session for the graph. We'll just use the defaults for this: let session = Session::new(&SessionOptions::new(), &graph)?;</code></pre>Running a session</h2> Before we run a session, we have a few other inputs the mtcnn model is expecting. We're gonna use the same defaults the mtcnn library for these values: let min_size = Tensor::new(&[]).with_values(&[40f32])?; let thresholds = Tensor::new(&[3]).with_values(&[0.6f32, 0.7f32, 0.7f32])?; let factor = Tensor::new(&[]).with_values(&[0.709f32])?;</code></pre> The graph can define multiple inputs/outputs that are required before running and it depends on the specific neural net as to what these are. For MTCNN, these are all described in the original implementation. Probably the most important one is the min_size</code> which describes the minimum size to find faces. Now we build the session arguments for the inputs: let mut args = SessionRunArgs::new(); //Load our parameters for the model args.add_feed(&graph.operation_by_name_required("min_size")?, 0, &min_size); args.add_feed(&graph.operation_by_name_required("thresholds")?, 0, &thresholds); args.add_feed(&graph.operation_by_name_required("factor")?, 0, &factor); //Load our input image args.add_feed(&graph.operation_by_name_required("input")?, 0, &input);</code></pre> Ok, what about outputs? There are two that we are going to request when the session is finished: bounding boxes and probabilities: let bbox = args.request_fetch(&graph.operation_by_name_required("box")?, 0); let prob = args.request_fetch(&graph.operation_by_name_required("prob")?, 0);</code></pre> Ok cool, we have our inputs, and our outputs, let's run it! session.run(&mut args)?;</code></pre>The BBox struct</h3> The model outputs the following values: Bounding box of the faces</li> Landmarks of the faces</li> Probability that it's a face from 0</code> to 1</code></li> </ul> In order to make it a bit more easy to work with, we'll define a bounding box struct which encodes those values back in a more easy to read fashion: #[derive(Copy, Clone, Debug)] pub struct BBox { pub x1: f32, pub y1: f32, pub x2: f32, pub y2: f32, pub prob: f32, }</code></pre> We'll omit landmarks for simplicity, but can always add them back if we need. Our job is to convert the arrays we get back from the tensorflow session into this struct so it's more meaningful. Saving the Output</h3> Right, now let's grab back the outputs. Just like inputs, outputs are Tensors</code> too: let bbox_res: Tensor<f32> = args.fetch(bbox)?; let prob_res: Tensor<f32> = args.fetch(prob)?;</code></pre> What is the shape of bbox</code>? Well, it's a multi-dimensional flattened array that includes 4 floats per bounding box representing the bounding box extents. The prob</code> is an array with a single float value per face: the probability from 0</code> to 1</code>. So we should expect the bbox_res</code> length to be the number of faces</code> x 4</code>, and prob_res</code> to be equal to the number of faces. Let's do some basic iteration and store our results into a Vec</code>: //Let's store the results as a Vec<BBox> let mut bboxes = Vec::new(); let mut i = 0; let mut j = 0; //While we have responses, iterate through while i < bbox_res.len() { //Add in the 4 floats from the `bbox_res` array. //Notice the y1, x1, etc.. is ordered differently to our struct definition. bboxes.push(BBox { y1: bbox_res[i], x1: bbox_res[i + 1], y2: bbox_res[i + 2], x2: bbox_res[i + 3], prob: prob_res[j], // Add in the facial probability }); //Step `i` ahead by 4. i += 4; //Step `i` ahead by 1. j += 1; }</code></pre>Combinators</h5> Alternatively, we could use combinators to replace the one above. let bboxes: Vec<_> = bbox_res .chunks_exact(4) // Split into chunks of 4 .zip(prob_res.iter()) // Combine it with prob_res .map(|(bbox, &prob)| BBox { y1: bbox[0], x1: bbox[1], y2: bbox[2], x2: bbox[3], prob, }) .collect();</code></pre>Printing out the bounding boxes</h3> Ok, we haven't encoded the bounding boxes into an image, yet. but let's debug to make sure we're getting back something: println!("BBox Length: {}, Bboxes:{:#?}", bboxes.len(), bboxes);</code></pre> Running this, what do we get back: BBox Length: 120, BBoxes:[ BBox { x1: 471.4591, y1: 287.59888, x2: 495.3053, y2: 317.25327, prob: 0.9999908 }, ....</code></pre> Whoa! 120 faces! Awesome! Drawing the Bounding Boxes</h2> Great, we have some bounding boxes. Let's draw them on the image, and save the output into a file. To draw bounding boxes, we can use the imageproc</a> library to draw simple borders around the bounding boxes. Firstly, we'll make the line colour constant green outside of our main()</code> function: const LINE_COLOUR: Rgba<u8> = Rgba { data: [0, 255, 0, 0], };</code></pre> Now stepping back into it. We are feeding in the input image read only, so let's use the first image to draw onto: let mut output_image = input_image;</code></pre> Then we iterate through our bounding boxes: for bbox in bboxes { //Drawing Happens Here! }</code></pre> Next, we use the draw_hollow_rect_mut</code> function. This will take a mutable image reference, and draw a hollow rectangle (outline) specified by the input Rect</code>, overwriting any existing pixels. The Rect</code> takes an x and y coordinate with the at</code> function, and then a width and height with the of_size</code> function. We use a bit of geometry to convert our bounding box to this format: let rect = Rect::at(bbox.x1 as i32, bbox.y1 as i32) .of_size((bbox.x2 - bbox.x1) as u32, (bbox.y2 - bbox.y1) as u32);</code></pre> Then draw the rect: draw_hollow_rect_mut(&mut output_image, rect, LINE_COLOUR);</code></pre> Once the for loop is done, we save it in the output file: output_image.save(&opt.output)?;</code></pre> And we're done. Let's run it! $ cargo run rustfest.jpg output.jpg Compiling mtcnn v0.1.0 (~/mtcnn) Finished dev [unoptimized + debuginfo] target(s) in 5.12s Running `target/debug/mtcnn rustfest.jpg output.jpg` 2019-03-28 16:15:48.194933: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.2 AVX AVX2 FMA BBox Length: 154, BBoxes:[ BBox { x1: 951.46875, y1: 274.00577, x2: 973.68304, y2: 301.93915, prob: 0.9999999 }, ....</code></pre> Looks good, no errors! Wrapping up</h2> Let's step through what we've done in this little application: Loaded a pre-trained tensorflow graph</li> Parse the command line arguments</li> Read in image data</li> Extracted faces by running a tensorflow session</li> Saved the results of that session back to the image</li> Wrote out the file</li> </ul> If you got stuck at any point, have a look at the repository here: https://github.com/cetra3/mtcnn</a> Hopefully this gives you a good introduction to using tensorflow in rust TMQ 0.1.0 Release: ZeroMQ bindings for Tokio Unknown — Thu, 07 Feb 2019 00:00:00 +0000 TMQ</a> is a rust library to use ZeroMQ</a> within the Tokio</a> ecosystem. ZeroMQ is a distributed messaging queue written in C supporting a number of different messaging patterns. While there are other options out there (including gRPC, etc..), I settled on ZeroMQ due to its cross-language support, great documentation and battle-tested nature. Version 0.1.0</code></a> is an alpha release which implements request</code>, response</code>, publish</code> and subscribe</code> style sockets. Usage Example</h2> This example uses tmq</code> to subscribe to messages, and then prints them out via pretty-env-logger</a>: extern crate failure; extern crate futures; #[macro_use] extern crate log; extern crate pretty_env_logger; extern crate tmq; extern crate tokio; use futures::{Future, Stream}; use tmq::*; use std::env; fn main() { if let Err(_) = env::var("RUST_LOG") { env::set_var("RUST_LOG", "subscribe=DEBUG"); } pretty_env_logger::init(); let request = subscribe(&Context::new()) .connect("tcp://127.0.0.1:7899") .expect("Couldn't connect") .subscribe("") .for_each(|val| { info!("Subscribe: {}", val.as_str().unwrap_or("")); Ok(()) }) .map_err(|e| { error!("Error Subscribing: {}", e); }); tokio::run(request); }</code></pre> Other examples of usage can be found in the project itself: https://github.com/cetra3/tmq/tree/master/examples</a> Existing Rust Crates</h2> To find out where tmq</code> sits within the rust ecosystem, it makes sense to discuss this in terms of other ZeroMQ crates. zmq</a> - Rust ZeroMQ Bindings</h3> This library has bindings to the C API and provides a great way to use ZeroMQ within rust. You still need a copy of ZeroMQ on your system and need it compiled in, but that difficulty is pretty much on par with using OpenSSL. I have managed to use this library in OSX, CentOS and Ubuntu with not many issues. The only downside of this library is the lack of bindings to tokio, and is one of the primary motivators for creating tmq</code>. zmq.rs</a> - A Native Implementation of ZeroMQ in Rust</h3> This library was a Rewrite in Rust attempt at a full ZeroMQ reimplementation, which hasn't seen any changes since 2015</a> and can probably be considered abandoned. Regardless: this API is still the old blocking style, which would still predicate a need for tmq</code> zmq-tokio</a> - Run ØMQ sockets using tokio reactors, futures, etc.</h3> Not even published on crates.io but a great initial attempt at bridging the async gap. This crate used the historic tokio-core</a> library which is a bit of a pain to adapt to the new tokio</a> crate. Unfortunately it looks like it's since been abandoned</a>. tokio-zmq</a> - ZeroMQ Concepts with futures on Tokio's runtime</h3> This library is probably the most polished version for tokio</code> and one that would be compared mostly to tmq</code>. I would have chosen this library, but the one major roadblock is the GPL</a> license. The GPL within a rust project is viral. You can't use this library without making the rest of your project GPL or GPL Compatible. If this doesn't concern you, then I would consider using this library. Comparison between tmq</h3> In comparison to tokio-zmq</code>, tmq</code> has less boilerplate to acheive the same thing, but does use a couple of custom traits to speed things along. This makes tmq</code> a bit more opinionated, but less verbose. Both styles have their merits, tokio-zmq</code> does give you more control over things wheras tmq</code> is, in my opinion, easier to write and reason about, but more restrictive in how you use it. To do a direct comparison of both libraries we're using the response</code> example. This example is a simple echo response, which when it receives a message, it echos it back verbatim to the requester. tokio-zmq</h4> Here is the excerpt</a> from tokio-zmq</code>: let ctx = Arc::new(zmq::Context::new()); let req_fut = Req::builder(ctx).connect("tcp://localhost:5560").build(); let runner = req_fut.and_then(|req| { req.send(build_multipart(0)).and_then(|req| { let (sink, stream) = req.sink_stream(25).split(); stream .zip(iter_ok(1..10_000)) .map(|(multipart, i)| { for msg in multipart { if let Some(msg) = msg.as_str() { println!("Received: {}", msg); } } build_multipart(i) }) .forward(sink) }) });</code></pre> On of the things you will notice is there is a lot of standard futures</code> and tokio</code> types used here: sink</code>, stream</code>, etc.. While this does make it more verbose, you are using constructs that you are familiar with in the futures style. tmq</h3> Here's a similar example</a> (although not using multipart</code> messages) from tmq</code>: let responder = respond(&Context::new()) .bind("tcp://127.0.0.1:7899")? .with(|msg: Message| { info!("Request: {}", msg.as_str().unwrap_or("")); Ok(msg) });</code></pre> You'll notice that the library has a bit less boilerplate, but is more opinionated on how you structure a response. Instead of a sink/stream</code> approach, the responder is a stream but has a with</code> method. The with</code> method takes anything that implements the Responder</code> trait, of which there is a blanket implemenation for closures and functions that take a message and return a message. Future Changes</h2> While currently usable, the tmq</code> library is far from finished and requires some work to bring it out of an alpha state. In no particular order here are the plans for the library: Testing</h4> Unit tests and integration tests need to be added in order to confirm and wire up connections. As the library is rather lightweight now the need for integration tests hasn't really arisen, but this will become more complicated in the future as more socket types are added. Benchmarking</h4> Benchmarks need to be added to the library in order to show the performance of using this over standard zmq</code>. Running up some dummy benchmarks show that it is perfomant enough for my current use case, and provides less overhead than your standard REST API calls. Windows Support</h4> I cheated a little by using the Evented</a> trait for mio, which makes it dead easy to use in tokio via PollEvented2</a>. Unfortunately the async story on windows is a bit different, and I haven't had a need to deploy on windows just yet. But it is something that has been considered. Documentation</h4> The library is mostly undocumented besides from the examples. Documentation should be pretty easy to do at this stage and won't take too long. More Socket Types</h4> Implementing more socket types, to make this feature complete with the standard zmq</code> library. There a plethora of different socket types for different use cases that give different guarantees. The 4 standard ones implemented are enough for me to start using this library today, but could easily be expanded. Multipart messages</h4> Multipart support for messages. Currently this is not supported, but should be mostly easy to implement Further thoughts</h2> I'm currently using tmq</code> within an actix-web</code> application</a> to bridge some messages and audit logs between a polyglot backend (including Java & Python). It has been quite solid so far, and I have plans to remove an existing ActiveMQ service to be replaced with tmq</code> where appropriate. Version 0.1.0</code> is the first real release, with previous releases having to vendor in the zmq</code> library in order to publish. While it is alpha, I don't plan to change what is there currently unless there is a compelling reason to do so. Please give tmq</a> a try and let me know your thoughts! My Wishes for Rust 2019 Unknown — Fri, 14 Dec 2018 00:00:00 +0000 2018 was a massive year for rust, and it's great to reflect back on how far we've all come. My main wish for the new year is to put on the breaks a little bit, and finish off what has been started without taking on anything majorly new. There are a couple of areas that I feel need more attention than most (purely for selfish reasons of course!): Compiler Times, IDE Support and RLS</h2> I think RLS is great. I think cargo check</code> is great too. Sometimes when I am waiting for RLS to finish in VSCode, I run cargo check</code> in a terminal to see who wins the race. Sometimes I get results faster from the cargo check</code> than I do in RLS. I'm not sure why that is, it might be that I'm making a lot of small edits, and RLS is constantly restarting. I am assuming that the plumbing for both methods is rather similar. On average, if I just adjust whitespace on one of my projects, it takes at least 10 seconds to recompile. If I change any trivial bit of code, this doubles to 20 seconds. I would love to have dynamic loading in rust as a first class citizen, but I fear the ABI stabilisation is a far way off</a> (I know you can cheat</a> by using the C ABI and libloading</a> but there is a fair amount of unsafe</code> code, which I'm not comfortable with). This would alleviate some of the issues around compilation times, by allowing compilation to be broken up into smaller chunks. For instance: I implemented a WebDAV handler with actix-web</a>. This added a significant portion of time to compilation after the implementation was finished, and the code is mostly decoupled from the rest of the app and sits in its own little route (XML instead of JSON, etc..). I'd love to be able to just hot-load this as a module, rather than compile it in each time if I'm not even touching this part. Incremental compilation does help a little bit here. Better IDE Support</h3> I have been using VSCode</a> as my primary rust IDE for a while now, which is great, but I'd love to be able to use IntelliJ as well. Each time there is an update, I do give it another shot, and find that there are a couple of problems with it: </iframe></div> Weeeeeeeeeeeeeeeeeeeeeee Admittedly, each release of the IntelliJ rust plugin is better than the last, but it is not even close to the support that Java has. I am hoping that this changes in the future. I'd also love to be able to give Oni</a> a go, but currently it's in limbo until I, or someone else, has enough time to work out what is wrong</a> with the Language Server Integration. Async/Await</h2> I would really love to see the async/await story come to a close in 2019. The rust team has laid the groundwork with the 2018 edition, and together with the new pinning API</a> I feel like it's just around the corner. This has the opportunity to really change the landscape and make rust a much more productive language. I've had a crack at writing both a multipart client</a> & ZeroMQ library</a>, but I can see that when the async stuff reaches stable, it's going to make life a lot easier. Kudos to the team for moving this forward (I also happen to think the logo</a> for futures-rs</code> is pretty rad!) Documentation Churn</h3> With these changes to async, there is going to be some challenges around documentation. I feel in tokio-land there is a high degree of tribal knowledge that, if you don't keep up with things, you are left behind. There is the new futures 0.3</code> crate, which is not currently in use by tokio crate, you have to use futures 0.1</code>. No-one knows what happened to 0.2</code>, it's a mystery left for future rust historians, and 0.3</code> is still very much in alpha</a>. Earlier on in the year if you went straight to the docs.rs page</a> for futures, this would direct you to the 0.3</code> version, which isn't helpful if you are using this in conjunction with tokio (this appears to be resolved now, by some clever use of yanking). Another example is the move away from tokio-core</a> into the standard tokio</a> crate. This is something that is a great evolution for tokio, but has left a number of libraries</a> outdated. A lot of the great blogs by pioneering devs have also become outdated when dealing with this, which increases the frustration for any newcomers. I am hoping that now things are more settled that this won't be much of an issue. All of these changes adds to a high level of fatigue for developers. If you're not lucky enough to be working in this space day-to-day, coming back after 6 months would be daunting. There is a push to help modernise the documentation</a>, but this hasn't seen much activity in a couple of months. I'm hoping to find some time in the new year to help out with this, as I think there is a lot of work to be done. Conclusion</h2> I think rust has come a long way and still has a long way to go. I focused a lot of energy on writing apps</a> in rust this year, and will undoubtably accelerate the usage as time goes on. I hope that the rust team has time to put on the breaks, and apply a bit of polish before doing anything major. I also hope I have more time to contribute some bits and pieces here and there to make the community better.

The New Profile</h3>
OK, running again with a `10GB</code> limit, let's have a look at the new flame graph profile:</p>`
`</a></p>`
`OK, that looks a lot</em> better!</p>`

WebSockets</h2>
We are going to use websockets as our primary means of communication between the front and backend, providing a bi-directional channel with which we can send messages.</p>

GraphiQL Playground</h4>
Navigating to `http://localhost:8080/playground</code> in a browser you will get the` `GraphiQL playground</a>:</p>`
`</a></p>`

Second Bonus: Scuba Dive Graph</h3>
I have also used a similar method to plot a dive, displaying the Depth</strong> and Air</strong> on a split axis, with my maximum depth</em> displayed:</p>
</p>

Conclusion</h2>
We used tera, as a template engine, to generate an SVG chart with rust. Some basic geometry was needed, but the results show for themselves. I hope you have found this educational and gives you some ideas.</p>
Also: try out lorikeet-dash</a> and let me know what you think!</p>

Implementing Clients</h2>
Not much has changed here as well. We could get away with not using our own struct definition here at all and just set up a connection, but would be great if we had something we could plop in where the old `tmq::dealer</code> was used.</p>`

The Server Loop</h2>
The server loop</a> is the main loop responsible for coordinating tasks & otherwise tasks are finished in the order they are supposed to. The server here responds to messages from the ZeroMQ socket & acts according to the message received.</p>

Other Tools</h2>
There have been some tools to both manage breaking changes in the past, a lot of the things I found were unmaintained or not in active use. There might also be some great language features that I haven't discovered yet that crate authors are already using.</p>

Rust Breaking Changes</h3>
This also doesn't appear to be maintained, but was an automatic listing of breaking changes to rust: https://killercup.github.io/bitrust/</a>. I do wonder though how many of those changes are internal, as I don't believe there have been many external breaking changes.</p>

Reflecting on Last Year's Wishes</h2>
My last blog</a> I had 3 main areas of improvement that I would like to have seen a focus on.</p>

Handling a Request</h2>
Actix Web is an Async framework and uses tokio. Our function is Synchronous and takes some time to complete.</p>
In other words, our request is blocking. Can we mix and match sync and async? Absolutely, but it is a little more cumbersome as you'll see.</p>

Benchmarking</h4>
Benchmarks need to be added to the library in order to show the performance of using this over standard `zmq</code>. Running up some dummy benchmarks show that it is perfomant enough for my current use case, and provides less overhead than your standard REST API calls.</p>`

Cetra's Thoughts

The State of Allocators in 2026

Creating MCP Servers in Rust

A tale of two lengths

Creating your own custom derive macro

Synchronizing state with Websockets and JSON Patch

Publishing Rust types to a TypeScript frontend

Implementing a Job queue with SQLx and Postgres

Drawing SVG Graphs with Rust

Implementing a Job queue with Tokio-Serde

Lorikeet 0.11.0 - Upgrading to async

Implementing a Job queue with Tokio, PostgreSQL & ZeroMQ

Handling Breaking API Changes

Lessons learnt updating a library to std::future

My Wishes for Rust 2020

Face Detection with Actix Web

Face Detection with Tensorflow Rust

TMQ 0.1.0 Release: ZeroMQ bindings for Tokio

My Wishes for Rust 2019

Cetra's Thoughts

The State of Allocators in 2026

C++ Allocator story</h3> C++ is an older language, and so you expect it to have some warts, moreso than rust.</p>

Conclusions</h2> As it stands, custom allocators are still not in a state to stabilise, and it would be good to stabilise them soon.</p>

Creating MCP Servers in Rust

A tale of two lengths

The New Profile</h3> OK, running again with a 10GB</code> limit, let's have a look at the new flame graph profile:</p>

Creating your own custom derive macro

Crate Layout</h2> You will need at minimum 2 crates to do this:</p>

Emitting our TokenStream</h3> From here, we want to iterate over the fields, and for each field emit a bit of code like:</p>

Standard Crate</h2> Now we have the macro crate, the core crate, we can simply use</code> this to derive on our own types:</p>

Synchronizing state with Websockets and JSON Patch

The Todo Struct</h2> Let's start shaping our todo struct:</p>

WebSockets</h2> We are going to use websockets as our primary means of communication between the front and backend, providing a bi-directional channel with which we can send messages.</p>

Shared WsState</h3> In the main()</code> function, we added a reference counted WsState</code> struct via AddExtensionLayer</code>. This represents our shared state between all websocket sessions.</p> Inside this struct, we are going to keep two things:</p>

Adding Sessions</h4> When we add a session, we want to send a full state to get it caught up, and then add it to our list:</p>

Broadcasting Our Changes</h4> Generating changes to state is one half of this method, the other half is to broadcast this out to every session.</p> We will first check whether there were any changes worth broadcasting, and then will serialize it to a string</p>

Axum Websocket Interface</h3> Axum provides a WebSocketUpgrade</code> struct, with which you can feed it a function (or closure) to start an async task when the websocket upgrade happens. We use this to feed in our WsState</code> value (behind an Arc</code>):</p>

Publishing Rust types to a TypeScript frontend

TypeScript from OpenAPI</h3> With an openapi.json</code> file you can use a tool such as openapi-generator-cli</code> to generate the requisite definitions:</p>

Implementing a Job queue with SQLx and Postgres

SQLx ORM</h3> I still consider SQLx</code> an ORM, but with SQL as a first-class citizen.</p> Here's how you'd do the same with SQLx</strong>:</p>

Mapping a postgresql query to rust structs</h3> sqlx has a query_as</code> method which allows you to convert returned result rows into rust structs, a bit like serde is used to serialize/deserialize data.</p>

Drawing SVG Graphs with Rust

Defining the Graph</h2> First we want to define what a graph is. A minimal example of what a graph would have is:</p> A List of Points</li> A Name/Description</li>

Second Bonus: Scuba Dive Graph</h3> I have also used a similar method to plot a dive, displaying the Depth</strong> and Air</strong> on a split axis, with my maximum depth</em> displayed:</p> </p>

Implementing a Job queue with Tokio-Serde

The TCP Stream</h3> The TCP stream is your standard tcp connection. In tokio, you can either bind</code> or connect</code> to a TCP socket.</p> With connect</code>, you will get a straight tcp stream, if successful:</p>

Existing Clients</h4> We'll do the same trick</em> for the client HashMap</code>: swap it out for an empty one as we're processing.</p> We'll put all received messages into an internal buffer, which we will use later to send the actual response from the stream.</p>

Returning Messages</h4> We've buffered up the returned messages into a Vec</code> and we can return one. But if there are more after popping we can let the waker know we may have more stuff to process:</p>

Implementing Clients</h2> Not much has changed here as well. We could get away with not using our own struct definition here at all and just set up a connection, but would be great if we had something we could plop in where the old tmq::dealer</code> was used.</p>

Lorikeet 0.11.0 - Upgrading to async

Using tokio sync primitives</h2> Tokio has a few primitives that are very close to their std counterparts.</p>

Conclusion</h2> I was happy with the ease at which it was to convert existing sections to async by sprinkling a few keywords here and there. I am concerned that it is too easy to have blocking sections, so I hope there is some tooling/linting around that in the future.</p>

Implementing a Job queue with Tokio, PostgreSQL & ZeroMQ

The New Profile</h3>
OK, running again with a `10GB</code> limit, let's have a look at the new flame graph profile:</p>`
`</a></p>`
`OK, that looks a lot</em> better!</p>`

WebSockets</h2>
We are going to use websockets as our primary means of communication between the front and backend, providing a bi-directional channel with which we can send messages.</p>

Second Bonus: Scuba Dive Graph</h3>
I have also used a similar method to plot a dive, displaying the Depth</strong> and Air</strong> on a split axis, with my maximum depth</em> displayed:</p>
</p>

Implementing Clients</h2>
Not much has changed here as well. We could get away with not using our own struct definition here at all and just set up a connection, but would be great if we had something we could plop in where the old `tmq::dealer</code> was used.</p>`