SVC16 Proposal External Extensions For Enhanced Functionality And Customization

Hey guys, let's dive into an exciting proposal for external extensions in SVC16! This idea could really open up a world of possibilities for developers and the platform itself. Imagine a system where developers can create and share their own extensions, kind of like the wild west days of early DOS clones. It's all about fostering innovation and expanding what SVC16 can do!

The Issue: Why External Extensions?

Currently, as far as we know, there aren't any extensions available for SVC16. This feels like a missed opportunity, especially considering the SVC16 theme. The beauty of extensions lies in their ability to add new features and functionalities without altering the core virtual machine (VM) code. Think of it as plug-and-play for software – a concept that can lead to a vibrant ecosystem of competing, semi-compatible "hardware," much like the early days of DOS clones.

External extensions can truly revolutionize SVC16. By allowing developers to create their own extensions, we open the door to a whole new level of customization and functionality. This would not only encourage community involvement but also allow for rapid innovation and experimentation. It's like giving SVC16 a superpower – the ability to evolve and adapt without major overhauls to the core system. Furthermore, SVC16 could officially recognize certain extensions that align with its spirit. This recognition would serve as a stamp of approval, highlighting valuable and well-made additions to the platform. This system allows for non-breaking functionality implementations without altering the VM's core code. This means developers can add features and improvements without the risk of destabilizing the entire system.

This approach is especially beneficial for a project like SVC16, where stability and core functionality are paramount. Extensions provide a safe space for experimentation and the introduction of new ideas. Imagine a developer creating a new graphics rendering engine as an extension or someone implementing a custom input method. These additions can significantly enhance the SVC16 experience without requiring changes to the fundamental VM architecture. By embracing external extensions, SVC16 can foster a dynamic and evolving ecosystem while maintaining a stable core.

The Proposal: How It Works

The core of this proposal revolves around creating an interface that the SVC16 VM can understand. This interface will act as a bridge between the VM and external extensions, allowing them to communicate and interact seamlessly.

Here's a glimpse of the Rust code defining the Extension trait:

trait Extension {
 fn api_version(&self) -> usize;
 fn on_init(&mut self);
 fn on_deinit(&mut self);
 fn extension_triggered(&mut self, buffer: &mut [u16; MEMSIZE]);
}

Let's break down what each part of this trait does:

• api_version(&self) -> usize: This function allows the extension to declare its API version. This is crucial for compatibility. The VM can use this information to ensure that the extension is compatible with the current version of the VM. It acts as a handshake, preventing issues that might arise from using an outdated or incompatible extension.
• on_init(&mut self): Think of this as the extension's "hello world" moment. This function is called when the extension is first loaded into the VM. It provides an opportunity for the extension to initialize its internal state, set up any necessary resources, or perform any other setup tasks before it starts interacting with the VM. It’s the extension’s chance to get ready for action.
• on_deinit(&mut self): This is the counterpart to on_init. When the extension is unloaded or the VM shuts down, this function is called. It allows the extension to gracefully clean up any resources it has allocated, save its state, or perform any other necessary teardown operations. This ensures that the extension doesn't leave any lingering effects after it's unloaded, contributing to the overall stability of the system.
• extension_triggered(&mut self, buffer: &mut [u16; MEMSIZE]): This is where the magic happens. This function is the main entry point for the extension's functionality. The VM calls this function when the extension needs to perform a task. The buffer parameter provides the extension with a mutable reference to the VM's memory. This allows the extension to read data from the VM, process it, and write the results back into the VM's memory. This direct access to memory is what allows extensions to add new functionalities and interact deeply with the VM.

To bring this trait to life, we need an implementation that can load external libraries. This is where libloading (or a similar crate) comes into play. It allows us to dynamically load shared libraries (.so and/or .dll files) at runtime. This means that extensions can be developed and distributed separately from the VM, and the VM can load them as needed. This is the key to creating a truly extensible system.

Here's an example of how the ExternalExtension struct might look:

use libloading::{Library, Symbol};
use std::path::Path;
use std::ffi::os_str::OsStr;

struct ExternalExtension {
 lib: Library,
 sym_api_version: Symbol<unsafe extern fn() -> usize>,
 sym_on_init: Symbol<unsafe extern fn() -> c_void>,
 sym_on_deinit: Symbol<unsafe extern fn() -> c_void>,
 sym_extension_triggered: Symbol<unsafe extern fn(&mut [u16; MEMSIZE]) -> c_void>,
}

impl ExternalExtension {
 pub fn from_lib<P: AsRef<OsStr>>(path: &Path) -> Self {
 // lots of error handling omitted here for brevity. Also there are lifetime considerations on Symbol's, which are also omitted here.
 let lib = Library::new(path);

 let sym_api_version: Symbol<unsafe extern fn() -> usize> = lib.get("svc16_extension_api_version");

 // it might be necessary to check the API version here, rather than later. Incase symbol names or signatures change in a way that makes later calls UB

 let sym_on_init: Symbol<unsafe extern fn() -> c_void> = lib.get("svc16_extension_on_init");
 let sym_on_deinit: Symbol<unsafe extern fn() -> c_void> = lib.get("svc16_extension_on_deinit");
 let sym_extension_triggered: Symbol<unsafe extern fn(&mut [u16; MEMSIZE]) -> c_void> = lib.get("svc16_extension_on_deinit");

 Self {
 lib,
 sym_api_version,
 sym_on_init,
 sym_on_deinit,
 sym_on_extension_triggered,
 }
 }
}

impl Extension for ExternalExtension {
 fn api_version(&self) -> usize {
 unsafe { self.sym_api_version() }
 }
 fn on_init(&mut self) {
 unsafe { self.sym_on_init() }
 }
 fn on_deinit(&mut self) {
 unsafe { self.sym_on_deinit() }
 }
 fn extension_triggered(&mut self, buffer: &mut [u16; MEMSIZE]) {
 unsafe { self.sym_extension_triggered(buffer) }
 }
}

This code demonstrates how an external library can be loaded and its functions called through the Extension trait. The ExternalExtension struct holds the loaded library and the symbols (functions) that the extension exposes. The from_lib function is responsible for loading the library and retrieving the symbols. The implementation of the Extension trait then uses these symbols to call the extension's functions.

Let's break down the code snippet:

• The code starts by importing necessary modules from the libloading, std::path, and std::ffi::os_str crates. These modules provide the functionality needed to load dynamic libraries and work with file paths.
• The ExternalExtension struct is defined, which will hold the loaded library and the symbols (functions) that the extension exposes. The lib field stores the loaded Library object, while the sym_* fields store Symbol objects, each representing a function exposed by the extension.
• The from_lib function is the heart of the extension loading process. It takes a path to a dynamic library as input and attempts to load the library using Library::new. It then retrieves the symbols for the required functions using lib.get. This function returns a Symbol object, which is a pointer to the function in memory.
• The code includes a crucial step: retrieving symbols for functions like svc16_extension_api_version, svc16_extension_on_init, svc16_extension_on_deinit, and svc16_extension_on_trigger. These symbols represent the entry points that the SVC16 VM will use to interact with the extension.
• Error handling: The comments in the code acknowledge the omission of error handling for brevity. In a production environment, robust error handling is crucial to ensure that loading and interacting with extensions is safe and reliable. This would involve checking for errors during library loading and symbol retrieval and handling them appropriately.
• API Version Check: The code suggests the importance of checking the API version of the extension. This check is vital for ensuring compatibility between the extension and the SVC16 VM. If the API versions don't match, it could lead to unexpected behavior or crashes. This check should ideally be performed early in the loading process.
• Lifetime considerations: The code also mentions lifetime considerations for Symbol objects. This is a critical aspect of Rust's memory safety model. Symbol objects hold references to memory within the loaded library. The lifetime of these references must be carefully managed to prevent dangling pointers or memory corruption.
• The impl Extension for ExternalExtension block implements the Extension trait for the ExternalExtension struct. This is where the actual calls to the extension's functions happen. Each function in the trait implementation simply calls the corresponding symbol using unsafe. The unsafe keyword is necessary because we're dealing with raw function pointers and external code, which can potentially violate Rust's safety guarantees.
• The implementation of the api_version function retrieves the API version of the extension by calling the sym_api_version symbol. The on_init and on_deinit functions call the corresponding symbols to initialize and deinitialize the extension. The extension_triggered function calls the sym_extension_triggered symbol, passing the mutable buffer to the extension. This is where the extension's core logic is executed.

The mut array reference in the extension_triggered function will likely need to be expressed as a mutable pointer for safety and compatibility reasons.

Why C Dynamic Libraries? The Interoperability Advantage

You might be wondering, why use C dynamic libraries instead of Rust dynamic libraries? The answer lies in stability and interoperability.

Rust's dynamic libraries are not yet ABI (Application Binary Interface) stable. This means that different versions of the Rust compiler can produce incompatible binaries. In simpler terms, an extension compiled with one version of Rust might not work with a VM compiled with a different version. This can lead to a fragmented ecosystem and compatibility nightmares.

C libraries, on the other hand, have a stable ABI. This means that they are compatible across different compilers and languages. By using C libraries, we ensure that extensions built today will continue to work with future versions of the SVC16 VM, regardless of the Rust compiler version used to build the VM.

This choice also unlocks a significant advantage: language flexibility. By targeting the C ABI, we allow developers to write extensions in any language that can produce C-compatible libraries. This includes popular languages like C, C++, Zig, Go, and even Rust itself. This opens up the extension ecosystem to a wider range of developers and skill sets.

However, this approach does come with a trade-off. While Rust provides strong safety guarantees, C and other languages might not. This means that extension developers need to be extra careful to avoid memory leaks, buffer overflows, and other common C programming pitfalls. But the benefits of stability and interoperability outweigh the risks, especially considering that the core VM remains written in safe Rust.

Conclusion: A Bright Future for SVC16 Extensions

This proposal for external extensions has the potential to transform SVC16 into a highly customizable and extensible platform. By allowing developers to create and share their own extensions, we can foster innovation, expand the functionality of SVC16, and create a vibrant ecosystem around the platform.

The use of C dynamic libraries ensures stability and interoperability, while the Extension trait provides a clear and consistent interface for extensions to interact with the VM. While there are challenges to address, such as ensuring the safety of extensions written in languages other than Rust, the potential benefits of this approach are immense.

Let's embrace the power of external extensions and unlock the full potential of SVC16!