docs: moved around documentation

on building and using

Co-authored-by: zaaarf <me@zaaarf.foo>
This commit is contained in:
əlemi 2024-09-13 22:55:32 +02:00
parent ac94fb86fd
commit 97c57a81d6
Signed by: alemi
GPG key ID: A4895B84D311642C
6 changed files with 90 additions and 83 deletions

46
dist/README.md vendored Normal file
View file

@ -0,0 +1,46 @@
# Compiling and Distributing FFI-compatible binaries
`codemp` aims to target as many platforms as possible, while remaining maintainable and performant.
To guarantee this, it can compile to a bare rust lib but also 4 different FFI-compatible shared objects: JavaScript, Python, Lua, Java.
> We also plan to offer bare C bindings for every other language which can do C interop, but it's not our top priority right now.
To compile the bare FFI-compatible shared object, just `cargo build --release --features=<lang>`, replacing `<lang>` with either `js`, `py`, `java`, `lua` or `luajit`.
In most languages, just importing the resulting shared object will work, however refer to each language's section below for more in-depth information.
## JavaScript
To build a npm package, `napi-cli` must first be installed: `npm install napi-cli`.
You can then `npx napi build` in the project root to compile the native extension and create the type annotations (`index.d.ts`).
A package.json is provided for publishing, but will require some tweaking.
## Python
To distribute the native extension we can leverage python wheels. It will be necessary to build the relevant wheels with [`maturin`](https://github.com/PyO3/maturin).
After installing with `pip install maturin`, run `maturin build` to obtain an `import`able package and installable wheels.
## Lua
Built Lua bindings are valid lua modules and require no extra steps to be used.
## Java
`codemp`'s Java bindings are implemented using the [JNI](https://docs.oracle.com/javase/8/docs/technotes/guides/jni/).
On the Rust side, all Java-related code is gated behind the `java` feature, and is implemented using [`jni`](https://github.com/jni-rs/jni-rs) crate.
Unlike other supported languages, Java is statically typed and requires knowing all foreign function types at compile time.
This means that, to use `codemp` through the JNI, all functions who need to be called must also be declared on the Java side, marked as `native`.
Thus, we also provide pre-made Java glue code, wrapping all native calls and defining classes to hold `codemp` types.
The Java bindings have no known major quirk. However, here are a list of facts that are useful to know when developing with these:
* Memory management is entirely delegated to the JVM's garbage collector.
* A more elegant solution than `Object.finalize()`, who is deprecated in newer Java versions, may be coming eventually.
* Exceptions coming from the native side have generally been made checked to imitate Rust's philosophy with `Result`.
* `JNIException`s are however unchecked: there is nothing you can do to recover from them, as they usually represent a severe error in the glue code. If they arise, it's probably a bug.
### Using
`codemp` **will be available soon** as an artifact on [Maven Central](https://mvnrepository.com)
### Building
This is a [Gradle](https://gradle.org/) project: building requires having both Gradle and Cargo installed, as well as the JDK (any non-abandoned version).
Once you have all the requirements, building is as simple as running `gradle build`: the output is going to be a JAR under `build/libs`, which you can import into your classpath with your IDE of choice.

5
dist/java/README.md vendored
View file

@ -2,7 +2,10 @@
`codemp`'s Java bindings are implemented using the [JNI](https://docs.oracle.com/javase/8/docs/technotes/guides/jni/).
On the Rust side, all Java-related code is gated behind the `java` feature, and is implemented using[`jni-rs`](https://github.com/jni-rs/jni-rs).
Unlike other languages, Java requires glue code on both sides: as a result, a Java component is necessary.
Unlike other supported languages, Java is statically typed and requires knowing all foreign function types at compile time.
This means that, to use `codemp` from Java, all functions which will be used must be declared (as `native`), making using our Java binding without extra glue extremely tedious.
We provide glue code also on the Java side, wrapping all native calls and defining classes to hold `codemp` types.
## Building
This is a [Gradle](https://gradle.org/) project: building requires having both Gradle and Cargo installed, as well as the JDK (any non-abandoned version).

18
dist/js/README.md vendored
View file

@ -1,18 +0,0 @@
# JavaScript bindings
NodeJS allows directly `require`ing properly formed shared objects, so the glue can live mostly on the Rust side.
Our JavaScript glue is built with [`napi`](https://napi.rs).
To get a usable shared object just `cargo build --release --features=js`, however preparing a proper javascript package to be included as dependency requires more steps.
## `npm`
`codemp` is directly available on `npm` as [`codemp`](https://npmjs.org/package/codemp).
## Building
To build a node package, `napi-cli` must first be installed: `npm install napi-cli`.
You can then `npx napi build` in the project root to compile the native extension and create the type annotations (`index.d.ts`).
A package.json is provided for publishing, but will require some tweaking.

19
dist/lua/README.md vendored
View file

@ -1,19 +0,0 @@
# Lua bindings
Lua allows directly `require`ing properly constructed shared objects, so glue code can live completely on the Rust side.
The Lua-compatible wrappers are built with [`mlua`](https://github.com/mlua-rs/mlua).
To build, just `cargo build --release --features=lua` and rename the resulting `libcodemp.so` / `codemp.dll` / `codemp.dylib` in `codemp_native.so/dll/dylib`.
This is important because Lua looks up the constructor symbol based on filename.
Type hints are provided in `annotations.lua`, just include them in your language server: `---@module 'annotations'`.
## LuaRocks
`codemp` is available as a rock on [LuaRocks](https://luarocks.org/modules/alemi/codemp)
## Manual bundling
LuaRocks compiles from source, which only works if have the rust toolchain available. To provide a reasonable NeoVim experience, we provide pre-built binaries.
> Download latest build and annotations from [here](https://codemp.dev/releases/lua/)
You will need a loader file to provide annotations: you can use provided `codemp.lua`

14
dist/py/README.md vendored
View file

@ -1,14 +0,0 @@
# Python bindings
Python allows directly `import`ing properly formed shared objects, so the glue can live mostly on the Rust side.
Our Python glue is built with [`PyO3`](https://pyo3.rs).
To get a usable shared object just `cargo build --release --features=python`, however preparing a proper python package to be included as dependency requires more steps.
## `PyPI`
`codemp` is directly available on `PyPI` as [`codemp`](https://pypi.org/project/codemp).
## Building
To distribute the native extension we can leverage python wheels. It will be necessary to build the relevant wheels with [`maturin`](https://github.com/PyO3/maturin).
After installing with `pip install maturin`, run `maturin build` to obtain an `import`able package.

View file

@ -1,38 +1,47 @@
//! # FFI
//! The glue code for FFI (Foreign Function Interface) in various languages, each gated behind
//! a feature flag.
//!
//! For all except Java, the resulting shared object is ready to use, but external packages are
//! available to simplify dependency management and provide type hints in editor.
//!
//! ## Lua
//! Using [mlua](https://docs.rs/mlua) it's possible to map almost perfectly the entirety of `codemp` API.
//! Notable outliers are functions that receive `codemp` objects: these instead receive arguments
//! to build the object instead (such as [`crate::api::Controller::send`])
//!
//! Note that async operations are carried out on a [tokio] current_thread runtime, so it is
//! necessary to drive it. A separate driver thread can be spawned with `spawn_runtime_driver`
//! function.
//!
//! To work with callbacks, the main Lua thread must periodically stop and poll for callbacks via
//! `poll_callback`, otherwise those will never run. This is necessary to allow safe concurrent
//! access to the global Lua state, so minimize callback execution time as much as possible.
//!
//! ## Python
//! Using [pyo3](https://docs.rs/pyo3) it's possible to map perfectly the entirety of `codemp` API.
//! Async operations run on a dedicated [tokio] runtime
//!
//! # Foreign Function Interface
//! `codemp` aims to be available as a library from as many programming languages as possible.
//! To achieve this, we rely on Foreign Function Interface.
//!
//! ## JavaScript
//! Using [napi](https://docs.rs/napi) it's possible to map perfectly the entirety of `codemp` API.
//! Async operations run on a dedicated [tokio] runtime and the result is sent back to main thread
//! Our JavaScript glue is built with [`napi`](https://napi.rs).
//!
//! All async operations are handled on a separate tokio runtime, automatically managed by `napi`.
//! Callbacks are safely scheduled to be called on the main loop thread.
//!
//! ## Python
//! Our Python glue is built with [`PyO3`](https://pyo3.rs).
//!
//! All async operations return a `Promise`, which can we `.wait()`-ed to block and get the return
//! value. The `Future` itself is run on a `tokio` runtime in a dedicated thread, which must be
//! stared with `codemp.init()` before doing any async operations.
//!
//! ## Lua
//! Our Lua glue is built with [`mlua`](https://github.com/mlua-rs/mlua).
//!
//! Lua bindings run all async code on a current thread tokio runtime, which should be driven with
//! a dedicated thread.
//!
//! All async functions will return a `Promise`, which can be `:await()`-ed to block and get the
//! return value.
//!
//! Note as Lua uses filename to locate entrypoint symbol, so shared object can't just have any name.
//! Accepted filenames are `libcodemp.___`, `codemp.___`, `codemp_native.___`, `codemp_lua.___` (extension depends on your platform: `so` on linux, `dll` on windows, `dylib` on macos).
//! Type hints are provided in `dist/lua/annotations.lua`, just include them in your language server: `---@module 'annotations'`.
//!
//! `codemp` is available as a rock on [LuaRocks](https://luarocks.org/modules/alemi/codemp),
//! however LuaRocks compiles from source and will require having `cargo` installed.
//! We provide pre-built binaries at [codemp.dev/releases/lua](https://codemp.dev/releases/lua/).
//! **Please do not rely on this link, as our built binaries will likely move somewhere else soon!**.
//!
//! ## Java
//! Since for java it is necessary to deal with the JNI and no complete FFI library is available,
//! java glue directly writes JNI functions leveraging [jni](https://docs.rs/jni) rust bindings.
//! Our Java glue is built with [`jni`](https://github.com/jni-rs/jni-rs).
//!
//! To have a runnable `jar`, some extra Java code must be compiled (available under `dist/java`)
//! and bundled together with the shared object. Such extra wrapper provides classes and methods
//! loading the native extension and invoking the underlying native functions.
//! Memory management is entirely delegated to the JVM's garbage collector.
//! A more elegant solution than `Object.finalize()`, who is deprecated in newer Java versions, may be coming eventually.
//!
//! Exceptions coming from the native side have generally been made checked to imitate Rust's philosophy with `Result`.
//! `JNIException`s are however unchecked: there is nothing you can do to recover from them, as they usually represent a severe error in the glue code. If they arise, it's probably a bug.
//!
/// java bindings, built with [jni]
#[cfg(feature = "java")]