simon/moonpool
1
0
Fork 0
mirror of https://github.com/c-cube/moonpool.git synced 2025-12-06 03:05:30 -05:00
Code Issues Projects Releases Packages Wiki Activity Actions

deploy: 75e528413b

Browse source
This commit is contained in:
c-cube 2025-11-15 03:59:13 +00:00
parent 338ed1fdfc
commit 4e309961a8
2 changed files with 30 additions and 40 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

35
moonpool-lwt/_doc-dir/README.md
View file

@ -16,16 +16,13 @@ In addition, some concurrency and parallelism primitives are provided:
- `Moonpool.Chan` provides simple cooperative and thread-safe channels - `Moonpool.Chan` provides simple cooperative and thread-safe channels
to use within pool-bound tasks. They're essentially re-usable futures. to use within pool-bound tasks. They're essentially re-usable futures.
On OCaml 5 (meaning there's actual domains and effects, not just threads), Moonpool now requires OCaml 5 (meaning there's actual domains and effects, not just threads),
a `Fut.await` primitive is provided. It's simpler and more powerful so the `Fut.await` primitive is always provided. It's simpler and more powerful
than the monadic combinators. than the monadic combinators.
- `Moonpool_forkjoin`, in the library `moonpool.forkjoin` - `Moonpool_forkjoin`, in the library `moonpool.forkjoin`
provides the fork-join parallelism primitives provides the fork-join parallelism primitives
to use within tasks running in the pool. to use within tasks running in the pool.
On OCaml 4.xx, there is only one domain; all threads run on it, but the
pool abstraction is still useful to provide preemptive concurrency.
## Usage ## Usage
The user can create several thread pools (implementing the interface `Runner.t`). The user can create several thread pools (implementing the interface `Runner.t`).
@ -182,7 +179,7 @@ scope).
### Fork-join ### Fork-join
On OCaml 5, again using effect handlers, the sublibrary `moonpool.forkjoin` The sub-library `moonpool.forkjoin`
provides a module `Moonpool_forkjoin` provides a module `Moonpool_forkjoin`
implements the [fork-join model](https://en.wikipedia.org/wiki/Fork%E2%80%93join_model). implements the [fork-join model](https://en.wikipedia.org/wiki/Fork%E2%80%93join_model).
It must run on a pool (using `Runner.run_async` or inside a future via `Fut.spawn`). It must run on a pool (using `Runner.run_async` or inside a future via `Fut.spawn`).
@ -296,21 +293,18 @@ You are assuming that, if pool P1 has 5000 tasks, and pool P2 has 10 other tasks
## OCaml versions ## OCaml versions
This works for OCaml >= 4.08. This works for OCaml >= 5.00.
- On OCaml 4.xx, there are no domains, so this is just a library for regular thread pools
with not actual parallelism (except for threads that call C code that releases the runtime lock, that is).
C calls that do release the runtime lock (e.g. to call [Z3](https://github.com/Z3Prover/z3), hash a file, etc.)
will still run in parallel.
- on OCaml 5.xx, there is a fixed pool of domains (using the recommended domain count).
These domains do not do much by themselves, but we schedule new threads on them, and form pools
of threads that contain threads from each domain.
Each domain might thus have multiple threads that belong to distinct pools (and several threads from
the same pool, too — this is useful for threads blocking on IO); Each pool will have threads
running on distinct domains, which enables parallelism.
A useful analogy is that each domain is a bit like a CPU core, and `Thread.t` is a logical thread running on a core. Internally, there is a fixed pool of domains (using the recommended domain count).
Multiple threads have to share a single core and do not run in parallel on it[^2]. These domains do not do much by themselves, but we schedule new threads on them, and form pools
We can therefore build pools that spread their worker threads on multiple cores to enable parallelism within each pool. of threads that contain threads from each domain.
Each domain might thus have multiple threads that belong to distinct pools (and several threads from
the same pool, too — this is useful for threads blocking on IO); Each pool will have threads
running on distinct domains, which enables parallelism.
A useful analogy is that each domain is a bit like a CPU core, and `Thread.t` is a logical thread running on a core.
Multiple threads have to share a single core and do not run in parallel on it[^2].
We can therefore build pools that spread their worker threads on multiple cores to enable parallelism within each pool.
TODO: actually use https://github.com/haesbaert/ocaml-processor to pin domains to cores, TODO: actually use https://github.com/haesbaert/ocaml-processor to pin domains to cores,
possibly optionally using `select` in dune. possibly optionally using `select` in dune.
@ -326,3 +320,4 @@ $ opam install moonpool
``` ```
[^2]: ignoring hyperthreading for the sake of the analogy. [^2]: ignoring hyperthreading for the sake of the analogy.

35
moonpool/_doc-dir/README.md
View file

@ -16,16 +16,13 @@ In addition, some concurrency and parallelism primitives are provided:
- `Moonpool.Chan` provides simple cooperative and thread-safe channels - `Moonpool.Chan` provides simple cooperative and thread-safe channels
to use within pool-bound tasks. They're essentially re-usable futures. to use within pool-bound tasks. They're essentially re-usable futures.
On OCaml 5 (meaning there's actual domains and effects, not just threads), Moonpool now requires OCaml 5 (meaning there's actual domains and effects, not just threads),
a `Fut.await` primitive is provided. It's simpler and more powerful so the `Fut.await` primitive is always provided. It's simpler and more powerful
than the monadic combinators. than the monadic combinators.
- `Moonpool_forkjoin`, in the library `moonpool.forkjoin` - `Moonpool_forkjoin`, in the library `moonpool.forkjoin`
provides the fork-join parallelism primitives provides the fork-join parallelism primitives
to use within tasks running in the pool. to use within tasks running in the pool.
On OCaml 4.xx, there is only one domain; all threads run on it, but the
pool abstraction is still useful to provide preemptive concurrency.
## Usage ## Usage
The user can create several thread pools (implementing the interface `Runner.t`). The user can create several thread pools (implementing the interface `Runner.t`).
@ -182,7 +179,7 @@ scope).
### Fork-join ### Fork-join
On OCaml 5, again using effect handlers, the sublibrary `moonpool.forkjoin` The sub-library `moonpool.forkjoin`
provides a module `Moonpool_forkjoin` provides a module `Moonpool_forkjoin`
implements the [fork-join model](https://en.wikipedia.org/wiki/Fork%E2%80%93join_model). implements the [fork-join model](https://en.wikipedia.org/wiki/Fork%E2%80%93join_model).
It must run on a pool (using `Runner.run_async` or inside a future via `Fut.spawn`). It must run on a pool (using `Runner.run_async` or inside a future via `Fut.spawn`).
@ -296,21 +293,18 @@ You are assuming that, if pool P1 has 5000 tasks, and pool P2 has 10 other tasks
## OCaml versions ## OCaml versions
This works for OCaml >= 4.08. This works for OCaml >= 5.00.
- On OCaml 4.xx, there are no domains, so this is just a library for regular thread pools
with not actual parallelism (except for threads that call C code that releases the runtime lock, that is).
C calls that do release the runtime lock (e.g. to call [Z3](https://github.com/Z3Prover/z3), hash a file, etc.)
will still run in parallel.
- on OCaml 5.xx, there is a fixed pool of domains (using the recommended domain count).
These domains do not do much by themselves, but we schedule new threads on them, and form pools
of threads that contain threads from each domain.
Each domain might thus have multiple threads that belong to distinct pools (and several threads from
the same pool, too — this is useful for threads blocking on IO); Each pool will have threads
running on distinct domains, which enables parallelism.
A useful analogy is that each domain is a bit like a CPU core, and `Thread.t` is a logical thread running on a core. Internally, there is a fixed pool of domains (using the recommended domain count).
Multiple threads have to share a single core and do not run in parallel on it[^2]. These domains do not do much by themselves, but we schedule new threads on them, and form pools
We can therefore build pools that spread their worker threads on multiple cores to enable parallelism within each pool. of threads that contain threads from each domain.
Each domain might thus have multiple threads that belong to distinct pools (and several threads from
the same pool, too — this is useful for threads blocking on IO); Each pool will have threads
running on distinct domains, which enables parallelism.
A useful analogy is that each domain is a bit like a CPU core, and `Thread.t` is a logical thread running on a core.
Multiple threads have to share a single core and do not run in parallel on it[^2].
We can therefore build pools that spread their worker threads on multiple cores to enable parallelism within each pool.
TODO: actually use https://github.com/haesbaert/ocaml-processor to pin domains to cores, TODO: actually use https://github.com/haesbaert/ocaml-processor to pin domains to cores,
possibly optionally using `select` in dune. possibly optionally using `select` in dune.
@ -326,3 +320,4 @@ $ opam install moonpool
``` ```
[^2]: ignoring hyperthreading for the sake of the analogy. [^2]: ignoring hyperthreading for the sake of the analogy.