Skip to content

T
typon-concurrency
Commit 81845140 authored by Tom Niget's avatar Tom Niget
Browse files
Options

Add runtime submodule, move stdlib, add README

parent 968bedc0
Hide whitespace changes
Inline Side-by-side
Showing with 1151 additions and 1422 deletions
.gitmodules 0 → 100644
View file @ 81845140
[submodule "runtime"]
path = runtime
url = https://lab.nexedi.com/typon/typon-concurrency
README.md
View file @ 81845140
# (*Working Title*) Typon
# Typon compiler
Typon is a three-part project to bring practical GIL-free concurrency to Python:
1. (*in progress*) [`typon/rt`](#typonrt-a-concurrency-runtime), a C++ concurrency runtime
2. (*not started*) [`typon/compiler`](#typoncompiler-a-python-to-c-compiler), a compiler from Python syntax into C++
3. (*not started*) [`typon/bindings`](#typonbindings-pythonc-bindings-for-typon), C++/Python bindings for interoperability with actual Python
This repository hosts the source code of the Typon compiler and standard library.
## Directory structure
## `typon/rt`, A Concurrency Runtime
- `docs`: documentation
- `include`: standard library and compiler primitives/intrisics
- `runtime`: concurrency runtime (Git submodule)
- `trans`: transpiler
A continuation-stealing concurrency runtime using cutting-edge C++20 coroutines,
featuring both `fork`/`sync` structured concurrency and `future`-based unbounded
concurrency.
## Development setup
Install the dependencies using `pip3 install -r requirements.txt`.
### Status
- [x] structured concurrency with `fork`/`sync`
- [x] systematic exception propagation from `fork`ed tasks
- [x] unbounded concurrency with the `future` primitive
- [x] asynchronous waiting on `Future` and `Promise` objects
- [x] mutexes (with asynchronous blocking)
- [x] asynchronous I/O (with [liburing](https://github.com/axboe/liburing))
- [ ] task and I/O cancellation
- [ ] channels
### `fork`/`sync`, Structured Concurrency
```C++
using namespace typon;
Join<int> fibo(int n) {
if (n < 2) {
co_return n;
}
// Start two potentially concurrent tasks
Forked a = co_await fork(fibo(n - 1));
Forked b = co_await fork(fibo(n - 2));
// Wait until they both complete
co_await Sync();
// Access the results
co_return a.get() + b.get();
}
```
[Structured concurrency](https://en.wikipedia.org/wiki/Structured_concurrency)
is a concurrency paradigm where parallel execution paths are always joined
before exiting the function which created them.
It makes it possible to reason locally about concurrent code, since concurrent
tasks will not outlive the current function. This makes concurrent code more
intuitive and readable, and makes it much easier to reason about memory safety
and thread safety. It also enables systematic exception propagation from
spawned tasks to the parent scope.
The `fork`/`sync` primitives mirror the `spawn`/`sync` paradigm of [Cilk](http://cilk.mit.edu/).
`fork` introduces a potentially parallel nested task. `sync` waits for all
the tasks forked so far in the current scope. `fork`/`sync` can only be used
inside a `Join` coroutine, which guarantees all forked tasks complete before
the parent `Join` coroutine returns, even when there is no explicit `sync`.
### Exception Propagation with `fork`/`sync`
When an exception escapes out of a `fork`, execution of the body of the `Join`
coroutine may have already resumed concurrently and continued past the point
where the `fork` occured.
Therefore propagating an exception from a `fork` is not as easy as in the case
of a simple function call: first the body of the`Join` coroutine must reach a
point where it can stop executing, and then all concurrent forks must complete.
The `Join` coroutine may stop executing at the site of the original `fork` (if
the continuation has not been resumed concurrently), or at any subsequent call
to `fork` after the exception occurs, or at the latest at the next explicit or
implicit `sync`.
Once all concurrent forks have completed, execution jumps directly to the call
site of the `Join` coroutine, where the exception is immediately rethrown as if
the `Join` coroutine had thrown the exception itself.
There is no way to catch that exception directly inside the body of the `Join`
coroutine.
```C++
using namespace typon;
Task<void> throw_exception() {
// ...
throw std::exception();
co_return; // so that this is a coroutine
}
Join<void> parallel() {
co_await fork(throw_exception());
// 1. execution may resume concurrently here, or jump straight to 6.
for (int i = 0; i < 5; i++) {
// 2. execution may stop abruptly here and jump straight to 6.
co_await fork(some_task());
}
// 3. execution might reach this point
co_await Sync(); // 4. after this, execution will jump to 6.
// 5. execution will never reach here
}
Task<void> caller() {
// ...
co_await parallel(); // 6. exception is rethrown here
// ...
}
```
### `future`, A Primitive for Unbounded Concurrency
```
using namespace typon;
Task<int> fibo(int n) {
if (n < 2) {
co_return n;
}
// Start two potentially concurrent tasks
Future a = co_await future(fibo(n - 1));
Future b = co_await future(fibo(n - 2));
// Wait for each future and retrieve the results
co_return co_await a.get() + co_await b.get();
}
```
In cases when structured concurrency is too constraining, the `future` primitive
creates concurrent tasks that may outlive the parent scope, more like the `go`
statement in Go. Unlike `go`, `future` immediately returns a `Future` object
that can be used to wait for the task to complete. If the `Future` object is
not used, the task becomes "detached" and lives independantly.
### Examples
See `rt/examples`.
### Using `rt/typon`
`typon/rt` is a headers-only library, so no preliminary build step is required.
Programs only need to include `rt/include/typon/typon.hpp`.
Compiling requires a compiler supporting C++20, such as `gcc++-11` or
`clang++-14` or more recent.
See `rt/examples` for compilation flags.
### References
##### Structured Concurrency
- Nathaniel J. Smith's [Notes on structured concurrency](https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/)
- Martin Sústrik's blog articles on [structured concurrency](https://250bpm.com/blog:137/index.html)
- Lewis Baker's talk on [structured concurrency with C++20 coroutines](https://www.youtube.com/watch?v=1Wy5sq3s2rg)
##### C++20 Coroutines
- Lewis Baker's [Asymmetric Transfer blog on C++20 coroutines](https://lewissbaker.github.io/)
- [C++20 coroutines at cppreference.com](https://en.cppreference.com/w/cpp/language/coroutines)
##### Papers
- N. S. Arora, R. D. Blumofe, and C. G. Plaxton. 1998.
Thread scheduling for multiprogrammed multiprocessors.
https://doi.org/10.1145/277651.277678
- D. Chase and Y. Lev. 2005.
Dynamic circular work-stealing deque.
https://doi.org/10.1145/1073970.1073974
- N. M. Lê, A. Pop, A. Cohen, and F. Zappa Nardelli. 2013.
Correct and efficient work-stealing for weak memory models.
https://doi.org/10.1145/2517327.2442524
- Kyle Singer, Yifan Xu, and I-Ting Angelina Lee. 2019.
Proactive work stealing for futures.
https://doi.org/10.1145/3293883.3295735
- C. X. Lin, T .W Huang, and M. D. F. Wong. 2020.
An efficient work-stealing scheduler for task dependency graph.
https://tsung-wei-huang.github.io/papers/icpads20.pdf
- F. Schmaus, N. Pfeiffer, W. Schröder-Preikschat, T. Hönig, and J. Nolte.
2021. Nowa: a wait-free continuation-stealing concurrency platform.
https://www4.cs.fau.de/Publications/2021/schmaus2021nowa.pdf
## `typon/compiler`, A Python to C++ compiler
### Status
This part has not been started yet.
### Driving Idea
Several projects show that it is quite feasible to use Python's `ast` module
to write compilers that take Python syntax as input language. The abstract
syntax tree can then be used to generate C++ output code. The compiler may
freely reinterpret the input syntax and give it different semantic meanings
than in the original Python. As such it is a basis to create an independent
language with a Python-compatible syntax.
In particular such a compiler may use Python's `type hints` syntax for type
annotations to introduce static typing in the language and generate required
typing annotations in the C++ output.
### References
##### Language Design
- Abilian's [design note on syntax for a Python-derived language](https://github.com/abilian/cythonplus-sandbox/blob/devel/sandbox/comparison.md)
##### Technical References
- Python's [`ast` module](https://docs.python.org/3/library/ast.html)
- [PEP 484 - Type Hints](https://peps.python.org/pep-0484/)
- [PEP 526 – Syntax for Variable Annotations](https://peps.python.org/pep-0526/)
##### Related Works
- Lukas Martinelli's [`py14` project](https://github.com/lukasmartinelli/py14)
- [The `py2many` project](https://github.com/py2many/py2many)
- [`mypyc`](https://github.com/mypyc/mypyc)
- [`mycpp`, a Python to C++ translator](https://www.oilshell.org/blog/2022/05/mycpp.html)
## `typon/bindings`, Python/C++ bindings for Typon
### Status
This part has not been started yet.
### Driving Idea
The previous part `typon/compiler`, aims to produce a language that takes
Python syntax as input, but is otherwise completely independent of Python.
Using Python/C++ bindings, this language can be made to be interoperable
with true Python, allowing it to be called from standard Python and to call
standard Python code.
Using the first part `typon/rt`, concurrent code may acquire the Python GIL
asynchronously so as not to block the underlying worker thread, in order to
safely call standard Python code.
### References
##### Technical References
- [The Python Global Interpreter Lock](https://wiki.python.org/moin/GlobalInterpreterLock)
##### Related Works
- [`nanobind`, a C++17/Python bindings library](https://github.com/wjakob/nanobind)
- [`pybind11`, a C++11/Python bindings library](https://github.com/pybind/pybind11)
- [Cython](https://cython.org/)
- [The Cython+ project](https://www.cython.plus/)
- [`mypyc`](https://github.com/mypyc/mypyc)
## Test harness
From inside the `trans` directory, run `python3 test_runner.py`.
\ No newline at end of file
rt/include/python/basedef.hpp → include/python/basedef.hpp
View file @ 81845140
//
// Created by Tom on 24/03/2023.
//
#ifndef TYPON_BASEDEF_HPP
#define TYPON_BASEDEF_HPP
template<typename Self>
class TyBuiltin {
template <typename... Args>
auto sync_wrapper(Args&&... args)
{
return static_cast<Self*>(this)->sync(std::forward<Args>(args)...);
}
public:
template <typename... Args>
auto operator()(Args&&... args) -> decltype(sync_wrapper(std::forward<Args>(args)...))
{
return sync_wrapper(std::forward<Args>(args)...);
}
};
struct method {};
template <typename Func, typename Self>
struct boundmethod {
[[no_unique_address]] Func func;
Self self;
boundmethod(Func func, Self self) : func(func), self(self) {}
template <typename... Args>
auto operator()(Args &&... args) const {
return func(self, std::forward<Args>(args)...);
}
};
template <typename Obj, std::derived_from<method> Attr>
auto dot_bind(Obj obj, Attr attr) {
return boundmethod(attr, obj);
}
template <typename Obj, typename Attr>
requires (! std::derived_from<Attr, method>)
auto dot_bind(Obj, Attr attr) {
return attr;
}
#define dot(OBJ, NAME) [](auto && obj) -> auto { return dot_bind(obj, obj.NAME); }(OBJ)
#define dotp(OBJ, NAME) [](auto && obj) -> auto { return dot_bind(obj, obj->NAME); }(OBJ)
#define dots(OBJ, NAME) [](auto && obj) -> auto { return std::remove_reference<decltype(obj)>::type::py_type::NAME; }(OBJ)
#endif // TYPON_BASEDEF_HPP
//
// Created by Tom on 24/03/2023.
//
#ifndef TYPON_BASEDEF_HPP
#define TYPON_BASEDEF_HPP
template<typename Self>
class TyBuiltin {
template <typename... Args>
auto sync_wrapper(Args&&... args)
{
return static_cast<Self*>(this)->sync(std::forward<Args>(args)...);
}
public:
template <typename... Args>
auto operator()(Args&&... args) -> decltype(sync_wrapper(std::forward<Args>(args)...))
{
return sync_wrapper(std::forward<Args>(args)...);
}
};
struct method {};
template <typename Func, typename Self>
struct boundmethod {
[[no_unique_address]] Func func;
Self self;
boundmethod(Func func, Self self) : func(func), self(self) {}
template <typename... Args>
auto operator()(Args &&... args) const {
return func(self, std::forward<Args>(args)...);
}
};
template <typename Obj, std::derived_from<method> Attr>
auto dot_bind(Obj obj, Attr attr) {
return boundmethod(attr, obj);
}
template <typename Obj, typename Attr>
requires (! std::derived_from<Attr, method>)
auto dot_bind(Obj, Attr attr) {
return attr;
}
#define dot(OBJ, NAME) [](auto && obj) -> auto { return dot_bind(obj, obj.NAME); }(OBJ)
#define dotp(OBJ, NAME) [](auto && obj) -> auto { return dot_bind(obj, obj->NAME); }(OBJ)
#define dots(OBJ, NAME) [](auto && obj) -> auto { return std::remove_reference<decltype(obj)>::type::py_type::NAME; }(OBJ)
#endif // TYPON_BASEDEF_HPP
rt/include/python/builtins.hpp → include/python/builtins.hpp
View file @ 81845140
//
// Created by Tom on 07/03/2023.
//
#ifndef TYPON_BUILTINS_HPP
#define TYPON_BUILTINS_HPP
#include <fmt/format.h>
#include <iostream>
#include <optional>
#include <ostream>
#include <string>
#include <python/basedef.hpp>
#include <typon/typon.hpp>
#ifdef __cpp_lib_unreachable
#include <utility>
[[noreturn]] inline void TYPON_UNREACHABLE() { std::unreachable(); }
#else
#include <cstdlib>
[[noreturn]] inline void TYPON_UNREACHABLE() { std::abort(); }
#endif
#define _Args(...) __VA_ARGS__
#define COMMA() ,
#define METHOD(ret, name, args, ...) \
static constexpr struct name##_s : method { \
template <typename Self> ret operator() args const __VA_ARGS__ \
} name{};
#define METHOD_GEN(gen, ret, name, args, ...) \
static constexpr struct name##_s : method { \
template <typename Self, _Args gen> ret operator() args const __VA_ARGS__ \
} name{};
#define FUNCTION(ret, name, args, ...) \
struct { \
ret operator() args const __VA_ARGS__ \
} static constexpr name;
using namespace std::literals;
template <typename T>
concept PySmartPtr = requires { typename T::element_type; };
template <typename T>
concept PyUserType = requires { typename T::py_type; };
template <typename T> struct RealType {
using type = T;
};
template <PyUserType T> struct RealType<T> {
using type = typename T::py_type;
};
template <PySmartPtr T> struct RealType<T> {
using type = typename T::element_type;
};
//template <typename T> using PyObj = std::shared_ptr<typename RealType<T>::type>;
template<typename T>
class PyObj : public std::shared_ptr<typename RealType<T>::type> {
public:
using inner = typename RealType<T>::type;
template<typename... Args>
PyObj(Args&&... args) : std::shared_ptr<inner>(std::make_shared<inner>(std::forward<Args>(args)...)) {}
PyObj() : std::shared_ptr<inner>() {}
PyObj(std::nullptr_t) : std::shared_ptr<inner>(nullptr) {}
PyObj(inner *ptr) : std::shared_ptr<inner>(ptr) {}
PyObj(const std::shared_ptr<inner> &ptr) : std::shared_ptr<inner>(ptr) {}
PyObj(std::shared_ptr<inner> &&ptr) : std::shared_ptr<inner>(ptr) {}
PyObj(const PyObj &ptr) : std::shared_ptr<inner>(ptr) {}
PyObj( PyObj &ptr) : std::shared_ptr<inner>(ptr) {}
PyObj(PyObj &&ptr) : std::shared_ptr<inner>(ptr) {}
PyObj &operator=(const PyObj &ptr) { std::shared_ptr<inner>::operator=(ptr); return *this; }
PyObj &operator=(PyObj &&ptr) { std::shared_ptr<inner>::operator=(ptr); return *this; }
PyObj &operator=(std::nullptr_t) { std::shared_ptr<inner>::operator=(nullptr); return *this; }
PyObj &operator=(inner *ptr) { std::shared_ptr<inner>::operator=(ptr); return *this; }
PyObj &operator=(const std::shared_ptr<inner> &ptr) { std::shared_ptr<inner>::operator=(ptr); return *this; }
template<typename U>
PyObj(const PyObj<U> &ptr) : std::shared_ptr<inner>(ptr) {}
//PyObj(PyObj<U> &&ptr) : std::shared_ptr<inner>(ptr) {}
// using make_shared
/*template<class U>
PyObj(U&& other) : std::shared_ptr<inner>(std::make_shared<inner>(other)) {}*/
template<class U>
bool operator==(const PyObj<U> &other) const {
// check null
if (this->get() == other.get()) {
return true;
}
if (this->get() == nullptr || other.get() == nullptr) {
return false;
}
return *this->get() == *other.get();
}
template<class U>
bool operator==(const U& other) const {
if (this->get() == nullptr) {
return false;
}
return *this->get() == other;
}
template<class U>
bool py_is(const PyObj<U> &other) const {
return this->get() == other.get();
}
};
template <typename T, typename... Args> auto pyobj(Args &&...args) -> PyObj<typename RealType<T>::type> {
return std::make_shared<typename RealType<T>::type>(
std::forward<Args>(args)...);
}
template <typename T, typename... Args> auto pyobj_agg(Args &&...args) -> PyObj<T> {
return std::make_shared<typename RealType<T>::type>((typename RealType<T>::type) { std::forward<Args>(args)... });
}
// typon_len
template <typename T>
concept PyIterator = requires(T t) {
{ t.py_next() } -> std::same_as<std::optional<T>>;
};
template <typename T>
concept PyIterable = requires(T t) {
{ t.py_iter() } -> PyIterator;
};
template <PyIterable T, PyIterator U> U iter(const T &t) { return t.py_iter(); }
template <typename T>
concept CppSize = requires(const T &t) {
{ t.size() } -> std::same_as<size_t>;
};
template <typename T>
concept PyLen = requires(const T &t) {
{ t.py_len() } -> std::same_as<size_t>;
};
template <CppSize T>
requires(!PyLen<T>)
size_t len(const T &t) {
return t.size();
}
template <PyLen T> size_t len(const T &t) { return t.py_len(); }
template <typename T>
concept PyNext = requires(T t) {
{ t.py_next() } -> std::same_as<std::optional<typename T::value_type>>;
};
struct {
template <PyNext T>
std::optional<typename T::value_type>
operator()(T &t, std::optional<typename T::value_type> def = std::nullopt) {
auto opt = t.py_next();
return opt ? opt : def;
}
} next;
template <typename T>
std::ostream &operator<<(std::ostream &os, std::optional<T> const &opt) {
return opt ? os << opt.value() : os << "None";
}
bool is_cpp() { return true; }
static constexpr auto PyNone = std::nullopt;
#define system_error(err, message) \
do { \
puts(message); \
throw fmt::system_error(err, message); \
} while (0)
#include "builtins/bool.hpp"
#include "builtins/complex.hpp"
#include "builtins/dict.hpp"
#include "builtins/list.hpp"
#include "builtins/print.hpp"
#include "builtins/range.hpp"
#include "builtins/set.hpp"
#include "builtins/slice.hpp"
#include "builtins/str.hpp"
struct file_s {
struct py_type {
METHOD(
typon::Task<PyStr>, read, (Self self, size_t size = -1), {
if (size == -1) {
size = self->len;
}
PyStr buf(size, '\0');
int nbytes = co_await typon::io::read(self->fd, buf.data(), size);
if (nbytes < 0) {
system_error(-nbytes, "read()");
}
buf.resize(nbytes);
co_return std::move(buf);
})
METHOD(
typon::Task<int>, write, (Self self, const std::string &buf), {
int nbytes = co_await typon::io::write(self->fd, buf);
if (nbytes < 0) {
system_error(-nbytes, "write()");
}
co_return nbytes;
})
METHOD(
typon::Task<void>, close, (Self self),
{ co_await typon::io::close(self->fd); })
METHOD(
typon::Task<void>, flush, (Self self),
{ co_await typon::io::fsync(self->fd); })
py_type(int fd = -1, size_t len = 0) : fd(fd), len(len) {}
py_type(const py_type &other)
: fd(other.fd), len(other.len) {}
METHOD(
auto, py_enter, (Self self), { return self; })
METHOD(
typon::Task<bool>, py_exit, (Self self), {
co_await dotp(self, close)();
co_return true;
})
int fd;
size_t len;
};
} file;
namespace typon {
using PyFile = PyObj<decltype(file)::py_type>;
}
typon::Task<typon::PyFile> open(const PyStr &path, std::string_view mode) {
const char *path_c = path.c_str();
size_t len = 0;
struct statx statxbuf;
if (int err = co_await typon::io::statx(AT_FDCWD, path_c, 0, STATX_SIZE,
&statxbuf)) {
// new file
} else {
len = statxbuf.stx_size;
}
int flags = 0;
bool created = false, writable = false, readable = false;
if (mode.find('x') != std::string_view::npos) {
created = true;
writable = true;
flags = O_EXCL | O_CREAT;
} else if (mode.find('r') != std::string_view::npos) {
readable = true;
flags = 0;
} else if (mode.find('w') != std::string_view::npos) {
writable = true;
flags = O_CREAT | O_TRUNC;
} else if (mode.find('a') != std::string_view::npos) {
writable = true;
flags = O_APPEND | O_CREAT;
}
if (mode.find('+') != std::string_view::npos) {
readable = true;
writable = true;
}
if (readable && writable) {
flags |= O_RDWR;
} else if (readable) {
flags |= O_RDONLY;
} else {
flags |= O_WRONLY;
}
flags |= O_CLOEXEC;
int fd = co_await typon::io::openat(AT_FDCWD, path_c, flags, 0666);
if (fd < 0) {
std::cerr << path << "," << flags << std::endl;
system_error(-fd, "openat()");
}
co_return pyobj<typon::PyFile>(fd, len);
}
#include "../typon/generator.hpp"
#include <pybind11/embed.h>
#include <pybind11/stl.h>
namespace py = pybind11;
#include <utility>
template <typename Ref>
struct lvalue_or_rvalue {
Ref &&ref;
template <typename Arg>
constexpr lvalue_or_rvalue(Arg &&arg) noexcept
: ref(std::move(arg))
{ }
constexpr operator Ref& () const & noexcept { return ref; }
constexpr operator Ref&& () const && noexcept { return std::move(ref); }
constexpr Ref& operator*() const noexcept { return ref; }
constexpr Ref* operator->() const noexcept { return &ref; }
};
namespace typon {
template< class... Types >
using PyTuple = std::tuple<Types...>;
}
template<typename T>
auto& iter_fix_ref(T& obj) { return obj; }
template<PySmartPtr T>
auto& iter_fix_ref(T& obj) { return *obj; }
namespace std {
template <class T> auto begin(std::shared_ptr<T> &obj) { return dotp(obj, begin)(); }
template <class T> auto end(std::shared_ptr<T> &obj) { return dotp(obj, end)(); }
}
template <typename T>
struct AlwaysTrue { // (1)
constexpr bool operator()(const T&) const {
return true;
}
};
template <typename Seq>
struct ValueTypeEx {
using type = decltype(*std::begin(std::declval<Seq&>()));
};
// (2)
template <typename Map, typename Seq, typename Filt = AlwaysTrue<typename ValueTypeEx<Seq>::type>>
auto mapFilter(Map map, Seq seq, Filt filt = Filt()) {
//typedef typename Seq::value_type value_type;
using value_type = typename ValueTypeEx<Seq>::type;
using return_type = decltype(map(std::declval<value_type>()));
std::vector<return_type> result{};
for (auto i : seq | std::views::filter(filt)
| std::views::transform(map)) result.push_back(i);
return typon::PyList(std::move(result));
}
#endif // TYPON_BUILTINS_HPP
//
// Created by Tom on 07/03/2023.
//
#ifndef TYPON_BUILTINS_HPP
#define TYPON_BUILTINS_HPP
#include <fmt/format.h>
#include <iostream>
#include <optional>
#include <ostream>
#include <string>
#include <python/basedef.hpp>
#include <typon/typon.hpp>
#ifdef __cpp_lib_unreachable
#include <utility>
[[noreturn]] inline void TYPON_UNREACHABLE() { std::unreachable(); }
#else
#include <cstdlib>
[[noreturn]] inline void TYPON_UNREACHABLE() { std::abort(); }
#endif
#define _Args(...) __VA_ARGS__
#define COMMA() ,
#define METHOD(ret, name, args, ...) \
static constexpr struct name##_s : method { \
template <typename Self> ret operator() args const __VA_ARGS__ \
} name{};
#define METHOD_GEN(gen, ret, name, args, ...) \
static constexpr struct name##_s : method { \
template <typename Self, _Args gen> ret operator() args const __VA_ARGS__ \
} name{};
#define FUNCTION(ret, name, args, ...) \
struct { \
ret operator() args const __VA_ARGS__ \
} static constexpr name;
using namespace std::literals;
template <typename T>
concept PySmartPtr = requires { typename T::element_type; };
template <typename T>
concept PyUserType = requires { typename T::py_type; };
template <typename T> struct RealType {
using type = T;
};
template <PyUserType T> struct RealType<T> {
using type = typename T::py_type;
};
template <PySmartPtr T> struct RealType<T> {
using type = typename T::element_type;
};
//template <typename T> using PyObj = std::shared_ptr<typename RealType<T>::type>;
template<typename T>
class PyObj : public std::shared_ptr<typename RealType<T>::type> {
public:
using inner = typename RealType<T>::type;
template<typename... Args>
PyObj(Args&&... args) : std::shared_ptr<inner>(std::make_shared<inner>(std::forward<Args>(args)...)) {}
PyObj() : std::shared_ptr<inner>() {}
PyObj(std::nullptr_t) : std::shared_ptr<inner>(nullptr) {}
PyObj(inner *ptr) : std::shared_ptr<inner>(ptr) {}
PyObj(const std::shared_ptr<inner> &ptr) : std::shared_ptr<inner>(ptr) {}
PyObj(std::shared_ptr<inner> &&ptr) : std::shared_ptr<inner>(ptr) {}
PyObj(const PyObj &ptr) : std::shared_ptr<inner>(ptr) {}
PyObj( PyObj &ptr) : std::shared_ptr<inner>(ptr) {}
PyObj(PyObj &&ptr) : std::shared_ptr<inner>(ptr) {}
PyObj &operator=(const PyObj &ptr) { std::shared_ptr<inner>::operator=(ptr); return *this; }
PyObj &operator=(PyObj &&ptr) { std::shared_ptr<inner>::operator=(ptr); return *this; }
PyObj &operator=(std::nullptr_t) { std::shared_ptr<inner>::operator=(nullptr); return *this; }
PyObj &operator=(inner *ptr) { std::shared_ptr<inner>::operator=(ptr); return *this; }
PyObj &operator=(const std::shared_ptr<inner> &ptr) { std::shared_ptr<inner>::operator=(ptr); return *this; }
template<typename U>
PyObj(const PyObj<U> &ptr) : std::shared_ptr<inner>(ptr) {}
//PyObj(PyObj<U> &&ptr) : std::shared_ptr<inner>(ptr) {}
// using make_shared
/*template<class U>
PyObj(U&& other) : std::shared_ptr<inner>(std::make_shared<inner>(other)) {}*/
template<class U>
bool operator==(const PyObj<U> &other) const {
// check null
if (this->get() == other.get()) {
return true;
}
if (this->get() == nullptr || other.get() == nullptr) {
return false;
}
return *this->get() == *other.get();
}
template<class U>
bool operator==(const U& other) const {
if (this->get() == nullptr) {
return false;
}
return *this->get() == other;
}
template<class U>
bool py_is(const PyObj<U> &other) const {
return this->get() == other.get();
}
};
template <typename T, typename... Args> auto pyobj(Args &&...args) -> PyObj<typename RealType<T>::type> {
return std::make_shared<typename RealType<T>::type>(
std::forward<Args>(args)...);
}
template <typename T, typename... Args> auto pyobj_agg(Args &&...args) -> PyObj<T> {
return std::make_shared<typename RealType<T>::type>((typename RealType<T>::type) { std::forward<Args>(args)... });
}
// typon_len
template <typename T>
concept PyIterator = requires(T t) {
{ t.py_next() } -> std::same_as<std::optional<T>>;
};
template <typename T>
concept PyIterable = requires(T t) {
{ t.py_iter() } -> PyIterator;
};
template <PyIterable T, PyIterator U> U iter(const T &t) { return t.py_iter(); }
template <typename T>
concept CppSize = requires(const T &t) {
{ t.size() } -> std::same_as<size_t>;
};
template <typename T>
concept PyLen = requires(const T &t) {
{ t.py_len() } -> std::same_as<size_t>;
};
template <CppSize T>
requires(!PyLen<T>)
size_t len(const T &t) {
return t.size();
}
template <PyLen T> size_t len(const T &t) { return t.py_len(); }
template <typename T>
concept PyNext = requires(T t) {
{ t.py_next() } -> std::same_as<std::optional<typename T::value_type>>;
};
struct {
template <PyNext T>
std::optional<typename T::value_type>
operator()(T &t, std::optional<typename T::value_type> def = std::nullopt) {
auto opt = t.py_next();
return opt ? opt : def;
}
} next;
template <typename T>
std::ostream &operator<<(std::ostream &os, std::optional<T> const &opt) {
return opt ? os << opt.value() : os << "None";
}
bool is_cpp() { return true; }
static constexpr auto PyNone = std::nullopt;
#define system_error(err, message) \
do { \
puts(message); \
throw fmt::system_error(err, message); \
} while (0)
#include "builtins/bool.hpp"
#include "builtins/complex.hpp"
#include "builtins/dict.hpp"
#include "builtins/list.hpp"
#include "builtins/print.hpp"
#include "builtins/range.hpp"
#include "builtins/set.hpp"
#include "builtins/slice.hpp"
#include "builtins/str.hpp"
struct file_s {
struct py_type {
METHOD(
typon::Task<PyStr>, read, (Self self, size_t size = -1), {
if (size == -1) {
size = self->len;
}
PyStr buf(size, '\0');
int nbytes = co_await typon::io::read(self->fd, buf.data(), size);
if (nbytes < 0) {
system_error(-nbytes, "read()");
}
buf.resize(nbytes);
co_return std::move(buf);
})
METHOD(
typon::Task<int>, write, (Self self, const std::string &buf), {
int nbytes = co_await typon::io::write(self->fd, buf);
if (nbytes < 0) {
system_error(-nbytes, "write()");
}
co_return nbytes;
})
METHOD(
typon::Task<void>, close, (Self self),
{ co_await typon::io::close(self->fd); })
METHOD(
typon::Task<void>, flush, (Self self),
{ co_await typon::io::fsync(self->fd); })
py_type(int fd = -1, size_t len = 0) : fd(fd), len(len) {}
py_type(const py_type &other)
: fd(other.fd), len(other.len) {}
METHOD(
auto, py_enter, (Self self), { return self; })
METHOD(
typon::Task<bool>, py_exit, (Self self), {
co_await dotp(self, close)();
co_return true;
})
int fd;
size_t len;
};
} file;
namespace typon {
using PyFile = PyObj<decltype(file)::py_type>;
}
typon::Task<typon::PyFile> open(const PyStr &path, std::string_view mode) {
const char *path_c = path.c_str();
size_t len = 0;
struct statx statxbuf;
if (int err = co_await typon::io::statx(AT_FDCWD, path_c, 0, STATX_SIZE,
&statxbuf)) {
// new file
} else {
len = statxbuf.stx_size;
}
int flags = 0;
bool created = false, writable = false, readable = false;
if (mode.find('x') != std::string_view::npos) {
created = true;
writable = true;
flags = O_EXCL | O_CREAT;
} else if (mode.find('r') != std::string_view::npos) {
readable = true;
flags = 0;
} else if (mode.find('w') != std::string_view::npos) {
writable = true;
flags = O_CREAT | O_TRUNC;
} else if (mode.find('a') != std::string_view::npos) {
writable = true;
flags = O_APPEND | O_CREAT;
}
if (mode.find('+') != std::string_view::npos) {
readable = true;
writable = true;
}
if (readable && writable) {
flags |= O_RDWR;
} else if (readable) {
flags |= O_RDONLY;
} else {
flags |= O_WRONLY;
}
flags |= O_CLOEXEC;
int fd = co_await typon::io::openat(AT_FDCWD, path_c, flags, 0666);
if (fd < 0) {
std::cerr << path << "," << flags << std::endl;
system_error(-fd, "openat()");
}
co_return pyobj<typon::PyFile>(fd, len);
}
#include <typon/generator.hpp>
#include <pybind11/embed.h>
#include <pybind11/stl.h>
namespace py = pybind11;
#include <utility>
template <typename Ref>
struct lvalue_or_rvalue {
Ref &&ref;
template <typename Arg>
constexpr lvalue_or_rvalue(Arg &&arg) noexcept
: ref(std::move(arg))
{ }
constexpr operator Ref& () const & noexcept { return ref; }
constexpr operator Ref&& () const && noexcept { return std::move(ref); }
constexpr Ref& operator*() const noexcept { return ref; }
constexpr Ref* operator->() const noexcept { return &ref; }
};
namespace typon {
template< class... Types >
using PyTuple = std::tuple<Types...>;
}
template<typename T>
auto& iter_fix_ref(T& obj) { return obj; }
template<PySmartPtr T>
auto& iter_fix_ref(T& obj) { return *obj; }
namespace std {
template <class T> auto begin(std::shared_ptr<T> &obj) { return dotp(obj, begin)(); }
template <class T> auto end(std::shared_ptr<T> &obj) { return dotp(obj, end)(); }
}
template <typename T>
struct AlwaysTrue { // (1)
constexpr bool operator()(const T&) const {
return true;
}
};
template <typename Seq>
struct ValueTypeEx {
using type = decltype(*std::begin(std::declval<Seq&>()));
};
// (2)
template <typename Map, typename Seq, typename Filt = AlwaysTrue<typename ValueTypeEx<Seq>::type>>
auto mapFilter(Map map, Seq seq, Filt filt = Filt()) {
//typedef typename Seq::value_type value_type;
using value_type = typename ValueTypeEx<Seq>::type;
using return_type = decltype(map(std::declval<value_type>()));
std::vector<return_type> result{};
for (auto i : seq | std::views::filter(filt)
| std::views::transform(map)) result.push_back(i);
return typon::PyList(std::move(result));
}
#endif // TYPON_BUILTINS_HPP
rt/include/python/builtins/bool.hpp → include/python/builtins/bool.hpp
View file @ 81845140
//
// Created by Tom on 08/03/2023.
//
#ifndef TYPON_BOOL_HPP
#define TYPON_BOOL_HPP
#include <ostream>
#include "print.hpp"
template <> void print_to<bool>(const bool &x, std::ostream &s) {
s << (x ? "True" : "False");
}
#endif // TYPON_BOOL_HPP
//
// Created by Tom on 08/03/2023.
//
#ifndef TYPON_BOOL_HPP
#define TYPON_BOOL_HPP
#include <ostream>
#include "print.hpp"
template <> void print_to<bool>(const bool &x, std::ostream &s) {
s << (x ? "True" : "False");
}
#endif // TYPON_BOOL_HPP
rt/include/python/builtins/complex.hpp → include/python/builtins/complex.hpp
View file @ 81845140
//
// Created by Tom on 08/03/2023.
//
#ifndef TYPON_COMPLEX_HPP
#define TYPON_COMPLEX_HPP
#include <complex>
#include <ostream>
#include "print.hpp"
using PyComplex = std::complex<double>;
PyComplex operator+(int a, const PyComplex &b) { return PyComplex(a) + b; }
PyComplex operator-(int a, const PyComplex &b) { return PyComplex(a) - b; }
template <> void print_to<PyComplex>(const PyComplex &x, std::ostream &s) {
if (x.real() == 0) {
s << x.imag() << "j";
} else {
s << '(' << x.real() << "+" << x.imag() << "j)";
}
}
#endif // TYPON_COMPLEX_HPP
//
// Created by Tom on 08/03/2023.
//
#ifndef TYPON_COMPLEX_HPP
#define TYPON_COMPLEX_HPP
#include <complex>
#include <ostream>
#include "print.hpp"
using PyComplex = std::complex<double>;
PyComplex operator+(int a, const PyComplex &b) { return PyComplex(a) + b; }
PyComplex operator-(int a, const PyComplex &b) { return PyComplex(a) - b; }
template <> void print_to<PyComplex>(const PyComplex &x, std::ostream &s) {
if (x.real() == 0) {
s << x.imag() << "j";
} else {
s << '(' << x.real() << "+" << x.imag() << "j)";
}
}
#endif // TYPON_COMPLEX_HPP
rt/include/python/builtins/dict.hpp → include/python/builtins/dict.hpp
View file @ 81845140
//
// Created by Tom on 08/03/2023.
//
#ifndef TYPON_DICT_HPP
#define TYPON_DICT_HPP
#include <unordered_map>
/*
template <typename K, typename V>
class PyDict : public std::unordered_map<K, V> {
public:
PyDict(std::unordered_map<K, V> &&m)
: std::unordered_map<K, V>(std::move(m)) {}
PyDict(std::initializer_list<std::pair<const K, V>> m)
: std::unordered_map<K, V>(m) {}
operator std::unordered_map<K, V>() const {
return std::unordered_map<K, V>(this->begin(), this->end());
}
operator std::unordered_map<K, V> &() {
return *reinterpret_cast<std::unordered_map<K, V> *>(this);
}
std::size_t py_len() const { return this->size(); }
bool py_contains(const K &k) const { return this->find(k) != this->end(); }
class iterator {
public:
using value_type = std::pair<K, V>;
using difference_type = std::ptrdiff_t;
using pointer = std::pair<K, V> *;
using reference = std::pair<K, V> &;
using iterator_category = std::forward_iterator_tag;
iterator(typename std::unordered_map<K, V>::iterator it) : _it(it) {}
iterator &operator++() {
_it++;
return *this;
}
iterator operator++(int) {
iterator tmp = *this;
_it++;
return tmp;
}
bool operator==(const iterator &rhs) const { return _it == rhs._it; }
bool operator!=(const iterator &rhs) const { return _it != rhs._it; }
const std::pair<K, V> &operator*() const { return *_it; }
const std::pair<K, V> *operator->() const { return &*_it; }
private:
typename std::unordered_map<K, V>::iterator _it;
};
iterator py_iter() const { return this->begin(); }
void py_print(std::ostream &s) const {
s << '{';
if (this->size() > 0) {
print_to(this->begin()->first, s);
s << ": ";
print_to(this->begin()->second, s);
for (auto it = ++this->begin(); it != this->end(); it++) {
s << ", ";
print_to(it->first, s);
s << ": ";
print_to(it->second, s);
}
}
s << '}';
}
};
template <typename K, typename V>
PyDict(std::initializer_list<std::pair<K, V>>) -> PyDict<K, V>;*/
namespace typon {
template <typename K, typename V> class PyDict {
public:
PyDict(std::shared_ptr<std::unordered_map<K, V>> &&m) : _m(std::move(m)) {}
PyDict(std::unordered_map<K, V> &&m)
: _m(std::move(
std::make_shared<std::unordered_map<K, V>>(std::move(m)))) {}
PyDict(std::initializer_list<std::pair<const K, V>> &&m)
: _m(std::make_shared<std::unordered_map<K, V>>(std::move(m))) {}
PyDict() : _m(std::make_shared<std::unordered_map<K, V>>()) {}
template <typename... Args>
PyDict(Args &&...args)
: _m(std::make_shared<std::unordered_map<K, V>>(
std::forward<Args>(args)...)) {}
auto begin() const { return _m->begin(); }
auto end() const { return _m->end(); }
auto py_contains(const K &k) const { return _m->find(k) != _m->end(); }
constexpr const V &operator[](const K &k) const { return (*_m)[k]; }
constexpr V &operator[](const K &k) { return (*_m)[k]; }
size_t py_len() const { return _m->size(); }
void py_repr(std::ostream &s) const {
s << '{';
if (_m->size() > 0) {
repr_to(_m->begin()->first, s);
s << ": ";
repr_to(_m->begin()->second, s);
for (auto it = ++_m->begin(); it != _m->end(); it++) {
s << ", ";
repr_to(it->first, s);
s << ": ";
repr_to(it->second, s);
}
}
s << '}';
}
void py_print(std::ostream &s) const { py_repr(s); }
private:
std::shared_ptr<std::unordered_map<K, V>> _m;
};
}
#endif // TYPON_DICT_HPP
//
// Created by Tom on 08/03/2023.
//
#ifndef TYPON_DICT_HPP
#define TYPON_DICT_HPP
#include <unordered_map>
/*
template <typename K, typename V>
class PyDict : public std::unordered_map<K, V> {
public:
PyDict(std::unordered_map<K, V> &&m)
: std::unordered_map<K, V>(std::move(m)) {}
PyDict(std::initializer_list<std::pair<const K, V>> m)
: std::unordered_map<K, V>(m) {}
operator std::unordered_map<K, V>() const {
return std::unordered_map<K, V>(this->begin(), this->end());
}
operator std::unordered_map<K, V> &() {
return *reinterpret_cast<std::unordered_map<K, V> *>(this);
}
std::size_t py_len() const { return this->size(); }
bool py_contains(const K &k) const { return this->find(k) != this->end(); }
class iterator {
public:
using value_type = std::pair<K, V>;
using difference_type = std::ptrdiff_t;
using pointer = std::pair<K, V> *;
using reference = std::pair<K, V> &;
using iterator_category = std::forward_iterator_tag;
iterator(typename std::unordered_map<K, V>::iterator it) : _it(it) {}
iterator &operator++() {
_it++;
return *this;
}
iterator operator++(int) {
iterator tmp = *this;
_it++;
return tmp;
}
bool operator==(const iterator &rhs) const { return _it == rhs._it; }
bool operator!=(const iterator &rhs) const { return _it != rhs._it; }
const std::pair<K, V> &operator*() const { return *_it; }
const std::pair<K, V> *operator->() const { return &*_it; }
private:
typename std::unordered_map<K, V>::iterator _it;
};
iterator py_iter() const { return this->begin(); }
void py_print(std::ostream &s) const {
s << '{';
if (this->size() > 0) {
print_to(this->begin()->first, s);
s << ": ";
print_to(this->begin()->second, s);
for (auto it = ++this->begin(); it != this->end(); it++) {
s << ", ";
print_to(it->first, s);
s << ": ";
print_to(it->second, s);
}
}
s << '}';
}
};
template <typename K, typename V>
PyDict(std::initializer_list<std::pair<K, V>>) -> PyDict<K, V>;*/
namespace typon {
template <typename K, typename V> class PyDict {
public:
PyDict(std::shared_ptr<std::unordered_map<K, V>> &&m) : _m(std::move(m)) {}
PyDict(std::unordered_map<K, V> &&m)
: _m(std::move(
std::make_shared<std::unordered_map<K, V>>(std::move(m)))) {}
PyDict(std::initializer_list<std::pair<const K, V>> &&m)
: _m(std::make_shared<std::unordered_map<K, V>>(std::move(m))) {}
PyDict() : _m(std::make_shared<std::unordered_map<K, V>>()) {}
template <typename... Args>
PyDict(Args &&...args)
: _m(std::make_shared<std::unordered_map<K, V>>(
std::forward<Args>(args)...)) {}
auto begin() const { return _m->begin(); }
auto end() const { return _m->end(); }
auto py_contains(const K &k) const { return _m->find(k) != _m->end(); }
constexpr const V &operator[](const K &k) const { return (*_m)[k]; }
constexpr V &operator[](const K &k) { return (*_m)[k]; }
size_t py_len() const { return _m->size(); }
void py_repr(std::ostream &s) const {
s << '{';
if (_m->size() > 0) {
repr_to(_m->begin()->first, s);
s << ": ";
repr_to(_m->begin()->second, s);
for (auto it = ++_m->begin(); it != _m->end(); it++) {
s << ", ";
repr_to(it->first, s);
s << ": ";
repr_to(it->second, s);
}
}
s << '}';
}
void py_print(std::ostream &s) const { py_repr(s); }
private:
std::shared_ptr<std::unordered_map<K, V>> _m;
};
}
#endif // TYPON_DICT_HPP
rt/include/python/builtins/list.hpp → include/python/builtins/list.hpp
View file @ 81845140
//
// Created by Tom on 08/03/2023.
//
#ifndef TYPON_LIST_HPP
#define TYPON_LIST_HPP
#include <algorithm>
#include <ostream>
#include <vector>
//#include <nanobind/stl/detail/nb_list.h>
#include <pybind11/stl.h>
namespace typon {
template <typename T> class PyList {
public:
using value_type = T;
PyList(std::shared_ptr<std::vector<T>> &&v) : _v(std::move(v)) {}
PyList(std::vector<T> &&v)
: _v(std::move(std::make_shared<std::vector<T>>(std::move(v)))) {}
PyList(std::initializer_list<T> &&v)
: _v(std::make_shared<std::vector<T>>(std::move(v))) {}
PyList() : _v(std::make_shared<std::vector<T>>()) {}
auto begin() const { return _v->begin(); }
auto end() const { return _v->end(); }
METHOD(auto, append, (Self self, const T &x), {
self._v->push_back(x);
})
METHOD(auto, py_contains, (Self self, const T &x), {
return std::find(self.begin(), self.end(), x) != self.end();
})
auto size() const { return _v->size(); }
void push_back(const T &value) { _v->push_back(value); }
void clear() { _v->clear(); }
/*operator std::vector<T>() const {
return std::vector<T>(this->begin(), this->end());
}
operator std::vector<T> &() {
return *reinterpret_cast<std::vector<T> *>(this);
}*/
constexpr const T &operator[](size_t i) const { return _v->operator[](i); }
constexpr T &operator[](size_t i) { return _v->operator[](i); }
size_t py_len() const { return _v->size(); }
void py_repr(std::ostream &s) const {
s << '[';
if (_v->size() > 0) {
repr_to(_v->operator[](0), s);
for (size_t i = 1; i < _v->size(); i++) {
s << ", ";
repr_to(_v->operator[](i), s);
}
}
s << ']';
}
void py_print(std::ostream &s) const { py_repr(s); }
PyList<T> operator+(const PyList<T> &other) const {
std::vector<T> v;
v.reserve(_v->size() + other._v->size());
v.insert(v.end(), _v->begin(), _v->end());
v.insert(v.end(), other._v->begin(), other._v->end());
return PyList<T>(std::move(v));
}
PyList<T> operator*(size_t n) const {
PyList<T> v{};
v._v->reserve(this->_v->size() * n);
for (size_t i = 0; i < n; i++) {
v._v->insert(v._v->end(), this->_v->begin(), this->_v->end());
}
return v;
}
private:
std::shared_ptr<std::vector<T>> _v;
};
}
template <typename T> typon::PyList<T> list(std::initializer_list<T> &&v) {
return typon::PyList<T>(std::move(v));
}
namespace PYBIND11_NAMESPACE {
namespace detail {
template <typename Type>
struct type_caster<typon::PyList<Type>> : list_caster<typon::PyList<Type>, Type> {};
}}
/*NAMESPACE_BEGIN(NB_NAMESPACE)
NAMESPACE_BEGIN(detail)
template <typename Type> struct type_caster<PyList<Type>>
: list_caster<PyList<Type>, Type> { };
NAMESPACE_END(detail)
NAMESPACE_END(NB_NAMESPACE)
*/
#endif // TYPON_LIST_HPP
//
// Created by Tom on 08/03/2023.
//
#ifndef TYPON_LIST_HPP
#define TYPON_LIST_HPP
#include <algorithm>
#include <ostream>
#include <vector>
//#include <nanobind/stl/detail/nb_list.h>
#include <pybind11/stl.h>
namespace typon {
template <typename T> class PyList {
public:
using value_type = T;
PyList(std::shared_ptr<std::vector<T>> &&v) : _v(std::move(v)) {}
PyList(std::vector<T> &&v)
: _v(std::move(std::make_shared<std::vector<T>>(std::move(v)))) {}
PyList(std::initializer_list<T> &&v)
: _v(std::make_shared<std::vector<T>>(std::move(v))) {}
PyList() : _v(std::make_shared<std::vector<T>>()) {}
auto begin() const { return _v->begin(); }
auto end() const { return _v->end(); }
METHOD(auto, append, (Self self, const T &x), {
self._v->push_back(x);
})
METHOD(auto, py_contains, (Self self, const T &x), {
return std::find(self.begin(), self.end(), x) != self.end();
})
auto size() const { return _v->size(); }
void push_back(const T &value) { _v->push_back(value); }
void clear() { _v->clear(); }
/*operator std::vector<T>() const {
return std::vector<T>(this->begin(), this->end());
}
operator std::vector<T> &() {
return *reinterpret_cast<std::vector<T> *>(this);
}*/
constexpr const T &operator[](size_t i) const { return _v->operator[](i); }
constexpr T &operator[](size_t i) { return _v->operator[](i); }
size_t py_len() const { return _v->size(); }
void py_repr(std::ostream &s) const {
s << '[';
if (_v->size() > 0) {
repr_to(_v->operator[](0), s);
for (size_t i = 1; i < _v->size(); i++) {
s << ", ";
repr_to(_v->operator[](i), s);
}
}
s << ']';
}
void py_print(std::ostream &s) const { py_repr(s); }
PyList<T> operator+(const PyList<T> &other) const {
std::vector<T> v;
v.reserve(_v->size() + other._v->size());
v.insert(v.end(), _v->begin(), _v->end());
v.insert(v.end(), other._v->begin(), other._v->end());
return PyList<T>(std::move(v));
}
PyList<T> operator*(size_t n) const {
PyList<T> v{};
v._v->reserve(this->_v->size() * n);
for (size_t i = 0; i < n; i++) {
v._v->insert(v._v->end(), this->_v->begin(), this->_v->end());
}
return v;
}
private:
std::shared_ptr<std::vector<T>> _v;
};
}
template <typename T> typon::PyList<T> list(std::initializer_list<T> &&v) {
return typon::PyList<T>(std::move(v));
}
namespace PYBIND11_NAMESPACE {
namespace detail {
template <typename Type>
struct type_caster<typon::PyList<Type>> : list_caster<typon::PyList<Type>, Type> {};
}}
/*NAMESPACE_BEGIN(NB_NAMESPACE)
NAMESPACE_BEGIN(detail)
template <typename Type> struct type_caster<PyList<Type>>
: list_caster<PyList<Type>, Type> { };
NAMESPACE_END(detail)
NAMESPACE_END(NB_NAMESPACE)
*/
#endif // TYPON_LIST_HPP
rt/include/python/builtins/print.hpp → include/python/builtins/print.hpp
View file @ 81845140
//
// Created by Tom on 09/03/2023.
//
#ifndef TYPON_PRINT_HPP
#define TYPON_PRINT_HPP
#include <iostream>
#include <ostream>
#include <functional>
#include "str.hpp"
#include <typon/typon.hpp>
template <typename T>
concept Streamable = requires(const T &x, std::ostream &s) {
{ s << x } -> std::same_as<std::ostream &>;
};
template <Streamable T> void print_to(const T &x, std::ostream &s) { s << x; }
template <Streamable T> void repr_to(const T &x, std::ostream &s) { s << x; }
template <typename T>
concept PyPrint = requires(const T &x, std::ostream &s) {
{ x.py_print(s) } -> std::same_as<void>;
};
template <PyPrint T> void print_to(const T &x, std::ostream &s) {
x.py_print(s);
}
template <typename T>
concept PyRepr = requires(const T &x, std::ostream &s) {
{ x.py_repr(s) } -> std::same_as<void>;
};
template <PyRepr T> void repr_to(const T &x, std::ostream &s) { x.py_repr(s); }
template <typename T>
concept Printable = requires(const T &x, std::ostream &s) {
{ print_to(x, s) } -> std::same_as<void>;
};
template <typename T>
concept Reprable = requires(const T &x, std::ostream &s) {
{ repr_to(x, s) } -> std::same_as<void>;
};
template <typename T>
concept FunctionPointer =
std::is_function_v<T> or std::is_member_function_pointer_v<T> or
std::is_function_v<std::remove_pointer_t<T>>;
template <Streamable T>
requires(FunctionPointer<T>)
void repr_to(const T &x, std::ostream &s) {
s << "<function at 0x" << std::hex << (size_t)x << std::dec << ">";
}
template <typename T>
void repr_to(const std::function<T> &x, std::ostream &s) {
s << "<function at 0x" << std::hex << (size_t)x.template target<T*>() << std::dec
<< ">";
}
template <> void repr_to(const PyStr &x, std::ostream &s) {
s << '"' << x << '"';
}
template <Streamable T>
requires(Reprable<T>)
void print_to(const T &x, std::ostream &s) {
repr_to(x, s);
}
template <Printable T>
void print_to(const std::shared_ptr<T> &x, std::ostream &s) {
print_to(*x, s);
}
template <Reprable T>
void repr_to(const std::shared_ptr<T> &x, std::ostream &s) {
repr_to(*x, s);
}
template <> void print_to<PyStr>(const PyStr &x, std::ostream &s) { s << x; }
/*
template <Printable T, Printable... Args>
typon::Task<void> print(T const &head, Args const &...args) {
print_to(head, std::cout);
(((std::cout << ' '), print_to(args, std::cout)), ...);
std::cout << '\n'; co_return;
}*/
struct {
void operator()() { std::cout << '\n'; }
template <Printable T, Printable... Args>
void operator()(T const &head, Args const &...args) {
print_to(head, std::cout);
(((std::cout << ' '), print_to(args, std::cout)), ...);
std::cout << '\n';
}
} print;
// typon::Task<void> print() { std::cout << '\n'; co_return; }
struct {
PyStr operator()(const PyStr& s = ""_ps) {
std::cout << s;
PyStr input;
std::getline(std::cin, input);
return input;
}
} input;
#endif // TYPON_PRINT_HPP
//
// Created by Tom on 09/03/2023.
//
#ifndef TYPON_PRINT_HPP
#define TYPON_PRINT_HPP
#include <iostream>
#include <ostream>
#include <functional>
#include "str.hpp"
#include <typon/typon.hpp>
template <typename T>
concept Streamable = requires(const T &x, std::ostream &s) {
{ s << x } -> std::same_as<std::ostream &>;
};
template <Streamable T> void print_to(const T &x, std::ostream &s) { s << x; }
template <Streamable T> void repr_to(const T &x, std::ostream &s) { s << x; }
template <typename T>
concept PyPrint = requires(const T &x, std::ostream &s) {
{ x.py_print(s) } -> std::same_as<void>;
};
template <PyPrint T> void print_to(const T &x, std::ostream &s) {
x.py_print(s);
}
template <typename T>
concept PyRepr = requires(const T &x, std::ostream &s) {
{ x.py_repr(s) } -> std::same_as<void>;
};
template <PyRepr T> void repr_to(const T &x, std::ostream &s) { x.py_repr(s); }
template <typename T>
concept Printable = requires(const T &x, std::ostream &s) {
{ print_to(x, s) } -> std::same_as<void>;
};
template <typename T>
concept Reprable = requires(const T &x, std::ostream &s) {
{ repr_to(x, s) } -> std::same_as<void>;
};
template <typename T>
concept FunctionPointer =
std::is_function_v<T> or std::is_member_function_pointer_v<T> or
std::is_function_v<std::remove_pointer_t<T>>;
template <Streamable T>
requires(FunctionPointer<T>)
void repr_to(const T &x, std::ostream &s) {
s << "<function at 0x" << std::hex << (size_t)x << std::dec << ">";
}
template <typename T>
void repr_to(const std::function<T> &x, std::ostream &s) {
s << "<function at 0x" << std::hex << (size_t)x.template target<T*>() << std::dec
<< ">";
}
template <> void repr_to(const PyStr &x, std::ostream &s) {
s << '"' << x << '"';
}
template <Streamable T>
requires(Reprable<T>)
void print_to(const T &x, std::ostream &s) {
repr_to(x, s);
}
template <Printable T>
void print_to(const std::shared_ptr<T> &x, std::ostream &s) {
print_to(*x, s);
}
template <Reprable T>
void repr_to(const std::shared_ptr<T> &x, std::ostream &s) {
repr_to(*x, s);
}
template <> void print_to<PyStr>(const PyStr &x, std::ostream &s) { s << x; }
/*
template <Printable T, Printable... Args>
typon::Task<void> print(T const &head, Args const &...args) {
print_to(head, std::cout);
(((std::cout << ' '), print_to(args, std::cout)), ...);
std::cout << '\n'; co_return;
}*/
struct {
void operator()() { std::cout << '\n'; }
template <Printable T, Printable... Args>
void operator()(T const &head, Args const &...args) {
print_to(head, std::cout);
(((std::cout << ' '), print_to(args, std::cout)), ...);
std::cout << '\n';
}
} print;
// typon::Task<void> print() { std::cout << '\n'; co_return; }
struct {
PyStr operator()(const PyStr& s = ""_ps) {
std::cout << s;
PyStr input;
std::getline(std::cin, input);
return input;
}
} input;
#endif // TYPON_PRINT_HPP
rt/include/python/builtins/range.hpp → include/python/builtins/range.hpp
View file @ 81845140
//
// Created by Tom on 13/03/2023.
//
#ifndef TYPON_RANGE_HPP
#define TYPON_RANGE_HPP
#include <ranges>
namespace view = std::views;
#include <python/basedef.hpp>
auto stride = [](int n) {
return [s = -1, n](auto const&) mutable { s = (s + 1) % n; return !s; };
};
// todo: proper range support
struct range_s : TyBuiltin<range_s>
{
template <typename T>
auto sync(T stop) { return sync(0, stop); }
template <typename T>
auto sync(T start, T stop, T step = 1) {
// https://www.modernescpp.com/index.php/c-20-pythons-map-function/
if(step == 0) {
throw std::invalid_argument("Step cannot be 0");
}
auto Step = start < stop ? step : -step;
auto Begin = std::min(start, stop);
auto End = Step < 0 ? Begin : std::max(start, stop);
return view::iota(Begin, End)
| view::filter(stride(std::abs(Step)))
| view::transform([start, stop](std::size_t i){
return start < stop ? i : stop - (i - start);
});
}
} range;
#endif // TYPON_RANGE_HPP
//
// Created by Tom on 13/03/2023.
//
#ifndef TYPON_RANGE_HPP
#define TYPON_RANGE_HPP
#include <ranges>
namespace view = std::views;
#include <python/basedef.hpp>
auto stride = [](int n) {
return [s = -1, n](auto const&) mutable { s = (s + 1) % n; return !s; };
};
// todo: proper range support
struct range_s : TyBuiltin<range_s>
{
template <typename T>
auto sync(T stop) { return sync(0, stop); }
template <typename T>
auto sync(T start, T stop, T step = 1) {
// https://www.modernescpp.com/index.php/c-20-pythons-map-function/
if(step == 0) {
throw std::invalid_argument("Step cannot be 0");
}
auto Step = start < stop ? step : -step;
auto Begin = std::min(start, stop);
auto End = Step < 0 ? Begin : std::max(start, stop);
return view::iota(Begin, End)
| view::filter(stride(std::abs(Step)))
| view::transform([start, stop](std::size_t i){
return start < stop ? i : stop - (i - start);
});
}
} range;
#endif // TYPON_RANGE_HPP
rt/include/python/builtins/set.hpp → include/python/builtins/set.hpp
View file @ 81845140
//
// Created by Tom on 08/03/2023.
//
#ifndef TYPON_SET_HPP
#define TYPON_SET_HPP
#include <unordered_set>
namespace typon {
template <typename T> class PySet : public std::unordered_set<T> {
public:
PySet(std::unordered_set<T> &&s) : std::unordered_set<T>(std::move(s)) {}
PySet(std::initializer_list<T> &&s) : std::unordered_set<T>(std::move(s)) {}
operator std::unordered_set<T>() const {
return std::unordered_set<T>(this->begin(), this->end());
}
std::size_t py_len() const { return this->size(); }
bool py_contains(const T &t) const { return this->find(t) != this->end(); }
class iterator {
public:
using value_type = T;
using difference_type = std::ptrdiff_t;
using pointer = T *;
using reference = T &;
using iterator_category = std::forward_iterator_tag;
iterator(typename std::unordered_set<T>::iterator it) : _it(it) {}
iterator &operator++() {
_it++;
return *this;
}
iterator operator++(int) {
iterator tmp = *this;
_it++;
return tmp;
}
bool operator==(const iterator &rhs) const { return _it == rhs._it; }
bool operator!=(const iterator &rhs) const { return _it != rhs._it; }
const T &operator*() const { return *_it; }
const T *operator->() const { return &*_it; }
private:
typename std::unordered_set<T>::iterator _it;
};
iterator py_iter() const { return this->begin(); }
void py_print(std::ostream &s) const {
s << '{';
if (this->size() > 0) {
print_to(*this->begin(), s);
for (auto it = ++this->begin(); it != this->end(); it++) {
s << ", ";
print_to(*it, s);
}
}
s << '}';
}
};
}
template <typename T> typon::PySet<T> set(std::initializer_list<T> &&s) {
return typon::PySet<T>(std::move(s));
}
#endif // TYPON_SET_HPP
//
// Created by Tom on 08/03/2023.
//
#ifndef TYPON_SET_HPP
#define TYPON_SET_HPP
#include <unordered_set>
namespace typon {
template <typename T> class PySet : public std::unordered_set<T> {
public:
PySet(std::unordered_set<T> &&s) : std::unordered_set<T>(std::move(s)) {}
PySet(std::initializer_list<T> &&s) : std::unordered_set<T>(std::move(s)) {}
operator std::unordered_set<T>() const {
return std::unordered_set<T>(this->begin(), this->end());
}
std::size_t py_len() const { return this->size(); }
bool py_contains(const T &t) const { return this->find(t) != this->end(); }
class iterator {
public:
using value_type = T;
using difference_type = std::ptrdiff_t;
using pointer = T *;
using reference = T &;
using iterator_category = std::forward_iterator_tag;
iterator(typename std::unordered_set<T>::iterator it) : _it(it) {}
iterator &operator++() {
_it++;
return *this;
}
iterator operator++(int) {
iterator tmp = *this;
_it++;
return tmp;
}
bool operator==(const iterator &rhs) const { return _it == rhs._it; }
bool operator!=(const iterator &rhs) const { return _it != rhs._it; }
const T &operator*() const { return *_it; }
const T *operator->() const { return &*_it; }
private:
typename std::unordered_set<T>::iterator _it;
};
iterator py_iter() const { return this->begin(); }
void py_print(std::ostream &s) const {
s << '{';
if (this->size() > 0) {
print_to(*this->begin(), s);
for (auto it = ++this->begin(); it != this->end(); it++) {
s << ", ";
print_to(*it, s);
}
}
s << '}';
}
};
}
template <typename T> typon::PySet<T> set(std::initializer_list<T> &&s) {
return typon::PySet<T>(std::move(s));
}
#endif // TYPON_SET_HPP
rt/include/python/builtins/str.hpp → include/python/builtins/str.hpp
View file @ 81845140
//
// Created by Tom on 08/03/2023.
//
#ifndef TYPON_STR_HPP
#define TYPON_STR_HPP
#include <sstream>
#include <string>
#include <algorithm>
using namespace std::literals;
#include "bytes.hpp"
#include "print.hpp"
#include "slice.hpp"
// #include <format>
#include <fmt/format.h>
class PyStr : public std::string {
public:
PyStr() : std::string() {}
PyStr(const std::string &s) : std::string(s) {}
PyStr(std::string &&s) : std::string(std::move(s)) {}
constexpr PyStr(const char *s, size_t count) : std::string(s, count) {}
constexpr PyStr(size_t count, char ch) : std::string(count, ch) {}
template<typename... Args>
PyStr(Args&&... args) : std::string(std::forward<Args>(args)...) {}
template <class InputIterator>
PyStr(InputIterator first, InputIterator last) : std::string(first, last) {}
METHOD(PyBytes, encode, (Self self, const std::string &encoding = "utf-8"), {
return PyBytes(self.begin(), self.end());
})
METHOD_GEN((typename... T), PyStr, format, (Self self, T &&...args), {
return PyStr(fmt::format(fmt::runtime(self), std::forward<T>(args)...));
})
METHOD(bool, startswith, (Self self, const std::string &s), {
return self.starts_with(s);
})
METHOD(int, find, (Self self, const std::string &s), {
auto pos = self.std::string::find(s);
return pos == std::string::npos ? -1 : pos;
})
METHOD(bool, isspace, (Self self), {
return self.find_first_not_of(' ') == std::string::npos;
})
METHOD(auto, py_contains, (Self self, const std::string &x), {
return self.std::string::find(x) != std::string::npos;
})
PyStr operator[](PySlice slice) const {
auto [len, new_slice] = slice.adjust_indices(this->size());
PyStr result;
result.reserve(len);
if (new_slice.start < new_slice.stop) {
if (new_slice.step > 0) {
for (auto i = new_slice.start; i < new_slice.stop;
i += new_slice.step) {
result.push_back(this->char_at(i));
}
}
} else {
if (new_slice.step < 0) {
for (auto i = new_slice.start; i > new_slice.stop;
i += new_slice.step) {
result.push_back(this->char_at(i));
}
}
}
return result;
}
PyStr operator[](ssize_t index) const {
if (index < 0) {
index += this->size();
}
return PyStr(1, std::string::operator[](index));
}
char char_at(ssize_t index) const {
if (index < 0) {
index += this->size();
}
return this->begin()[index];
}
operator int() const {
return std::stoi(*this);
}
};
inline constexpr PyStr operator""_ps(const char *s, size_t len) noexcept {
return PyStr(s, len);
}
template<typename Self>
PyStr PyBytes::decode_s::operator()(Self self, const std::string &encoding) const {
return PyStr(self.begin(), self.end());
}
template <typename T> PyStr str(const T &x) {
std::stringstream s;
print_to(x, s);
return s.str();
}
template <typename T> PyStr repr(const T &x) {
std::stringstream s;
repr_to(x, s);
return s.str();
}
template <> struct std::hash<PyStr> {
std::size_t operator()(const PyStr &s) const noexcept {
return std::hash<std::string>()(s);
}
};
#endif // TYPON_STR_HPP
//
// Created by Tom on 08/03/2023.
//
#ifndef TYPON_STR_HPP
#define TYPON_STR_HPP
#include <sstream>
#include <string>
#include <algorithm>
using namespace std::literals;
#include "bytes.hpp"
#include "print.hpp"
#include "slice.hpp"
// #include <format>
#include <fmt/format.h>
class PyStr : public std::string {
public:
PyStr() : std::string() {}
PyStr(const std::string &s) : std::string(s) {}
PyStr(std::string &&s) : std::string(std::move(s)) {}
constexpr PyStr(const char *s, size_t count) : std::string(s, count) {}
constexpr PyStr(size_t count, char ch) : std::string(count, ch) {}
template<typename... Args>
PyStr(Args&&... args) : std::string(std::forward<Args>(args)...) {}
template <class InputIterator>
PyStr(InputIterator first, InputIterator last) : std::string(first, last) {}
METHOD(PyBytes, encode, (Self self, const std::string &encoding = "utf-8"), {
return PyBytes(self.begin(), self.end());
})
METHOD_GEN((typename... T), PyStr, format, (Self self, T &&...args), {
return PyStr(fmt::format(fmt::runtime(self), std::forward<T>(args)...));
})
METHOD(bool, startswith, (Self self, const std::string &s), {
return self.starts_with(s);
})
METHOD(int, find, (Self self, const std::string &s), {
auto pos = self.std::string::find(s);
return pos == std::string::npos ? -1 : pos;
})
METHOD(bool, isspace, (Self self), {
return self.find_first_not_of(' ') == std::string::npos;
})
METHOD(auto, py_contains, (Self self, const std::string &x), {
return self.std::string::find(x) != std::string::npos;
})
PyStr operator[](PySlice slice) const {
auto [len, new_slice] = slice.adjust_indices(this->size());
PyStr result;
result.reserve(len);
if (new_slice.start < new_slice.stop) {
if (new_slice.step > 0) {
for (auto i = new_slice.start; i < new_slice.stop;
i += new_slice.step) {
result.push_back(this->char_at(i));
}
}
} else {
if (new_slice.step < 0) {
for (auto i = new_slice.start; i > new_slice.stop;
i += new_slice.step) {
result.push_back(this->char_at(i));
}
}
}
return result;
}
PyStr operator[](ssize_t index) const {
if (index < 0) {
index += this->size();
}
return PyStr(1, std::string::operator[](index));
}
char char_at(ssize_t index) const {
if (index < 0) {
index += this->size();
}
return this->begin()[index];
}
operator int() const {
return std::stoi(*this);
}
};
inline constexpr PyStr operator""_ps(const char *s, size_t len) noexcept {
return PyStr(s, len);
}
template<typename Self>
PyStr PyBytes::decode_s::operator()(Self self, const std::string &encoding) const {
return PyStr(self.begin(), self.end());
}
template <typename T> PyStr str(const T &x) {
std::stringstream s;
print_to(x, s);
return s.str();
}
template <typename T> PyStr repr(const T &x) {
std::stringstream s;
repr_to(x, s);
return s.str();
}
template <> struct std::hash<PyStr> {
std::size_t operator()(const PyStr &s) const noexcept {
return std::hash<std::string>()(s);
}
};
#endif // TYPON_STR_HPP
rt/include/python/sys.hpp → include/python/sys.hpp
View file @ 81845140
//
// Created by Tom on 09/03/2023.
//
#ifndef TYPON_SYS_HPP
#define TYPON_SYS_HPP
#include <iostream>
#include "builtins.hpp"
namespace py_sys {
struct sys_t {
static constexpr auto &stdin = std::cin;
static constexpr auto &stdout = std::cout;
static constexpr auto &stderr = std::cerr;
typon::PyList<PyStr> argv;
FUNCTION(void, exit, (int code), {
std::exit(code);
})
} all;
auto& get_all() {
return all;
}
} // namespace py_sys
#endif // TYPON_SYS_HPP
//
// Created by Tom on 09/03/2023.
//
#ifndef TYPON_SYS_HPP
#define TYPON_SYS_HPP
#include <iostream>
#include "builtins.hpp"
namespace py_sys {
struct sys_t {
static constexpr auto &stdin = std::cin;
static constexpr auto &stdout = std::cout;
static constexpr auto &stderr = std::cerr;
typon::PyList<PyStr> argv;
FUNCTION(void, exit, (int code), {
std::exit(code);
})
} all;
auto& get_all() {
return all;
}
} // namespace py_sys
#endif // TYPON_SYS_HPP
runtime @ 9194f47b
Subproject commit 9194f47b6a9fc96b3d77b932033b9e3395be4cb8
trans/__main__.py
View file @ 81845140
......@@ -11,7 +11,8 @@ from pathlib import Path
import sys
compiler_path = Path(__file__).parent
runtime_path = compiler_path.parent / "rt"
stdlib_path = compiler_path.parent / "include"
runtime_path = compiler_path.parent / "runtime" / "rt" / "include"
sys.path.insert(0, str(compiler_path))
......@@ -40,7 +41,8 @@ if args.cpp_flags:
import sysconfig
include_dirs = [
str(runtime_path / "include"),
str(stdlib_path),
str(runtime_path),
sysconfig.get_path("include"),
sysconfig.get_path("platinclude"),
pybind11.commands.get_include()
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7