- 12 Mar, 2024 1 commit
-
-
Titouan Soulard authored
logrotate test for the rapid-cdn SR have been failing for quite some time because of a squid.log file created (and henceforth rotated) on any slave instance of the CDN: ``` Traceback (most recent call last): self.assertEqual( AssertionError: Items in the second set but not the first: 'squid.log_10.0.160.212.20240306.23h57m09s-20240307.00h00m02s.old.xz' ``` Since we do not want to test this file in that specific test, this commit instead loosely check the content of the rotated files directory. In other words, the test now checks for the presence of the two files to be tested, but avoid failing if other files exist. This goes in line with the two following lines of that same test, loosely checking for file absence instead of an empty directory. /cc @tomo @luke /approved-by @luke /reviewed-on !1541
-
- 08 Mar, 2024 3 commits
-
-
Julien Muchembled authored
This saves 380MB, by removing: - bin/mysql_client_test (and the bin/mariadb-client-test symlink) - mysql-test/
-
Ivan Tyagov authored
See merge request !1543
-
Ivan Tyagov authored
OPENSSL's libraries in this case SlapOS ones.
-
- 05 Mar, 2024 1 commit
-
-
Thomas Gambier authored
The SR didn't compile without libssl-dev package. Correctly use the openssl lib from slapos.
-
- 01 Mar, 2024 1 commit
-
-
Jérome Perrin authored
Every restricted python code on python2 will be compiled as if it had `from __future__ import print_function`, to ease transition away from python2. To update project code, 2to3 from python2.7 seems to do a good job. Invoking like from the root of a repository rewrite all scripts: 2to3 --write --nobackups --no-diffs --fix=print .
-
- 29 Feb, 2024 3 commits
-
-
Łukasz Nowak authored
When the backend to redirect to uses scheme standard port it's cleaner to redirect to URL without the port. See https://www.rfc-editor.org/rfc/rfc9110.html#name-https-normalization-and-com: "If the port is equal to the default port for a scheme, the normal form is to omit the port subcomponent."
-
Łukasz Nowak authored
-
Titouan Soulard authored
I forgot to update hashes after commits b6959875 and 7906b01c. This commit solves the problem by updating slapos-master hashes.
-
- 27 Feb, 2024 2 commits
-
-
Titouan Soulard authored
-
Thomas Gambier authored
I used the following commands: autopep8 test_ors.py --select=E101 --ignore=E121 --indent-size=2 --in-place autopep8 test.py --select=E101 --ignore=E121 --indent-size=2 --in-place
-
- 26 Feb, 2024 2 commits
-
-
Thomas Gambier authored
-
Titouan Soulard authored
-
- 22 Feb, 2024 2 commits
-
-
Jérome Perrin authored
This parameter no longer exists, this was not removed correctly
-
Nicolas Wavrant authored
"repozo --verify" is not working as this code expects it to: it simply prints errors in stdout, and doesn't return an error code in case of error. Thus, running it had absolutely no effect, except wasting IO and CPU time. This commit introduces the use of "repozo --recover --with-verify", which runs the verify and the recover in a same step, and has the advantage to raise (it doesn't exit with 0) in case of error. Also, as it does the verification and the recovery at the same time, it uses half the IO for the read. On a production server using SSDs, with a ZODB of 1Tb, runner-import-restore now takes 14h instead of 26h, iow a performance increase of 46%.
-
- 21 Feb, 2024 1 commit
-
-
Rafael Monnerat authored
See merge request nexedi/slapos!1534
-
- 20 Feb, 2024 1 commit
-
-
Rafael Monnerat authored
-
- 19 Feb, 2024 1 commit
-
-
Thomas Gambier authored
This is needed since version up of pim-dm in cfb05d82
-
- 18 Feb, 2024 2 commits
-
-
Kirill Smelkov authored
Hello up there. This merge-request brings in major update to ors-amarisoft software release: first eNB is significantly restructured to prepare base for further changes, and then we add support for working with multiple radio units and multiple cells with all LTE/NR and FDD/TDD simultaneously. All kinds of Carrier Aggregation - LTE+LTE, NR+NR and LTE+NR are now supported. All kinds of Handover - Intra-ENB, Inter-ENB with LTE→NR and NR→LTE are now supported as well. UE simulator is also updated to support multiple radio units, cells and UEs. In the new system configuration of RU, CELL, PEERCELL, PEER and UE objects are done via shared instances attached to the main eNB or UEsim instance. Most of the parameters become runtime settings instead of being static choice of particular software template. There is no longer multiple rendered softwares - all that remain is 1. `software.cfg` for generic software, and 2. `software-ors.cfg` for ORS. Switching to configuring things at runtime became possible because SlapOS Master recently switched to new JSON-editor with support for `oneOf`, arrays and conditionals - bits that make it possible to configure settings in the WEB UI with multiple choices for e.g. RF mode, cell or radio unit type. For ORS full backward compatibility is preserved via special proxy which translates ORS input schema to configuration objects of the new generic eNB. Since most our current ORS deployments are TDD, `software-tdd-ors.cfg` link to `software-ors.cfg` is also provided to preserve backward compatibility at software-release URL level for those instances. eNB and gNB are merged along the way. Unittests are improved. JSON schemas become primary source for defaults(*). Unnecessary parameters are removed and are now computed automatically. For example it is no longer needed to explicitly specify SSB NR-ARFCN for peer NR cell, or `txa0cc00_center_frequency` for Lopcomm RU. `tx_gain` and `rx_gain` become generic parameters that semantically apply uniformly to all Radio Units. A protection against buildout code injection via specially-crafted references of shared instances is installed. The problem was noticed because instantiation was failing with spaces in the references - a condition that is present by default on the testnodes. Solving the problem generally via custom "buildout encoding" was not hard and probably the solution might be useful not only for ors-amarisoft software release. Please see the patch `"Protect from buildout code injection"` for details. There are more minor enhancements and bug fixes in there. Please see individual patches for details. Kirill /cc @jhuge, @lu.xu, @xavier_thompson, @Daetalus /approved-by @tomo /reviewed-on !1533 (*) this goes in line with similar design choice to make JSON schemas primary source of defaults in Rapid-CDN: !1380 .
-
Kirill Smelkov authored
To run tapsplit we use plone.recipe.command with both command and update-command set to `tapsplit ...`. But tapsplit, when run, currently fully recreates and reinitializes subtap interfaces, which leads to interfering with running enb because subtap interfaces, that enb started to use, are removed. This is not desirable behaviour. What we need: 1) create subtap interfaces only once and keep them stable 2) until configuration changes which should lead to * subtaps recreated, and * enb restarted 3) if subtap interfaces disappear for any reason, recreate it -> Rework tapsplit to keep its promise, that it "brings tap interface into state with several children interfaces each covering part of original interface address space", without recreating those children on every run and instead doing any action only if their state is not what is desired. In other words those interfaces now are only created when they do not exist before. Addresses and routes are added only if they are not there before tapsplit is run, etc. After the patch the first run of tapsplit to split by 2 looks like # ./pythonwitheggs ru/tapsplit slaptap16 2 slaptap16: split 2401:5180:0:66:a200::/71 by 2 preserve 2401:5180:0:66:a200::/73 -> slaptap16-1 2401:5180:0:66:a280::/73 # ip tuntap add dev slaptap16-1 mode tap user slapuser16 # ip link set slaptap16-1 up # ip addr add 2401:5180:0:66:a280::/73 dev slaptap16-1 noprefixroute # ip route add 2401:5180:0:66:a280::1 dev slaptap16-1 # ip route add 2401:5180:0:66:a280::/73 dev slaptap16-1 via 2401:5180:0:66:a280::1 -> slaptap16-2 2401:5180:0:66:a300::/73 # ip tuntap add dev slaptap16-2 mode tap user slapuser16 # ip link set slaptap16-2 up # ip addr add 2401:5180:0:66:a300::/73 dev slaptap16-2 noprefixroute # ip route add 2401:5180:0:66:a300::1 dev slaptap16-2 # ip route add 2401:5180:0:66:a300::/73 dev slaptap16-2 via 2401:5180:0:66:a300::1 The second run with the same arguments looks as # ./pythonwitheggs ru/tapsplit slaptap16 2 slaptap16: split 2401:5180:0:66:a200::/71 by 2 preserve 2401:5180:0:66:a200::/73 -> slaptap16-1 2401:5180:0:66:a280::/73 # slaptap16-1: already exists # slaptap16-1: already up # slaptap16-1: already has 2401:5180:0:66:a280::/73 addr # slaptap16-1: already has 2401:5180:0:66:a280::1 route # slaptap16-1: already has 2401:5180:0:66:a280::/73 route -> slaptap16-2 2401:5180:0:66:a300::/73 # slaptap16-2: already exists # slaptap16-2: already up # slaptap16-2: already has 2401:5180:0:66:a300::/73 addr # slaptap16-2: already has 2401:5180:0:66:a300::1 route # slaptap16-2: already has 2401:5180:0:66:a300::/73 route where it could be seen that no actions had been taken. And if, for example, the user manipulates slaptap16-2 and manually sets it down, the third run restores it to desired 'UP' state and readds the address and routes because the kernel removed them when link went down: # ip -6 addr show dev slaptap16-2 157: slaptap16-2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_codel state DOWN group default qlen 1000 inet6 2401:5180:0:66:a300::/73 scope global tentative noprefixroute valid_lft forever preferred_lft forever # ip -6 route show dev slaptap16-2 2401:5180:0:66:a300::1 metric 1024 linkdown pref medium 2401:5180:0:66:a300::/73 via 2401:5180:0:66:a300::1 metric 1024 linkdown pref medium # ip link set slaptap16-2 down # ip -6 addr show dev slaptap16-2 # ip -6 route show dev slaptap16-2 # ./pythonwitheggs ru/tapsplit slaptap16 2 slaptap16: split 2401:5180:0:66:a200::/71 by 2 preserve 2401:5180:0:66:a200::/73 -> slaptap16-1 2401:5180:0:66:a280::/73 # slaptap16-1: already exists # slaptap16-1: already up # slaptap16-1: already has 2401:5180:0:66:a280::/73 addr # slaptap16-1: already has 2401:5180:0:66:a280::1 route # slaptap16-1: already has 2401:5180:0:66:a280::/73 route -> slaptap16-2 2401:5180:0:66:a300::/73 # slaptap16-2: already exists # ip link set slaptap16-2 up # ip addr add 2401:5180:0:66:a300::/73 dev slaptap16-2 noprefixroute # ip route add 2401:5180:0:66:a300::1 dev slaptap16-2 # ip route add 2401:5180:0:66:a300::/73 dev slaptap16-2 via 2401:5180:0:66:a300::1 The first version of this patch tried to solve the problem by setting update-command to be noop instead of reworking tapsplit itself. But as Thomas noted this does not satisfy requirement "3". Amends 49ce8ef5 (software/ors-amarisoft: Provide dedicated TAP interface for each Radio Unit) /helped-by @tomo /cc @jhuge, @lu.xu, @xavier_thompson, @Daetalus /reviewed-on nexedi/slapos!1508
-
- 16 Feb, 2024 11 commits
-
-
Jérome Perrin authored
- use caucase for balancer certificate - move virtual host logic on the backend - change "frontend" parameter to request "" type (and no longer "zope") See merge request nexedi/slapos!1504
-
Jérome Perrin authored
The strategy for compatibility is that: - haproxy still listen on the same port as before, without rewrite rule. This is called "legacy" port. - for each frontend from request parameters, we introduce an haproxy frontend with a rewrite for the corresponding `internal-path` parameter. - the shared frontend instance is updated to use this new frontend entry from haproxy. This will cause a small downtime until the shared frontend is updated to the new URL on ERP5, but since this feature was not used, it's OK. Technical details are that we: - split haproxy config to have frontends and backends. - introduce one frontend in haproxy for each frontend from request parameters. - routing-rule-list argument is still honored the same way, globally and after path from frontend. - change the shared frontend requests to use "" type, no longer "zope" type. - we don't do automatic detection of /VirtualHostRoot in URL but always add it, because it could be used to trick zope into thinking it serves requests for an arbitrary host and do open redirects - before using the request's host header in virtualhost path, we check that it does not contain /, to prevent injection of virutalhost path elements through the host header. - we don't use the "path" parameter from shared frontend, because we want the frontend to be simple, so we don't want it to rewrite the request path (which is also the reason why we deprecated "zope" type) - the tests have changed a lot, because they were using what's now the "legacy" URL types, so we updated it to use the new URL types with all the /VirtualHostRoot/../ in path and also because they use IPv6 URL, no longer IPv4
-
Jérome Perrin authored
-
Jérome Perrin authored
and save the already allocated ports in a state file, so that requesting new families does not change already allocated ports.
-
Jérome Perrin authored
This reverts commit 620c9332 (stack/erp5: stop using caucase managed certificate for balancer, 2020-11-10) with an updated design. We add a caucase service for balancer in the balancer partition. The caucase service from the root partition (that was not used) is removed. The underlying idea is that the default configuration should use multiple caucases with limited scope, here we have one caucase to manage the certificate used by haproxy server in the balancer partition, so we put one caucase to manage this certificate and the caucase is configured to auto-accept one certificate only. The plan is that when we will add a certificate for mariadb server, we'll add another caucase inside this mariadb server. For more advanced usage and also to support the cases where a new certificate needs to be re-emitted for some reason, users can request with an existing caucase URL. In that case, they will have to accept the certificate requests. Notable changes: balancer/ssl/caucase-url is no longer documented in parameters, this is an internal parameter, users can pass one global caucase service to manage all partition CAUCASE environment variable is no longer set when running zope. There was no identified use case and with this new approach of multiple caucases, the term "caucase" alone became ambiguous.
-
Jérome Perrin authored
This is not documented in schema and has no effect in erp5 (but this is still used for slapos-master)
-
Jérome Perrin authored
-
Jérome Perrin authored
This change the format or the (mostly) unused frontend parameter to support requesting more than one frontend and also enable the request of a frontend by default, so that requesting a frontend separately is no longer needed. The `frontend` parameter now also supports requesting frontends for specific paths on the ERP5 backend, the example below requests a frontend serving directly a web site, with the necessary rewrite rules: ```js { "frontend": { "default": { "internal-path": "/erp5/web_site_module/renderjs_runner/" } } } ``` The example below requests a default frontend to the erp5 root, to access the ZMI or erp5_xhtml_style interface and two web sites: ```js { "frontend": { "default": {}, "erp5js": { "internal-path": "/erp5/web_site_module/renderjs_runner/" }, "crm": { "internal-path": "/erp5/web_site_module/erp5_officejs_support_request_ui/" } } } ``` The example below has an explicit definition of the zope families using `zope-partition-dict` parameter, because there is more than one zope family, no frontend is requested by default: ```js { "zope-partition-dict": { "backoffice": { "family": "backoffice" }, "web": { "family": "web" }, "activities": { "family": "activities" } } } ``` Continuing this example, to have frontends for backoffice and web families, the frontend request can specify the families, like it is demonstrated in the example below. In this example, we don't specify an entry for "activities" family, so no frontend will be requested for this family. ```js { "frontend": { "backoffice": { "zope-family": "backoffice" }, "web": { "zope-family": "web", "internal-path": "/erp5/web_site_module/web_site/" } } "zope-partition-dict": { "backoffice": { "family": "backoffice" }, "web": { "family": "web" }, "activities": { "family": "activities" } } } ```
-
Jérome Perrin authored
-
Kirill Smelkov authored
Going 0.140 -> 0.142 introduces the following changes: nexedi/slapos.toolbox@0.140...0.142 Test results: https://erp5js.nexedi.net/#/test_result_module/20240215-D4F3CE96 (all ok)
-
Jérome Perrin authored
-
- 15 Feb, 2024 1 commit
-
-
Thomas Gambier authored
-
- 14 Feb, 2024 1 commit
-
-
Jérome Perrin authored
-
- 13 Feb, 2024 7 commits
-
-
Kirill Smelkov authored
Typos, whitespace changes, etc...
-
Kirill Smelkov authored
Noticed accidentaly: ssb_pos_bitmap is string, not integer - the default should be of the same type.
-
Kirill Smelkov authored
Rework UEsim to be able to work with multiple cells, multiple radio units(*), multiple UE all at the same time. RU, CELLs and UEs are now configured, simiularly to eNB, via shared instances. Add tests. Contrary to ORS don't care about backward compatibility here because currently we have just a few UEsim deployments and migrating them should be easy. Please see added schemas, tests and updates slapos-render-config on how to use the new system. (*) contrary to eNB UEsim does not allow to use one RU for two cells,
-
Kirill Smelkov authored
It should be already generally supported, but many parameters needs in NR case are currently hardcoded to their TDD values. I've adjusted only a few of them and stopped, becuase there is currently no practical case at hand for me to test it. Still I think it makes sense to save this first step. For ORS, who uses TDD, rendered config stays practically the same: ``` $ ./pythonwitheggs slapos-render-config.py && xdiff -C -C config/{old,out}/ors/ ``` ```diff diff --git a/config/old/ors/gnb/enb.cfg b/config/out/ors/gnb/enb.cfg index ead7f0160..9a260d73f 100644 --- a/config/old/ors/gnb/enb.cfg +++ b/config/out/ors/gnb/enb.cfg @@ -97,6 +97,7 @@ prach: { + ra_response_window: 20, }, pdcch: { @@ -180,7 +181,6 @@ preamble_received_target_power: -110, preamble_trans_max: 7, power_ramping_step: 4, - ra_response_window: 20, restricted_set_config: "unrestricted_set", ra_contention_resolution_timer: 64, ssb_per_prach_occasion: 1, ```
-
Kirill Smelkov authored
If a base station has multiple cells, it should be possible for UE to be handed over from one cell into another, for example when UE crosses border of sectors. So far we had only Inter-ENB HO and now we are also adding configuration for HO in between all own cells. Add tests for everything. NOTE: we use allowed_meas_bandwidth, antenna_port_1 in dst=LTE case because otherwise, e.g. for NR->LTE HO if they are not present, lteenb complains: enb.cfg:260: expecting 'allowed_meas_bandwidth' field enb.cfg:260: expecting 'antenna_port_1' field for both Intra-ENB and Inter-ENB handovers. For Intra-ENB case we can compute those numbers from cell definition. For Inter-ENB case, when we don't know target context, we use conservative values.
-
Kirill Smelkov authored
We already had CA for LTE+LTE case. Let's also setup it for NR+NR and LTE+NR cases as well. Add tests for everything.
-
Kirill Smelkov authored
Currently when setting up cells we allow users to input - dl_earfcn for LTE, and - dl_nr_arfcn and nr_band for NR and from that lteenb automatically computes - ul_earfcn for LTE, and - ul_nr_arfcn and ssb_nr_arfcn for NR everything kind of works out of the box for eNB case when there is simple SDR Radio Unit. Then there are also the following cases: 1. we also need to set UL frequency when configuring Lopcomm RU, or any other ORAN-based Radio Unit. 2. we also need to specify DL/UL frequencies in MHz when configuring Lopcomm RU. 3. when configuring NR peercell it is required to set ssb_nr_arfcn. It is also required to set both dl_nr_arfcn and ul_nr_arfcn, and so far we were setting only DL one and using it for both assuming TDD band. 4. when configuring UEsim we need to specify both dl_nr_arfcn and ssb_nr_arfcn for NR cells. So the problem is that even though lteenb automatically computes UL and SSB frequencies, there is no way for us to reuse results of that computation for scenarios outside of "simple enb" case. As the result we kind of workaround that currently with exposing additional parameters and asking users to look into enb, probably its `cell phy` output, with the following: ---- 8< ---- (from UEsim) "ssb_nr_arfcn": { "title": "SSB NR ARFCN", "description": "SSB NR ARFCN, you can retrieve from ENB/GNB side", or use additional parameters just for ul_earfcn and frequency-in-MHz: ---- 8< ---- (from ru/lopcomm) "txa0cc00_center_frequency": { "title": "Lopcomm ORAN DL Center Frequency in MHz (TXA0CC00)", "description": "Lopcomm ORAN Center Frequency in MHz (TXA0CC00)", "type": "number", "default": 2140 }, "rxa0cc00_center_frequency_earfcn": { "title": "Lopcomm ORAN UL Center Frequency EARFCN (RXA0CC00)", "description": "Lopcomm ORAN Center Frequency EARFCN (RXA0CC00)", "type": "number", "default": 18300 }, "rxa0cc00_center_frequency": { "title": "Lopcomm ORAN UL Center Frequency in MHz (RXA0CC00)", "description": "Lopcomm ORAN Center Frequency in MHz (RXA0CC00)", "type": "number", "default": 1950 }, which prevents automation, opens the door for inconsistencies and puts the load to resolve all that on users. The root cause of the problem is that there is no way to access at instantiation time what lteenb would compute internally. But DL<->UL conversion and DL->SSB conversion is not a difficult task and we can do that on our own. -> Do that here and solve all the problems listed above in one go. For frequency computations we use my xlte.earfcn and xlte.nrarfcn modules and upstream nrarfcn egg. For this xlte is updated(*) to primarily pick up kirr/xlte@6cb9d37f kirr/xlte@b8065120 and the rest of the conversion is in slaplte to use corresponding dl2ul and dl2ssb routines. For consistency ul_earfcn and ul_nr_arfcn now become input parameters for LTE and NR cells, but optional. If they are absent - they are computed with defaults, but a user can now control them explicitly. The same applies for ssb_nr_arfcn. This patch does not convert UEsim yet, because UEsim does not use rulib yet and will be handled all in one go in a follow-up step. The computation routines are thoroughly tested. First they have unit tests inside XLTE itself, then we also update our tests in generic test/test.py here with explicitly checking that correct numbers are emitted for UL and SSB frequencies, and third I've also verified SSB computation results with respect to https://tech-academy.amarisoft.com/OutOfBox_UEsim_SA.html#Tips_SSB_Frequency . All this creates a base to be sure that the computations are correct and we are indeed safe to switch our frequencies computation modules. (*) full upgrade brings kirr/xlte@e716ab51...8e606c64 ; nrarfcn egg: https://pypi.org/project/nrarfcn/
-