- 11 Feb, 2016 14 commits
-
-
Kirill Smelkov authored
Upstream added `pages` and `elasticsearch` queues to sidekiq https://gitlab.com/gitlab-org/omnibus-gitlab/commit/be6844ff https://gitlab.com/gitlab-org/omnibus-gitlab/commit/801d6ed7 but since those queues are used for EE-only services we do not enable them. /cc @kazuhiko, @jerome
-
Kirill Smelkov authored
As gitlab-workhorse is now serving everything, it proxies unknown-to-it requests to unicorn, and that returns redirect to /users/sign_in for /non-existent, not 403. Adjust the promise and just verify whether we can get /static.css NOTE nginx promise now effectively depends on gitlab-workhorse being up and running - because nginx is now configured to be only an SSL terminator and all requests processing is done by gitlab-workhorse and services below it. If in the future we'll need a pure nginx-is-up promise, we can introduce something to nginx config and verify that particular http endpoint. /cc @kazuhiko, @jerome
-
Kirill Smelkov authored
add `-documentRoot ...` argument to gitlab-workhorse start as upstream started to do: https://gitlab.com/gitlab-org/omnibus-gitlab/commit/d769f751 ( gitlab-workhorse now needs to know where document root is as it now serves all content - e.g. static files too this option was introduced here: https://gitlab.com/gitlab-org/gitlab-workhorse/commit/1a7009e4 ) /cc @kazuhiko, @jerome
-
Kirill Smelkov authored
I manually reviewed files/gitlab-config-template/gitlab.rb.template files/gitlab-cookbooks/gitlab/attributes/default.rb from omnibus-gitlab 8.4.2+ce.0-3-g68d5ee8 and picked up changes: - unicorn OOM killer memory limits were raised up (because gitlab is now more memory hungry according to talks on gitlab.com) https://gitlab.com/gitlab-org/omnibus-gitlab/commit/f8ee4a19 - there is no default limit for nginx client_max_body_size - it just goes to gitlab-workhorse as is at any size https://gitlab.com/gitlab-org/omnibus-gitlab/commit/78862837 /cc @kazuhiko, @jerome
-
Kirill Smelkov authored
Direct merge for gitconfig.erb from omnibus-gitlab did not brought up updates. pack.threads=1 and receive.fsckObjects=true picked up from https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-cookbooks/gitlab/attributes/default.rb in 264d785a stay the same. /cc @kazuhiko, @jerome
-
Kirill Smelkov authored
No changes. /cc @kazuhiko, @jerome
-
Kirill Smelkov authored
- there is a section for gitlab pages, which we stub-out; - there is no longer a need to add /raw/... handling to nginx - as now nginx is just an SSL terminator for gitlab-workhorse, all URL handling is done inside gitlab-workhorse and is dealt with ok by our patches. - as now nginx does not directly connect to unicorn, there is no need to pass unicorn section to nginx's template. /cc @kazuhiko, @jerome
-
Kirill Smelkov authored
Like wih gitlab configs, this does only pure merge. We will slaposify / adjust to updates in the following patches. /cc @kazuhiko, @jerome
-
Kirill Smelkov authored
Slapos'ify config updates brought by sync with omnibus-gitlab (see merge in the previous patch). Changes: - default visibility levels gone away (see merge commit and https://gitlab.com/gitlab-org/omnibus-gitlab/commit/b5ebbab3 ) - there are new settings for GitLab Pages and Elasticsearch, which are EE only and thus we do not support them. - there are new settings for auxiliary cron-like jobs, which we do not support for now, since they are used either for CI (not supported by us) or EE features. Configuration files that were synced, but did not changed are also marked as updated, so it is easier to track their changes to upstream in the future. /cc @kazuhiko, @jerome
-
Kirill Smelkov authored
This does only pure merge. We will slaposify / adjust config and corresponding md5sum in the following patches. /cc @kazuhiko, @jerome
-
Kirill Smelkov authored
Like 8f945bd2 - pristine copy from omnibus 8.4.4+ce.0-0-g1680742. Nothing changed at all - gitconfig.erb stays the same. /cc @kazuhiko, @jerome
-
Kirill Smelkov authored
Like 61544d87 - pristine copy from omnibus 8.4.4+ce.0-0-g1680742. Most of http handling is moving to gitlab-workhorse. Nginx remains only as ssl terminator. /cc @kazuhiko, @jerome
-
Kirill Smelkov authored
Like 6fd7b987 - pristine copy from omnibus 8.4.4+ce.0-0-g1680742. The only change is in gitlab.yml The following files stay the same: database.yml.erb gitlab-rails-config.ru.erb gitlab-shell-config.yml.erb rack_attack.rb.erb resque.yml.erb smtp_settings.rb.erb unicorn.rb.erb /cc @kazuhiko, @jerome
-
Kirill Smelkov authored
Update GitLab software to - gitlab-ce 8.4 + NXD patches https://lab.nexedi.com/kirr/gitlab-ce/commits/8-4-nxd - gitlab-shell to 2.6.10 which now is pure upstream, as all NXD patches were merged. - gitlab-workhorse 0.6.X + NXD patches. https://lab.nexedi.com/kirr/gitlab-workhorse/commits/y/blobraw-3 https://gitlab.com/gitlab-org/gitlab-workhorse/merge_requests/17 This only updates software and begins SR update to 8.4 - for now gitlab instance becomes non-working -- we'll pull in configuration files updates and fixups in the following patches. /cc @kazuhiko, @jerome
-
- 10 Feb, 2016 1 commit
-
-
Jérome Perrin authored
slapos.recipe.cmmi executes command lines by line, so this build options were ignored.
-
- 09 Feb, 2016 1 commit
-
-
Rafael Monnerat authored
older versions are failing on ARM arch.
-
- 02 Feb, 2016 2 commits
-
-
Kazuhiko Shiozaki authored
version up : OpenBLAS 0.2.15. First try with auto-detected target and if it fails try TARGET=GENERIC. You can also spcify the target explicitly by build-ext-options. reviewd-by @Tyagov (on nexedi/slapos!42).
-
Kazuhiko Shiozaki authored
-
- 01 Feb, 2016 4 commits
-
-
Kirill Smelkov authored
This patch series teaches ERP5 software release to automatically instantiate Jupyter notebook web UI and tune it to connect to ERP5 by default. When Jupyter is enabled, it also installs on-server erp5_data_notebook bt5 (see nexedi/erp5!29 and nexedi/erp5@f662b5a2) which handles code execution requested for Jupyter. For ERP5 - for security and backward compatibility reasons - Jupyter instantiation and erp5_data_notebook bt5 install happen only if jupyter is explicitly enabled in instance parameters. The default is not to have Jupyter out of the box. On the other hand for Wendelin SR, which inherits from ERP5 SR, the default is to have Jupyter out of the box, because Wendelin SR is fresh enough without lots of backward compatibility needs, and Jupyter is usually very handy for people who use Wendelin. -------- NOTE Currently erp5-data-notebook bt5 has the following limitations (see details on nexedi/slapos!43 and nexedi/erp5!29): - errors are not reported properly to users; - state is not fully saved to ZODB. the latter point means notebook works only if it is connected to Zope family with only 1 zope process. Hopefully this will be resolved some day. Technical overview about how the integration is done itself on slapos part and other notes are in 0a446263. /proposed-for-review-on nexedi/slapos!43
-
Douglas authored
Query strings used on GET requests have size limitations on servers and this causes big code cells to not be executed at all, returning only an Internal Server Error with no further explanation. /reviewed-by @kirr, @Tyagov (on nexedi/slapos!43)
-
Kirill Smelkov authored
Pandas and scikit-learn are handy to have for data processing which we are going to do more and more in ERP5 context. Matplotlib is very handy to have when one works with Jupyter, but also can be used by just backend code to generate graphs. From this point of view it makes sense to have this eggs always available out of the box. To do so move them from Wendelin to ERP5. /suggested-by @tiwariayush /reviewed-by @Tyagov (on !43)
-
Kirill Smelkov authored
This patch teaches ERP5 software release to automatically instantiate Jupyter notebook web UI and tune it to connect to ERP5 by default. When Jupyter is enabled, it also installs on-server erp5_data_notebook bt5 (erp5!29) which handles code execution requested for Jupyter. For ERP5 - for security and backward compatibility reasons - Jupyter instantiation and erp5_data_notebook bt5 install happen only if jupyter is explicitly enabled in instance parameters. The default is not to have Jupyter out of the box. On the other hand for Wendelin SR, which inherits from ERP5 SR, the default is to have Jupyter out of the box, because Wendelin SR is fresh enough without lots of backward compatibility needs, and Jupyter is usually very handy for people who use Wendelin. ~~~~ For integration, we reuse already established in ERP5 infrastructure, to request various slave instances, and request Jupyter in a way so it automatically tunes and connects to balancer of one of Zope family. Jupyter code itself is compiled by reusing software/ipython_notebook/software.cfg, and Jupyter instance code is reused by hooking software/ipython_notebook/instance.cfg.in into ERP5 SR properly (the idea to override instance-jupyter not to render into default template.cfg is taken from previous work by @tiwariayush). ~~~~ I tested this patch inside webrunner with create-erp5-site software type and various configurations (whether to have or not have jupyter, to which zope family to connect it, etc). I have not tested frontend instantiation fully - because tests were done only in webrunner, but I've tried to make sure generated buildout code is valid for cases with frontend. NOTE the code in this patch depends erp5_data_notebook bt5 (erp5!29) which just got merged to erp5.git recently (see erp5@f662b5a2) NOTE even when erp5_data_notebook bt5 is installed, on a freshly installed ERP5, it is required to "check site consistency" first, so that initial bt5(s) are actually installed and erp5 is ready to function. /cc @vpelletier, @Tyagov, @klaus, @Camata, @tiwariayush, @Kreisel, @jerome, @nexedi /proposed-for-review-on !43
-
- 31 Jan, 2016 6 commits
-
-
Kazuhiko Shiozaki authored
-
Kazuhiko Shiozaki authored
-
Kazuhiko Shiozaki authored
-
Kazuhiko Shiozaki authored
-
Kazuhiko Shiozaki authored
-
Kirill Smelkov authored
Gitlab uses github-markup to render various text-based markups (markdown, rst, ...) into html. For rst github-markup wants to run python and have docutils egg available: https://github.com/github/markup/blob/5393ae93/lib/github/markups.rb#L36 as we were not having docutils installed and path to proper python interpreter setup, rst documents were not automatically rendered and were show just as plain text. We do a lot of documents in rst - that case is important for us. So fix it by providing gitlab with properly setup python interpreter with all needed eggs installed. /cc @kazuhiko, @jerome /proposed-for-review-on nexedi/slapos!39
-
- 27 Jan, 2016 1 commit
-
-
Kirill Smelkov authored
The reason is: we are going to integrate Jupyter into ERP5 SR, and in ERP5 SR code, there is already established infrastructure, which requests slave instances. That infrastructure passes parameters in serialized (= json) form, that's why we need to switch. On the other hand, slapos.cookbook:slapconfiguration.serialised handles non-json instance parameters as well: when there parameters format is not {'_' -> <json>} it decodes them as slapos.cookbook:slapconfiguration would do: https://lab.nexedi.com/nexedi/slapos/blob/69229988/slapos/recipe/slapconfiguration.py#L232 https://lab.nexedi.com/nexedi/slapos/blob/69229988/slapos/recipe/librecipe/__init__.py#L51 So from this point of view, though we don't really need it here (Jupyter SR is relatively new), we are staying backward compatible. /cc @tiwariayush
-
- 26 Jan, 2016 2 commits
-
-
Kazuhiko Shiozaki authored
-
Kirill Smelkov authored
Both numpy and ipython are included in ERP5 SR which wendelin inherits from, and are pinned there. Here are e.g. latest pin-ups for numpy and ipython in erp5: e3144a8a (version up eggs.) 135570c9 (version up eggs.) Furthermore: this is not only a cleanup. As e3144a8a shows current version of numpy in erp5 is 1.10.4 and in wendelin we still have 1.9.2 which is unintentional downgrade compared to erp5. Don't do that. /cc @kazuhiko /reviewed-by @Tyagov (on nexedi/slapos!41)
-
- 25 Jan, 2016 2 commits
-
-
Julien Muchembled authored
-
-
- 24 Jan, 2016 1 commit
-
-
Kirill Smelkov authored
@rafael added this in 971d0bb7 (erp5: Make possible extent the list of initial business templates to install), but we dropped that change while merging erp5-cluster to master - see: 6bbb61a8 "Merge branch 'master' into erp5-cluster", and e84d5e83 "Merge branch 'erp5-cluster'" 6bbb61a8 claimed that it Dropped commit 971d0bb7 ("erp5: Make possible extent the list of initial business templates to install"). but it actually dropped changes only under stack/erp5/ , not software/wendelin/ Fix it. /cc @rafael, @jm, @Tyagov /reviewed-by TrustMe
-
- 22 Jan, 2016 1 commit
-
-
Julien Muchembled authored
-
- 21 Jan, 2016 1 commit
-
-
Jérome Perrin authored
-
- 20 Jan, 2016 3 commits
-
-
Julien Muchembled authored
In slapos.package.git/obs, we need to build binaries for specific paths, without changing where buildout actually install them.
-
Kazuhiko Shiozaki authored
-
Kazuhiko Shiozaki authored
-
- 17 Jan, 2016 1 commit
-
-
Kirill Smelkov authored
Hello up there, Here comes SlapOS port of GitLab. We start from GitLab 8.2.X as that is what we currently run on KVM on lab.nexedi.com, so that our data can be straightforwardly migrated. The SR compiles all needed software and organized all (sub-)services in one partition and interconnects them with unix sockets for security and speed reasons (see patch "gitlab: Make a plan to base instance layout on gitlab-omnibus and to interconnect all internal services"). Services configuration files are originally taken from omnibus-gitlab "distribution" and incrementally ported to slapos variant. This way we establish a (imho) good path on how to track upstream changes and minimize our delta & effort supporting it. GitLab itself is patched (above patches that were already applied by upstream): - to support HTTP(S) only - to show site's ICP number - to speedup raw blob downloading ~ 17x times ( see patch "gitlab: Optimize raw blob downloading" for details and https://gitlab.com/gitlab-org/gitlab-workhorse/merge_requests/17 ) Overall it should work and we should finally be able to migrate slapos.git (because of raw blob downloading is not slow now) to GitLab and all other Nexedi git repositories. Thanks, Kirill P.S. Somewhat outdated, but this picture on GitLab architecture might help to understand how parts are glued together: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/architecture.md P.P.S. Native resiliency is not implemented yet, but we should be able to use gitlab inside resilient webrunner already. /proposed-for-review-on !39 /partly-reviewed-by @kazuhiko, @jerome, @Yanni, @jp /cc @rafael, @jm
-