- 19 Jan, 2021 5 commits
-
-
Ivan Tyagov authored
Add set category for a Data Stream. A data Stream can belong to many Data Sets (or any other 'collections' or 'sets').
-
Ivan Tyagov authored
During ingestion the relation between Data Set and Data Stream is set on Data Ingestion Line. This seemed enough for small Data Sets (few thousands of Data Streams). With large Data Sets this is inefficient as getting list of Data Streams for a Data Set takes sometimes tens of minutes. Thus we needed a faster way to loop up: which is an explicitl relation over set category and proper MySQL indexing. This commit is a trade-off between cost of storage (i.e. a category) compared to scalability and performance of a system.
-
Ivan Tyagov authored
This reverts commit 793fe76f.
-
Ivan Tyagov authored
-
Ivan Tyagov authored
-
- 18 Jan, 2021 2 commits
-
-
Ivan Tyagov authored
-
Ivan Tyagov authored
Add an utility script to check md5sum of all Data Streams for a Data Set (uploaded from file system) and what we have in reallity.
-
- 15 Jan, 2021 2 commits
-
-
Ivan Tyagov authored
-
Ivan Tyagov authored
-
- 14 Jan, 2021 1 commit
-
-
Ivan Tyagov authored
Rather than read Data Streams' properties from ZODB object do read them from ZSQL's brain object - much faster when data sets contain hunreds of thousands of files.
-
- 13 Jan, 2021 2 commits
-
-
Ivan Tyagov authored
-
Ivan Tyagov authored
Index directly the relation between a Data Stream and its logical set (in this case of default Wendelin model - Data Set).
-
- 12 Jan, 2021 1 commit
-
-
Ivan Tyagov authored
-
- 21 Dec, 2020 1 commit
-
-
Ivan Tyagov authored
See merge request !73
-
- 14 Dec, 2020 2 commits
- 01 Dec, 2020 2 commits
-
-
Ivan Tyagov authored
See merge request nexedi/wendelin!72
-
Roque authored
-
- 29 Oct, 2020 2 commits
-
-
Roque authored
- 28 Oct, 2020 1 commit
-
-
Ivan Tyagov authored
See merge request nexedi/wendelin!70
-
- 27 Oct, 2020 1 commit
-
-
Roque authored
-
- 15 Oct, 2020 1 commit
-
-
Ivan Tyagov authored
See merge request nexedi/wendelin!69
-
- 14 Oct, 2020 1 commit
-
-
Roque authored
-
- 28 Sep, 2020 1 commit
-
-
Ivan Tyagov authored
See merge request nexedi/wendelin!68
-
- 25 Sep, 2020 1 commit
-
-
Roque authored
-
- 24 Sep, 2020 1 commit
-
-
Ivan Tyagov authored
-
- 17 Sep, 2020 2 commits
-
-
Ivan Tyagov authored
-
Ivan Tyagov authored
-
- 15 Sep, 2020 3 commits
-
-
Ivan Tyagov authored
See merge request nexedi/wendelin!67
-
Roque authored
- show also validated data streams
-
Roque authored
-
- 11 Sep, 2020 1 commit
-
-
Ivan Tyagov authored
See merge request nexedi/wendelin!66
-
- 10 Sep, 2020 1 commit
-
-
Roque authored
-
- 09 Sep, 2020 2 commits
-
-
Roque authored
-
Ivan Tyagov authored
-
- 07 Sep, 2020 1 commit
-
-
Roque authored
-
- 02 Sep, 2020 1 commit
-
-
Ivan Tyagov authored
See merge request nexedi/wendelin!65
-
- 01 Sep, 2020 2 commits