Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Support
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
W
wendelin
Project overview
Project overview
Details
Activity
Releases
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Issues
0
Issues
0
List
Boards
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Analytics
Analytics
CI / CD
Repository
Value Stream
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
Léo-Paul Géneau
wendelin
Commits
85d24341
Commit
85d24341
authored
Jul 28, 2020
by
Roque
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
erp5_wendelin_data_lake_ingestion: rename data supply
parent
d2600881
Changes
8
Hide whitespace changes
Inline
Side-by-side
Showing
8 changed files
with
19 additions
and
19 deletions
+19
-19
bt5/erp5_wendelin_data_lake_ingestion/PathTemplateItem/data_supply_module/default_ebulk.xml
...ion/PathTemplateItem/data_supply_module/default_ebulk.xml
+5
-5
bt5/erp5_wendelin_data_lake_ingestion/PathTemplateItem/data_supply_module/default_ebulk/1.xml
...n/PathTemplateItem/data_supply_module/default_ebulk/1.xml
+2
-2
bt5/erp5_wendelin_data_lake_ingestion/PathTemplateItem/data_supply_module/default_ebulk/2.xml
...n/PathTemplateItem/data_supply_module/default_ebulk/2.xml
+2
-2
bt5/erp5_wendelin_data_lake_ingestion/SkinTemplateItem/portal_skins/erp5_wendelin_data_lake/IngestionPolicy_getIngestionOperationAndParameterDictEbulk.py
...stionPolicy_getIngestionOperationAndParameterDictEbulk.py
+1
-1
bt5/erp5_wendelin_data_lake_ingestion/TestTemplateItem/portal_components/test.erp5.testDataLakeIngestion.py
...Item/portal_components/test.erp5.testDataLakeIngestion.py
+3
-3
bt5/erp5_wendelin_data_lake_ingestion/bt/template_keep_last_workflow_history_only_path_list
...ion/bt/template_keep_last_workflow_history_only_path_list
+2
-2
bt5/erp5_wendelin_data_lake_ingestion/bt/template_keep_workflow_path_list
...n_data_lake_ingestion/bt/template_keep_workflow_path_list
+2
-2
bt5/erp5_wendelin_data_lake_ingestion/bt/template_path_list
bt5/erp5_wendelin_data_lake_ingestion/bt/template_path_list
+2
-2
No files found.
bt5/erp5_wendelin_data_lake_ingestion/PathTemplateItem/data_supply_module/
em
bulk.xml
→
bt5/erp5_wendelin_data_lake_ingestion/PathTemplateItem/data_supply_module/
default_e
bulk.xml
View file @
85d24341
...
@@ -86,7 +86,7 @@
...
@@ -86,7 +86,7 @@
</item>
</item>
<item>
<item>
<key>
<string>
default_reference
</string>
</key>
<key>
<string>
default_reference
</string>
</key>
<value>
<string>
em
bulk
</string>
</value>
<value>
<string>
default_e
bulk
</string>
</value>
</item>
</item>
<item>
<item>
<key>
<string>
description
</string>
</key>
<key>
<string>
description
</string>
</key>
...
@@ -132,7 +132,7 @@
...
@@ -132,7 +132,7 @@
</item>
</item>
<item>
<item>
<key>
<string>
id
</string>
</key>
<key>
<string>
id
</string>
</key>
<value>
<string>
em
bulk
</string>
</value>
<value>
<string>
default_e
bulk
</string>
</value>
</item>
</item>
<item>
<item>
<key>
<string>
language
</string>
</key>
<key>
<string>
language
</string>
</key>
...
@@ -146,7 +146,7 @@
...
@@ -146,7 +146,7 @@
</item>
</item>
<item>
<item>
<key>
<string>
title
</string>
</key>
<key>
<string>
title
</string>
</key>
<value>
<string>
E
m
bulk
</string>
</value>
<value>
<string>
Ebulk
</string>
</value>
</item>
</item>
<item>
<item>
<key>
<string>
version
</string>
</key>
<key>
<string>
version
</string>
</key>
...
@@ -329,7 +329,7 @@
...
@@ -329,7 +329,7 @@
</item>
</item>
<item>
<item>
<key>
<string>
serial
</string>
</key>
<key>
<string>
serial
</string>
</key>
<value>
<string>
9
60.15698.49376.7287
</string>
</value>
<value>
<string>
9
85.36429.29664.46233
</string>
</value>
</item>
</item>
<item>
<item>
<key>
<string>
state
</string>
</key>
<key>
<string>
state
</string>
</key>
...
@@ -347,7 +347,7 @@
...
@@ -347,7 +347,7 @@
</tuple>
</tuple>
<state>
<state>
<tuple>
<tuple>
<float>
1
498225133.32
</float>
<float>
1
595958580.55
</float>
<string>
UTC
</string>
<string>
UTC
</string>
</tuple>
</tuple>
</state>
</state>
...
...
bt5/erp5_wendelin_data_lake_ingestion/PathTemplateItem/data_supply_module/
em
bulk/1.xml
→
bt5/erp5_wendelin_data_lake_ingestion/PathTemplateItem/data_supply_module/
default_e
bulk/1.xml
View file @
85d24341
...
@@ -214,7 +214,7 @@
...
@@ -214,7 +214,7 @@
</item>
</item>
<item>
<item>
<key>
<string>
serial
</string>
</key>
<key>
<string>
serial
</string>
</key>
<value>
<string>
9
59.7816.8415.26299
</string>
</value>
<value>
<string>
9
85.17805.27243.30754
</string>
</value>
</item>
</item>
<item>
<item>
<key>
<string>
state
</string>
</key>
<key>
<string>
state
</string>
</key>
...
@@ -232,7 +232,7 @@
...
@@ -232,7 +232,7 @@
</tuple>
</tuple>
<state>
<state>
<tuple>
<tuple>
<float>
1
498225125.16
</float>
<float>
1
595958567.14
</float>
<string>
UTC
</string>
<string>
UTC
</string>
</tuple>
</tuple>
</state>
</state>
...
...
bt5/erp5_wendelin_data_lake_ingestion/PathTemplateItem/data_supply_module/
em
bulk/2.xml
→
bt5/erp5_wendelin_data_lake_ingestion/PathTemplateItem/data_supply_module/
default_e
bulk/2.xml
View file @
85d24341
...
@@ -220,7 +220,7 @@
...
@@ -220,7 +220,7 @@
</item>
</item>
<item>
<item>
<key>
<string>
serial
</string>
</key>
<key>
<string>
serial
</string>
</key>
<value>
<string>
9
59.7816.8415.26299
</string>
</value>
<value>
<string>
9
85.17805.27243.30754
</string>
</value>
</item>
</item>
<item>
<item>
<key>
<string>
state
</string>
</key>
<key>
<string>
state
</string>
</key>
...
@@ -238,7 +238,7 @@
...
@@ -238,7 +238,7 @@
</tuple>
</tuple>
<state>
<state>
<tuple>
<tuple>
<float>
1
498225125
.15
</float>
<float>
1
595958567
.15
</float>
<string>
UTC
</string>
<string>
UTC
</string>
</tuple>
</tuple>
</state>
</state>
...
...
bt5/erp5_wendelin_data_lake_ingestion/SkinTemplateItem/portal_skins/erp5_wendelin_data_lake/IngestionPolicy_getIngestionOperationAndParameterDictEbulk.py
View file @
85d24341
...
@@ -33,7 +33,7 @@ if data_ingestion is not None:
...
@@ -33,7 +33,7 @@ if data_ingestion is not None:
specialise_value_list
=
[
x
.
getObject
()
for
x
in
portal_catalog
.
searchResults
(
specialise_value_list
=
[
x
.
getObject
()
for
x
in
portal_catalog
.
searchResults
(
portal_type
=
'Data Supply'
,
portal_type
=
'Data Supply'
,
reference
=
'
em
bulk'
,
reference
=
'
default_e
bulk'
,
validation_state
=
'validated'
)]
validation_state
=
'validated'
)]
# create a new data ingestion
# create a new data ingestion
...
...
bt5/erp5_wendelin_data_lake_ingestion/TestTemplateItem/portal_components/test.erp5.testDataLakeIngestion.py
View file @
85d24341
...
@@ -216,11 +216,11 @@ class TestDataIngestion(SecurityTestCase):
...
@@ -216,11 +216,11 @@ class TestDataIngestion(SecurityTestCase):
self
.
assertNotEqual
(
None
,
self
.
assertNotEqual
(
None
,
getattr
(
self
.
portal
.
portal_ingestion_policies
,
"default_ebulk"
,
None
))
getattr
(
self
.
portal
.
portal_ingestion_policies
,
"default_ebulk"
,
None
))
self
.
assertNotEqual
(
None
,
self
.
assertNotEqual
(
None
,
getattr
(
self
.
portal
.
data_supply_module
,
"
em
bulk"
,
None
))
getattr
(
self
.
portal
.
data_supply_module
,
"
default_e
bulk"
,
None
))
def
test_04_D
efaultModelSecurityModel
(
self
):
def
test_04_D
atasetAndDatastreamsConsistency
(
self
):
"""
"""
Test
default security model : 'All can download, only contributors can upload.'
Test
that data set state transition also changes its data streams states
"""
"""
data_set
,
data_stream_list
=
self
.
stepIngest
(
self
.
CSV
,
","
,
randomize_ingestion_reference
=
True
)
data_set
,
data_stream_list
=
self
.
stepIngest
(
self
.
CSV
,
","
,
randomize_ingestion_reference
=
True
)
self
.
tic
()
self
.
tic
()
...
...
bt5/erp5_wendelin_data_lake_ingestion/bt/template_keep_last_workflow_history_only_path_list
View file @
85d24341
...
@@ -3,8 +3,8 @@ data_operation_module/transform_fif
...
@@ -3,8 +3,8 @@ data_operation_module/transform_fif
data_product_module/fif_array
data_product_module/fif_array
data_product_module/fif_data
data_product_module/fif_data
data_product_module/fif_descriptor
data_product_module/fif_descriptor
data_supply_module/
em
bulk
data_supply_module/
default_e
bulk
data_supply_module/
em
bulk/**
data_supply_module/
default_e
bulk/**
portal_ingestion_policies/wendelin_embulk
portal_ingestion_policies/wendelin_embulk
portal_ingestion_policies/default_ebulk
portal_ingestion_policies/default_ebulk
portal_categories/function/**
portal_categories/function/**
...
...
bt5/erp5_wendelin_data_lake_ingestion/bt/template_keep_workflow_path_list
View file @
85d24341
...
@@ -3,8 +3,8 @@ data_operation_module/transform_fif
...
@@ -3,8 +3,8 @@ data_operation_module/transform_fif
data_product_module/fif_array
data_product_module/fif_array
data_product_module/fif_data
data_product_module/fif_data
data_product_module/fif_descriptor
data_product_module/fif_descriptor
data_supply_module/
em
bulk
data_supply_module/
default_e
bulk
data_supply_module/
em
bulk/**
data_supply_module/
default_e
bulk/**
portal_ingestion_policies/wendelin_embulk
portal_ingestion_policies/wendelin_embulk
portal_ingestion_policies/default_ebulk
portal_ingestion_policies/default_ebulk
portal_categories/use/**
portal_categories/use/**
\ No newline at end of file
bt5/erp5_wendelin_data_lake_ingestion/bt/template_path_list
View file @
85d24341
...
@@ -3,8 +3,8 @@ data_operation_module/transform_fif
...
@@ -3,8 +3,8 @@ data_operation_module/transform_fif
data_product_module/fif_array
data_product_module/fif_array
data_product_module/fif_data
data_product_module/fif_data
data_product_module/fif_descriptor
data_product_module/fif_descriptor
data_supply_module/
em
bulk
data_supply_module/
default_e
bulk
data_supply_module/
em
bulk/**
data_supply_module/
default_e
bulk/**
portal_alarms/wendelin_check_datastream_consistency
portal_alarms/wendelin_check_datastream_consistency
portal_alarms/wendelin_check_datastream_consistency/**
portal_alarms/wendelin_check_datastream_consistency/**
portal_alarms/wendelin_data_lake_handle_analysis
portal_alarms/wendelin_data_lake_handle_analysis
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment