Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Support
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
W
wendelin
Project overview
Project overview
Details
Activity
Releases
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Issues
0
Issues
0
List
Boards
Labels
Milestones
Merge Requests
4
Merge Requests
4
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Analytics
Analytics
CI / CD
Repository
Value Stream
Wiki
Wiki
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
nexedi
wendelin
Commits
ea2af2de
Commit
ea2af2de
authored
Jun 09, 2023
by
Martin Manchev
Browse files
Options
Browse Files
Download
Plain Diff
Merge remote-tracking branch 'upstream/master' into coding-styles
parents
76255fb0
5782850e
Changes
9
Show whitespace changes
Inline
Side-by-side
Showing
9 changed files
with
14 additions
and
38 deletions
+14
-38
bt5/erp5_wendelin_data_lake_ingestion/SkinTemplateItem/portal_skins/erp5_wendelin_data_lake/Alarm_dataLakeHandleAnalysis.py
...s/erp5_wendelin_data_lake/Alarm_dataLakeHandleAnalysis.py
+3
-3
bt5/erp5_wendelin_data_lake_ingestion/SkinTemplateItem/portal_skins/erp5_wendelin_data_lake/ERP5Site_createDataAnalysisListForDataLake.py
...n_data_lake/ERP5Site_createDataAnalysisListForDataLake.py
+0
-0
bt5/erp5_wendelin_data_lake_ingestion/SkinTemplateItem/portal_skins/erp5_wendelin_data_lake/ERP5Site_createDataAnalysisListForDataLake.xml
..._data_lake/ERP5Site_createDataAnalysisListForDataLake.xml
+1
-1
bt5/erp5_wendelin_data_lake_ingestion/SkinTemplateItem/portal_skins/erp5_wendelin_data_lake/ERP5Site_executeDataAnalysisListForDataLake.py
..._data_lake/ERP5Site_executeDataAnalysisListForDataLake.py
+0
-0
bt5/erp5_wendelin_data_lake_ingestion/SkinTemplateItem/portal_skins/erp5_wendelin_data_lake/ERP5Site_executeDataAnalysisListForDataLake.xml
...data_lake/ERP5Site_executeDataAnalysisListForDataLake.xml
+1
-1
bt5/erp5_wendelin_data_lake_ingestion/SkinTemplateItem/portal_skins/erp5_wendelin_data_lake/ERP5Site_stopIngestionListForDataLake.py
...ndelin_data_lake/ERP5Site_stopIngestionListForDataLake.py
+0
-0
bt5/erp5_wendelin_data_lake_ingestion/SkinTemplateItem/portal_skins/erp5_wendelin_data_lake/ERP5Site_stopIngestionListForDataLake.xml
...delin_data_lake/ERP5Site_stopIngestionListForDataLake.xml
+1
-1
bt5/erp5_wendelin_data_lake_ingestion/TestTemplateItem/portal_components/test.erp5.testDataLakeIngestion.py
...Item/portal_components/test.erp5.testDataLakeIngestion.py
+4
-4
bt5/erp5_wendelin_data_lake_ingestion/TestTemplateItem/portal_components/test.erp5.testDataLakeIngestion.xml
...tem/portal_components/test.erp5.testDataLakeIngestion.xml
+4
-28
No files found.
bt5/erp5_wendelin_data_lake_ingestion/SkinTemplateItem/portal_skins/erp5_wendelin_data_lake/Alarm_dataLakeHandleAnalysis.py
View file @
ea2af2de
portal
=
context
.
getPortalObject
()
portal
=
context
.
getPortalObject
()
portal
.
DataLake_stopIngestionList
()
portal
.
ERP5Site_stopIngestionListForDataLake
()
portal
.
DataLake_createDataAnalysisList
()
portal
.
ERP5Site_createDataAnalysisListForDataLake
()
portal
.
DataLake_executeDataAnalysisList
()
portal
.
ERP5Site_executeDataAnalysisListForDataLake
()
bt5/erp5_wendelin_data_lake_ingestion/SkinTemplateItem/portal_skins/erp5_wendelin_data_lake/ERP5Site_createDataAnalysisList.py
→
bt5/erp5_wendelin_data_lake_ingestion/SkinTemplateItem/portal_skins/erp5_wendelin_data_lake/ERP5Site_createDataAnalysisList
ForDataLake
.py
View file @
ea2af2de
File moved
bt5/erp5_wendelin_data_lake_ingestion/SkinTemplateItem/portal_skins/erp5_wendelin_data_lake/ERP5Site_
stopIngestionList
.xml
→
bt5/erp5_wendelin_data_lake_ingestion/SkinTemplateItem/portal_skins/erp5_wendelin_data_lake/ERP5Site_
createDataAnalysisListForDataLake
.xml
View file @
ea2af2de
...
@@ -54,7 +54,7 @@
...
@@ -54,7 +54,7 @@
</item>
</item>
<item>
<item>
<key>
<string>
id
</string>
</key>
<key>
<string>
id
</string>
</key>
<value>
<string>
ERP5Site_
stopIngestionList
</string>
</value>
<value>
<string>
ERP5Site_
createDataAnalysisListForDataLake
</string>
</value>
</item>
</item>
</dictionary>
</dictionary>
</pickle>
</pickle>
...
...
bt5/erp5_wendelin_data_lake_ingestion/SkinTemplateItem/portal_skins/erp5_wendelin_data_lake/ERP5Site_executeDataAnalysisList.py
→
bt5/erp5_wendelin_data_lake_ingestion/SkinTemplateItem/portal_skins/erp5_wendelin_data_lake/ERP5Site_executeDataAnalysisList
ForDataLake
.py
View file @
ea2af2de
File moved
bt5/erp5_wendelin_data_lake_ingestion/SkinTemplateItem/portal_skins/erp5_wendelin_data_lake/ERP5Site_executeDataAnalysisList.xml
→
bt5/erp5_wendelin_data_lake_ingestion/SkinTemplateItem/portal_skins/erp5_wendelin_data_lake/ERP5Site_executeDataAnalysisList
ForDataLake
.xml
View file @
ea2af2de
...
@@ -54,7 +54,7 @@
...
@@ -54,7 +54,7 @@
</item>
</item>
<item>
<item>
<key>
<string>
id
</string>
</key>
<key>
<string>
id
</string>
</key>
<value>
<string>
ERP5Site_executeDataAnalysisList
</string>
</value>
<value>
<string>
ERP5Site_executeDataAnalysisList
ForDataLake
</string>
</value>
</item>
</item>
</dictionary>
</dictionary>
</pickle>
</pickle>
...
...
bt5/erp5_wendelin_data_lake_ingestion/SkinTemplateItem/portal_skins/erp5_wendelin_data_lake/ERP5Site_stopIngestionList.py
→
bt5/erp5_wendelin_data_lake_ingestion/SkinTemplateItem/portal_skins/erp5_wendelin_data_lake/ERP5Site_stopIngestionList
ForDataLake
.py
View file @
ea2af2de
File moved
bt5/erp5_wendelin_data_lake_ingestion/SkinTemplateItem/portal_skins/erp5_wendelin_data_lake/ERP5Site_
createDataAnalysisList
.xml
→
bt5/erp5_wendelin_data_lake_ingestion/SkinTemplateItem/portal_skins/erp5_wendelin_data_lake/ERP5Site_
stopIngestionListForDataLake
.xml
View file @
ea2af2de
...
@@ -54,7 +54,7 @@
...
@@ -54,7 +54,7 @@
</item>
</item>
<item>
<item>
<key>
<string>
id
</string>
</key>
<key>
<string>
id
</string>
</key>
<value>
<string>
ERP5Site_
createDataAnalysisList
</string>
</value>
<value>
<string>
ERP5Site_
stopIngestionListForDataLake
</string>
</value>
</item>
</item>
</dictionary>
</dictionary>
</pickle>
</pickle>
...
...
bt5/erp5_wendelin_data_lake_ingestion/TestTemplateItem/portal_components/test.erp5.testDataLakeIngestion.py
View file @
ea2af2de
...
@@ -181,7 +181,7 @@ class TestDataIngestion(SecurityTestCase):
...
@@ -181,7 +181,7 @@ class TestDataIngestion(SecurityTestCase):
self
.
tic
()
self
.
tic
()
# call explicitly alarm so all 4 Data Streams are validated and published
# call explicitly alarm so all 4 Data Streams are validated and published
self
.
portal
.
portal_alarms
.
wendelin_
handle_analysis
.
Alarm_h
andleAnalysis
()
self
.
portal
.
portal_alarms
.
wendelin_
data_lake_handle_analysis
.
Alarm_dataLakeH
andleAnalysis
()
self
.
tic
()
self
.
tic
()
# check resulting Data Streams
# check resulting Data Streams
...
@@ -306,7 +306,7 @@ class TestDataIngestion(SecurityTestCase):
...
@@ -306,7 +306,7 @@ class TestDataIngestion(SecurityTestCase):
self
.
tic
()
self
.
tic
()
# call explicitly alarm to process and validate the ingestion
# call explicitly alarm to process and validate the ingestion
self
.
portal
.
portal_alarms
.
wendelin_
handle_analysis
.
Alarm_h
andleAnalysis
()
self
.
portal
.
portal_alarms
.
wendelin_
data_lake_handle_analysis
.
Alarm_dataLakeH
andleAnalysis
()
self
.
tic
()
self
.
tic
()
ingestion_reference
=
self
.
REF_DATASET
+
self
.
REFERENCE_SEPARATOR
+
reference
+
self
.
CSV
ingestion_reference
=
self
.
REF_DATASET
+
self
.
REFERENCE_SEPARATOR
+
reference
+
self
.
CSV
...
@@ -329,9 +329,9 @@ class TestDataIngestion(SecurityTestCase):
...
@@ -329,9 +329,9 @@ class TestDataIngestion(SecurityTestCase):
Check all data analysis and data transformations are well configured
Check all data analysis and data transformations are well configured
"""
"""
# call two times the alarm (1. create analysis 2. run analysis)
# call two times the alarm (1. create analysis 2. run analysis)
self
.
portal
.
portal_alarms
.
wendelin_
handle_analysis
.
Alarm_h
andleAnalysis
()
self
.
portal
.
portal_alarms
.
wendelin_
data_lake_handle_analysis
.
Alarm_dataLakeH
andleAnalysis
()
self
.
tic
()
self
.
tic
()
self
.
portal
.
portal_alarms
.
wendelin_
handle_analysis
.
Alarm_h
andleAnalysis
()
self
.
portal
.
portal_alarms
.
wendelin_
data_lake_handle_analysis
.
Alarm_dataLakeH
andleAnalysis
()
self
.
tic
()
self
.
tic
()
def
test_10_checkDataSetDataStreamRelation
(
self
):
def
test_10_checkDataSetDataStreamRelation
(
self
):
...
...
bt5/erp5_wendelin_data_lake_ingestion/TestTemplateItem/portal_components/test.erp5.testDataLakeIngestion.xml
View file @
ea2af2de
...
@@ -6,12 +6,6 @@
...
@@ -6,12 +6,6 @@
</pickle>
</pickle>
<pickle>
<pickle>
<dictionary>
<dictionary>
<item>
<key>
<string>
_recorded_property_dict
</string>
</key>
<value>
<persistent>
<string
encoding=
"base64"
>
AAAAAAAAAAI=
</string>
</persistent>
</value>
</item>
<item>
<item>
<key>
<string>
default_reference
</string>
</key>
<key>
<string>
default_reference
</string>
</key>
<value>
<string>
testDataLakeIngestion
</string>
</value>
<value>
<string>
testDataLakeIngestion
</string>
</value>
...
@@ -45,10 +39,7 @@
...
@@ -45,10 +39,7 @@
<item>
<item>
<key>
<string>
text_content_warning_message
</string>
</key>
<key>
<string>
text_content_warning_message
</string>
</key>
<value>
<value>
<tuple>
<tuple/>
<string>
W:106, 34: Unused variable \'i\' (unused-variable)
</string>
<string>
W:106, 76: Unused variable \'j\' (unused-variable)
</string>
</tuple>
</value>
</value>
</item>
</item>
<item>
<item>
...
@@ -58,28 +49,13 @@
...
@@ -58,28 +49,13 @@
<item>
<item>
<key>
<string>
workflow_history
</string>
</key>
<key>
<string>
workflow_history
</string>
</key>
<value>
<value>
<persistent>
<string
encoding=
"base64"
>
AAAAAAAAAA
M
=
</string>
</persistent>
<persistent>
<string
encoding=
"base64"
>
AAAAAAAAAA
I
=
</string>
</persistent>
</value>
</value>
</item>
</item>
</dictionary>
</dictionary>
</pickle>
</pickle>
</record>
</record>
<record
id=
"2"
aka=
"AAAAAAAAAAI="
>
<record
id=
"2"
aka=
"AAAAAAAAAAI="
>
<pickle>
<global
name=
"PersistentMapping"
module=
"Persistence.mapping"
/>
</pickle>
<pickle>
<dictionary>
<item>
<key>
<string>
data
</string>
</key>
<value>
<dictionary/>
</value>
</item>
</dictionary>
</pickle>
</record>
<record
id=
"3"
aka=
"AAAAAAAAAAM="
>
<pickle>
<pickle>
<global
name=
"PersistentMapping"
module=
"Persistence.mapping"
/>
<global
name=
"PersistentMapping"
module=
"Persistence.mapping"
/>
</pickle>
</pickle>
...
@@ -92,7 +68,7 @@
...
@@ -92,7 +68,7 @@
<item>
<item>
<key>
<string>
component_validation_workflow
</string>
</key>
<key>
<string>
component_validation_workflow
</string>
</key>
<value>
<value>
<persistent>
<string
encoding=
"base64"
>
AAAAAAAAAA
Q
=
</string>
</persistent>
<persistent>
<string
encoding=
"base64"
>
AAAAAAAAAA
M
=
</string>
</persistent>
</value>
</value>
</item>
</item>
</dictionary>
</dictionary>
...
@@ -101,7 +77,7 @@
...
@@ -101,7 +77,7 @@
</dictionary>
</dictionary>
</pickle>
</pickle>
</record>
</record>
<record
id=
"
4"
aka=
"AAAAAAAAAAQ
="
>
<record
id=
"
3"
aka=
"AAAAAAAAAAM
="
>
<pickle>
<pickle>
<global
name=
"WorkflowHistoryList"
module=
"Products.ERP5Type.Workflow"
/>
<global
name=
"WorkflowHistoryList"
module=
"Products.ERP5Type.Workflow"
/>
</pickle>
</pickle>
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment