Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Support
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
W
wendelin
Project overview
Project overview
Details
Activity
Releases
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Issues
0
Issues
0
List
Boards
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Analytics
Analytics
CI / CD
Repository
Value Stream
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
Léo-Paul Géneau
wendelin
Commits
197fe2fb
Commit
197fe2fb
authored
Sep 07, 2020
by
Roque
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
erp5_wendelin_data_lake_ingestion: script to invalidate old data sets (<=2019)
parent
bc683f1e
Changes
2
Hide whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
92 additions
and
0 deletions
+92
-0
bt5/erp5_wendelin_data_lake_ingestion/SkinTemplateItem/portal_skins/erp5_wendelin_data_lake/ERP5Site_invalidateOldDatasets.py
...erp5_wendelin_data_lake/ERP5Site_invalidateOldDatasets.py
+30
-0
bt5/erp5_wendelin_data_lake_ingestion/SkinTemplateItem/portal_skins/erp5_wendelin_data_lake/ERP5Site_invalidateOldDatasets.xml
...rp5_wendelin_data_lake/ERP5Site_invalidateOldDatasets.xml
+62
-0
No files found.
bt5/erp5_wendelin_data_lake_ingestion/SkinTemplateItem/portal_skins/erp5_wendelin_data_lake/ERP5Site_invalidateOldDatasets.py
0 → 100644
View file @
197fe2fb
"""This script invalidate all data sets (and corresponding ingestion objects) older than wendelin.io release (<= 2019)"""
portal
=
context
.
getPortalObject
()
portal_catalog
=
portal
.
portal_catalog
from
DateTime
import
DateTime
old_date
=
DateTime
(
2019
,
12
,
31
)
catalog_kw
=
{
'modification_date'
:
{
'query'
:
old_date
,
'range'
:
'<='
}}
print
"Following Dataset were invalidated:"
for
data_set
in
portal_catalog
(
portal_type
=
"Data Set"
,
**
catalog_kw
):
print
print
"DATASET: "
+
data_set
.
getReference
()
print
"state: "
+
data_set
.
getValidationState
()
print
"date: "
+
str
(
data_set
.
getModificationDate
())
print
"len of datastream list: "
+
str
(
len
(
data_set
.
DataSet_getDataStreamList
()))
for
data_stream
in
data_set
.
DataSet_getDataStreamList
():
if
data_stream
is
not
None
:
portal
.
ERP5Site_invalidateIngestionObjects
(
data_stream
.
getReference
())
try
:
data_stream
.
invalidate
()
except
:
pass
# fails if it's already invalidated, draft or if it doens't allow invalidation (e.g. DI)
portal
.
ERP5Site_invalidateReference
(
data_set
)
try
:
data_set
.
invalidate
()
except
:
pass
# fails if it's already invalidated, draft or if it doens't allow invalidation (e.g. DI)
return
printed
bt5/erp5_wendelin_data_lake_ingestion/SkinTemplateItem/portal_skins/erp5_wendelin_data_lake/ERP5Site_invalidateOldDatasets.xml
0 → 100644
View file @
197fe2fb
<?xml version="1.0"?>
<ZopeData>
<record
id=
"1"
aka=
"AAAAAAAAAAE="
>
<pickle>
<global
name=
"PythonScript"
module=
"Products.PythonScripts.PythonScript"
/>
</pickle>
<pickle>
<dictionary>
<item>
<key>
<string>
Script_magic
</string>
</key>
<value>
<int>
3
</int>
</value>
</item>
<item>
<key>
<string>
_bind_names
</string>
</key>
<value>
<object>
<klass>
<global
name=
"NameAssignments"
module=
"Shared.DC.Scripts.Bindings"
/>
</klass>
<tuple/>
<state>
<dictionary>
<item>
<key>
<string>
_asgns
</string>
</key>
<value>
<dictionary>
<item>
<key>
<string>
name_container
</string>
</key>
<value>
<string>
container
</string>
</value>
</item>
<item>
<key>
<string>
name_context
</string>
</key>
<value>
<string>
context
</string>
</value>
</item>
<item>
<key>
<string>
name_m_self
</string>
</key>
<value>
<string>
script
</string>
</value>
</item>
<item>
<key>
<string>
name_subpath
</string>
</key>
<value>
<string>
traverse_subpath
</string>
</value>
</item>
</dictionary>
</value>
</item>
</dictionary>
</state>
</object>
</value>
</item>
<item>
<key>
<string>
_params
</string>
</key>
<value>
<string></string>
</value>
</item>
<item>
<key>
<string>
id
</string>
</key>
<value>
<string>
ERP5Site_invalidateOldDatasets
</string>
</value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment