WIP: erp5_web_renderjs_ui: Change date separator from a dash to a slash before sending query to Zope
Zope DateTime uses local timezone automatically if date seperator was a slash like 2020/02/08, but if it was a dash like 2020-02-08, Zope DateTime ignores local timezone and timezone become GMT+0. Then if server is not located in GMT+0 zone, date search does not work. Since HTML5 date input field uses dash as a separator, it must be changed to slash before sending query to Zope.
# Server timezone is GMT+9. >> print DateTime('2020-02-18') 2020/02/18 00:00:00 GMT+0 >> print DateTime('2020/02/18') 2020/02/18 00:00:00 GMT+9
# # Server timezone is GMT+9 # context.portal_catalog(effective_date='2020-02-18', src__=1) # SELECT DISTINCT catalog.path, catalog.uid FROM ( catalog AS `catalog` INNER JOIN versioning AS `versioning` ON `versioning`.`uid` = `catalog`.`uid` ) WHERE 1 = 1 AND ((`versioning`.`effective_date` >= "2020-02-18 00:00:00" AND `versioning`.`effective_date` < "2020-02-19 00:00:00") AND (`catalog`.`viewable_owner` = 'zope' OR `catalog`.`security_uid` IN (17, 20, 23, 24, 25, 28, 29, 30, 31, 32, 33, 34, 35, 36, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51)))
# # Server timezone is GMT+9 # context.portal_catalog(effective_date='2020/02/18', src__=1) # SELECT DISTINCT catalog.path, catalog.uid FROM ( catalog AS `catalog` INNER JOIN versioning AS `versioning` ON `versioning`.`uid` = `catalog`.`uid` ) WHERE 1 = 1 AND ((`versioning`.`effective_date` >= "2020-02-17 15:00:00" AND `versioning`.`effective_date` < "2020-02-18 15:00:00") AND (`catalog`.`viewable_owner` = 'zope' OR `catalog`.`security_uid` IN (17, 20, 23, 24, 25, 28, 29, 30, 31, 32, 33, 34, 35, 36, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51)))
@romain Could you review this MR?
marked as a Work In ProgressToggle commit list
As you show on your example, this issue also exists when calling
portal_catalogdirectly. For example, an user can get it when using the
erp5_xhtml_styleglobal search. Maybe we would like to get rid of it on
portal_catalogparsing configuration level.
One way could be to dynamically change the behaviour on all date columns string parsing. This topic must be discussed with more people probably.
What I'm sure is that changing ERP5JS search editor is only a workaround, which could break applications not relying on erp5 storage. Any other html5 form with a date input will lead to the same behaviour. One could imagine to handle it on jio erp5storage level. But I don't see how this can be correctly done without being aware of the portal_catalog full configuration.
Oh I see. Then I should fix portal_catalog date columns string parsing.
There's no DateTimeField here ? in many places it's DateTimeField validator which transforms whatever was submitted in request to a date that zope understands.
For reference, in old UI:
- Search dialog uses a date time field
- Listbox search columns also does something so that if a data column is rendered as a date time field, this date time field is used to pass a date or a date range to catalog
- in other contexts (like the global search), then I guess it's also passed as a string
Also, for the records in zodbtools!8 (merged) we have been using https://dateparser.readthedocs.io/en/latest/ to parse dates, taking as inspiration the way
git logallows to query
git log --since "last week". I was thinking that it might be cool to have it in ERP5, but if I remember correctly, dateparser does not really support parsing ranges, you cannot say "this month", you can only say from "4 weeks ago" until "yesterday". Anyway, this sounds like science fiction.
Maybe what we need is a proper widget to select a timeframe ( two datetimes, some presets like current month, current day etc ) ?
In reality DateTime behavoir is bad, although it breaks compatibility.
If you look at https://github.com/zopefoundation/DateTime/issues?q=is%3Aissue+is%3Aclosed close to 50% of bugs are this exact problem BTW :)
All the search editor logic only occurs on client side. There is no interaction with ERP5. Its only purpose is to change the browser URL.
The issue could be globally fixed on the catalog DateTimeKey level, so that we get the same parsing behaviour with
slash, leading to the same SQL query.
This will not change the
closedToggle commit list
reopenedToggle commit list
Why do we have to use a date as string and not keep JIO query objects ?
The search editor uses Query object to represent the final query. But, as its purpose is to modify the browser URL, the query is serialized as a search string.
When the panel is open a second time, the query parsing does not currently regenerate the contains logic. This issue must be fixed on jio query parsing level.
No. jIO specifications is to follow ERP5 query serialization format.