1. 12 Sep, 2018 5 commits
    • Vincent Pelletier's avatar
      SQLCatalog_deferFullTextIndex{,Activity}: Use serialization_tag . · b41dc8e0
      Vincent Pelletier authored
      Just like regular indexations, fulltext indexations are subject to last-commit-wins.
      Which means that it is possible to reach a state where the fulltext table is
      persistently desynchronised from ZODB:
      - start fulltext indexation activity on many documents (typically: 100)
      - modify one of the documents being indexed
      - start fulltext indexation activity caused by this edit, and assume indexation only
      happens for this object
      - commit the single-object indexation (because it is very fast to retrieve fulltext
      data from just one document)
      - commit the many-objects indexation later (because it is much slower to
      retrieve 100 fulltext representations)
      
      As a consequence, cod emust spawn one fulltext indexation activity per
      document, each with the appropriate serialisation tag. Serialisation tag which
      must not conflict with regular indexation, so use a fixed prefix.
      As a consequence of having to spawn one activity per document, use a
      grouping method to still index by batches to amortise transaction overhead.
      Keep the same method_id as before for backward-compatibility (maybe
      dependencies on this value exist, even though it is bad practice).
      Rewrite SQLCatalog_deferFullTextIndexActivity so ot works as a grouping
      method, simplifying it in the process:
      - build parameter_dict with all entries, as we already know all needed keys
      - None is not callable, so test "not None" in just one expression
      - remove whitespace at end of line
      - use GroupedMessage API
      b41dc8e0
    • Jérome Perrin's avatar
      fixup! fixup! runTestSuite: support multiple zserver addresses and frontends URLs · 43d16f61
      Jérome Perrin authored
      ''.split(',') is [''] which evaluates to false ...
      e80b37ff was still incorrect
      43d16f61
    • Jérome Perrin's avatar
      fixup! runTestSuite: support multiple zserver addresses and frontends URLs · e80b37ff
      Jérome Perrin authored
      This was not really backward compatible with old slapos versions not passing
      zserver_address_list
      e80b37ff
    • Jérome Perrin's avatar
      web_renderjs_ui_test: more robust way of waiting for CKEditor · 556a18fa
      Jérome Perrin authored
      To answer question in the comment:
      > How to inspect inside another iframe?
      
      use selectFrame (after waiting for the frame to be added in the DOM of course).
      
      Reference of selenium commands: https://web.archive.org/web/20160807232621/http://release.seleniumhq.org/selenium-core/1.0.1/reference.html#selectFrame (I'm not sure it's still on the web)
      556a18fa
    • Jérome Perrin's avatar
      fixup! fixup! web: no need initialize web site languages · f2361768
      Jérome Perrin authored
      Tests asserts that size have all languages available, so let's be
      compatible and just set the default language to the current selected
      one at site creation.
      f2361768
  2. 11 Sep, 2018 6 commits
  3. 07 Sep, 2018 6 commits
  4. 05 Sep, 2018 4 commits
  5. 04 Sep, 2018 7 commits
  6. 03 Sep, 2018 2 commits
  7. 01 Sep, 2018 5 commits
    • Vincent Pelletier's avatar
      SQLCatalog: Optimise and simplify transactional_cache_decorator. · 189db11c
      Vincent Pelletier authored
      Auto-generate cache_id, removing the risk of duplicating one.
      Inline getInstanceID as it is its only caller left.
      Avoid wrapping if getTransactionalVariable cannot be imported.
      Avoid intermediate variables, at the cost of dupicating logic but
      accelerating run-time.
      Use functools.wraps .
      Access _cache_sequence_number directly - these pieces of code are friends
      anyway, we already set a volatile property on Catalog instance. Saves one
      getattr and one call.
      189db11c
    • Vincent Pelletier's avatar
      SQLCatalog: Assorted cleanups and simplifications. · 66f13089
      Vincent Pelletier authored
      No backslash at EOL.
      Avoid splitting messages, to ease code lookup.
      LOG's "error" argument accepts a boolean, which removes the need to touch
      sys.exc_info.
      Use modern exception raising syntax.
      66f13089
    • Vincent Pelletier's avatar
      b0978eaf
    • Vincent Pelletier's avatar
      SQLCatalog: Drop unused function. · 1f2850cd
      Vincent Pelletier authored
      1f2850cd
    • Vincent Pelletier's avatar
      SQLExpression: Fold _getSelectDict into __init__, its only caller. · 028060e6
      Vincent Pelletier authored
      _getSelectDict was also calling itself on sub-sql_expressions, but this
      was useless: they would already have called it, and discarded one of its
      returned values. Instead, preserve both returned values in __init__ and
      access these attributes directly when iterating over sub-sql_expressions.
      Also, fold canMergeSelectDict back into its only 2 call places, saving one
      getattr and one call.
      Also, get rid of a few properties which were only used in _getSelectDict.
      Also, get rid of two iterable copies: sql_expression_list when it is
      already a list (no need to cast it again to a list, we just built one with
      the list comprehension), and select_dict being cast into a list when
      entering _getSelectDict, despite not being used (besides in extra
      _getSelectDict calls, which are now entirely avoided).
      028060e6
  8. 31 Aug, 2018 3 commits
  9. 29 Aug, 2018 2 commits
    • Vincent Pelletier's avatar
    • Vincent Pelletier's avatar
      ZMySQLDA: Optimise date & datetime parsing in results. · 01e14257
      Vincent Pelletier authored
      A pprofile run shows DateTime_or_None costing 3ms per value (including all
      callees).
      While this may seem low, it is common to receive hundreds of such values
      in single query results (ex: web mode document & webpage lookups),
      so the total time spent parting dates per query can become significantly
      larger than the time to execute the query itself.
      In the case of this measure setup, 504 dates were processed for a total of
      1.5 seconds.
      DateTime is slow to parse strings because it cannot rely on a pre-known
      format. With MySQL/MariaDB, the format is know for those column types, so
      implement specific parsing to get better performance.
      On the same setup, pprofile run shows a parsing time of 123µs per value in
      DATETIME_to_DateTime_or_None, or 65ms total.
      DATE_to_DateTime_or_None should not deviate significantly from this.
      01e14257