1. 12 Jul, 2017 36 commits
    • Ayush Tiwari's avatar
      f1170797
    • Ayush Tiwari's avatar
      a10d9f2e
    • Ayush Tiwari's avatar
      a68a1242
    • Ayush Tiwari's avatar
      5292e045
    • Ayush Tiwari's avatar
      bb50d453
    • Ayush Tiwari's avatar
      4df25a91
    • Ayush Tiwari's avatar
      1f140217
    • Ayush Tiwari's avatar
    • Ayush Tiwari's avatar
      erp5_catalog: [Migration] Exception while migrating catalog object · 504f1a71
      Ayush Tiwari authored
      As now, we expect our catalog object to be erp5 catalog and not indexable,
      so its important to not reindex or unindex the old object.
      504f1a71
    • Ayush Tiwari's avatar
      erp5_catalog: Solve name conflict between isIndexable property for ERP5Site and ERP5 Catalog · 6fe99692
      Ayush Tiwari authored
      Earlier, In ERP5Site, we use property 'isIndexable' to have control over indexing and unindexing.
      This was usefull specially in case of bootstrap. To have this control, we had a function
      `isIndexable` in SQLCatalog which we used to switch for enabling-disabling indexation. (769f9fb2)
      
      Now, we do have ERP5 Catalog, which itself is an ERP5 Object, hence it has an accessor `isIndexable`
      of itself which we plan to disable for now(as we don't want circular dependency while indexing
      catalog in itself). But as we did have a function `isIndexable` in SQLCatalog which had control
      over the indexing, we needed to rename this property for ERP5Site.
      
      Hence, now we use property 'isIndexingRequired' for ERP5Site and use to have overall control
      over indexing process.
      
      Please note, we do not intend to change the same property name for any other object. Its just
      for ERP5Site.
      6fe99692
    • Ayush Tiwari's avatar
      erp5_catalog: Change order in which we update Bussiness Template · d603c85d
      Ayush Tiwari authored
      Required as most of the changes we have are in erp5_core and that's
      why its important to install/update it before we install erp5_catalog_storage
      bt5.
      d603c85d
    • Ayush Tiwari's avatar
    • Ayush Tiwari's avatar
      erp5_catalog: Remove usage of default_sql_catalog_id attribute from ERP5CatalogTool class · 76e23d4b
      Ayush Tiwari authored
      For compatibilty, we have added 2 extra functions, but in no way we need
      to use this attribute as it doesn't give any meaning for ERP5CatalogTool
      to have a default SQL Catalog.
      76e23d4b
    • Ayush Tiwari's avatar
      sql_catalog: Drop usage of getattr from SQLCatalog · 8d06e273
      Ayush Tiwari authored
      This would increase compatibility between accessors in erp5_catalog and
      remove the need of copy and patch. At the same time, it also helps in
      increasing performance as using _getOb is always less costly than
      getattr, as it reduces the need to look in acquisition.
      8d06e273
    • Ayush Tiwari's avatar
    • Ayush Tiwari's avatar
      erp5_catalog: [WORKAROUND] Explicitly call reset for component_tool in test. · 00ee9ab4
      Ayush Tiwari authored
      After adding document component object, _p_oid was bing generated at the
      last step of commit hook. This led to problem that _registry_dict for
      dynamic_class 'erp5.component.document' wasn't getting updated with
      the _p_oid of the component created. This led to failing of test because
      to validate object with same reference, the checkConsistency function
      checks in the _registry_dict for reference and _p_oid.
      
      Explicilty calling commit here generates the oid before validation, hence
      make it available for the registry_dict in time.
      
      But, this clearly is a workaround which is clearly not solving the real
      problem of why the _p_oid isn't being generated at the right step.
      
      Also, me(Ayush), tried to recreate this error on a running instance, but
      it worked fine there.
      00ee9ab4
    • Ayush Tiwari's avatar
      erp5_catalog: Do not use lazy_class objects while generating UIDs · 292af552
      Ayush Tiwari authored
      Earlier, class for any Catalog object used to be Products.ZSQLCatalog.SQLCatalog,
      but now as we have shifted SQLCatalog to ERP5Catalog, the catalog objects have
      lazy_class 'erp5.portal_type.Catalog' as their 1st class in mro. This class
      don't have the required attributes.
      
      The lazy_class is dynamic, so its not possible for them to save reserved IDs
      as their attributes. So, we use the next class in mro, which would be
      class 'Products.ERP5Catalog.ERP5Catalog' and save attributes in it.
      
      Notice that this change of 'self.__klass__' is not required if
      we are starting a lock with '_reserved_uid_lock' and then acquire-release it
      consicutively in one go(transaction).
      292af552
    • Ayush Tiwari's avatar
      erp5_catalog: Refactor _catalogObject List to remove the need to copy and patch · 0ffaf172
      Ayush Tiwari authored
      Create FilterDict and Filter class which would be used to imitate the behaviour
      of filter_dict for Catalog.
      0ffaf172
    • Ayush Tiwari's avatar
      erp5_catalog: Dynamic migration · 99230fae
      Ayush Tiwari authored
      And, Patch changeObjectClass extension to remove useless attributes
      
      Copying __dict__ from one object to another brings us to situation where
      we don have many objects which we don't need at all, for example, migrating
      objects with subclasses who were initially OFS objects and later an ERP5
      object can lead to adding subobjects as attributes of the new object, which
      is completely undesirable. To handle this, it is important to delete the
      sub-objects as the attributes for those migrated classes.
      
      erp5_catalog: Refactor _bootstrap for Catalog Tool
      
      Old Catalog Tool didn't have portal_type attribute, so while migrating
      via synchronizeDynamicModule, after _bootstrap, we expect the tool to
      have a portal_type to finalize migration.
      
      This step is now being done only at the end of _bootstrap after we
      change the classes for portal_catalog and its sub-objects.
      
      And, changes to improve performance while dynamic migration.
      
      XXX: Still at this point we are not doing the same thing with
      sub-objects of catalog tool and wait till business template
      installation.
      
      erp5_catalog: Update attributes of the portal_catalog object after migration
      99230fae
    • Ayush Tiwari's avatar
    • Ayush Tiwari's avatar
      erp5_catalog: Use getattr to get catalog attributes · 7062eeb2
      Ayush Tiwari authored
      One of the difference between SQL and ERP5 Catalog is how they
      handle their properties. For ERP5Catalog, we use property_sheets
      which generate setters and getters, for SQLCatalog, properties
      just act as attributes(as one can expect from a property set
      on a class).
      
      This creates a difference on how we use them, especially for
      list_type properties, which have accessors like get<PropertyName>List
      to get the list of objects. For SQLCatalog, there is no such thing.
      This creates problem whereever 'get<PropertyName>' or getProperty(<name>)
      was being used to get multiple property types.
      
      So, we changed this to rely on getattr for the Python Scripts in this
      commit. Note that, getattr isn't good if we care about performance,
      but given that its used only twice, this can be better than adding
      extra function overhead.
      7062eeb2
    • Ayush Tiwari's avatar
    • Ayush Tiwari's avatar
      1e70e4cd
    • Ayush Tiwari's avatar
    • Ayush Tiwari's avatar
    • Ayush Tiwari's avatar
      erp5_catalog: Update tests according to the changes in ERP5Catalog · 973f12b3
      Ayush Tiwari authored
      Also, add test for migration in testERP5Catalog(as the first test to be run)
      and migrate Catalog in afterSetup in testERP5CatalogSecurityUidOptimization.
      973f12b3
    • Ayush Tiwari's avatar
    • Ayush Tiwari's avatar
    • Ayush Tiwari's avatar
    • Ayush Tiwari's avatar
      erp5_catalog: Conversion script to convert ZSQL Method and PythonScript to ERP5 objects · bd76f014
      Ayush Tiwari authored
      Also, Update BusinessTemplate installation with updated filter_dict
      This removes the need to copy-patch or if-else on meta_type of catalog.
      Use dynamic migration while installing the catalog method objects for
      bt5.
      
      Note : here we use conversion method for python script but dynamic migration for
      ZSQL Methods. This is necessary because after the conversion to erp5 object, we need
      to compile the body to generate code object for python script.
      By just changing the class, this was not possible.
      
      erp5_catalog: Simplify bt5 installation
      bd76f014
    • Ayush Tiwari's avatar
      erp5_catalog: Update erp5_core business template · 6dd33716
      Ayush Tiwari authored
      Contains views, property sheets,  portal types concerning migration of
      portal_catalog, sql catalog and their subobjects to erp5 object.
      
      Portal Types:
      1. Catalog : ERP5 Catalog object (an ERP5 Folder), which earlier used to be OFS Folder.
      	Actions:
      		Update Catalog
      		Clear Catalog
      		Clear Reserved
      		Export Properties
      
      2. Catalog Tool: Portal Catalog where we can add and use multiple ERP5 catalogs.
      	Actions:
      		Hot ReindexAll
      
      3. SQL Method: SQL methods with their views inside erp5.
      	Actions:
      		Run Method
      
      Property Sheet:
      1. Catalog
      2. CatalogTool
      3. SQLMethod
      	Containing properties for the various portal_types/classes respectively.
      4. CatalogFilter
      
      Also, filter_dict for erp5_catalog would now not be a Persistent Mapping
      object. The info inside filter_dict is being saved inside the SQL Method
      objects as their properties.
      
      Views:
      1. Catalog (View)
      2. CatalogTool(View, Properties, Filtered Items), Object Actions
      3. SQLMethod (View, Filter)
      4. Python Script (Filter)
      
      Extras:
      
      	- Dialog view for catalog before clear_catalog
      
      	- Add typeBaseMethod(s) for PythonScript and SQLMethod portal types:
      The methods <portal_type>_getRedirectParameterDictAfterAdd are required for these
      portal_types, cause their __call__ methods are overridden, so after addition
      of new objects for them, it was taking to the url concerning wherever __call__
      methods directed them.
      Now, we change them to redirect to 'abosulte_url+'/view'' which is basically
      the homepage for these catalog methods.
      
      	- Do not use default ERP5 Catalog to show properties:
      Earlier, in tales for properties for any erp5 catalog, we showed
      the values for the default_erp5_catalog. But keeping in mind that
      we can have multiple catalogs at same time, the old approach
      was wrong. Hence, we now display properties for the current catalog
      
      	- Search fo Catalog_viewContentList and Catalog_viewFilterList
      6dd33716
    • Ayush Tiwari's avatar
      erp5_catalog: Update testCopySupport according to changes in portal_catalog · d43e6930
      Ayush Tiwari authored
      Its better to change the tests where they need to call getpath function of
      portal_catalog to use portal_catalog.getpath instead of portal_catalog.getPath,
      as we have overridden the 'getPath' function for CatalogTool class due to change
      in inherited class.
      d43e6930
    • Ayush Tiwari's avatar
      erp5_catalog: Rename reindexObject method to use them as new methods for CatalogTool. · 3f7abc5b
      Ayush Tiwari authored
      - This step is needed due to the use of BaseTool as Base class for CatalogTool
        due to which there were conflict between reindexObject from the Base and the one
        from the BaseTool.
      3f7abc5b
    • Ayush Tiwari's avatar
      erp5_catalog: New ERP5CatalogTool based on BaseTool from ERP5Type · 682e4912
      Ayush Tiwari authored
      	- Remove copy-pasting all code from CatalogTool, better to rely on inheritence
      	- Remove unnecessary imports
      	- Add argument id in __init__ class
      	- Add functions _isBootstrapRequired and _bootstrap
      	- Update BusinessTemplate installation according to changes made in ERP5Catalog and Tool
      682e4912
    • Ayush Tiwari's avatar
      erp5_catalog: New Catalog for ERP5. · c43ce942
      Ayush Tiwari authored
      We try to move from using SQLCatalog to ERP5Catalog as the default Catalog inside
      ERP5. The major difference is use of Folder class as the base for Catalog from
      Products.ERP5Type.Core
      
      Significant changes:
      	-Add dtml form for creating ERP5Catalog object
      	-Inherit from Catalog class from Products.ZSQLCatalog.SQLCatalog instead of copy-pasting the whole code again.
      	-Add allowed_types for ERP5Catalog tool
      	-Monkey patch some property setters and getters to maintain consistency
      	-Update id and title for ERP5Catlog while class initialization
      	-Set declarative securities and solve some inheritance conflicts
      	-Add isRADContent for ERP5Catalog Class
       	-Solve inheritence conflict for _setPropValue function in ERP5Catalog class
      	-Add SQL Method portal_type in allowed_types for ERP5Catalog class
      	-Override getCatalogMethodIds cause it uses global variable in SQLCatalog.Catalog
      	-Redefine security declarations
      	-Add functions for object_actions of Catalog portal_type in ERP5Catalog object
      	-Add filter_dict and _properties attributes for compatibilty
      	-isIndexable should be function instead of Accessor property for Catalog object
      c43ce942
    • Ayush Tiwari's avatar
      erp5_catalog: Add SQLMethod class as ERP5 based class · 7fd32fc0
      Ayush Tiwari authored
      Here, we are creating the SQLMethod class based on ZSQLMethods.SQL
      class and XMLObject. Also, we are moving important attributes to
      property in 'SQL Method' property sheets.
      7fd32fc0
  2. 11 Jul, 2017 2 commits
  3. 10 Jul, 2017 1 commit
    • Jérome Perrin's avatar
      configure cloudooo by URL in default preference · 2cf95f9e
      Jérome Perrin authored
      Since !306 we have configured system preference to use cloudoo from the Slapos SR, but this configuration was still old style configuration setting host / port.
      
      It caused a test to fail 
      ```
      ======================================================================
      FAIL: test_01_PreferenceSetup (testIngestion.TestIngestion)
      ----------------------------------------------------------------------
      Traceback (most recent call last):
        File "/srv/slapgrid/slappart7/srv/testnode/aai/soft/f030becfa6e3878c62ccb9bccc43a291/parts/erp5/Products/ERP5OOo/tests/testIngestion.py", line 1048, in test_01_PreferenceSetup
          self.assertEqual(preference_tool.getPreferredOoodocServerAddress(), conversion_dict['hostname'])
      AssertionError: 'erp5-cloudooo' != 'localhost'
      ```
      
      and that was a problem with the test, we don't have to care about `getPreferredOoodocServerAddress`, as only URL matters now (it was just working because both were set to localhost).
      
      /reviewed-on nexedi/erp5!315
      2cf95f9e
  4. 07 Jul, 2017 1 commit
    • Julien Muchembled's avatar
      tests: clean up code related to configuration of cloudooo/memcached connectors · f84f4cdb
      Julien Muchembled authored
      - The conversion server is supposed to be configured in a system preference,
        so do this instead of using a normal preference.
      - _getConversionServerDict -> _getConversionServerUrl, to make clear that
        cloudooo is now configured by a url, instead of a host/port couple.
      - Refactoring: From the moment where setUpERP5Site() sets up things
        automatically, we don't the "same" duplicated code throughout many test to
        redo cloudooo configuration.
      - In the promise file, the volatile/persistent memcached url were swapped.
      f84f4cdb