WIP: erp5: bump version for pylint and astroid to the latest ones with Python2 support (!648) · Merge Requests · nexedi / slapos

WIP: erp5: bump version for pylint and astroid to the latest ones with Python2 support

After Jérome told me that several false-positive warnings we have had with pylint have been fixed in newest version of pylint and astroid, I have updated versions to the latest python2 version (1.9.5) and ran Unit Tests against my own ERP5 branch. As you can see from the Test Result, there are not so many failures and all of them are easy to fixed. Moreover, it seems that 1.9 branch is still maintained (the last release was on 16th of July 2019).

Next, I checked pylint performances by calling checkPythonSource() 10 times on each ZODB Components in erp5_base, erp5_crm and erp5_forge and as you can see the first results I've got so far are not good at all (my laptop is quite slow but still...):

ID	1.9.5 slowness factor	1.4.4 stddev	1.4.4 avg (ms)	1.9.5 avg (ms)	1.9.5 stddev
document.erp5.GeographicalArea	4.8	589	866.7	4161.3	47
document.erp5.GeographicalPoint	8.3	3	335.4	2790.6	243
document.erp5.Login	8.0	22	127.6	1018.8	30
document.erp5.ScriptConstraint	7.7	78	475.8	3664.2	40
document.erp5.TransformDocToDocx	13.8	7	57.6	793.2	45
document.erp5.TransformDocToHtml	14.0	1	55.3	775.2	6
document.erp5.TransformDocxToDocy	13.8	1	56.5	778.6	16
document.erp5.TransformDocxToOdt	13.4	2	57.9	774.2	7
document.erp5.TransformDocyToDocx	14.0	1	55.5	776.1	11
document.erp5.TransformHtmlToPdf	18.2	5	41.5	755.6	10
document.erp5.TransformImageToBmp	29.6	2	27.8	825.5	340
document.erp5.TransformImageToPcx	26.6	1	26.9	713.3	9
document.erp5.TransformOdpToHtml	13.5	2	57.4	774.0	5
document.erp5.TransformOdpToPdf	13.9	0	55.7	772.5	5
document.erp5.TransformOdpToPptx	15.5	2	56.8	877.5	305
document.erp5.TransformOdpToSxi	13.5	1	57.5	774.7	9
document.erp5.TransformOdsToHtml	13.8	1	56.2	773.7	5
document.erp5.TransformOdsToPdf	14.0	0	55.4	774.9	7
document.erp5.TransformOdsToXlsx	13.8	1	56.7	783.0	14
document.erp5.TransformOdtToDocx	13.4	2	58.5	782.4	11
document.erp5.TransformOdtToHtml	13.9	1	55.8	775.5	8
document.erp5.TransformPdfToBmp	14.1	0	54.9	777.5	13
document.erp5.TransformPptToPptx	13.8	2	56.5	777.3	13
document.erp5.TransformPptxToOdp	13.3	2	57.9	772.7	6
document.erp5.TransformPptxToPpty	14.0	1	55.7	777.8	7
document.erp5.TransformPptyToPptx	16.3	0	55.2	899.2	367
document.erp5.TransformSxiToHtml	13.7	1	56.7	777.4	8
document.erp5.TransformSxiToOdp	13.4	2	57.8	776.6	6
document.erp5.TransformXlsToOds	14.0	1	55.6	776.3	8
document.erp5.TransformXlsToXlsx	15.8	0	55.1	868.5	270
document.erp5.TransformXlsxToOds	13.3	1	58.4	778.2	11
document.erp5.TransformXlsxToXlsy	13.9	1	55.8	775.6	7
document.erp5.TransformXlsyToXlsx	14.0	1	55.3	776.5	6
extension.erp5.BarcodeUtils	6.7	31	124.5	833.2	57
extension.erp5.BaseMigration	10.5	117	76.9	809.1	218
extension.erp5.CRMMigration	14.0	30	55.7	778.5	118
extension.erp5.CRMUtils	19.2	2	37.3	713.9	7
extension.erp5.DocumentExtraction	5.2	42	188.6	984.0	53
extension.erp5.FolderWorkflowActionUtils	4.5	3	222.1	1010.3	11
extension.erp5.Glossary	4.5	156	276.3	1247.0	377
extension.erp5.InventoryBrain	5.1	155	1125.1	5705.0	63
extension.erp5.PersonLoginMigration	21.4	0	33.2	709.7	7
extension.erp5.ProxyFieldUtils	19.2	2	38.5	737.7	14
extension.erp5.PythonScriptParserUtility	5.2	213	184.3	949.7	338
extension.erp5.SearchUtility	7.7	29	113.3	873.9	55
extension.erp5.StandardSecurity	5.1	204	160.4	815.4	10
extension.erp5.ZODBHistory	4.7	206	181.6	859.3	9
module.erp5.Git	3.2	27	938.5	2989.5	144
module.erp5.Subversion	2.6	490	1081.2	2827.2	71
module.erp5.SubversionClient	3.3	14	507.9	1655.4	20
module.erp5.TransformLib	8.4	17	101.7	855.0	28
module.erp5.WorkingCopy	3.4	35	772.4	2593.3	271
test.erp5.testActivityTool	4.1	366	715.8	2903.7	403
test.erp5.testCRMSupportRequest	3.2	5	752.6	2412.5	68
tool.erp5.CallableTool	10.4	1	269.7	2814.3	28
tool.erp5.DiffTool	5.6	49	342.5	1909.2	385

I have not yet investigated why it is that much slower (1.9.5 has many more new checks so this may be one of the reason) so take these results with a pinch of salt. I will check further and add some more info to this MR.

Aside from this, there is also a pylint/astroid bug which I encountered (and this is why not all the Components of the aforementioned bt5 are listed here): https://github.com/PyCQA/pylint/issues/2188 . This will need to be fixed or workaround'ed as this checking source code of any ZODB Components using Folder class (Products.ERP5Type.Core.Folder) fails with a InconsistentMroError error. I started writing a patch but it is still too dirty to be usable.

After Jérome told me that several false-positive warnings we have had with pylint have been fixed in newest version of pylint and astroid, I have updated versions to the latest python2 version (1.9.5) and ran Unit Tests against my own ERP5 branch. As you can see from the [Test Result](https://nexedi.erp5.net/test_result_module/20191031-6CFDE412), there are not so many failures and all of them are easy to fixed. Moreover, it seems that 1.9 branch is still maintained (the last release was on 16th of July 2019).

Next, I checked pylint performances by calling `checkPythonSource()` 10 times on each ZODB Components in `erp5_base`, `erp5_crm` and `erp5_forge` and as you can see the first results I've got so far are not good at all (my laptop is quite slow but still...):

| ID | 1.9.5 slowness factor | 1.4.4 stddev | 1.4.4 avg (ms) | 1.9.5 avg (ms) | 1.9.5 stddev |
| --- | ---: | ---: | ---: | ---: | ---: |
| document.erp5.GeographicalArea | 4.8 | 589 | 866.7 | 4161.3 | 47 |
| document.erp5.GeographicalPoint | 8.3 | 3 | 335.4 | 2790.6 | 243 |
| document.erp5.Login | 8.0 | 22 | 127.6 | 1018.8 | 30 |
| document.erp5.ScriptConstraint | 7.7 | 78 | 475.8 | 3664.2 | 40 |
| document.erp5.TransformDocToDocx | 13.8 | 7 | 57.6 | 793.2 | 45 |
| document.erp5.TransformDocToHtml | 14.0 | 1 | 55.3 | 775.2 | 6 |
| document.erp5.TransformDocxToDocy | 13.8 | 1 | 56.5 | 778.6 | 16 |
| document.erp5.TransformDocxToOdt | 13.4 | 2 | 57.9 | 774.2 | 7 |
| document.erp5.TransformDocyToDocx | 14.0 | 1 | 55.5 | 776.1 | 11 |
| document.erp5.TransformHtmlToPdf | 18.2 | 5 | 41.5 | 755.6 | 10 |
| document.erp5.TransformImageToBmp | 29.6 | 2 | 27.8 | 825.5 | 340 |
| document.erp5.TransformImageToPcx | 26.6 | 1 | 26.9 | 713.3 | 9 |
| document.erp5.TransformOdpToHtml | 13.5 | 2 | 57.4 | 774.0 | 5 |
| document.erp5.TransformOdpToPdf | 13.9 | 0 | 55.7 | 772.5 | 5 |
| document.erp5.TransformOdpToPptx | 15.5 | 2 | 56.8 | 877.5 | 305 |
| document.erp5.TransformOdpToSxi | 13.5 | 1 | 57.5 | 774.7 | 9 |
| document.erp5.TransformOdsToHtml | 13.8 | 1 | 56.2 | 773.7 | 5 |
| document.erp5.TransformOdsToPdf | 14.0 | 0 | 55.4 | 774.9 | 7 |
| document.erp5.TransformOdsToXlsx | 13.8 | 1 | 56.7 | 783.0 | 14 |
| document.erp5.TransformOdtToDocx | 13.4 | 2 | 58.5 | 782.4 | 11 |
| document.erp5.TransformOdtToHtml | 13.9 | 1 | 55.8 | 775.5 | 8 |
| document.erp5.TransformPdfToBmp | 14.1 | 0 | 54.9 | 777.5 | 13 |
| document.erp5.TransformPptToPptx | 13.8 | 2 | 56.5 | 777.3 | 13 |
| document.erp5.TransformPptxToOdp | 13.3 | 2 | 57.9 | 772.7 | 6 |
| document.erp5.TransformPptxToPpty | 14.0 | 1 | 55.7 | 777.8 | 7 |
| document.erp5.TransformPptyToPptx | 16.3 | 0 | 55.2 | 899.2 | 367 |
| document.erp5.TransformSxiToHtml | 13.7 | 1 | 56.7 | 777.4 | 8 |
| document.erp5.TransformSxiToOdp | 13.4 | 2 | 57.8 | 776.6 | 6 |
| document.erp5.TransformXlsToOds | 14.0 | 1 | 55.6 | 776.3 | 8 |
| document.erp5.TransformXlsToXlsx | 15.8 | 0 | 55.1 | 868.5 | 270 |
| document.erp5.TransformXlsxToOds | 13.3 | 1 | 58.4 | 778.2 | 11 |
| document.erp5.TransformXlsxToXlsy | 13.9 | 1 | 55.8 | 775.6 | 7 |
| document.erp5.TransformXlsyToXlsx | 14.0 | 1 | 55.3 | 776.5 | 6 |
| extension.erp5.BarcodeUtils | 6.7 | 31 | 124.5 | 833.2 | 57 |
| extension.erp5.BaseMigration | 10.5 | 117 | 76.9 | 809.1 | 218 |
| extension.erp5.CRMMigration | 14.0 | 30 | 55.7 | 778.5 | 118 |
| extension.erp5.CRMUtils | 19.2 | 2 | 37.3 | 713.9 | 7 |
| extension.erp5.DocumentExtraction | 5.2 | 42 | 188.6 | 984.0 | 53 |
| extension.erp5.FolderWorkflowActionUtils | 4.5 | 3 | 222.1 | 1010.3 | 11 |
| extension.erp5.Glossary | 4.5 | 156 | 276.3 | 1247.0 | 377 |
| extension.erp5.InventoryBrain | 5.1 | 155 | 1125.1 | 5705.0 | 63 |
| extension.erp5.PersonLoginMigration | 21.4 | 0 | 33.2 | 709.7 | 7 |
| extension.erp5.ProxyFieldUtils | 19.2 | 2 | 38.5 | 737.7 | 14 |
| extension.erp5.PythonScriptParserUtility | 5.2 | 213 | 184.3 | 949.7 | 338 |
| extension.erp5.SearchUtility | 7.7 | 29 | 113.3 | 873.9 | 55 |
| extension.erp5.StandardSecurity | 5.1 | 204 | 160.4 | 815.4 | 10 |
| extension.erp5.ZODBHistory | 4.7 | 206 | 181.6 | 859.3 | 9 |
| module.erp5.Git | 3.2 | 27 | 938.5 | 2989.5 | 144 |
| module.erp5.Subversion | 2.6 | 490 | 1081.2 | 2827.2 | 71 |
| module.erp5.SubversionClient | 3.3 | 14 | 507.9 | 1655.4 | 20 |
| module.erp5.TransformLib | 8.4 | 17 | 101.7 | 855.0 | 28 |
| module.erp5.WorkingCopy | 3.4 | 35 | 772.4 | 2593.3 | 271 |
| test.erp5.testActivityTool | 4.1 | 366 | 715.8 | 2903.7 | 403 |
| test.erp5.testCRMSupportRequest | 3.2 | 5 | 752.6 | 2412.5 | 68 |
| tool.erp5.CallableTool | 10.4 | 1 | 269.7 | 2814.3 | 28 |
| tool.erp5.DiffTool | 5.6 | 49 | 342.5 | 1909.2 | 385 |

Aside from this, there is also a pylint/astroid bug which I encountered (and this is why not all the Components of the aforementioned bt5 are listed here): https://github.com/PyCQA/pylint/issues/2188 . This will need to be fixed or workaround'ed as this checking source code of any ZODB Components using `Folder` class (`Products.ERP5Type.Core.Folder`) fails with a `InconsistentMroError` error. I started writing a patch but it is still too dirty to be usable.